GNU bug report logs - #53803
[PATCH v2] doc: Add Writing Service Configuration section.

Previous Next

To reply to this bug, email your comments to 53803 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Xinglu Chen <public <at> yoctocell.xyz>
To: Andrew Tropin <andrew <at> trop.in>, guix-devel <at> gnu.org,
 guix-patches <at> gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [PATCH v2] doc: Add Writing Service Configuration section.
Date: Sat, 05 Feb 2022 15:25:06 +0100

[Message part 1 (text/plain, inline)]

Andrew schrieb am Donnerstag der 23. Dezember 2021 um 16:16 +03:

>> Also, is should there be any preference for using alists or list of
>> lists or vice versa?
>
> Now it should be clear that a -configuration record is preferable as a
> service value, lists and alists are special cases for auxiliray
> services and shouldn't be used in most cases.

But what is the preference between a list and an alist?  Why does
‘etc-service-type’ use

  '((file contents))

and ‘home-file-service-type’ use

  '((file . contents))

?

>>> +There is a module @code{gnu service configuration}, which contains
>>> +helpers simplifying configuration definition process.  Take a look at
>>> +@code{gnu services docker} module or grep for
>>> +@code{define-configuration} to find usage examples.
>>> +
>>> +@c Provide some examples, tips, and rationale behind @code{gnu service
>>> +@c configuration} module.
>>
>> Note that I already sent a patch that (at least tries to) document (gnu
>> service configuration)[1].
>>
>> One thing that is lacking is when to use (guix records) (which isn’t
>> documented yet) vs (gnu service configuration).  There should probably
>> be one or two paragraphs about that.
>>
>
> Saw it, I'll try to review and comment on it, when I'll get some spare
> time.  I'll keep this comment for now, and after the section about gnu
> service configuration module is merged, we will add links to it and
> provide more info and examples on implementing actual configurations.

It has already been merged now.  :-)

>>> +configuration file name, but in kebab-case: bashrc for bashrc,
>>
>> Not everyone might familiar with what exactly “kebab-case” means; we
>> should probably leave a footnote or something.
>>
>> “…@code{bashrc} for @file{.bashrc}…”
>>
>> It should also mention that preceding dots should be removed as well.
>> What should happend with files named ‘file.ext’?  Should the field be
>> named ‘file-ext’?
>
> Added a footnote, provided more expressive examples
> @code{bashrc} for @file{.bashrc},
> @code{bash-profile} for @file{.bash_profile},
> @code{tmux-conf} for @file{tmux.conf}, etc.

I suggested adding a footnote for the meaning of “kebab-case”.  I don’t
think these examples should just be in a footnote.  Footnotes are
usually used for stuff that might be distracting if put in the text; I
wouldn’t consider these examples distracting; they are very valuable.

>>> +bash-profile for bash_profile, etc.  The implementation for such fields
>>
>> “…@code{bash-profile} for @file{.bash_profile}.
>>
>> Also, many services have an ‘extra-content’, ‘extra-config’, or
>> ‘extra-options’ field.  In most cases these just take a string and
>> appends it to some configuration file.  Should these instead be named
>> ‘sshd_config’, ‘xserver-conf’, and ‘asound-config’, respectively?
>>
>
> I find this pattern purely-established (content vs conf vs options),
> unclear (you can never know where this extra content will be inserted
> until you take a look at implementation of serialization function)

That’s why we have documentation for all the fields.  Moreover, if it
wasn’t documented, the order of the contents of the fields of
‘home-bash-configuration’ aren’t obvious to the user either.

>>> +There is no any special requirements or
>>> +recommendations here, but it's necessary to make it possible to disable
>>> +all the effects of such fields to provide a user with an empty
>>> +configuration and let them generate it from scratch with only field for
>>> +configuration file.
>>
>> I don’t really understand what is meant by “let them generate it from
>> scratch with only field for configuration file”.  
>
> The good examples of the bad behavior are alsa and nginx service types,
> they always provide some boilerplate with reasonably good default
> configuration, but you can't alter it by setting some fields to #f or
> some other values.

That applies for the Bash service as well; it unconditionally adds stuff
to ~/.bash_profile.

> For nginx it's only partially true, you actually can use `file` field,
> but it will alter the effect of all other fields and will just use the
> file as nginx.conf, kinda conforms what I'm asking here, but makes all
> other fields useless.
>
> Added the following explanation to this item:
>
> --8<---------------cut here---------------start------------->8---
> For example, setting @code{guix-defaults?} to
> @code{#f} and @code{aliases} to @code{'()} will give user an ability to
> control the content of @file{.bashrc} solely by setting the value of
> @code{bashrc} field.
> --8<---------------cut here---------------end--------------->8---
>
>
>>
>> It doesn’t mention if a configuration record should cover all the
>> configuration options available in a configuration file.  For example,
>> the current ‘openssh-configuration’ has quite a few options, but these
>> obviously don’t cover all the options available in /etc/ssh/sshd_config,
>> which is why there is an “escape hatch”, ‘extra-content’ field.
>>
>> In some cases a program might have too many configuration fields for us
>> to map using configuration records, e.g., Git.  In rde, the approach we
>> took was to use nested lists to represent the INI configuration.  I
>> think this approach could also be mentioned here.
>>
>
> This is mentioned below, as well as the problem of closed-world
> assumption.  Software should be fully configurable with field for
> respective config file, escape hatch should be a part of this field.
>
> Escape hatch is necessary to allow to reuse already existing
> configuration, but not to provide configuration, which can't be
> expressed by respective configuration field.

But the Git service in rde uses a ‘config-extra-content’ field, which is
basically the same thing as ‘extra-content’ or ‘extra-config’, right?

> and uneccesary (we do not need extra-* fields because we can add any
> extra content using G-expression inside our primary configuration, see
> sway example below).

>>> +simple, it requires to write serializer once for one configuration
>>                       ^ “one” or “you”
>
> Sounds better for me without one or you.

I don’t think it is grammatically correct to not put an object after
“requires to”.

>>> +configuration and resulting app config, including different field
>>> +casing, new semantic units.
>>
>> But it means that the syntax for configuring a program is more
>> Scheme-like.  For example, the Dovecot service provides a very
>> complicated but Schemeish interface for configuring Dovecot, though as
>> you have mentioned, it might be missing some fields since the Dovecot
>> configuration file might have changed since the creation of the service.
>>
>
> Yes it is more Scheme-flavored, but it doesn't mean good.  I can write a
> good rationale on this topic, but will do it next time, now I'll just
> give you an example, which should be relevant to you: Imagine writing an
> importer (for `guix home import` for example) from XML to SXML, now
> imagine that instead of SXML we have Scheme-like configuration.  It
> hours and days more work.  Implementing and maintaining such
> Scheme-flavored configuration is a big pain, especially if software
> still changes and config options isn't stable yet.

Sorry for being unclear, I meant that the docs were very one-sided and
only mentioned the advantages, and not the disadvantages.  It should,
IMO, give a more objective view of problem.

>>> +(which also can be used for i3).  The following value of @code{config}
>>> +field of @code{home-sway-configuration}:
>>
>> ‘home-sway-configuration’ isn’t in Guix as of now, so it probably
>> shouldn’t be mentioned, as least for now.
>
> Don't think it's a big problem.  We can treat it as an imaginary example
> for now.

Then it should be clearly stated that it currently doesn’t exist in
Guix.


A few minutes later, Andrew wrote:

> * guix.texi (Writing Service Configuration): New section.
> ---
>  doc/guix.texi | 252 +++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 248 insertions(+), 4 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 333cb4117a..29d85d3dc5 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -10363,6 +10363,7 @@ compiling modules.  It can be @code{#f}, @code{#t}, or @code{'detailed}.
>  The other arguments are as for @code{derivation} (@pxref{Derivations}).
>  @end deffn
>  
> +@anchor{file-like objects}
>  @cindex file-like objects
>  The @code{local-file}, @code{plain-file}, @code{computed-file},
>  @code{program-file}, and @code{scheme-file} procedures below return
> @@ -15942,6 +15943,7 @@ symlink:
>  Return a service that sets the host name to @var{name}.
>  @end deffn
>  
> +@anchor{console-font-service-type}
>  @defvr {Scheme Variable} console-font-service-type
>  Install the given fonts on the specified ttys (fonts are per
>  virtual console on the kernel Linux).  The value of this service is a list of
> @@ -33717,6 +33719,7 @@ a daemon that can execute application bundles (sometimes referred to as
>  
>  @end defvr
>  
> +@anchor{docker-configuration}
>  @deftp {Data Type} docker-configuration
>  This is the data type representing the configuration of Docker and Containerd.
>  
> @@ -35652,10 +35655,11 @@ them in an @code{operating-system} declaration.  But how do we define
>  them in the first place?  And what is a service anyway?
>  
>  @menu
> -* Service Composition::         The model for composing services.
> -* Service Types and Services::  Types and services.
> -* Service Reference::           API reference.
> -* Shepherd Services::           A particular type of service.
> +* Service Composition::            The model for composing services.
> +* Service Types and Services::     Types and services.
> +* Writing Service Configurations:: A guideline for writing guix services.
> +* Service Reference::              API reference.
> +* Shepherd Services::              A particular type of service.
>  @end menu
>  
>  @node Service Composition
> @@ -35851,6 +35855,245 @@ There can be only one instance of an extensible service type such as
>  Still here?  The next section provides a reference of the programming
>  interface for services.
>  
> +@node Writing Service Configurations
> +@subsection Writing Service Configurations
> +
> +Guix already contains a wide variety of system and home services, but
> +sometimes users might want to add new services.  This section contains
> +tips for simplifying this process, and should help to make service
> +configurations and their implementations more consistent.
> +
> +@quotation Note
> +If you find any exceptions or patterns missing in this section, please
> +send a patch with additions/changes to @email{guix-devel@@gnu.org}
> +mailing list or just start a discussion/ask a question.
> +@end quotation
> +
> +@subsubheading Configuration Itself
> +
> +As we know from previous sections, a Guix service can accept a service
> +value, usually some kind of configuration record and optionally, be
> +extended with additional values by other services (@pxref{Service
> +Composition}).
> +
> +When being extended, most services take some kind of configuration
> +record or a list thereof, but in some cases a simpler value is all
> +that is necessary.
> +
> +There are some cases, when the service accepts a list of pairs or some
> +other non-record values.  For example, @code{console-font-service-type}
> +(@pxref{console-font-service-type}) accepts an
> +association list, and @code{etc-service-type} (@pxref{etc-service-type})
> +accepts a list of lists.  Those services are kinda special, they do
> +auxiliary work of setting up some part of the operating system or home
> +environment, or just an intermediate helpers used by other Guix
> +services.  For example @code{etc-service-type} is not that useful on its
> +own, but it helps other services to create files in /etc directory, when
> +it necessary.
     ^ missing “is”

Use @file{/etc} instead of just /etc.

> +However, in most cases a Guix service is wrapping some software, which
> +consists of one or more packages, and configuration file or files.
> +Therefore, the value for such service is quite complicated and it's hard
> +to represent it with just a list or basic data type, in such cases we
> +use a record.  Each such record (@pxref{SRFI-9 Records, Scheme Records,,
> +guile, GNU Guile Reference Manual}) have @samp{-configuration} suffix,

s/have/shoudl have a/

> +for example, the @code{docker-service-type} should accept a record type
> +named @code{docker-configuration}, which contains fields used to
> +configure Docker.  Configuration records for home services should also
> +have a @code{home-} prefix in their name.
> +
> +There is a module @code{gnu service configuration}, which contains
> +helpers simplifying configuration definition process.  Take a look at
> +@code{gnu services docker} module or grep for
> +@code{define-configuration} to find usage examples.

Since the docs for (gnu services configuration) already exists, a link
pointing to the node should be used.

> +After a configuration record has been properly named and defined let's
> +discuss how to name and define the fields, and which approach to use for
> +implementing the serialization code related to them.
> +
> +In this context, the @dfn{serialization} is a process of converting

spurious “the”

> +values of the fields defined in service configuration into a string or
                                  ^ “a”
                                  
> +strings of a target config format, which will be put to the

s/config/configuration/

> +configuration file or files used by the program.

s/the/a/

> +@subsubheading Configuration Record Fields
> +
> +@enumerate
> +@item
> +It's a good idea to have one or more fields for specifying the package
> +or packages that will be installed by a service.  For example,
> +@code{docker-configuration} has @code{docker}, @code{docker-cli},
> +@code{containerd} fields (@pxref{docker-configuration}).  Sometimes it
> +make sense to make a field, which accepts a list of packages for cases,
> +where an arbitrary list of plugins can be passed to the configuration.
> +There are some services, which provide a field called @code{package} in
> +their configuration, which is ok, but the way it done in
                                                   ^ “is”
                                                   
> +@code{docker-configuration} is more flexible and thus preferable.

Just to make things clear, it would be preferable to call the field
‘git’ instead of ‘package’ even if there is only one packages that will
be installed?

> +@item
> +Fields for configuration files, should be name the same as target
> +configuration file name, but in kebab-case <at> footnote{The case used for
> +identifiers in languages of Lisp family, example:
> +@code{this-is-kebab-case}.}: @code{bashrc} for @file{.bashrc},
> +@code{bash-profile} for @file{.bash_profile},
> +@code{tmux-conf} for @file{tmux.conf}, etc.  The implementation
> +for such fields will be discussed in the next subsubsection.
> +
> +@item
> +Other fields in most cases add some boilerplates/reasonable defaults to
> +configuration files, enable/disable installation of some packages or
> +provide other custom behavior, for example @code{guix-defaults?} or
> +@code{aliases} fields in @code{home-bash-configuration}
> +(@pxref{home-bash-configuration}).  There is no any special requirements
> +or recommendations here, but it's necessary to make it possible to
> +disable all the effects of such fields to provide a user with an empty
> +configuration and let them generate it from scratch with only field for
> +configuration file.  For example, setting @code{guix-defaults?} to
> +@code{#f} and @code{aliases} to @code{'()} will give user an ability to
> +control the content of @file{.bashrc} solely by setting the value of
> +@code{bashrc} field.
> +
> +
> +@end enumerate
> +
> +@subsubheading Fields for Configuration Files
> +
> +The field should accept a data structure (preferably a combination of
> +simple lists, alists, @ref{Vectors, vectors,, guile,},
> +@ref{G-Expressions, gexps} and basic data types), which will be
> +serialized to target configuration format, in other words, it should
                ^ “the”
                
> +provide an alternative Lisp syntax, which can be later translated to a
> +target one, like SXML to XML.  Such approach is quite flexible and
> +simple, it requires to write serializer once for one configuration
                      ^ “one”  ^ “a”
                      
> +format and can be reused multiple times in different Guix services.
> +
> +Let's take a look at JSON: we implement serialization function, which
                                          ^ “a”
                                          
> +converts vectors to arrays, alists to objects (AKA dictionaries or
> +associative arrays), numbers to numbers, gexps to the strings,

s/the//

> +@ref{file-like objects} (@pxref{G-Expressions}) to the strings, which

Likewise

> +contains the path to the file in the store, @code{#t} to @code{true} and

s/contains/contain/

> +so on, and now we have all programs using JSON as a format for
> +configurations covered.  Maybe some fine-tunning will be needed for
> +particular application, but the primary serialization part is already

s/application/programs/

> +finished.
> +
> +The pros and cons of such approach is inherited from open-world
> +assumption.  It doesn't matter if the underlying applications provides
> +new configuration options, we don't need to change anything in the
> +service configuration and its serialization code, it will work perfectly
> +fine.  On the other hand, it is harder to type check and structure check
> +at ``compile-time'', and we can end up with a configuration, which won't
> +be accepted by the target program due to unexisting, misspelled or
> +wrongly-typed options.  It's possible to add those checks, but we will
> +get the drawbacks of closed-world assumption: we need to keep the

A link or footnote for “closed-world assumption” should be provided.

> +service implementation in-sync with app config options, and it will make

“…with the configuration options of the program, and it will make it…”

> +impossible to use the same service with older/newer package version,

s|/|or|
s/version/versions/

> +which has a slightly different list of available options and will add an
> +excessive maintanence load.
> +
> +However, for some applications with really stable configuration those
> +checks can be helpful and should be implemented if possible, for some
> +others we can implement them only partially.
> +
> +The alternative approach applied in some exitsting services is to use
> +records for defining the structure of configuration field, it has the
> +same downsides of closed-world assumption and a few more problems:
> +
> +@enumerate
> +@item
> +It has to replicate all the available options for the app (sometimes
> +hundreds or thousands) to allow the user express any configuration they
> +want.
> +@item
> +Having a few records adds one more layer of abstraction between service
> +configuration and resulting app config, including different field

s/app/program/

> +casing, new semantic units.
> +@c provide examples?
> +@item
> +It harder to implement optional settings, serialization becomes very
     ^ “is”
> +ad-hoc and hard to reuse among other services with the same target
> +config format.
> +@end enumerate
> +
> +Exceptions can exist, but the overall idea is to provide a lispy syntax
> +for target configuration.  Take a look at Sway example configuration
> +(which also can be used for i3).  The following value of @code{config}
> +field of @code{home-sway-configuration}:
> +
> +@example
> +`((include ,(local-file "./sway/config"))
> +  (bindsym $mod+Ctrl+Shift+a exec emacsclient -c --eval "'(eshell)'")
> +  (bindsym $mod+Ctrl+Shift+o "[class=\"IceCat\"]" kill)
> +  (input * ((xkb_layout us,ru)
> +            (xkb_variant dvorak,))))
> +@end example
> +
> +would yield something like:
> +
> +@example
> +include /gnu/store/408jwvh6wxxn1j85lj95fniih05gx5xj-config
> +bindsym $mod+Ctrl+Shift+a exec emacsclient -c --eval '(eshell)'
> +bindsym $mod+Ctrl+Shift+o [class="IceCat"] kill
> +input * @{
> +    xkb_layout us,ru
> +    xkb_variant dvorak,
> +@}
> +@end example
> +
> +The mapping between Scheme code and resulting configuration is quite
> +obvious.  The serialization code with some type and structure checks
> +takes less than 70 lines and every possible Sway/i3 configuration can be
> +expressed using this field.
> +
> +@subsubheading Let User Escape
                     ^ “The”
                     
> +Sometimes a user already has a configuration file for an program, make

s/an/a/

> +sure that it is possible to reuse it directly without rewriting.  In the
> +example above, the following snippet allows one to include already an
> +existing config to the newly generated one utilizing @code{include}
> +directive of i3/Sway config language:
> +
> +@lisp
> +(include ,(local-file "./sway/config"))
> +@end lisp
> +
> +When building a resulting config the file-like objects are substituted
> +with a path of the file in the store and Sway's @code{include} loads
> +this file during startup.  The way file-like objects are treated here
> +also allows one to specify paths to plugins or other binary files like:
> +
> +@lisp
> +(load-plugin ,(file-append plugin-package "/share/plugin.so"))
> +@end lisp
> +
> +(the example value for imaginary service configuration config file
> +field).
> +
> +In some cases the target configuration language may not have such an
> +@code{include} directive and can't provide such a functionallity, to
> +workaround it we can do the following trick:
> +
> +@lisp
> +#~(call-with-input-file
> +   #$(local-file "./sway/config")
> +   (@@ (ice-9 textual-ports) get-string-all))
> +@end lisp
> +
> +The ‘get-string-all’ procedure will read the contents of the

@code{get-string-all}

> +@file{./sway/config} file (to be more preciese the copy of this file
> +placed in the store), and return a string containing the contents.  Once
> +serialized, the G-expression will thus be turn into the contents of the
> +Sway configuration file in @file{./sway/config}.  This code can be
> +easily combined with the rest of Sway's configuration, additionally, we
> +can control the place where the content of @file{./sway/config} will
> +appear in resulting file by moving this snippet around.
> +
> +Following these simple rules will help to make simple, consistent and
> +maintainable service configurations, and will let users express any
> +possible needs and reuse existing configuration files.
> +
>  @node Service Reference
>  @subsection Service Reference
>  
> @@ -36076,6 +36319,7 @@ The type of the ``boot service'', which produces the @dfn{boot script}.
>  The boot script is what the initial RAM disk runs when booting.
>  @end defvr
>  
> +@anchor{etc-service-type}
>  @defvr {Scheme Variable} etc-service-type
>  The type of the @file{/etc} service.  This service is used to create
>  files under @file{/etc} and can be extended by
> -- 
> 2.34.0

Another point that might be worth bringing up is what we should do with
existing services which do not adhere to these guidelines.  Should we
re-write them or just keep them as-is?

Sorry for the long delay!  I will take a look at the “Return back
original implementation for text-config serialization” thread soon.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.