GNU bug report logs - #46043
[PATCH] Allow booting with custom shepherd package (fixed)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46043 in the body.
You can then email your comments to 46043 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: <guix-patches <at> gnu.org>, <maximedevos <at> telenet.be>
Subject: [PATCH] Allow booting with custom shepherd package (fixed)
Date: Fri, 22 Jan 2021 20:57:03 +0100

[Message part 1 (text/plain, inline)]

[previous e-mail was bogus. ggrr emacs, why can't I open e-mail
I was composing and saved with C-x C-f and continue later,
even if I set mml-mode]

Hi Guix,

If I didn't miss anything, this patch allows for booting with a non-standard
shepherd package, which may be useful for using features from a not-yet
released shepherd (presuming you've packaged the custom shepherd first :)).

I'll send a follow-up e-mail to confirm/disconfirm whether this works
well.

Maxime

[0001-services-shepherd-allow-custom-shepherd-package.patch (text/x-patch, inline)]

From e4f6743a5502eb6c878ee2c3bf7f7153f8006e01 Mon Sep 17 00:00:00 2001
From: Maxime Devos <maximedevos <at> telenet.be>
Date: Fri, 22 Jan 2021 20:06:55 +0100
Subject: [PATCH] services: shepherd: allow custom 'shepherd' package

* gnu/services/shepherd.scm
  (<shepherd-configuration>): New record.
  (shepherd-boot-gexp, shepherd-root-service-type): Use it.
  (scm->go, shepherd-configuration-file): Allow passing custom
  shepherd package.
* doc/guix.texi (Shepherd Services). Document it.
---
 doc/guix.texi             | 17 +++++++++++++-
 gnu/services/shepherd.scm | 48 +++++++++++++++++++++++++++++----------
 2 files changed, 52 insertions(+), 13 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index a765b3792e..344ffcfea5 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -32707,9 +32707,24 @@ The service type for the Shepherd ``root service''---i.e., PID <at> tie{}1.
 
 This is the service type that extensions target when they want to create
 shepherd services (@pxref{Service Types and Services}, for an example).
-Each extension must pass a list of @code{<shepherd-service>}.
+Each extension must pass a list of @code{<shepherd-service>}. Its
+value must be a @code{shepherd-configuration}, as described below:
 @end defvr
 
+@deftp {Data Type} shepherd-configuration
+This data type represents shepherd's configuration.
+
+@table @code
+@item shepherd (default: @code{shepherd})
+The shepherd package to use.
+
+@item services (default: @code{'()})
+A list of @code{<shepherd-service>} to start.
+You should probably use the service extension
+mechanism instead (@pxref{Shepherd Services}).
+@end table
+@end deftp
+
 @defvr {Scheme Variable} %shepherd-root-service
 This service represents PID <at> tie{}1.
 @end defvr
diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm
index d2f9776288..748b19957a 100644
--- a/gnu/services/shepherd.scm
+++ b/gnu/services/shepherd.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2017 Clément Lassieur <clement <at> lassieur.org>
 ;;; Copyright © 2018 Carlo Zancanaro <carlo <at> zancanaro.id.au>
 ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
+;;; Copyright © 2021 Maxime Devos <maximedevos <at> telenet.be>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -36,7 +37,12 @@
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
-  #:export (shepherd-root-service-type
+  #:export (shepherd-configuration
+            shepherd-configuration?
+            shepherd-configuration-shepherd
+            shepherd-configuration-services
+
+            shepherd-root-service-type
             %shepherd-root-service
             shepherd-service-type
 
@@ -76,7 +82,18 @@
 ;;; Code:
 
 
-(define (shepherd-boot-gexp services)
+(define-record-type* <shepherd-configuration>
+  shepherd-configuration make-shepherd-configuration
+  shepherd-configuration?
+  (shepherd shepherd-configuration-shepherd
+            (default shepherd)) ; package
+  (services shepherd-configuration-services
+            (default '()))) ; list of <shepherd-service>
+
+(define (shepherd-boot-gexp config)
+  "Return a gexp starting the shepherd service."
+  (let ((shepherd (shepherd-configuration-shepherd config))
+        (services (shepherd-configuration-services config)))
   #~(begin
       ;; Keep track of the booted system.
       (false-if-exception (delete-file "/run/booted-system"))
@@ -95,7 +112,7 @@
       ;; Start shepherd.
       (execl #$(file-append shepherd "/bin/shepherd")
              "shepherd" "--config"
-             #$(shepherd-configuration-file services))))
+             #$(shepherd-configuration-file services #:shepherd shepherd)))))
 
 (define shepherd-root-service-type
   (service-type
@@ -103,20 +120,25 @@
    ;; Extending the root shepherd service (aka. PID 1) happens by
    ;; concatenating the list of services provided by the extensions.
    (compose concatenate)
-   (extend append)
+   (extend (lambda (config extra-services)
+             (shepherd-configuration
+               (inherit config)
+               (services (append (shepherd-configuration-services config)
+                                 extra-services)))))
    (extensions (list (service-extension boot-service-type
                                         shepherd-boot-gexp)
                      (service-extension profile-service-type
                                         (const (list shepherd)))))
+   (default-value (shepherd-configuration))
    (description
     "Run the GNU Shepherd as PID 1---i.e., the operating system's first
 process.  The Shepherd takes care of managing services such as daemons by
 ensuring they are started and stopped in the right order.")))
 
 (define %shepherd-root-service
-  ;; The root shepherd service, aka. PID 1.  Its parameter is a list of
-  ;; <shepherd-service> objects.
-  (service shepherd-root-service-type '()))
+  ;; The root shepherd service, aka. PID 1.  Its parameter is a
+  ;; <shepherd-configuration>.
+  (service shepherd-root-service-type))
 
 (define-syntax shepherd-service-type
   (syntax-rules (description)
@@ -270,9 +292,9 @@ stored."
                                    #~(#$name #$doc #$proc)))
                                 (shepherd-service-actions service))))))))
 
-(define (scm->go file)
+(define* (scm->go file #:key (shepherd shepherd))
   "Compile FILE, which contains code to be loaded by shepherd's config file,
-and return the resulting '.go' file."
+and return the resulting '.go' file. SHEPHERD is used as shepherd package."
   (let-system (system target)
     (with-extensions (list shepherd)
       (computed-file (string-append (basename (scheme-file-name file) ".scm")
@@ -294,11 +316,13 @@ and return the resulting '.go' file."
                      #:options '(#:local-build? #t
                                  #:substitutable? #f)))))
 
-(define (shepherd-configuration-file services)
-  "Return the shepherd configuration file for SERVICES."
+(define* (shepherd-configuration-file services #:key (shepherd shepherd))
+  "Return the shepherd configuration file for SERVICES. SHEPHERD is used
+as shepherd package."
   (assert-valid-graph services)
 
-  (let ((files (map shepherd-service-file services)))
+  (let ((files (map shepherd-service-file services))
+        (scm->go (cute scm->go <> #:shepherd shepherd)))
     (define config
       #~(begin
           (use-modules (srfi srfi-34)
-- 
2.30.0

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Leo Prikler <leo.prikler <at> student.tugraz.at>
To: Maxime Devos <maximedevos <at> telenet.be>, 46043 <at> debbugs.gnu.org
Subject: Re: [PATCH] Allow booting with custom shepherd package (fixed)
Date: Sat, 23 Jan 2021 09:25:52 +0100

Hi Maxime,

Am Freitag, den 22.01.2021, 20:57 +0100 schrieb Maxime Devos:
>     (compose concatenate)
> -   (extend append)
> +   (extend (lambda (config extra-services)
> +             (shepherd-configuration
> +               (inherit config)
> +               (services (append (shepherd-configuration-services
> config)
> +                                 extra-services)))))
I think you should also patch compose or drop it.  I sadly don't know,
what impact that would have, but IIUC you should only spawn one
shepherd-root-service anyway.

Regards,
Leo

Message #11 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Leo Prikler <leo.prikler <at> student.tugraz.at>, 46043 <at> debbugs.gnu.org
Subject: Re: [PATCH] Allow booting with custom shepherd package (fixed)
Date: Sat, 23 Jan 2021 16:11:52 +0100

[Message part 1 (text/plain, inline)]

Hi Leo,
On Sat, 2021-01-23 at 09:25 +0100, Leo Prikler wrote:
> Hi Maxime,
> 
> Am Freitag, den 22.01.2021, 20:57 +0100 schrieb Maxime Devos:
> >     (compose concatenate)
> > -   (extend append)
> > +   (extend (lambda (config extra-services)
> > +             (shepherd-configuration
> > +               (inherit config)
> > +               (services (append (shepherd-configuration-services
> > config)
> > +                                 extra-services)))))
> I think you should also patch compose or drop it.  I sadly don't know,
> what impact that would have, but IIUC you should only spawn one
> shepherd-root-service anyway.

Only one shepherd-root-service-type should be spawned, correct.
In practice, this means you'll need to use a construct like this
(if I didn't make any errors):

  (modify-services %base-services
    (shepherd-root-service-type c =>
      (shepherd-configuration
        (inherit c)
        (shepherd a-custom-shepherd))))

For clarification: shepherd services should be added
via the service-extension mechanism, not by
manually changing the 'services' field
(though the latter is a possibility now).

About compose and concatenate: I think it is correct as-is.
If compose is dropped, then shepherd-root-service-type wouldn't be
extensible anymore. I don't see how it could be patched
in a way that makes sense.

Compare with udev-service-type as described in the manual,
which follows the same structure.

Regards,
Maxime
-- 
Maxime Devos <maximedevos <at> telenet.be>
PGP Key: C1F3 3EE2 0C52 8FDB 7DD7  011F 49E3 EE22 1917 25EE
Freenode handle: mdevos

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Leo Prikler <leo.prikler <at> student.tugraz.at>
To: 46043 <at> debbugs.gnu.org
Cc: Maxime Devos <maximedevos <at> telenet.be>
Subject: Re: [PATCH] Allow booting with custom shepherd package (fixed)
Date: Sat, 23 Jan 2021 16:25:12 +0100

Hi,

Am Samstag, den 23.01.2021, 16:11 +0100 schrieb Maxime Devos:
> [...]
> 
> About compose and concatenate: I think it is correct as-is.
> If compose is dropped, then shepherd-root-service-type wouldn't be
> extensible anymore. I don't see how it could be patched
> in a way that makes sense.
Ahh, my bad.  It seems I don't have a strong enough understanding of
the difference between compose and extend yet.  Nvm then.

Regards,
Leo

Message #17 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: 46043 <at> debbugs.gnu.org
Cc: Maxime Devos <maximedevos <at> telenet.be>
Subject: Re: [PATCH] Allow booting with custom shepherd package (fixed)
Date: Mon, 25 Jan 2021 09:48:36 +0100

[Message part 1 (text/plain, inline)]

Version 2 of the patch is attached. This version also uses
the custom package for /run/current-system/profile/bin/shepherd.

[0001-services-shepherd-allow-custom-shepherd-package.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 46043 <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Tue, 26 Jan 2021 21:49:57 +0100

Hi Maxime,

Maxime Devos <maximedevos <at> telenet.be> skribis:

> Version 2 of the patch is attached. This version also uses
> the custom package for /run/current-system/profile/bin/shepherd.
>
> From b3102f72cd66dee4442596d89e29900b8215e1be Mon Sep 17 00:00:00 2001
> From: Maxime Devos <maximedevos <at> telenet.be>
> Date: Fri, 22 Jan 2021 20:06:55 +0100
> Subject: [PATCH] services: shepherd: allow custom 'shepherd' package
>
> * gnu/services/shepherd.scm
>   (<shepherd-configuration>): New record.
>   (shepherd-boot-gexp, shepherd-root-service-type): Use it.
>   (scm->go, shepherd-configuration-file): Allow passing custom
>   shepherd package.
> * doc/guix.texi (Shepherd Services). Document it.

Nice!  Overall LGTM.  Nitpicking:

>  shepherd services (@pxref{Service Types and Services}, for an example).
> -Each extension must pass a list of @code{<shepherd-service>}.
> +Each extension must pass a list of @code{<shepherd-service>}. Its

Please leave two spaces after end-of-sentence periods.

> +value must be a @code{shepherd-configuration}, as described below:

End with period rather than colon.

> +@deftp {Data Type} shepherd-configuration
> +This data type represents shepherd's configuration.

s/shepherd/the Shepherd/ (referring to the software).

> +@table @code
> +@item shepherd (default: @code{shepherd})
> +The shepherd package to use.

“Shepherd” (capitalized).

> +(define* (scm->go file #:key (shepherd shepherd))

[...]

> +(define* (shepherd-configuration-file services #:key (shepherd shepherd))

I’d made the second argument compulsory instead of a keyword parameter.

That’s all.  Could you send an updated patch?

Thanks for addressing this longstanding shortcoming!

Ludo’.

Message #23 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 46043 <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Wed, 27 Jan 2021 14:24:58 +0100

[Message part 1 (text/plain, inline)]

> That’s all.  Could you send an updated patch?
See attachment.

[0001-services-shepherd-allow-custom-shepherd-package.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #28 received at 46043-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 46043-done <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Thu, 28 Jan 2021 00:09:19 +0100

Hi,

Maxime Devos <maximedevos <at> telenet.be> skribis:

> From 1b1e49f4aad95b766126fadd4166acf5ab6fbd31 Mon Sep 17 00:00:00 2001
> From: Maxime Devos <maximedevos <at> telenet.be>
> Date: Fri, 22 Jan 2021 20:06:55 +0100
> Subject: [PATCH] services: shepherd: allow custom 'shepherd' package
>
> * gnu/services/shepherd.scm
>   (<shepherd-configuration>): New record.
>   (shepherd-boot-gexp, shepherd-root-service-type): Use it.
>   (scm->go, shepherd-configuration-file): Allow passing custom
>   shepherd package.
> * doc/guix.texi (Shepherd Services). Document it.

Applied, thanks!

Ludo’.

Message #31 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 46043 <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Thu, 28 Jan 2021 00:15:51 +0100

Maxime Devos <maximedevos <at> telenet.be> skribis:

> From 1b1e49f4aad95b766126fadd4166acf5ab6fbd31 Mon Sep 17 00:00:00 2001
> From: Maxime Devos <maximedevos <at> telenet.be>
> Date: Fri, 22 Jan 2021 20:06:55 +0100
> Subject: [PATCH] services: shepherd: allow custom 'shepherd' package
>
> * gnu/services/shepherd.scm
>   (<shepherd-configuration>): New record.
>   (shepherd-boot-gexp, shepherd-root-service-type): Use it.
>   (scm->go, shepherd-configuration-file): Allow passing custom
>   shepherd package.
> * doc/guix.texi (Shepherd Services). Document it.

Oops, I spoke too fast: “make check-system TESTS=basic” shows one
problem, which stems from this:

--8<---------------cut here---------------start------------->8---
(define (operating-system-shepherd-service-names os)
  "Return the list of Shepherd service names for OS."
  (append-map shepherd-service-provision
              (service-value
               (fold-services (operating-system-services os)
                              #:target-type
                              shepherd-root-service-type))))
--8<---------------cut here---------------end--------------->8---

This procedure needs to be adjusted because the value of
‘shepherd-root-service-type’ instances is no longer a list of
<shepherd-service>.

I suspect ‘guix system search’ and ‘guix system shepherd-graph’ are
affected as well, though I didn’t check (“make check” should catch it, I
think.)

Could you take a look?

TIA,
Ludo’.

Message #38 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 46043 <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Thu, 28 Jan 2021 09:20:00 +0100

[Message part 1 (text/plain, inline)]

> Could you take a look?
Revised patch is attached.
'make check-system TESTS=basic' now succeeds, I'm now
running 'make check-system' just in case.

[0001-services-shepherd-allow-custom-shepherd-package.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 46043 <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Thu, 28 Jan 2021 14:18:17 +0100

Maxime Devos <maximedevos <at> telenet.be> skribis:

> Revised patch is attached.

I think that’s the same as before (there are no changes to
gnu/system.scm).  :-)

Ludo’.

Message #44 received at 46043 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 46043 <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Thu, 28 Jan 2021 17:32:08 +0100

[Message part 1 (text/plain, inline)]

> I think that’s the same as before (there are no changes to
> gnu/system.scm).  :-)
This time for real :-)

[0001-services-shepherd-allow-custom-shepherd-package.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #49 received at 46043-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 46043-done <at> debbugs.gnu.org
Subject: Re: bug#46043: [PATCH] Allow booting with custom shepherd package
 (fixed)
Date: Sat, 30 Jan 2021 15:36:39 +0100

Hi,

Maxime Devos <maximedevos <at> telenet.be> skribis:

> From 085708344c4a9d7f819a399e666e2bab2ae7a54d Mon Sep 17 00:00:00 2001
> From: Maxime Devos <maximedevos <at> telenet.be>
> Date: Fri, 22 Jan 2021 20:06:55 +0100
> Subject: [PATCH] services: shepherd: allow custom 'shepherd' package
>
> * gnu/services/shepherd.scm
>   (<shepherd-configuration>): New record.
>   (shepherd-boot-gexp, shepherd-root-service-type): Use it.
>   (scm->go, shepherd-configuration-file): Allow passing custom
>   shepherd package.
> * gnu/system.scm
>   (operating-system-shepherd-service-names): Use the new record.
> * doc/guix.texi (Shepherd Services). Document it.

Applied!  I had to fix ‘guix system shepherd-graph’, as explained
before, but apart from that it all looks good.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.