GNU bug report logs - #66156
[PATCH 00/12] Introducing Smart Hurdloading

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 66156 in the body.
You can then email your comments to 66156 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Josselin Poiret <dev <at> jpoiret.xyz>, Janneke Nieuwenhuizen <janneke <at> gnu.org>
Subject: [PATCH 00/12] Introducing Smart Hurdloading
Date: Fri, 22 Sep 2023 14:52:27 +0200

This patch series, my friends, sets up Smart Hurdloading™, sometimes
referred to as “Hurd auto-offloading”.  (Yeah I worked hard on branding…)

Concretely, if you add (service hurd-vm-service-type), you can now run:

  guix build libreoffice -s i586-gnu

on your machine, and the childhurd will take care of keeping one CPU core
busy for a while.  There’s no configuration involved beyond this one
(service hurd-vm-service-type) line: no need to exchange SSH keys,
substitute keys, write /etc/guix/machines.scm, and all that.  It’s all
taken care off automatically.

One key element here is support for declarative and extensible offloading
configuration—meaning that /etc/guix/machines.scm is now optionally generated
straight from your ‘operating-system’ config.  The rest is about generating
keys, authorizing them, etc.

The series is a bit of a potpourri: I fixed Hurd issues here and there
(currently offloading to a childhurd doesn’t work on ‘master’, because
of the locale issue, for instance), improved documentation, etc.  I also
took a couple of long detours not shown here that might lead to further
improvements in the future.

My goal is to extend this mechanism beyond the Hurd, to have a generic
mechanism to spin up Guix System VMs we can easily offload to.

Thoughts?

Ludo’.

Ludovic Courtès (12):
  system: vm: Remove unused variable.
  secret-service: Increase default handshake timeout.
  services: hurd-vm: Use the default SSH port number.
  gnu: glibc-utf8-locales: Reintroduce input labels.
  services: guix: Use the right locale package on GNU/Hurd.
  services: guix: Support declarative offloading setup.
  services: childhurd: Authorize the childhurd’s key on the host.
  services: hurd-vm: ‘image’ field has to be an <image> record.
  tests: hurd-vm: Remove custom disk image configuration.
  services: hurd-vm: Disable password-based authentication for root.
  doc: Give an example showing how to add an account in the childhurd.
  services: hurd-vm: Implement zero-configuration offloading.

 doc/guix.texi                   | 167 ++++++++++++++++++++++++++------
 gnu/build/secret-service.scm    |   2 +-
 gnu/packages/base.scm           |   4 +-
 gnu/services/base.scm           |  58 ++++++++++-
 gnu/services/virtualization.scm | 153 ++++++++++++++++++++++++++---
 gnu/system/vm.scm               |  41 ++------
 gnu/tests/virtualization.scm    |  68 ++++++++++---
 7 files changed, 394 insertions(+), 99 deletions(-)


base-commit: 3d8d67ef6928f5d81118c97f03372cd341eab8b0
-- 
2.41.0

Message #8 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 01/12] system: vm: Remove unused variable.
Date: Fri, 22 Sep 2023 14:54:02 +0200

This is a followup to 05a9d1f378e2e13e8f759be926ea368358afc27c, which
removed its sole user.

* gnu/system/vm.scm (%linux-vm-file-systems): Remove.
(mapping->file-system): Add comment about “cache=loose”.
---
 gnu/system/vm.scm | 41 ++++++-----------------------------------
 1 file changed, 6 insertions(+), 35 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index b7bccd72a4..70f7b00116 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -87,41 +87,6 @@ (define-module (gnu system vm)
 ;; conservative default.
 (define %default-msize-value (* 100 (expt 2 20))) ;100 MiB
 
-(define %linux-vm-file-systems
-  ;; File systems mounted for 'derivation-in-linux-vm'.  These are shared with
-  ;; the host over 9p.
-  ;;
-  ;; The 9p documentation says that cache=loose is "intended for exclusive,
-  ;; read-only mounts", without additional details.  It's much faster than the
-  ;; default cache=none, especially when copying and registering store items.
-  ;; Thus, use cache=loose, except for /xchg where we want to ensure
-  ;; consistency.
-  (list (file-system
-          (mount-point (%store-prefix))
-          (device "store")
-          (type "9p")
-          (needed-for-boot? #t)
-          (flags '(read-only))
-          (options (format #f "trans=virtio,cache=loose,msize=~a"
-                           %default-msize-value))
-          (check? #f))
-        (file-system
-          (mount-point "/xchg")
-          (device "xchg")
-          (type "9p")
-          (needed-for-boot? #t)
-          (options (format #f "trans=virtio,msize=~a" %default-msize-value))
-          (check? #f))
-        (file-system
-          (mount-point "/tmp")
-          (device "tmp")
-          (type "9p")
-          (needed-for-boot? #t)
-          (options (format #f "trans=virtio,cache=loose,msize=~a"
-                           %default-msize-value))
-          (check? #f))))
-
-
 ;;;
 ;;; VMs that share file systems with the host.
 ;;;
@@ -145,6 +110,12 @@ (define (mapping->file-system mapping)
        (device (file-system->mount-tag source))
        (type "9p")
        (flags (if writable? '() '(read-only)))
+
+       ;; The 9p documentation says that cache=loose is "intended for
+       ;; exclusive, read-only mounts", without additional details.  It's
+       ;; faster than the default cache=none, especially when copying and
+       ;; registering store items.  Thus, use cache=loose, except for writable
+       ;; mounts, to ensure consistency.
        (options (string-append "trans=virtio"
                                (if writable? "" ",cache=loose")
                                ",msize=" (number->string %default-msize-value)))
-- 
2.41.0

Message #11 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 02/12] secret-service: Increase default handshake timeout.
Date: Fri, 22 Sep 2023 14:54:03 +0200

* gnu/build/secret-service.scm (secret-service-send-secrets):
Increase #:handshake-timeout.
---
 gnu/build/secret-service.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/build/secret-service.scm b/gnu/build/secret-service.scm
index c53145c8e7..e13fd4eef3 100644
--- a/gnu/build/secret-service.scm
+++ b/gnu/build/secret-service.scm
@@ -95,7 +95,7 @@ (define (wait-for-readable-fd port timeout)
 
 (define* (secret-service-send-secrets port secret-root
                                       #:key (retry 60)
-                                      (handshake-timeout 120))
+                                      (handshake-timeout 180))
   "Copy all files under SECRET-ROOT using TCP to secret-service listening at
 local PORT.  If connect fails, sleep 1s and retry RETRY times; once connected,
 wait for at most HANDSHAKE-TIMEOUT seconds for handshake to complete.  Return
-- 
2.41.0

Message #14 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 03/12] services: hurd-vm: Use the default SSH port number.
Date: Fri, 22 Sep 2023 14:54:04 +0200

* gnu/services/virtualization.scm (%hurd-vm-operating-system): Remove
‘port-number’ from ‘openssh-configuration’.
(hurd-vm-net-options): Change 2222 to 22 in port forwarding.
---
 gnu/services/virtualization.scm | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index e1a206e0eb..fd153dd051 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1078,7 +1078,6 @@ (define %hurd-vm-operating-system
                         (openssh-configuration
                          (openssh openssh-sans-x)
                          (use-pam? #f)
-                         (port-number 2222)
                          (permit-root-login #t)
                          (allow-empty-passwords? #t)
                          (password-authentication? #t)))
@@ -1146,7 +1145,7 @@ (define (hurd-vm-net-options config)
                     "-:1004"
                     ",hostfwd=tcp:127.0.0.1:"
                     (number->string (hurd-vm-port config %hurd-vm-ssh-port))
-                    "-:2222"
+                    "-:22"
                     ",hostfwd=tcp:127.0.0.1:"
                     (number->string (hurd-vm-port config %hurd-vm-vnc-port))
                     "-:5900")))
-- 
2.41.0

Message #17 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 04/12] gnu: glibc-utf8-locales: Reintroduce input labels.
Date: Fri, 22 Sep 2023 14:54:05 +0200

Since 2f73ea3487b3bf6eb055c08aae7c53713d61a4d7,
‘make-glibc-utf8-locales’ couldn’t be passed a package with a name other
than “glibc” (in particular “glibc-hurd”) since the builder expects the
name “glibc” in ‘%build-inputs’.

* gnu/packages/base.scm (make-glibc-utf8-locales): Reintroduce labels in
‘native-inputs’.
---
 gnu/packages/base.scm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 0b6cb2ddc5..c0813f7de0 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1382,7 +1382,9 @@ (define*-public (make-glibc-utf8-locales glibc #:key
                                                          locale ".UTF-8")))
                                ',locales)
                      #t))))
-    (native-inputs (list glibc gzip))
+    (native-inputs
+     `(("glibc" ,glibc)
+       ("gzip" ,gzip)))
     (synopsis (if default-locales?
                   (P_ "Small sample of UTF-8 locales")
                   (P_ "Customized sample of UTF-8 locales")))
-- 
2.41.0

Message #20 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 05/12] services: guix: Use the right locale package on
 GNU/Hurd.
Date: Fri, 22 Sep 2023 14:54:06 +0200

Fixes a bug introduced in 0dd293b4d9095137c9952e16ca951f887b7e7018
whereby guix-daemon on GNU/Hurd would have ‘GUIX_LOCPATH’ set to the
“wrong” locale data (2.35 instead of 2.37).

Consequently, it would fail to setlocale(3) and calls to
‘std::stoi’ (when reading the output of ‘guix authenticate’) would
throw, leading to this error message of guix-daemon:

  unexpected build daemon error: stoi

This would manifest when sending store items to a childhurd:

  $ guix copy --to=localhost:10022 sed
  guix copy: sending 1 store item (1 MiB) to 'localhost'...
  guix copy: error: unknown error while sending files over SSH

The “unknown error” is the ‘stoi’ exception.

This commit fixes that, but for the ‘guix-daemon’ service only.

* gnu/services/base.scm (guix-shepherd-service)[locales]: New variable.
Use it instead of ‘glibc-utf8-locales’.
---
 gnu/services/base.scm | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index b3f2d2e8b8..10e0d4cf9d 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -62,8 +62,9 @@ (define-module (gnu services base)
                           util-linux xfsprogs))
   #:use-module (gnu packages bash)
   #:use-module ((gnu packages base)
-                #:select (coreutils glibc glibc-utf8-locales tar
-                          canonical-package))
+                #:select (coreutils glibc glibc/hurd
+                          glibc-utf8-locales make-glibc-utf8-locales
+                          tar canonical-package))
   #:use-module ((gnu packages compression) #:select (gzip))
   #:use-module (gnu packages fonts)
   #:autoload   (gnu packages guile-xyz) (guile-netlink)
@@ -87,6 +88,7 @@ (define-module (gnu services base)
   #:use-module ((guix self) #:select (make-config.scm))
   #:use-module (guix diagnostics)
   #:use-module (guix i18n)
+  #:autoload   (guix utils) (target-hurd?)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
@@ -1831,6 +1833,12 @@ (define shepherd-discover-action
 
 (define (guix-shepherd-service config)
   "Return a <shepherd-service> for the Guix daemon service with CONFIG."
+  (define locales
+    (let-system (system target)
+      (if (target-hurd? (or target system))
+          (make-glibc-utf8-locales glibc/hurd)
+          glibc-utf8-locales)))
+
   (match-record config <guix-configuration>
     (guix build-group build-accounts authorize-key? authorized-keys
           use-substitutes? substitute-urls max-silent-time timeout
@@ -1912,8 +1920,7 @@ (define (guix-shepherd-service config)
                                  ;; 'nss-certs'.  See
                                  ;; <https://bugs.gnu.org/32942>.
                                  (string-append "GUIX_LOCPATH="
-                                                #$glibc-utf8-locales
-                                                "/lib/locale")
+                                                #$locales "/lib/locale")
                                  "LC_ALL=en_US.utf8"
                                  ;; Make 'tar' and 'gzip' available so
                                  ;; that 'guix perform-download' can use
-- 
2.41.0

Message #23 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 06/12] services: guix: Support declarative offloading setup.
Date: Fri, 22 Sep 2023 14:54:07 +0200

* gnu/services/base.scm (guix-machines-files-installation): New
procedure.
(<guix-configuration>)[build-machines]: New field.
(guix-activation): Call ‘ guix-machines-files-installation’.
(<guix-extension>)[build-machines]: New field.
(guix-extension-merge): Handle it.
(guix-service-type)[extend]: Likewise.
* doc/guix.texi (Daemon Offload Setup): Add note linking to
‘guix-configuration’.
(Base Services): Document ‘build-machines’ field of <guix-configuration>
and of <guix-extension>.
(Virtualization Services): Add ‘hurd-vm’ anchor.
---
 doc/guix.texi         | 42 +++++++++++++++++++++++++++++++++++++++++-
 gnu/services/base.scm | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 84 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 50c4984d71..ca48d6c404 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1484,6 +1484,14 @@ Daemon Offload Setup
 @end table
 @end deftp
 
+@quotation Note
+On Guix System, instead of managing @file{/etc/guix/machines.scm}
+independently, you can choose to specify build machines directly in the
+@code{operating-system} declaration, in the @code{build-machines} field
+of @code{guix-configuration}.  @xref{guix-configuration-build-machines,
+@code{build-machines} field of @code{guix-configuration}}.
+@end quotation
+
 The @command{guix} command must be in the search path on the build
 machines.  You can check whether this is the case by running:
 
@@ -19259,6 +19267,28 @@ Base Services
 Whether to discover substitute servers on the local network using mDNS
 and DNS-SD.
 
+@anchor{guix-configuration-build-machines}
+@item @code{build-machines} (default: @code{#f})
+This field must be either @code{#f} or a list of gexps evaluating to a
+@code{build-machine} record (@pxref{Daemon Offload Setup}).
+
+When it is @code{#f}, the @file{/etc/guix/machines.scm} file is left
+untouched.  Otherwise, the list of of gexps is written to
+@file{/etc/guix/machines.scm}; if a previously-existing file is found,
+it is backed up as @file{/etc/guix/machines.scm.bak}.  This allows you
+to declare build machines for offloading directly in the operating
+system declaration, like so:
+
+@lisp
+(guix-configuration
+  (build-machines
+    (list #~(build-machine (name "foo.example.org") @dots{})
+          #~(build-machine (name "bar.example.org") @dots{}))))
+@end lisp
+
+Additional build machines may be added @i{via} the @code{guix-extension}
+mechanism (see below).
+
 @item @code{extra-options} (default: @code{'()})
 List of extra command-line options for @command{guix-daemon}.
 
@@ -19296,7 +19326,6 @@ Base Services
 @end deftp
 
 @deftp {Data Type} guix-extension
-
 This data type represents the parameters of the Guix build daemon that
 are extendable. This is the type of the object that must be used within
 a guix service extension.
@@ -19309,6 +19338,16 @@ Base Services
 @item @code{substitute-urls} (default: @code{'()})
 A list of strings where each element is a substitute URL.
 
+@item @code{build-machines} (default: @code{'()})
+A list of gexps that evaluate to @code{build-machine} records
+(@pxref{Daemon Offload Setup}).
+
+Using this field, a service may add new build machines to receive builds
+offloaded by the daemon.  This is useful for a service such as
+@code{hurd-vm-service-type}, which can make a GNU/Hurd virtual machine
+directly usable for offloading (@pxref{hurd-vm,
+@code{hurd-vm-service-type}}).
+
 @item @code{chroot-directories} (default: @code{'()})
 A list of file-like objects or strings pointing to additional directories the build daemon can use.
 @end table
@@ -35650,6 +35689,7 @@ Virtualization Services
 @end deftp
 
 
+@anchor{hurd-vm}
 @subsubheading The Hurd in a Virtual Machine
 
 @cindex @code{hurd}
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 10e0d4cf9d..98d59fd36d 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1743,6 +1743,31 @@ (define %default-authorized-guix-keys
   (list (file-append guix "/share/guix/berlin.guix.gnu.org.pub")
         (file-append guix "/share/guix/bordeaux.guix.gnu.org.pub")))
 
+(define (guix-machines-files-installation machines)
+  "Return a gexp to install MACHINES, a list of gexps, as
+/etc/guix/machines.scm, which is used for offloading."
+  (with-imported-modules '((guix build utils))
+    #~(begin
+        (use-modules (guix build utils))
+
+        (define machines-file
+          "/etc/guix/machines.scm")
+
+        ;; If MACHINES-FILE already exists, move it out of the way.
+        ;; Create a backup if it's a regular file: it's likely that the
+        ;; user manually updated it.
+        (if (file-exists? machines-file)
+            (if (and (symbolic-link? machines-file)
+                     (store-file-name? (readlink machines-file)))
+                (delete-file machines-file)
+                (rename-file machines-file
+                             (string-append machines-file ".bak")))
+            (mkdir-p (dirname machines-file)))
+
+        ;; Installed the declared machines file.
+        (symlink #+(scheme-file "machines.scm" machines)
+                 machines-file))))
+
 (define-record-type* <guix-configuration>
   guix-configuration make-guix-configuration
   guix-configuration?
@@ -1780,6 +1805,8 @@ (define-record-type* <guix-configuration>
                     (default #f))
   (tmpdir           guix-tmpdir                   ;string | #f
                     (default #f))
+  (build-machines   guix-build-machines           ;list of gexps | #f
+                    (default #f))
   (environment      guix-configuration-environment  ;list of strings
                     (default '())))
 
@@ -1965,8 +1992,15 @@ (define (guix-activation config)
           (system* #$(file-append guix "/bin/guix") "archive"
                    "--generate-key"))
 
+        ;; Optionally install /etc/guix/acl...
         #$(if authorize-key?
               (substitute-key-authorization authorized-keys guix)
+              #~#f)
+
+        ;; ... and /etc/guix/machines.scm.
+        #$(if (guix-build-machines config)
+              (guix-machines-files-installation
+               #~(list #$@(guix-build-machines config)))
               #~#f))))
 
 (define-record-type* <guix-extension>
@@ -1976,6 +2010,8 @@ (define-record-type* <guix-extension>
                     (default '()))
   (substitute-urls guix-extension-substitute-urls ;list of strings
                     (default '()))
+  (build-machines  guix-extension-build-machines  ;list of gexps
+                   (default '()))
   (chroot-directories guix-extension-chroot-directories ;list of file-like/strings
                       (default '())))
 
@@ -1985,6 +2021,8 @@ (define (guix-extension-merge a b)
                             (guix-extension-authorized-keys b)))
    (substitute-urls (append (guix-extension-substitute-urls a)
                             (guix-extension-substitute-urls b)))
+   (build-machines (append (guix-extension-build-machines a)
+                           (guix-extension-build-machines b)))
    (chroot-directories (append (guix-extension-chroot-directories a)
                                (guix-extension-chroot-directories b)))))
 
@@ -2008,6 +2046,11 @@ (define guix-service-type
                                        (guix-configuration-authorized-keys config)))
               (substitute-urls (append (guix-extension-substitute-urls extension)
                                        (guix-configuration-substitute-urls config)))
+              (build-machines
+               (and (or (guix-build-machines config)
+                        (pair? (guix-extension-build-machines extension)))
+                    (append (or (guix-build-machines config) '())
+                            (guix-extension-build-machines extension))))
               (chroot-directories
                (append (guix-extension-chroot-directories extension)
                        (guix-configuration-chroot-directories config))))))
-- 
2.41.0

Message #26 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 07/12] services: childhurd: Authorize the childhurd’s key on the host.
Date: Fri, 22 Sep 2023 14:54:08 +0200

This partly automates setting up a childhurd for offloading purposes.

* gnu/services/virtualization.scm (authorize-guest-substitutes-on-host):
New procedure.
(hurd-vm-activation): Use it.
---
 gnu/services/virtualization.scm | 51 ++++++++++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index fd153dd051..ca000f5d28 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -28,6 +28,7 @@ (define-module (gnu services virtualization)
   #:use-module (gnu image)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages gdb)
+  #:autoload   (gnu packages gnupg) (guile-gcrypt)
   #:use-module (gnu packages package-management)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages virtualization)
@@ -50,6 +51,7 @@ (define-module (gnu services virtualization)
   #:use-module (guix records)
   #:use-module (guix store)
   #:use-module (guix utils)
+  #:autoload   (guix self) (make-config.scm)
 
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
@@ -1271,6 +1273,50 @@ (define (initialize-hurd-vm-substitutes)
 
   (program-file "initialize-hurd-vm-substitutes" run))
 
+(define (authorize-guest-substitutes-on-host)
+  "Return a program that authorizes the guest's archive signing key (passed as
+an argument) on the host."
+  (define not-config?
+    (match-lambda
+      ('(guix config) #f)
+      (('guix _ ...) #t)
+      (('gnu _ ...) #t)
+      (_ #f)))
+
+  (define run
+    (with-extensions (list guile-gcrypt)
+      (with-imported-modules `(((guix config) => ,(make-config.scm))
+                               ,@(source-module-closure
+                                  '((guix pki)
+                                    (guix build utils))
+                                  #:select? not-config?))
+        #~(begin
+            (use-modules (ice-9 match)
+                         (ice-9 textual-ports)
+                         (gcrypt pk-crypto)
+                         (guix pki)
+                         (guix build utils))
+
+            (match (command-line)
+              ((_ guest-config-directory)
+               (let ((guest-key (string-append guest-config-directory
+                                               "/signing-key.pub")))
+                 (if (file-exists? guest-key)
+                     ;; Add guest key to the host's ACL.
+                     (let* ((key (string->canonical-sexp
+                                  (call-with-input-file guest-key
+                                    get-string-all)))
+                            (acl (public-keys->acl
+                                  (cons key (acl->public-keys (current-acl))))))
+                       (with-atomic-file-replacement %acl-file
+                         (lambda (_ port)
+                           (write-acl acl port))))
+                     (format (current-error-port)
+                             "warning: guest key missing from '~a'~%"
+                             guest-key)))))))))
+
+  (program-file "authorize-guest-substitutes-on-host" run))
+
 (define (hurd-vm-activation config)
   "Return a gexp to activate the Hurd VM according to CONFIG."
   (with-imported-modules '((guix build utils))
@@ -1294,7 +1340,10 @@ (define (hurd-vm-activation config)
 
         (unless (file-exists? guix-directory)
           (invoke #$(initialize-hurd-vm-substitutes)
-                  guix-directory)))))
+                  guix-directory))
+
+        ;; Authorize the archive signing key from GUIX-DIRECTORY in the host.
+        (invoke #$(authorize-guest-substitutes-on-host) guix-directory))))
 
 (define hurd-vm-service-type
   (service-type
-- 
2.41.0

Message #29 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 10/12] services: hurd-vm: Disable password-based
 authentication for root.
Date: Fri, 22 Sep 2023 14:54:11 +0200

With offloading to a childhurd is enabled, allowing password-less root
login in the childhurd to anyone amounts to providing write access to
the host’s store to anyone.  Thus, disable password-based root logins in
the childhurd.

* gnu/services/virtualization.scm (%hurd-vm-operating-system): Change
‘permit-root-login’ to 'prohibit-password.
* gnu/tests/virtualization.scm (%childhurd-os): Provide a custom ‘os’
field for ‘hurd-vm-configuration’.
* doc/guix.texi (Virtualization Services): Remove mention of
password-less root login.
---
 doc/guix.texi                   |  5 -----
 gnu/services/virtualization.scm |  2 +-
 gnu/tests/virtualization.scm    | 15 ++++++++++++++-
 3 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 472e2e0958..95f29a2d19 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35718,11 +35718,6 @@ Virtualization Services
 The default configuration (see @code{hurd-vm-configuration} below)
 spawns a secure shell (SSH) server in your GNU/Hurd system, which QEMU
 (the virtual machine emulator) redirects to port 10222 on the host.
-Thus, you can connect over SSH to the childhurd with:
-
-@example
-ssh root@@localhost -p 10022
-@end example
 
 The childhurd is volatile and stateless: it starts with a fresh root
 file system every time you restart it.  By default though, all the files
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 258b503461..930c2ce702 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1080,7 +1080,7 @@ (define %hurd-vm-operating-system
                         (openssh-configuration
                          (openssh openssh-sans-x)
                          (use-pam? #f)
-                         (permit-root-login #t)
+                         (permit-root-login 'prohibit-password)
                          (allow-empty-passwords? #t)
                          (password-authentication? #t)))
 
diff --git a/gnu/tests/virtualization.scm b/gnu/tests/virtualization.scm
index 9e7928703e..599e58edf0 100644
--- a/gnu/tests/virtualization.scm
+++ b/gnu/tests/virtualization.scm
@@ -31,6 +31,7 @@ (define-module (gnu tests virtualization)
   #:use-module (gnu services)
   #:use-module (gnu services dbus)
   #:use-module (gnu services networking)
+  #:use-module (gnu services ssh)
   #:use-module (gnu services virtualization)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages virtualization)
@@ -228,7 +229,19 @@ (define %test-qemu-guest-agent
 (define %childhurd-os
   (simple-operating-system
    (service dhcp-client-service-type)
-   (service hurd-vm-service-type)))
+   (service hurd-vm-service-type
+            (hurd-vm-configuration
+             ;; Allow root login with an empty password to simplify the test
+             ;; below.
+             (os (operating-system
+                   (inherit %hurd-vm-operating-system)
+                   (services
+                    (modify-services (operating-system-user-services
+                                      %hurd-vm-operating-system)
+                      (openssh-service-type
+                       config => (openssh-configuration
+                                  (inherit config)
+                                  (permit-root-login #t)))))))))))
 
 (define (run-childhurd-test)
   (define os
-- 
2.41.0

Message #32 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 09/12] tests: hurd-vm: Remove custom disk image configuration.
Date: Fri, 22 Sep 2023 14:54:10 +0200

This was added in 18e76f89055f25f015fadb7c999b410f38a88cc6.  Presumably,
the problem was that using compressed QCOW2 images makes the childhurd
slower, so it’s eventually marked as failing to start.  By enabling KVM
inside the Guix System VM, we allow the childhurd to run on KVM, which
compensates the slowdown due to the use of a compressed image.

* gnu/tests/virtualization.scm (hurd-vm-disk-image-raw): Remove.
(%childhurd-os): Use default config for ‘hurd-vm-service-type’.
(run-childhurd-test)[test]: Pass “-cpu host” to the run-vm script.
---
 gnu/tests/virtualization.scm | 20 +++++---------------
 1 file changed, 5 insertions(+), 15 deletions(-)

diff --git a/gnu/tests/virtualization.scm b/gnu/tests/virtualization.scm
index 41253968e9..9e7928703e 100644
--- a/gnu/tests/virtualization.scm
+++ b/gnu/tests/virtualization.scm
@@ -225,23 +225,10 @@ (define %test-qemu-guest-agent
 ;;; GNU/Hurd virtual machines, aka. childhurds.
 ;;;
 
-;; Copy of `hurd-vm-disk-image', using plain disk-image for test
-(define (hurd-vm-disk-image-raw config)
-  (let ((os ((@@ (gnu services virtualization) secret-service-operating-system)
-             (hurd-vm-configuration-os config)))
-        (disk-size (hurd-vm-configuration-disk-size config)))
-    (image
-     (inherit hurd-disk-image)
-     (format 'disk-image)
-     (size disk-size)
-     (operating-system os))))
-
 (define %childhurd-os
   (simple-operating-system
    (service dhcp-client-service-type)
-   (service hurd-vm-service-type
-            (hurd-vm-configuration
-             (image (hurd-vm-disk-image-raw this-record))))))
+   (service hurd-vm-service-type)))
 
 (define (run-childhurd-test)
   (define os
@@ -292,7 +279,10 @@ (define (run-childhurd-test)
                        (ice-9 match))
 
           (define marionette
-            (make-marionette (list #$vm)))
+            ;; Emulate the host CPU so that KVM is available inside as well
+            ;; ("nested KVM"), provided
+            ;; /sys/module/kvm_intel/parameters/nested (or similar) allows it.
+            (make-marionette (list #$vm "-cpu" "host")))
 
           (test-runner-current (system-test-runner #$output))
           (test-begin "childhurd")
-- 
2.41.0

Message #35 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 11/12] doc: Give an example showing how to add an account in
 the childhurd.
Date: Fri, 22 Sep 2023 14:54:12 +0200

* doc/guix.texi (Virtualization Services): Give an example showing how
to add an account.
---
 doc/guix.texi | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 95f29a2d19..53b0ebd1db 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35727,6 +35727,57 @@ Virtualization Services
 substitute keys, and so on---see the explanation of @code{secret-root}
 below.
 
+You will probably find it useful to create an account for you in the
+GNU/Hurd virtual machine and to authorize logins with your SSH key.  To
+do that, you can define the GNU/Hurd system in the usual way
+(@pxref{Using the Configuration System}), and then pass that operating
+system as the @code{os} field of @code{hurd-vm-configuration}, as in
+this example:
+
+@lisp
+(define childhurd-os
+  ;; Definition of my GNU/Hurd system, derived from the default one.
+  (operating-system
+    (inherit %hurd-vm-operating-system)
+
+    ;; Add a user account.
+    (users (cons (user-account
+                  (name "charlie")
+                  (comment "This is me!")
+                  (group "users")
+                  (supplementary-groups '("wheel"))) ;for 'sudo'
+                 %base-user-accounts))
+
+    (services
+     ;; Modify the SSH configuration to allow login as "root"
+     ;; and as "charlie" using public key authentication.
+     (modify-services (operating-system-user-services
+                       %hurd-vm-operating-system)
+       (openssh-service-type
+        config => (openssh-configuration
+                   (inherit config)
+                   (authorized-keys
+                    `(("root"
+                       ,(local-file
+                         "/home/charlie/.ssh/id_rsa.pub"))
+                      ("charlie"
+                       ,(local-file
+                         "/home/charlie/.ssh/id_rsa.pub"))))))))))
+
+(operating-system
+  ;; @dots{}
+  (services
+    ;; Add the 'hurd-vm' service, configured to use the
+    ;; operating system configuration above.
+    (append (list (service hurd-vm-service-type
+                           (hurd-vm-configuration
+                             (os %childhurd-os))))
+            %base-services)))
+@end lisp
+
+That's it!  The remainder of this section provides the reference of the
+service configuration.
+
 @defvar hurd-vm-service-type
 This is the type of the Hurd in a Virtual Machine service.  Its value
 must be a @code{hurd-vm-configuration} object, which specifies the
-- 
2.41.0

Message #38 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 08/12] services: hurd-vm: ‘image’ field has to be an <image> record.
Date: Fri, 22 Sep 2023 14:54:09 +0200

* gnu/services/virtualization.scm (<hurd-vm-configuration>)[image]:
Document as being an <image> record.
(hurd-vm-disk-image): Remove call to ‘system-image’.
(hurd-vm-shepherd-service): Add call to ‘system-image’.
* gnu/tests/virtualization.scm (hurd-vm-disk-image-raw): Remove call to
‘system-image’.
* doc/guix.texi (Virtualization Services): Adjust accordingly.
---
 doc/guix.texi                   |  4 ++--
 gnu/services/virtualization.scm |  9 ++++-----
 gnu/tests/virtualization.scm    | 11 +++++------
 3 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index ca48d6c404..472e2e0958 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35766,8 +35766,8 @@ Virtualization Services
 The QEMU package to use.
 
 @item @code{image} (default: @var{hurd-vm-disk-image})
-The procedure used to build the disk-image built from this
-configuration.
+The image object representing the disk image of this virtual machine
+(@pxref{System Images}).
 
 @item @code{disk-size} (default: @code{'guess})
 The size of the disk image.
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index ca000f5d28..258b503461 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1101,7 +1101,7 @@ (define-record-type* <hurd-vm-configuration>
                (default %hurd-vm-operating-system))
   (qemu        hurd-vm-configuration-qemu               ;file-like
                (default qemu-minimal))
-  (image       hurd-vm-configuration-image              ;string
+  (image       hurd-vm-configuration-image              ;<image>
                (thunked)
                (default (hurd-vm-disk-image this-record)))
   (disk-size   hurd-vm-configuration-disk-size          ;number or 'guess
@@ -1126,9 +1126,8 @@ (define (hurd-vm-disk-image config)
          (disk-size (hurd-vm-configuration-disk-size config))
          (type      (lookup-image-type-by-name 'hurd-qcow2))
          (os->image (image-type-constructor type)))
-    (system-image
-     (image (inherit (os->image os))
-            (size disk-size)))))
+    (image (inherit (os->image os))
+           (size disk-size))))
 
 (define (hurd-vm-port config base)
   "Return the forwarded vm port for this childhurd config."
@@ -1170,7 +1169,7 @@ (define (hurd-vm-shepherd-service config)
                       "-m" (number->string #$memory-size)
                       #$@net-options
                       #$@options
-                      "--hda" #+image
+                      "--hda" #+(system-image image)
 
                       ;; Cause the service to be respawned if the guest
                       ;; reboots (it can reboot for instance if it did not
diff --git a/gnu/tests/virtualization.scm b/gnu/tests/virtualization.scm
index 73c8099b79..41253968e9 100644
--- a/gnu/tests/virtualization.scm
+++ b/gnu/tests/virtualization.scm
@@ -230,12 +230,11 @@ (define (hurd-vm-disk-image-raw config)
   (let ((os ((@@ (gnu services virtualization) secret-service-operating-system)
              (hurd-vm-configuration-os config)))
         (disk-size (hurd-vm-configuration-disk-size config)))
-    (system-image
-     (image
-      (inherit hurd-disk-image)
-      (format 'disk-image)
-      (size disk-size)
-      (operating-system os)))))
+    (image
+     (inherit hurd-disk-image)
+     (format 'disk-image)
+     (size disk-size)
+     (operating-system os))))
 
 (define %childhurd-os
   (simple-operating-system
-- 
2.41.0

Message #41 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 66156 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 12/12] services: hurd-vm: Implement zero-configuration
 offloading.
Date: Fri, 22 Sep 2023 14:54:13 +0200

This allows for zero-configuration offloading to a childhurd.

* gnu/services/virtualization.scm (operating-system-with-offloading-account):
New procedure.
(<hurd-vm-configuration>)[offloading?]: New field.
(hurd-vm-disk-image): Define ‘transform’ and use it.
(hurd-vm-activation): Generate SSH key for user ‘offloading’ and add
authorize it via /etc/childhurd/etc/ssh/authorized_keys.d.
(hurd-vm-configuration-offloading-ssh-key)
(hurd-vm-guix-extension): New procedures.
(hurd-vm-service-type): Add GUIX-SERVICE-TYPE extension.
* gnu/tests/virtualization.scm (run-childhurd-test)[import-module?]: New
procedure.
[os]: Add (gnu build install) and its closure to #:import-modules.
[test]: Add “copy-on-write store” and “offloading” tests.
* doc/guix.texi (Virtualization Services): Document it.
---
 doc/guix.texi                   | 71 +++++++++++++++----------
 gnu/services/virtualization.scm | 92 +++++++++++++++++++++++++++++++--
 gnu/tests/virtualization.scm    | 38 +++++++++++++-
 3 files changed, 169 insertions(+), 32 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 53b0ebd1db..35da58e59f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35718,6 +35718,15 @@ Virtualization Services
 The default configuration (see @code{hurd-vm-configuration} below)
 spawns a secure shell (SSH) server in your GNU/Hurd system, which QEMU
 (the virtual machine emulator) redirects to port 10222 on the host.
+By default, the service enables @dfn{offloading} such that the host
+@code{guix-daemon} automatically offloads GNU/Hurd builds to the
+childhurd (@pxref{Daemon Offload Setup}).  This is what happens when
+running a command like the following one, where @code{i586-gnu} is the
+system type of 32-bit GNU/Hurd:
+
+@example
+guix build emacs-minimal -s i586-gnu
+@end example
 
 The childhurd is volatile and stateless: it starts with a fresh root
 file system every time you restart it.  By default though, all the files
@@ -35851,6 +35860,41 @@ Virtualization Services
 @var{vnc-port}: @code{(+ 15900 (* 1000 @var{ID}))}
 @end example
 
+@cindex childhurd, offloading
+@cindex Hurd, offloading
+@item @code{offloading?} (default: @code{#t})
+Whether to automatically set up offloading of builds to the childhurd.
+
+When enabled, this lets you run GNU/Hurd builds on the host and have
+them transparently offloaded to the VM, for instance when running a
+command like this:
+
+@example
+guix build coreutils -s i586-gnu
+@end example
+
+This option automatically sets up offloading like so:
+
+@enumerate
+@item
+Authorizing the childhurd's key on the host so that the host accepts
+build results coming from the childhurd, which can be done like so
+(@pxref{Invoking guix archive, @command{guix archive --authorize}}, for
+more on that).
+
+@item
+Creating a user account called @code{offloading} dedicated to offloading
+in the childhurd.
+
+@item
+Creating an SSH key pair on the host and making it an authorized key of
+the @code{offloading} account in the childhurd.
+
+@item
+Adding the childhurd to @file{/etc/guix/machines.scm} (@pxref{Daemon
+Offload Setup}).
+@end enumerate
+
 @item @code{secret-root} (default: @file{/etc/childhurd})
 The root directory with out-of-band secrets to be installed into the
 childhurd once it runs.  Childhurds are volatile which means that on
@@ -35868,38 +35912,13 @@ Virtualization Services
 /etc/childhurd/etc/guix/acl
 /etc/childhurd/etc/guix/signing-key.pub
 /etc/childhurd/etc/guix/signing-key.sec
+/etc/childhurd/etc/ssh/authorized_keys.d/offloading
 /etc/childhurd/etc/ssh/ssh_host_ed25519_key
 /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
 /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
 /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
 @end example
 
-These files are automatically sent to the guest Hurd VM when it boots,
-including permissions.
-
-@cindex childhurd, offloading
-@cindex Hurd, offloading
-Having these files in place means that only a couple of things are
-missing to allow the host to offload @code{i586-gnu} builds to the
-childhurd:
-
-@enumerate
-@item
-Authorizing the childhurd's key on the host so that the host accepts
-build results coming from the childhurd, which can be done like so:
-
-@example
-guix archive --authorize < \
-  /etc/childhurd/etc/guix/signing-key.pub
-@end example
-
-@item
-Adding the childhurd to @file{/etc/guix/machines.scm} (@pxref{Daemon
-Offload Setup}).
-@end enumerate
-
-We're working towards making that happen automatically---get in touch
-with us at @email{guix-devel@@gnu.org} to discuss it!
 @end table
 @end deftp
 
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 930c2ce702..076eca7ea2 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -27,6 +27,7 @@ (define-module (gnu services virtualization)
   #:use-module (gnu bootloader grub)
   #:use-module (gnu image)
   #:use-module (gnu packages admin)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages gdb)
   #:autoload   (gnu packages gnupg) (guile-gcrypt)
   #:use-module (gnu packages package-management)
@@ -52,6 +53,7 @@ (define-module (gnu services virtualization)
   #:use-module (guix store)
   #:use-module (guix utils)
   #:autoload   (guix self) (make-config.scm)
+  #:autoload   (guix platform) (platform-system)
 
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
@@ -1063,6 +1065,26 @@ (define (secret-service-operating-system os)
 ;;; The Hurd in VM service: a Childhurd.
 ;;;
 
+(define (operating-system-with-offloading-account os)
+  (define accounts
+    (list (user-group
+           (name "offloading")
+           (system? #t))
+          (user-account
+           (name "offloading")
+           (group "offloading")
+           (system? #t)
+           (comment "Offloading privilege separation user")
+           (home-directory "/var/run/offloading")
+           (shell (file-append bash-minimal "/bin/sh")))))
+
+  (operating-system
+    (inherit os)
+    (services (cons (simple-service 'offloading-account
+                                    account-service-type
+                                    accounts)
+                    (operating-system-user-services os)))))
+
 (define %hurd-vm-operating-system
   (operating-system
     (inherit %hurd-default-operating-system)
@@ -1115,14 +1137,21 @@ (define-record-type* <hurd-vm-configuration>
   (net-options hurd-vm-configuration-net-options        ;list of string
                (thunked)
                (default (hurd-vm-net-options this-record)))
+  (offloading? hurd-vm-configuration-offloading?        ;Boolean
+               (default #t))
   (secret-root hurd-vm-configuration-secret-root        ;string
                (default "/etc/childhurd")))
 
 (define (hurd-vm-disk-image config)
   "Return a disk-image for the Hurd according to CONFIG.  The secret-service
 is added to the OS specified in CONFIG."
-  (let* ((os        (secret-service-operating-system
-                     (hurd-vm-configuration-os config)))
+  (define transform
+    (compose secret-service-operating-system
+             (if (hurd-vm-configuration-offloading? config)
+                 operating-system-with-offloading-account
+                 identity)))
+
+  (let* ((os        (transform (hurd-vm-configuration-os config)))
          (disk-size (hurd-vm-configuration-disk-size config))
          (type      (lookup-image-type-by-name 'hurd-qcow2))
          (os->image (image-type-constructor type)))
@@ -1331,18 +1360,71 @@ (define (hurd-vm-activation config)
         (define guix-directory
           (string-append secret-directory "/etc/guix"))
 
+        (define offloading-ssh-key
+          #$(hurd-vm-configuration-offloading-ssh-key config))
+
         (unless (file-exists? ssh-directory)
           ;; Generate SSH host keys under SSH-DIRECTORY.
           (mkdir-p ssh-directory)
           (invoke #$(file-append openssh "/bin/ssh-keygen")
                   "-A" "-f" secret-directory))
 
+        (unless (or (not #$(hurd-vm-configuration-offloading? config))
+                    (file-exists? offloading-ssh-key))
+          ;; Generate a user SSH key pair for the host to use when offloading
+          ;; to the guest.
+          (mkdir-p (dirname offloading-ssh-key))
+          (invoke #$(file-append openssh "/bin/ssh-keygen")
+                  "-t" "ed25519" "-N" ""
+                  "-f" offloading-ssh-key)
+
+          ;; Authorize it in the guest for user 'offloading'.
+          (let ((authorizations
+                 (string-append ssh-directory
+                                "/authorized_keys.d/offloading")))
+            (mkdir-p (dirname authorizations))
+            (copy-file (string-append offloading-ssh-key ".pub")
+                       authorizations)
+            (chmod (dirname authorizations) #o555)))
+
         (unless (file-exists? guix-directory)
           (invoke #$(initialize-hurd-vm-substitutes)
                   guix-directory))
 
-        ;; Authorize the archive signing key from GUIX-DIRECTORY in the host.
-        (invoke #$(authorize-guest-substitutes-on-host) guix-directory))))
+        (when #$(hurd-vm-configuration-offloading? config)
+          ;; Authorize the archive signing key from GUIX-DIRECTORY in the host.
+          (invoke #$(authorize-guest-substitutes-on-host) guix-directory)))))
+
+(define (hurd-vm-configuration-offloading-ssh-key config)
+  "Return the name of the file containing the SSH key of user 'offloading'."
+  (string-append "/etc/guix/offload/ssh/childhurd"
+                 (or (and=> (hurd-vm-configuration-id config)
+                            number->string)
+                     "")))
+
+(define (hurd-vm-guix-extension config)
+  "When offloading is enabled, add this childhurd to the list of offlading
+machines in /etc/guix/machines.scm."
+  (if (hurd-vm-configuration-offloading? config)
+      (let* ((image (hurd-vm-configuration-image config))
+             (platform (image-platform image))
+             (system (platform-system platform))
+             (vm-ssh-key (string-append
+                          (hurd-vm-configuration-secret-root config)
+                          "/etc/ssh/ssh_host_ed25519_key.pub"))
+             (host-ssh-key (hurd-vm-configuration-offloading-ssh-key config)))
+        (guix-extension
+         (build-machines
+          (list #~(build-machine
+                   (name "localhost")
+                   (port #$(hurd-vm-port config %hurd-vm-ssh-port))
+                   (systems '(#$system))
+                   (host-key (call-with-input-file #$vm-ssh-key
+                               (@ (ice-9 textual-ports)
+                                  get-string-all)))
+                   (user "offloading")
+                   (private-key #$host-ssh-key))))))
+      (guix-extension)))
 
 (define hurd-vm-service-type
   (service-type
@@ -1351,6 +1433,8 @@ (define hurd-vm-service-type
                                         hurd-vm-shepherd-service)
                      (service-extension account-service-type
                                         (const %hurd-vm-accounts))
+                     (service-extension guix-service-type
+                                        hurd-vm-guix-extension)
                      (service-extension activation-service-type
                                         hurd-vm-activation)))
    (default-value (hurd-vm-configuration))
diff --git a/gnu/tests/virtualization.scm b/gnu/tests/virtualization.scm
index 599e58edf0..b79164737b 100644
--- a/gnu/tests/virtualization.scm
+++ b/gnu/tests/virtualization.scm
@@ -38,6 +38,7 @@ (define-module (gnu tests virtualization)
   #:use-module (guix gexp)
   #:use-module (guix records)
   #:use-module (guix store)
+  #:use-module (guix modules)
   #:export (%test-libvirt
             %test-qemu-guest-agent
             %test-childhurd))
@@ -244,11 +245,19 @@ (define %childhurd-os
                                   (permit-root-login #t)))))))))))
 
 (define (run-childhurd-test)
+  (define (import-module? module)
+    ;; This module is optional and depends on Guile-Gcrypt, do skip it.
+    (and (guix-module-name? module)
+         (not (equal? module '(guix store deduplication)))))
+
   (define os
     (marionette-operating-system
      %childhurd-os
-     #:imported-modules '((gnu services herd)
-                          (guix combinators))))
+     #:imported-modules (source-module-closure
+                         '((gnu services herd)
+                           (guix combinators)
+                           (gnu build install))
+                         #:select? import-module?)))
 
   (define vm
     (virtual-machine
@@ -373,6 +382,31 @@ (define (run-childhurd-test)
                                    (pk 'drv (string-trim-right drv)))
                    drv)))
 
+          (test-assert "copy-on-write store"
+            ;; Set up a writable store.  The root partition is already an
+            ;; overlayfs, which is not suitable as the bottom part of this
+            ;; additional overlayfs; thus, create a tmpfs for the backing
+            ;; store.
+            ;; TODO: Remove this when <virtual-machine> creates a writable
+            ;; store.
+            (marionette-eval
+             '(begin
+                (use-modules (gnu build install)
+                             (guix build syscalls))
+
+                (mkdir "/run/writable-store")
+                (mount "none" "/run/writable-store" "tmpfs")
+                (mount-cow-store "/run/writable-store" "/backing-store")
+                (system* "df" "-hT"))
+             marionette))
+
+          (test-equal "offloading"
+            0
+            (marionette-eval
+             '(and (file-exists? "/etc/guix/machines.scm")
+                   (system* "guix" "offload" "test"))
+             marionette))
+
           (test-end))))
 
   (gexp->derivation "childhurd-test" test))
-- 
2.41.0

Message #44 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 66156 <at> debbugs.gnu.org
Subject: Re: [PATCH 00/12] Introducing Smart Hurdloading
Date: Fri, 22 Sep 2023 16:07:13 +0200

Ludovic Courtès writes:

Hi Ludo,

> This patch series, my friends, sets up Smart Hurdloading™, sometimes
> referred to as “Hurd auto-offloading”.  (Yeah I worked hard on branding…)
>
> Concretely, if you add (service hurd-vm-service-type), you can now run:
>
>   guix build libreoffice -s i586-gnu

Beautiful!  I'm wondering how this works wrt disk size; the default is
still 'guess?  In my childhurds I have

   (hurd-vm-configuration
    ..
    (disk-size (* 16 1024 (expt 2 20))) ;16GiB

which is pretty cheap now that we have qcow2.  I'm not sure how
expensive it is to set memory-size if you don't use it?

> The series is a bit of a potpourri: I fixed Hurd issues here and there
> (currently offloading to a childhurd doesn’t work on ‘master’, because
> of the locale issue, for instance), improved documentation, etc.  I also
> took a couple of long detours not shown here that might lead to further
> improvements in the future.

That's great, very nice.  And quite understandable afaic, the hurd-team
branch has also been quite a mixed set of work.

> My goal is to extend this mechanism beyond the Hurd, to have a generic
> mechanism to spin up Guix System VMs we can easily offload to.
>
> Thoughts?

I'm wondering if there is a way to/if we need a way to set the default
priority of the childhurd.  I'm usually offloading to another machine's
childhurd and will probably want to prioritize that when I'm at home.

(Making a copy of /etc/guix/machines.scm and editing that as required
would probably mork for me.)

I've added this patch set to the hurd-team branch and am testing it
now.  The only thing I found without testing:

> Subject: [bug#66156] [PATCH 06/12] services: guix: Support declarative offloading setup.
> To: 66156 <at> debbugs.gnu.org
> Cc: Ludovic Courtès <ludo <at> gnu.org>
> Date: Fri, 22 Sep 2023 14:54:07 +0200 (44 minutes, 21 seconds ago)
> Resent-From: Ludovic Courtès <ludo <at> gnu.org>
> 
> * gnu/services/base.scm (guix-machines-files-installation): New
> procedure.
> (<guix-configuration>)[build-machines]: New field.
> (guix-activation): Call ‘ guix-machines-files-installation’.
                           ^
Remove space.

Thanks for this amazing piece of work and bug-fixing!

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #47 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 66156 <at> debbugs.gnu.org
Subject: Re: bug#66156: [PATCH 00/12] Introducing Smart Hurdloading
Date: Fri, 22 Sep 2023 17:24:51 +0200

Hello!

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> Beautiful!  I'm wondering how this works wrt disk size; the default is
> still 'guess?  In my childhurds I have
>
>    (hurd-vm-configuration
>     ..
>     (disk-size (* 16 1024 (expt 2 20))) ;16GiB
>
> which is pretty cheap now that we have qcow2.  I'm not sure how
> expensive it is to set memory-size if you don't use it?

I guess setting ‘disk-size’ should still work (and as you write: with
QCOW2, it doesn’t cost much to ask for extra space, until you actually
use it); nothing has changed here.

> I'm wondering if there is a way to/if we need a way to set the default
> priority of the childhurd.  I'm usually offloading to another machine's
> childhurd and will probably want to prioritize that when I'm at home.

Good question.  Perhaps you could declare that other machine with a
higher ‘speed’ value?

Thanks for your feedback!

Ludo’.

Message #50 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 66156 <at> debbugs.gnu.org
Subject: Re: bug#66156: [PATCH 00/12] Introducing Smart Hurdloading
Date: Sat, 23 Sep 2023 15:44:41 +0200

Ludovic Courtès writes:

Hello!

Okay, after overcoming completely unreleated troubles I finally managed
to reconfigure a laptop to hurd-team to test it.  It works great!

> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> Beautiful!  I'm wondering how this works wrt disk size; the default is
>> still 'guess?  In my childhurds I have
>>
>>    (hurd-vm-configuration
>>     ..
>>     (disk-size (* 16 1024 (expt 2 20))) ;16GiB
>>
>> which is pretty cheap now that we have qcow2.  I'm not sure how
>> expensive it is to set memory-size if you don't use it?
>
> I guess setting ‘disk-size’ should still work (and as you write: with
> QCOW2, it doesn’t cost much to ask for extra space, until you actually
> use it); nothing has changed here.

Yes, works beautifully.

>> I'm wondering if there is a way to/if we need a way to set the default
>> priority of the childhurd.  I'm usually offloading to another machine's
>> childhurd and will probably want to prioritize that when I'm at home.
>
> Good question.  Perhaps you could declare that other machine with a
> higher ‘speed’ value?

That also works nicely.

I found that by default, only root can offload.  Is that intentional,
how would that work?  I changed permissions on
`/etc/guix/offload/ssh/childhurd' like so

--8<---------------cut here---------------start------------->8---
$ l /etc/guix/offload/ssh
total 8
-rw-r--r-- 1 root root   93 Sep 23 14:47 childhurd.pub
-rw-r----- 1 root wheel 399 Sep 23 14:47 childhurd
--8<---------------cut here---------------end--------------->8---

to "fix" that. WDYT?

Found another two commit message nitpicks

> Subject: Re: [bug#66156] [PATCH 12/12] services: hurd-vm: Implement zero-configuration offloading.
[..]

> > This allows for zero-configuration offloading to a childhurd.
>
> * gnu/services/virtualization.scm (operating-system-with-offloading-account):
> New procedure.
> (<hurd-vm-configuration>)[offloading?]: New field.
> (hurd-vm-disk-image): Define ‘transform’ and use it.
> (hurd-vm-activation): Generate SSH key for user ‘offloading’ and add
> authorize it via /etc/childhurd/etc/ssh/authorized_keys.d.

Remove "add".

> (hurd-vm-configuration-offloading-ssh-key)
> (hurd-vm-guix-extension): New procedures.

Rather use 

(hurd-vm-configuration-offloading-ssh-key,
hurd-vm-guix-extension): New procedures.

?

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #53 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 66156 <at> debbugs.gnu.org
Subject: Re: bug#66156: [PATCH 00/12] Introducing Smart Hurdloading
Date: Wed, 27 Sep 2023 19:35:59 +0200

Hello,

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> Okay, after overcoming completely unreleated troubles I finally managed
> to reconfigure a laptop to hurd-team to test it.  It works great!

Yay, thanks for testing!

> I found that by default, only root can offload.  Is that intentional,
> how would that work?

It’s is intentional: ‘guix offload’ is only ever invoked by
guix-daemon¹, as root.

¹ Except for ‘guix offload test’, which is meant to be spawned by users,
  as root too.

> I changed permissions on `/etc/guix/offload/ssh/childhurd' like so
>
> $ l /etc/guix/offload/ssh
> total 8
> -rw-r--r-- 1 root root   93 Sep 23 14:47 childhurd.pub
> -rw-r----- 1 root wheel 399 Sep 23 14:47 childhurd
>
> to "fix" that. WDYT?

What would be the use case?

> Found another two commit message nitpicks

Noted!

I’d like to see if there are other opinions in particular about
/etc/guix/machines.scm generation, then I guess we can merge?

Thanks,
Ludo’.

Message #56 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 66156 <at> debbugs.gnu.org
Subject: Re: bug#66156: [PATCH 00/12] Introducing Smart Hurdloading
Date: Wed, 27 Sep 2023 19:52:23 +0200

Ludovic Courtès writes:

Hi!

> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> Okay, after overcoming completely unreleated troubles I finally managed
>> to reconfigure a laptop to hurd-team to test it.  It works great!
>
> Yay, thanks for testing!

Happy to :)

>> I found that by default, only root can offload.  Is that intentional,
>> how would that work?
>
> It’s is intentional: ‘guix offload’ is only ever invoked by
> guix-daemon¹, as root.

...ah.  I missed...

> ¹ Except for ‘guix offload test’, which is meant to be spawned by users,
>   as root too.

...this bit.

>> I changed permissions on `/etc/guix/offload/ssh/childhurd' like so
>>
>> $ l /etc/guix/offload/ssh
>> total 8
>> -rw-r--r-- 1 root root   93 Sep 23 14:47 childhurd.pub
>> -rw-r----- 1 root wheel 399 Sep 23 14:47 childhurd
>>
>> to "fix" that. WDYT?
>
> What would be the use case?

Well, running `guix offload test' as "janneke" :)

I didn't "dare" (should I say "proceed"?) to test a real offload before
seeing "guix offload test" work.  That's how I usually go about
offloading.

Reading the manual, especially if you know it, I guess that it does say
so in a subtle way

--8<---------------cut here---------------start------------->8---
   To test whether your setup is operational, run this command on the
master node:

     # guix offload test
--8<---------------cut here---------------end--------------->8---

And sure enough, offloading does work when I revert permessions on the
private key, so yeah...

>> Found another two commit message nitpicks
>
> Noted!
>
> I’d like to see if there are other opinions in particular about
> /etc/guix/machines.scm generation, then I guess we can merge?

Sure!

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #59 received at 66156 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>,
 Florian Pelz <pelzflorian <at> pelzflorian.de>, 66156 <at> debbugs.gnu.org
Subject: Re: bug#66156: [PATCH 00/12] Introducing Smart Hurdloading
Date: Sun, 01 Oct 2023 23:06:19 +0200

Hello,

I pushed these as b9fae146d6cc4a6968a8eb18beef29aa1414a31e.

I forgot to amend the commit logs as you had suggested; apologies!

I also forgot to submit a news entry with the initial patch set, but
then thought it’d be nice to have, so I pushed the following as
b9fae146d6cc4a6968a8eb18beef29aa1414a31e (with French translation):

   (entry (commit "953c65ffdd43c02c934518fb7a1c68542584b223")
          (title
           (en "Declarative offloading on Guix System and childhurds"))
          (body
           (en "When configuring Guix System, it is now possible to declare
  builds machines to offload to directly from the @code{operating-system}
  declaration by specifying the @code{build-machines} field of
  @code{guix-configuration}.  When you do this, @command{guix system} generates
  a @file{/etc/guix/machines.scm} file by concatenating the @code{(build-machine
  @dots{})} expressions you specified.

  This mechanism is used by @code{hurd-vm-service-type}, also known as
  ``childhurd'', to create virtual machines running the GNU/Hurd operating
  system one can offload to, without additional configuration steps.

  Run @command{info \"(guix) Daemon Offload Setup\"} for more info on
  offloading; run @command{info \"(guix) Virtualization Services\"} for info on
  @code{hurd-vm-service-type}.")))

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.