GNU bug report logs - #41785
[PATCH] DRAFT services: Add 'hurd-in-vm service-type'.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41785 in the body.
You can then email your comments to 41785 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
To: guix-patches <at> gnu.org
Subject: [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Wed, 10 Jun 2020 10:54:41 +0200

TODO: Figure-out how to run this hurd VM inside a VM.

Using

--8<---------------cut here---------------start------------->8---
diff --git a/gnu/system/examples/bare-bones.tmpl b/gnu/system/examples/bare-bones.tmpl
index 1035ab1d60..40fb354ea4 100644
--- a/gnu/system/examples/bare-bones.tmpl
+++ b/gnu/system/examples/bare-bones.tmpl
@@ -5,6 +5,8 @@
 (use-service-modules networking ssh)
 (use-package-modules screen ssh)

+(use-service-modules hurd virtualization)
+
 (operating-system
   (host-name "komputilo")
   (timezone "Europe/Berlin")
@@ -44,8 +46,12 @@
   ;; Add services to the baseline: a DHCP client and
   ;; an SSH server.
   (services (append (list (service dhcp-client-service-type)
+                          (service hurd-in-vm-service-type)
                           (service openssh-service-type
                                    (openssh-configuration
                                     (openssh openssh-sans-x)
-                                    (port-number 2222))))
+                                    (port-number 2222)
+                                    (permit-root-login #t)
+                                    (allow-empty-passwords? #t)
+                                    (password-authentication? #t))))
                     %base-services)))
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 441b1eb7e0..e3560b80b7 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -878,7 +878,7 @@ functionality of the kernel Linux.")))
     (define vm-command
       #~(list
          (string-append #$qemu "/bin/qemu-system-i386")
-         #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+         ;;huh? breaks hurd in bare-bones VM #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
          "-m" (number->string #$memory-size)
          #$@options
          #+image))
--8<---------------cut here---------------end--------------->8---

and doing something like

    ./pre-inst-env guix system vm gnu/system/examples/bare-bones.tmpl --no-offload
    /gnu/store/96wh3jwsla4p6d4s547mmqxsi4qbbc0r-run-vm.sh -m 2G \
      --device rtl8139,netdev=net0 \
      --netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:5900

nicely starts a bare-bones VM with the the hurd-in-vm service inside, but I
cannot seem to connect to the Hurd VM it in any way.  Appending
",hostfwd=tcp:127.0.0.1:20022-:20022" (to directly ssh into the Hurd) even
blocks me from ssh'ing into the GNU/linux host VM.

hurd-in-vm works beautifully when added to my system configuration and
reconfiguring.

* gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
hurd-vm-disk-image): New procedures.
(%hurd-in-vm-operating-system, hurd-in-vm-service-type): New variable.
(<hurd-in-vm-configuration>): New record type.
* doc/guix.texi (Virtualization Services): Document it.
---
 doc/guix.texi                   |  61 ++++++++++++++
 gnu/services/virtualization.scm | 140 ++++++++++++++++++++++++++++++--
 2 files changed, 194 insertions(+), 7 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 15e077a41c..cae77288f4 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24583,6 +24583,67 @@ Return true if @var{obj} is a platform object.
 Return the name of @var{platform}---a string such as @code{"arm"}.
 @end deffn
 
+
+@subsubheading The Hurd in a Virtual Machine
+
+@cindex @code{hurd}
+@cindex the Hurd
+
+Service @code{hurd-in-vm} provides support for running a Virtual Machine
+with the GNU <at> tie{}Hurd.
+
+@defvr {Scheme Variable} hurd-in-vm-service-type
+This is the type of the Hurd in a Virtual Machine service.  Its value
+must be a @code{hurd-in-vm-configuration} object, which specifies the
+operating system (@pxref{operating-system Reference}) and the disk size
+for the Hurd Virtual Machine, the QEMU package to use as well as the
+options for running it.
+
+For example:
+
+@lisp
+(service hurd-in-vm-service-type
+         (hurd-in-vm-configuration
+          (disk-size (* 5000 (expt 2 20))) ;5G
+          (memory-size 1024)))             ;1024MiB
+@end lisp
+
+would create a disk image big enough to build GNU <at> tie{}Hello, with some
+extra memory.
+@end defvr
+
+@deftp {Data Type} hurd-in-vm-configuration
+The data type representing the configuration for
+@code{hurd-in-vm-service-type}.
+
+@table @asis
+@item @code{os} (default: @var{%hurd-in-vm-operating-system})
+The operating system to instantiate.  This default is bare-bones with a
+permissive OpenSSH secure shell daemon listening on port 2222
+(@pxref{Networking Services, @code{openssh-service-type}}).
+
+@item @code{qemu} (default: @code{qemu-minimal})
+The QEMU package to use.
+
+@item @code{image} (default: @var{hurd-vm-disk-image})
+The procedure used to build the disk-image built from this
+configuration.
+
+@item @code{disk-size} (default: @code{'guess})
+The size of the disk image.
+
+@item @code{memory-size} (default: @code{512})
+The memory size of the Virtual Machine in mebibytes.
+
+@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
+      @code{"--netdev"} @
+      @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
+      @code{"--snapshot"} @
+      @code{"--hda")})
+The extra options for running QEMU.
+@end table
+@end deftp
+
 @node Version Control Services
 @subsection Version Control Services
 
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 989e439d5d..441b1eb7e0 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Ryan Moe <ryan.moe <at> gmail.com>
 ;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -18,24 +19,41 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services virtualization)
-  #:use-module (gnu services)
-  #:use-module (gnu services configuration)
+  #:use-module (gnu bootloader)
+  #:use-module (gnu bootloader grub)
+  #:use-module (gnu image)
+  #:use-module (gnu packages admin)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu packages virtualization)
   #:use-module (gnu services base)
+  #:use-module (gnu services configuration)
   #:use-module (gnu services dbus)
   #:use-module (gnu services shepherd)
-  #:use-module (gnu system shadow)
+  #:use-module (gnu services ssh)
+  #:use-module (gnu services)
   #:use-module (gnu system file-systems)
-  #:use-module (gnu packages admin)
-  #:use-module (gnu packages virtualization)
-  #:use-module (guix records)
+  #:use-module (gnu system hurd)
+  #:use-module (gnu system image)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system)
+  #:use-module (guix derivations)
   #:use-module (guix gexp)
+  #:use-module (guix monads)
   #:use-module (guix packages)
+  #:use-module (guix records)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
 
-  #:export (libvirt-configuration
+  #:export (%hurd-in-vm-operating-system
+            hurd-in-vm-configuration
+            hurd-in-vm-service-type
+
+            libvirt-configuration
             libvirt-service-type
             virtlog-configuration
             virtlog-service-type
@@ -773,3 +791,111 @@ given QEMU package."
                  "This service supports transparent emulation of binaries
 compiled for other architectures using QEMU and the @code{binfmt_misc}
 functionality of the kernel Linux.")))
+
+
+;;;
+;;; The Hurd in VM service.
+;;;
+
+(define* (disk-image os #:key (image-size 'guess) target)
+  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
+  (with-store store
+    (run-with-store store
+      (let ((file-system-type "ext2"))
+        (mlet* %store-monad
+            ((base-image (find-image file-system-type))
+             (sys        (lower-object
+                          (system-image
+                           (image
+                            (inherit base-image)
+                            (size image-size)
+                            (operating-system os)))))
+             (drvs       (mapm/accumulate-builds lower-object (list sys)))
+             (%          (built-derivations drvs)))
+          (let ((output (derivation->output-path sys)))
+            (return output))))
+      #:target target)))
+
+(define %hurd-in-vm-operating-system
+  (operating-system
+    (inherit %hurd-default-operating-system)
+    (host-name "guixydevel")
+    (timezone "Europe/Amsterdam")
+    (bootloader (bootloader-configuration
+                 (bootloader grub-minimal-bootloader)
+                 (target "/dev/vda")
+                 (timeout 0)))
+    (services (cons*
+               (service openssh-service-type
+                        (openssh-configuration
+                         (openssh openssh-sans-x)
+                         (use-pam? #f)
+                         (port-number 2222)
+                         (permit-root-login #t)
+                         (allow-empty-passwords? #t)
+                         (password-authentication? #t)))
+               %base-services/hurd))))
+
+(define-record-type* <hurd-in-vm-configuration>
+  hurd-in-vm-configuration make-hurd-in-vm-configuration
+  hurd-in-vm-configuration?
+  (os          hurd-in-vm-configuration-os                 ;<operating-system>
+               (default %hurd-in-vm-operating-system))
+  (qemu        hurd-in-vm-configuration-qemu               ;<package>
+               (default qemu-minimal))
+  (image       hurd-in-vm-configuration-image              ;string
+               (thunked)
+               (default (hurd-vm-disk-image this-record)))
+  (disk-size   hurd-in-vm-configuration-disk-size          ;number or 'guess
+               (default 'guess))
+  (memory-size hurd-in-vm-configuration-memory-size        ;number
+               (default 512))
+  (options     hurd-in-vm-configuration-options            ;list of string
+               (default
+                 `("--device" "rtl8139,netdev=net0"
+                   "--netdev" (string-append
+                               "user,id=net0"
+                               ",hostfwd=tcp:127.0.0.1:20022-:2222"
+                               ",hostfwd=tcp:127.0.0.1:25900-:5900")
+                   "--snapshot"
+                   "--hda"))))
+
+(define (hurd-vm-disk-image config)
+  "Return a disk-image for the Hurd according to CONFIG."
+  (let ((os (hurd-in-vm-configuration-os config))
+        (disk-size (hurd-in-vm-configuration-disk-size config))
+        (target (and (not (%current-target-system)) "i586-pc-gnu")))
+    (disk-image os #:target target #:image-size disk-size)))
+
+(define (hurd-in-vm-shepherd-service config)
+  "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
+
+  (let ((image       (hurd-in-vm-configuration-image config))
+        (qemu        (hurd-in-vm-configuration-qemu config))
+        (memory-size (hurd-in-vm-configuration-memory-size config))
+        (options     (hurd-in-vm-configuration-options config)))
+
+    (define vm-command
+      #~(list
+         (string-append #$qemu "/bin/qemu-system-i386")
+         #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+         "-m" (number->string #$memory-size)
+         #$@options
+         #+image))
+
+    (list
+     (shepherd-service
+      (documentation "Run the Hurd in a Virtual Machine.")
+      (provision '(hurd-in-vm))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor #$vm-command))
+      (stop  #~(make-kill-destructor))))))
+
+(define hurd-in-vm-service-type
+  (service-type
+   (name 'hurd-in-vm)
+   (extensions (list (service-extension shepherd-root-service-type
+                                        hurd-in-vm-shepherd-service)))
+   (default-value (hurd-in-vm-configuration))
+   (description
+    "Provide a Virtual Machine running the GNU Hurd.")))
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #8 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Wed, 10 Jun 2020 13:34:01 +0200

Hello Jan,

> TODO: Figure-out how to run this hurd VM inside a VM.

Just to clarify things here, we now have the "disk-image" command that
produces a Hurd bootable disk-image. The "vm-image" command is almost
working, and should produce a very similar result.

The "vm" command that produces a vm-image and spawns a VM isn't working
yet with the Hurd. This command is for now passing directly "kernel" and
"initrd" arguments to QEMU, so a few adaptations will be needed.

Now the goad of hurd-in-vm-service-type is to provide a service that
does almost the same thing as the "vm" command, but as a service, right?

So, I don't get why would we need to run a Hurd VM inside a VM. I've
been struggling a lot with running nested layers of virtualization (for
system generation before the recent patches), and the result is often
too slow to be really usable.

Thanks,

Mathieu

Message #11 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Thu, 11 Jun 2020 21:43:28 +0200

Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

> Just to clarify things here, we now have the "disk-image" command that
> produces a Hurd bootable disk-image. The "vm-image" command is almost
> working, and should produce a very similar result.
>
> The "vm" command that produces a vm-image and spawns a VM isn't working
> yet with the Hurd. This command is for now passing directly "kernel" and
> "initrd" arguments to QEMU, so a few adaptations will be needed.
>
> Now the goad of hurd-in-vm-service-type is to provide a service that
> does almost the same thing as the "vm" command, but as a service, right?

That’s my understanding.

> So, I don't get why would we need to run a Hurd VM inside a VM. I've
> been struggling a lot with running nested layers of virtualization (for
> system generation before the recent patches), and the result is often
> too slow to be really usable.

I guess it’s useful if you want to test a GNU/Linux system that uses the
VM service, as in:

  guix system vm my-gnu+linux.scm

where ‘my-gnu+linux.scm’ contains:

   (services (cons (service hurd-vm-service-type)
                   …))

I suppose it should just work, though probably without ‘-enable-kvm’;
janneke?

Thanks,
Ludo’.

Message #14 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Thu, 11 Jun 2020 21:59:50 +0200

Hey!

That was fast!  :-)

"Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org> skribis:

> and doing something like
>
>     ./pre-inst-env guix system vm gnu/system/examples/bare-bones.tmpl --no-offload
>     /gnu/store/96wh3jwsla4p6d4s547mmqxsi4qbbc0r-run-vm.sh -m 2G \
>       --device rtl8139,netdev=net0 \
>       --netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:5900
>
> nicely starts a bare-bones VM with the the hurd-in-vm service inside, but I
> cannot seem to connect to the Hurd VM it in any way.  Appending
> ",hostfwd=tcp:127.0.0.1:20022-:20022" (to directly ssh into the Hurd) even
> blocks me from ssh'ing into the GNU/linux host VM.

Weird.

> hurd-in-vm works beautifully when added to my system configuration and
> reconfiguring.
>
> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
> hurd-vm-disk-image): New procedures.
> (%hurd-in-vm-operating-system, hurd-in-vm-service-type): New variable.
> (<hurd-in-vm-configuration>): New record type.
> * doc/guix.texi (Virtualization Services): Document it.

[…]

> +@subsubheading The Hurd in a Virtual Machine
> +
> +@cindex @code{hurd}
> +@cindex the Hurd
> +
> +Service @code{hurd-in-vm} provides support for running a Virtual Machine
> +with the GNU <at> tie{}Hurd.

“… support for running GNU/Hurd in a virtual machine (VM).  The virtual
machine is a Shepherd service that can be controlled with commands such
as:

@example
herd stop hurd-vm
@end example

The given GNU/Hurd operating system configuration is cross-compiled.”

Nitpick: I’d call it “hurd-vm”, because it runs a Hurd VM.  :-)

It’s a volatile VM, due to the use of ‘-snapshot’, right?

(The Hurd actually has “sub-Hurds”¹ and “neighborhurds”².  I wonder if
it’s our duty to coin another term… a guesthurd? a visithurd?)

¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
² https://www.gnu.org/software/hurd/hurd/neighborhurd.html

> +(define* (disk-image os #:key (image-size 'guess) target)
> +  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
> +  (with-store store
      ^
In general, procedures should talk to the user-provided store and never
open a new connection.  They should also never call ‘build-derivations’
explicitly, the only exception so far being the graft implementation.

So you can drop ‘with-store’ here, and then:

> +    (run-with-store store
> +      (let ((file-system-type "ext2"))
> +        (mlet* %store-monad
> +            ((base-image (find-image file-system-type))
> +             (sys        (lower-object
> +                          (system-image
> +                           (image
> +                            (inherit base-image)
> +                            (size image-size)
> +                            (operating-system os)))))
> +             (drvs       (mapm/accumulate-builds lower-object (list sys)))
> +             (%          (built-derivations drvs)))
> +          (let ((output (derivation->output-path sys)))
> +            (return output))))

Mathieu, can we make ‘find-image’ non-monadic?  It really shouldn’t be
because it doesn’t interact with the store.  It can take an optional
‘system’ parameter if we want.

So, assuming ‘find-image’ is non-monadic, the code above becomes
something like:

  (system-image
    (image (inherit base-image)
           (size image-size)
           (operating-system
            (with-parameters ((%current-target-system "i586-pc-gnu"))
              os))))

> +(define %hurd-in-vm-operating-system
> +  (operating-system
> +    (inherit %hurd-default-operating-system)
> +    (host-name "guixydevel")
> +    (timezone "Europe/Amsterdam")
> +    (bootloader (bootloader-configuration
> +                 (bootloader grub-minimal-bootloader)
> +                 (target "/dev/vda")
> +                 (timeout 0)))
> +    (services (cons*
> +               (service openssh-service-type
> +                        (openssh-configuration
> +                         (openssh openssh-sans-x)
> +                         (use-pam? #f)
> +                         (port-number 2222)
> +                         (permit-root-login #t)
> +                         (allow-empty-passwords? #t)
> +                         (password-authentication? #t)))
> +               %base-services/hurd))))

I understand the need to factorize useful configs, but IMO it doesn’t
belong here.  So I’d just leave it out.  There’s already
‘%hurd-default-operating-system’ that does the heavy lifting anyway.

> +(define hurd-in-vm-service-type
> +  (service-type
> +   (name 'hurd-in-vm)
> +   (extensions (list (service-extension shepherd-root-service-type
> +                                        hurd-in-vm-shepherd-service)))
> +   (default-value (hurd-in-vm-configuration))
> +   (description
> +    "Provide a Virtual Machine running the GNU Hurd.")))

Being pedantic: s|the GNU Hurd|GNU/Hurd|.  :-)

Otherwise looks great to me, thank you!

Ludo’.

Message #17 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Thu, 11 Jun 2020 21:59:59 +0200

Ludovic Courtès writes:

Hello,

> Mathieu Othacehe <othacehe <at> gnu.org> skribis:
>
>> Just to clarify things here, we now have the "disk-image" command that
>> produces a Hurd bootable disk-image. The "vm-image" command is almost
>> working, and should produce a very similar result.

(yes)

>> The "vm" command that produces a vm-image and spawns a VM isn't working
>> yet with the Hurd. This command is for now passing directly "kernel" and
>> "initrd" arguments to QEMU, so a few adaptations will be needed.
>>
>> Now the goad of hurd-in-vm-service-type is to provide a service that
>> does almost the same thing as the "vm" command, but as a service, right?
>
> That’s my understanding.

Indeed, mine too.

>> So, I don't get why would we need to run a Hurd VM inside a VM. I've
>> been struggling a lot with running nested layers of virtualization (for
>> system generation before the recent patches), and the result is often
>> too slow to be really usable.
>
> I guess it’s useful if you want to test a GNU/Linux system that uses the
> VM service, as in:
>
>   guix system vm my-gnu+linux.scm
>
> where ‘my-gnu+linux.scm’ contains:
>
>    (services (cons (service hurd-vm-service-type)
>                    …))
>
> I suppose it should just work, though probably without ‘-enable-kvm’;
> janneke?

Well -- yes.  That was my thinking.

We would not normally use such a VM-in-VM thing; when you want a Hurd VM
using the "vm" command would be great (or as now, create a disk image
and start QEMU).

However, for writing and debugging (testing?) this Hurd-in-VM service, I
thought adding it to a linux bare-bones.tmpl and using guix vm on that
was a nice and "trivial" way to test it.  That proved so troublesome
that I added it to my system.

As I understand it the intended use for this service is to have a real
simple way to add a Hurd VM to some build machines, so that we can
easily create a herd of Hurd-VMs to aid the Hurd work.  Ludo?

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #20 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>,
 "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Thu, 11 Jun 2020 22:01:25 +0200

[Message part 1 (text/plain, inline)]

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> So, I don't get why would we need to run a Hurd VM inside a VM. I've
> been struggling a lot with running nested layers of virtualization (for
> system generation before the recent patches), and the result is often
> too slow to be really usable.

Note that recent processors support nested layers of virtualization
natively with little overhead, but it's disabled by default.

For an Intel processor, it can be enabled by adding this to your system
configuration:

  (kernel-arguments (cons "kvm_intel.nested=1" %default-kernel-arguments))

The corresponding AMD kernel module is called "kvm_amd".

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Thu, 11 Jun 2020 23:57:06 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès writes:

Hello,

> That was fast!  :-)

Yeah...we need this, right ;)

> "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org> skribis:
>
>> and doing something like
>>
>>     ./pre-inst-env guix system vm gnu/system/examples/bare-bones.tmpl --no-offload
>>     /gnu/store/96wh3jwsla4p6d4s547mmqxsi4qbbc0r-run-vm.sh -m 2G \
>>       --device rtl8139,netdev=net0 \
>>       --netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:5900
>>
>> nicely starts a bare-bones VM with the the hurd-in-vm service inside, but I
>> cannot seem to connect to the Hurd VM it in any way.  Appending
>> ",hostfwd=tcp:127.0.0.1:20022-:20022" (to directly ssh into the Hurd) even
>> blocks me from ssh'ing into the GNU/linux host VM.
>
> Weird.
>
>> hurd-in-vm works beautifully when added to my system configuration and
>> reconfiguring.
>>
>> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
>> hurd-vm-disk-image): New procedures.
>> (%hurd-in-vm-operating-system, hurd-in-vm-service-type): New variable.
>> (<hurd-in-vm-configuration>): New record type.
>> * doc/guix.texi (Virtualization Services): Document it.
>
> […]
>
>> +@subsubheading The Hurd in a Virtual Machine
>> +
>> +@cindex @code{hurd}
>> +@cindex the Hurd
>> +
>> +Service @code{hurd-in-vm} provides support for running a Virtual Machine
>> +with the GNU <at> tie{}Hurd.
>
> “… support for running GNU/Hurd in a virtual machine (VM).  The virtual
> machine is a Shepherd service that can be controlled with commands such
> as:
>
> @example
> herd stop hurd-vm
> @end example
>
> The given GNU/Hurd operating system configuration is cross-compiled.”

Nice, thanks!

> Nitpick: I’d call it “hurd-vm”, because it runs a Hurd VM.  :-)

Done!

> It’s a volatile VM, due to the use of ‘-snapshot’, right?

By default: Yes.  That seemed more ready-to-use.  A stateful VM image
would need to an out-of-store, writable copy.  You can actually do that
and modify the hurd-vm-configuration.

> (The Hurd actually has “sub-Hurds”¹ and “neighborhurds”².  I wonder if
> it’s our duty to coin another term… a guesthurd? a visithurd?)
>
> ¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
> ² https://www.gnu.org/software/hurd/hurd/neighborhurd.html

Oh, that's cool!  Associating along from the neighborhurd pun, what
about a "childhurd" (as a pun on childhood -- only needed while the Hurd
is growing up)?

"herd start childhurd" -- hmm?  In the updated patch, I still have
hurd-vm.  If we do our duty and coin "childhurd", should I just
s/hurd-vm/childhurd/g ?

>> +(define* (disk-image os #:key (image-size 'guess) target)
>> +  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
>> +  (with-store store
>       ^
> In general, procedures should talk to the user-provided store and never
> open a new connection.  They should also never call ‘build-derivations’
> explicitly, the only exception so far being the graft implementation.
>
> So you can drop ‘with-store’ here, and then:
>
>> +    (run-with-store store
>> +      (let ((file-system-type "ext2"))
>> +        (mlet* %store-monad
>> +            ((base-image (find-image file-system-type))
>> +             (sys        (lower-object
>> +                          (system-image
>> +                           (image
>> +                            (inherit base-image)
>> +                            (size image-size)
>> +                            (operating-system os)))))
>> +             (drvs       (mapm/accumulate-builds lower-object (list sys)))
>> +             (%          (built-derivations drvs)))
>> +          (let ((output (derivation->output-path sys)))
>> +            (return output))))
>
> Mathieu, can we make ‘find-image’ non-monadic?  It really shouldn’t be
> because it doesn’t interact with the store.  It can take an optional
> ‘system’ parameter if we want.

It seems that "just works".  I've made that change in a separate patch
(attached).

> So, assuming ‘find-image’ is non-monadic, the code above becomes
> something like:
>
>   (system-image
>     (image (inherit base-image)
>            (size image-size)
>            (operating-system
>             (with-parameters ((%current-target-system "i586-pc-gnu"))
>               os))))

Hmm...I don't think that I understand.  This

--8<---------------cut here---------------start------------->8---
(define* (disk-image os #:key (image-size 'guess) target)
  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
  (let ((base-image (find-image "ext2")))
    (system-image
     (image (inherit base-image)
            (size image-size)
            (operating-system
              (with-parameters ((%current-target-system target))
                os))))))

--8<---------------cut here---------------end--------------->8---

gives

--8<---------------cut here---------------start------------->8---
$ ~/src/guix/master/pre-inst-env guix system build dundal.scm
%default-substitute-urls:("https://ci.guix.gnu.org")
Backtrace:
In ice-9/boot-9.scm:
  1736:10  4 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In unknown file:
           3 (apply-smob/0 #<thunk 7f4ce92e3980>)
In ice-9/boot-9.scm:
    718:2  2 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
In ice-9/eval.scm:
    619:8  1 (_ #(#(#<directory (guile-user) 7f4ce8f05f00>)))
In guix/ui.scm:
  1945:12  0 (run-guix-command _ . _)

guix/ui.scm:1945:12: In procedure run-guix-command:
In procedure operating-system-file-systems: Wrong type argument: #<<parameterized> bindings: ((#<<parameter> 7f4ce7c23740 proc: #<procedure 7f4ce7c28200 at ice-9/boot-9.scm:1299:5 () | (x)>> #<procedure 7f4cd32f83c0 at gnu/services/virtualization.scm:806:14 ()>)) thunk: #<procedure 7f4cd32f8340 at gnu/services/virtualization.scm:806:14 ()>>
--8<---------------cut here---------------end--------------->8---

...I could do with some help here.

>> +(define %hurd-in-vm-operating-system
[..]
>> +  (operating-system
>> +               (service openssh-service-type
>> +                        (openssh-configuration
>> +                         (openssh openssh-sans-x)
[..]
>> +               %base-services/hurd))))
>
> I understand the need to factorize useful configs, but IMO it doesn’t
> belong here.  So I’d just leave it out.  There’s already
> ‘%hurd-default-operating-system’ that does the heavy lifting anyway.

Sure, removed!  Users will most probably want to add an openssh server
using openssh-sans-x; but I guess that's something for a blog post or
cookbook then.

>> +(define hurd-in-vm-service-type
>> +  (service-type
>> +   (name 'hurd-in-vm)
>> +   (extensions (list (service-extension shepherd-root-service-type
>> +                                        hurd-in-vm-shepherd-service)))
>> +   (default-value (hurd-in-vm-configuration))
>> +   (description
>> +    "Provide a Virtual Machine running the GNU Hurd.")))
>
> Being pedantic: s|the GNU Hurd|GNU/Hurd|.  :-)
>
> Otherwise looks great to me, thank you!

Great; thanks...find two new patches attached.

Janneke

[v2-0001-image-Make-find-image-non-monadic.patch (text/x-patch, inline)]

From b01b8d2a46a6a04cb8f09d74c06cbbc82878f070 Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
Date: Thu, 11 Jun 2020 22:52:12 +0200
Subject: [PATCH v2 1/2] image: Make 'find-image' non-monadic.

* gnu/system/image.scm (find-image): Make non-monadic.
* gnu/tests/install.scm (run-install): Update caller.
* guix/scripts/system.scm (perform-action): Likewise.
---
 gnu/system/image.scm    | 21 ++++++++++-----------
 gnu/tests/install.scm   |  5 +++--
 guix/scripts/system.scm |  3 ++-
 3 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index a0e6bf31f1..66a9f6b335 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -551,16 +552,14 @@ image, depending on IMAGE format."
   "Find and return an image that could match the given FILE-SYSTEM-TYPE.  This
 is useful to adapt to interfaces written before the addition of the <image>
 record."
-  (mlet %store-monad ((target (current-target-system)))
-    (mbegin %store-monad
-      (return
-       (match file-system-type
-         ("iso9660" iso9660-image)
-         (_ (cond
-             ((and target
-                   (hurd-triplet? target))
-              hurd-disk-image)
-             (else
-              efi-disk-image))))))))
+  (let ((target (%current-target-system)))
+    (match file-system-type
+      ("iso9660" iso9660-image)
+      (_ (cond
+          ((and target
+                (hurd-triplet? target))
+           hurd-disk-image)
+          (else
+           efi-disk-image))))))
 
 ;;; image.scm ends here
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 6bd8c7d3d2..d18e33179f 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2017, 2019 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;; Copyright © 2020 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -228,8 +229,8 @@ packages defined in installation-os."
   (mlet* %store-monad ((_      (set-grafting #f))
                        (system (current-system))
                        (target (operating-system-derivation target-os))
-                       (base-image (find-image
-                                    installation-disk-image-file-system-type))
+                       (base-image -> (find-image
+                                       installation-disk-image-file-system-type))
 
                        ;; Since the installation system has no network access,
                        ;; we cheat a little bit by adding TARGET to its GC
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 3d7aa77cb7..fc92b9f07b 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -5,6 +5,7 @@
 ;;; Copyright © 2017, 2019 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;; Copyright © 2018 Ricardo Wurmus <rekado <at> elephly.net>
 ;;; Copyright © 2019 Christopher Baines <mail <at> cbaines.net>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -801,7 +802,7 @@ static checks."
       (check-initrd-modules os)))
 
   (mlet* %store-monad
-      ((image     (find-image file-system-type))
+      ((image ->  (find-image file-system-type))
        (sys       (system-derivation-for-action os image action
                                                 #:file-system-type file-system-type
                                                 #:image-size image-size
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[v2-0002-services-Add-hurd-vm-service-type.patch (text/x-patch, inline)]

From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
Date: Wed, 10 Jun 2020 00:10:28 +0200
Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.

* gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
hurd-vm-disk-image): New procedures.
(hurd-in-vm-service-type): New variable.
(<hurd-in-vm-configuration>): New record type.
* doc/guix.texi (Virtualization Services): Document it.
---
 doc/guix.texi                       |  66 +++++++++++++++++
 gnu/services/virtualization.scm     | 110 ++++++++++++++++++++++++++--
 gnu/system/examples/bare-bones.tmpl |   8 +-
 3 files changed, 176 insertions(+), 8 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 15e077a41c..2c924e5313 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24583,6 +24583,72 @@ Return true if @var{obj} is a platform object.
 Return the name of @var{platform}---a string such as @code{"arm"}.
 @end deffn
 
+
+@subsubheading The Hurd in a Virtual Machine
+
+@cindex @code{hurd}
+@cindex the Hurd
+
+Service @code{hurd-vm} provides support for running GNU/Hurd in a
+virtual machine (VM).  The virtual machine is a Shepherd service that
+can be controlled with commands such as:
+
+@example
+herd stop hurd-vm
+@end example
+
+The given GNU/Hurd operating system configuration is cross-compiled.
+
+@defvr {Scheme Variable} hurd-vm-service-type
+This is the type of the Hurd in a Virtual Machine service.  Its value
+must be a @code{hurd-vm-configuration} object, which specifies the
+operating system (@pxref{operating-system Reference}) and the disk size
+for the Hurd Virtual Machine, the QEMU package to use as well as the
+options for running it.
+
+For example:
+
+@lisp
+(service hurd-vm-service-type
+         (hurd-vm-configuration
+          (disk-size (* 5000 (expt 2 20))) ;5G
+          (memory-size 1024)))             ;1024MiB
+@end lisp
+
+would create a disk image big enough to build GNU <at> tie{}Hello, with some
+extra memory.
+@end defvr
+
+@deftp {Data Type} hurd-vm-configuration
+The data type representing the configuration for
+@code{hurd-vm-service-type}.
+
+@table @asis
+@item @code{os} (default: @var{%hurd-default-operating-system})
+The operating system to instantiate.
+
+@item @code{qemu} (default: @code{qemu-minimal})
+The QEMU package to use.
+
+@item @code{image} (default: @var{hurd-vm-disk-image})
+The procedure used to build the disk-image built from this
+configuration.
+
+@item @code{disk-size} (default: @code{'guess})
+The size of the disk image.
+
+@item @code{memory-size} (default: @code{512})
+The memory size of the Virtual Machine in mebibytes.
+
+@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
+      @code{"--netdev"} @
+      @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
+      @code{"--snapshot"} @
+      @code{"--hda")})
+The extra options for running QEMU.
+@end table
+@end deftp
+
 @node Version Control Services
 @subsection Version Control Services
 
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 989e439d5d..f2a5e7200e 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Ryan Moe <ryan.moe <at> gmail.com>
 ;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -18,24 +19,40 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services virtualization)
-  #:use-module (gnu services)
-  #:use-module (gnu services configuration)
+  #:use-module (gnu bootloader)
+  #:use-module (gnu bootloader grub)
+  #:use-module (gnu image)
+  #:use-module (gnu packages admin)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu packages virtualization)
   #:use-module (gnu services base)
+  #:use-module (gnu services configuration)
   #:use-module (gnu services dbus)
   #:use-module (gnu services shepherd)
-  #:use-module (gnu system shadow)
+  #:use-module (gnu services ssh)
+  #:use-module (gnu services)
   #:use-module (gnu system file-systems)
-  #:use-module (gnu packages admin)
-  #:use-module (gnu packages virtualization)
-  #:use-module (guix records)
+  #:use-module (gnu system hurd)
+  #:use-module (gnu system image)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system)
+  #:use-module (guix derivations)
   #:use-module (guix gexp)
+  #:use-module (guix monads)
   #:use-module (guix packages)
+  #:use-module (guix records)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
 
-  #:export (libvirt-configuration
+  #:export (hurd-vm-configuration
+            hurd-vm-service-type
+
+            libvirt-configuration
             libvirt-service-type
             virtlog-configuration
             virtlog-service-type
@@ -773,3 +790,82 @@ given QEMU package."
                  "This service supports transparent emulation of binaries
 compiled for other architectures using QEMU and the @code{binfmt_misc}
 functionality of the kernel Linux.")))
+
+
+;;;
+;;; The Hurd in VM service.
+;;;
+
+(define* (disk-image os #:key (image-size 'guess) target)
+  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
+  (let ((base-image (find-image "ext2")))
+    (system-image
+     (image (inherit base-image)
+            (size image-size)
+            (operating-system
+              (with-parameters ((%current-target-system target))
+                os))))))
+
+(define-record-type* <hurd-vm-configuration>
+  hurd-vm-configuration make-hurd-vm-configuration
+  hurd-vm-configuration?
+  (os          hurd-vm-configuration-os                 ;<operating-system>
+               (default %hurd-default-operating-system))
+  (qemu        hurd-vm-configuration-qemu               ;<package>
+               (default qemu-minimal))
+  (image       hurd-vm-configuration-image              ;string
+               (thunked)
+               (default (hurd-vm-disk-image this-record)))
+  (disk-size   hurd-vm-configuration-disk-size          ;number or 'guess
+               (default 'guess))
+  (memory-size hurd-vm-configuration-memory-size        ;number
+               (default 512))
+  (options     hurd-vm-configuration-options            ;list of string
+               (default
+                 `("--device" "rtl8139,netdev=net0"
+                   "--netdev" (string-append
+                               "user,id=net0"
+                               ",hostfwd=tcp:127.0.0.1:20022-:2222"
+                               ",hostfwd=tcp:127.0.0.1:25900-:5900")
+                   "--snapshot"
+                   "--hda"))))
+
+(define (hurd-vm-disk-image config)
+  "Return a disk-image for the Hurd according to CONFIG."
+  (let ((os (hurd-vm-configuration-os config))
+        (disk-size (hurd-vm-configuration-disk-size config))
+        (target (and (not (%current-target-system)) "i586-pc-gnu")))
+    (disk-image os #:target target #:image-size disk-size)))
+
+(define (hurd-vm-shepherd-service config)
+  "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
+
+  (let ((image       (hurd-vm-configuration-image config))
+        (qemu        (hurd-vm-configuration-qemu config))
+        (memory-size (hurd-vm-configuration-memory-size config))
+        (options     (hurd-vm-configuration-options config)))
+
+    (define vm-command
+      #~(list
+         (string-append #$qemu "/bin/qemu-system-i386")
+         #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+         "-m" (number->string #$memory-size)
+         #$@options
+         #+image))
+
+    (list
+     (shepherd-service
+      (documentation "Run the Hurd in a Virtual Machine.")
+      (provision '(hurd-vm))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor #$vm-command))
+      (stop  #~(make-kill-destructor))))))
+
+(define hurd-vm-service-type
+  (service-type
+   (name 'hurd-vm)
+   (extensions (list (service-extension shepherd-root-service-type
+                                        hurd-vm-shepherd-service)))
+   (default-value (hurd-vm-configuration))
+   (description
+    "Provide a Virtual Machine running the GNU/Hurd.")))
diff --git a/gnu/system/examples/bare-bones.tmpl b/gnu/system/examples/bare-bones.tmpl
index 1035ab1d60..1d4f7743ab 100644
--- a/gnu/system/examples/bare-bones.tmpl
+++ b/gnu/system/examples/bare-bones.tmpl
@@ -5,6 +5,8 @@
 (use-service-modules networking ssh)
 (use-package-modules screen ssh)
 
+(use-service-modules hurd virtualization)
+
 (operating-system
   (host-name "komputilo")
   (timezone "Europe/Berlin")
@@ -44,8 +46,12 @@
   ;; Add services to the baseline: a DHCP client and
   ;; an SSH server.
   (services (append (list (service dhcp-client-service-type)
+                          (service hurd-vm-service-type)
                           (service openssh-service-type
                                    (openssh-configuration
                                     (openssh openssh-sans-x)
-                                    (port-number 2222))))
+                                    (port-number 2222)
+                                    (permit-root-login #t)
+                                    (allow-empty-passwords? #t)
+                                    (password-authentication? #t))))
                     %base-services)))
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[Message part 4 (text/plain, inline)]

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #26 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Marius Bakke <marius <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 08:39:41 +0200

Marius Bakke writes:

Hello,

> Mathieu Othacehe <othacehe <at> gnu.org> writes:
>
>> So, I don't get why would we need to run a Hurd VM inside a VM. I've
>> been struggling a lot with running nested layers of virtualization (for
>> system generation before the recent patches), and the result is often
>> too slow to be really usable.
>
> Note that recent processors support nested layers of virtualization
> natively with little overhead, but it's disabled by default.

Ah!

> For an Intel processor, it can be enabled by adding this to your system
> configuration:
>
>   (kernel-arguments (cons "kvm_intel.nested=1" %default-kernel-arguments))

Is there an obvious downside to enabling this?

Great...So on the host I did

--8<---------------cut here---------------start------------->8---
root <at> dundal ~# rmmod kvm_intel
root <at> dundal ~# modprobe kvm_intel kvm_intel.nested=1
root <at> dundal ~# cat /sys/module/kvm_intel/parameters/nested
Y
--8<---------------cut here---------------end--------------->8---

and the interwebs told me that to start the VM, you have to add "-cpu
host"; so I started it using

--8<---------------cut here---------------start------------->8---
/gnu/store/k2b7nx34cwyi6yk49wgy4hg9mrwcmll5-run-vm.sh -cpu host -m 2G -device rtl8139,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:25900-:25900
--8<---------------cut here---------------end--------------->8---

and trying to "ssh -p 20022 localhost" from inside the bare-bones VM now
prints

--8<---------------cut here---------------start------------->8---
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
key_exchange_identification: read: Connection reset by peer
Connection reset by 127.0.0.1 port 20022
--8<---------------cut here---------------end--------------->8---

...something networky with QEMU.  Ideas?

Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #29 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 08:46:58 +0200

Jan Nieuwenhuizen writes:

> Ludovic Courtès writes:
>
> Hello,
>
>> That was fast!  :-)
>
> Yeah...we need this, right ;)

Sometimes...too fast :-(

So...there's still the run-with-store/with-parameters puzzle; but just
wanted to say that

>>From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
> Date: Wed, 10 Jun 2020 00:10:28 +0200
> Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.

[...]

>  gnu/system/examples/bare-bones.tmpl |   8 +-

Oops...


> diff --git a/gnu/system/examples/bare-bones.tmpl b/gnu/system/examples/bare-bones.tmpl
> index 1035ab1d60..1d4f7743ab 100644
> --- a/gnu/system/examples/bare-bones.tmpl
> +++ b/gnu/system/examples/bare-bones.tmpl

...without this bits, of course!!

Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #32 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Diego Nicola Barbato <dnbarbato <at> posteo.de>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org,
 Marius Bakke <marius <at> gnu.org>
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 12:51:29 +0200

Hey,

Jan Nieuwenhuizen <janneke <at> gnu.org> writes:

[...]

> and the interwebs told me that to start the VM, you have to add "-cpu
> host"; so I started it using
>
> /gnu/store/k2b7nx34cwyi6yk49wgy4hg9mrwcmll5-run-vm.sh -cpu host -m 2G -device rtl8139,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:25900-:25900
>
>
> and trying to "ssh -p 20022 localhost" from inside the bare-bones VM now
> prints
>
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> key_exchange_identification: read: Connection reset by peer
> Connection reset by 127.0.0.1 port 20022
>
> ...something networky with QEMU.  Ideas?

I've recently had intermittent issues with 'guix system vm' and hostfwd.
Does it help if you remove the '-nic user,model=virtio-net-pci' option
from (a copy of) the run-vm.sh script so that there's only one netdev?

Regards,

Diego

Message #35 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 16:42:55 +0200

Hello Ludo & janneke,

> Mathieu, can we make ‘find-image’ non-monadic?  It really shouldn’t be
> because it doesn’t interact with the store.  It can take an optional
> ‘system’ parameter if we want.

Yes, you're right, passing 'target' to 'find-image' should be enough to
make it non-monadic.

> So, assuming ‘find-image’ is non-monadic, the code above becomes
> something like:
>
>   (system-image
>     (image (inherit base-image)
>            (size image-size)
>            (operating-system
>             (with-parameters ((%current-target-system "i586-pc-gnu"))
>               os))))

I would prefer 'target' to be part of the image itself, as I proposed
here: https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.

There's no way for now, that the image is built without cross-compiling
for "i586-pc-gnu", so I think it could be part of the "image" record
itself.

WDYT?

>> +(define hurd-in-vm-service-type
>> +  (service-type
>> +   (name 'hurd-in-vm)
>> +   (extensions (list (service-extension shepherd-root-service-type
>> +                                        hurd-in-vm-shepherd-service)))
>> +   (default-value (hurd-in-vm-configuration))
>> +   (description
>> +    "Provide a Virtual Machine running the GNU Hurd.")))
>
> Being pedantic: s|the GNU Hurd|GNU/Hurd|.  :-)
>
> Otherwise looks great to me, thank you!

Looks really nice to me too :)

Thanks,

Mathieu

Message #38 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 16:45:04 +0200

Hi!

Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:

>> It’s a volatile VM, due to the use of ‘-snapshot’, right?
>
> By default: Yes.  That seemed more ready-to-use.  A stateful VM image
> would need to an out-of-store, writable copy.  You can actually do that
> and modify the hurd-vm-configuration.

It’s maybe worth mentioning in the manual.

>> (The Hurd actually has “sub-Hurds”¹ and “neighborhurds”².  I wonder if
>> it’s our duty to coin another term… a guesthurd? a visithurd?)
>>
>> ¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
>> ² https://www.gnu.org/software/hurd/hurd/neighborhurd.html
>
> Oh, that's cool!  Associating along from the neighborhurd pun, what
> about a "childhurd" (as a pun on childhood -- only needed while the Hurd
> is growing up)?

“Childhurd”, LOVE IT!

> "herd start childhurd" -- hmm?  In the updated patch, I still have
> hurd-vm.  If we do our duty and coin "childhurd", should I just
> s/hurd-vm/childhurd/g ?

Shepherd services can have more than one name, so I propose to have both!

>> So, assuming ‘find-image’ is non-monadic, the code above becomes
>> something like:
>>
>>   (system-image
>>     (image (inherit base-image)
>>            (size image-size)
>>            (operating-system
>>             (with-parameters ((%current-target-system "i586-pc-gnu"))
>>               os))))
>
> Hmm...I don't think that I understand.  This
>
> (define* (disk-image os #:key (image-size 'guess) target)
>   "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
>   (let ((base-image (find-image "ext2")))
>     (system-image
>      (image (inherit base-image)
>             (size image-size)
>             (operating-system
>               (with-parameters ((%current-target-system target))
>                 os))))))
>
>
> gives
>
> $ ~/src/guix/master/pre-inst-env guix system build dundal.scm
> %default-substitute-urls:("https://ci.guix.gnu.org")
> Backtrace:
> In ice-9/boot-9.scm:
>   1736:10  4 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
> In unknown file:
>            3 (apply-smob/0 #<thunk 7f4ce92e3980>)
> In ice-9/boot-9.scm:
>     718:2  2 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
> In ice-9/eval.scm:
>     619:8  1 (_ #(#(#<directory (guile-user) 7f4ce8f05f00>)))
> In guix/ui.scm:
>   1945:12  0 (run-guix-command _ . _)
>
> guix/ui.scm:1945:12: In procedure run-guix-command:
> In procedure operating-system-file-systems: Wrong type argument: #<<parameterized> bindings: ((#<<parameter> 7f4ce7c23740 proc: #<procedure 7f4ce7c28200 at ice-9/boot-9.scm:1299:5 () | (x)>> #<procedure 7f4cd32f83c0 at gnu/services/virtualization.scm:806:14 ()>)) thunk: #<procedure 7f4cd32f8340 at gnu/services/virtualization.scm:806:14 ()>>
>
> ...I could do with some help here.

Ooh, sadness.  We can’t do that because the image machinery really
expects an <operating-system>.

What if you move ‘with-parameters’ around (system-image …) instead?

>>> +(define %hurd-in-vm-operating-system
> [..]
>>> +  (operating-system
>>> +               (service openssh-service-type
>>> +                        (openssh-configuration
>>> +                         (openssh openssh-sans-x)
> [..]
>>> +               %base-services/hurd))))
>>
>> I understand the need to factorize useful configs, but IMO it doesn’t
>> belong here.  So I’d just leave it out.  There’s already
>> ‘%hurd-default-operating-system’ that does the heavy lifting anyway.
>
> Sure, removed!  Users will most probably want to add an openssh server
> using openssh-sans-x; but I guess that's something for a blog post or
> cookbook then.

Yeah.

Hmm, come to think about it, we need a default value here anyway, so
after all, we have a good reason to have it here.  (Says the guy who
changes his mind.)

> From b01b8d2a46a6a04cb8f09d74c06cbbc82878f070 Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
> Date: Thu, 11 Jun 2020 22:52:12 +0200
> Subject: [PATCH v2 1/2] image: Make 'find-image' non-monadic.
>
> * gnu/system/image.scm (find-image): Make non-monadic.
> * gnu/tests/install.scm (run-install): Update caller.
> * guix/scripts/system.scm (perform-action): Likewise.

[...]

>    "Find and return an image that could match the given FILE-SYSTEM-TYPE.  This
>  is useful to adapt to interfaces written before the addition of the <image>
>  record."
> -  (mlet %store-monad ((target (current-target-system)))
> -    (mbegin %store-monad
> -      (return
> -       (match file-system-type
> -         ("iso9660" iso9660-image)
> -         (_ (cond
> -             ((and target
> -                   (hurd-triplet? target))
> -              hurd-disk-image)
> -             (else
> -              efi-disk-image))))))))
> +  (let ((target (%current-target-system)))
> +    (match file-system-type
> +      ("iso9660" iso9660-image)
> +      (_ (cond
> +          ((and target
> +                (hurd-triplet? target))
> +           hurd-disk-image)
> +          (else
> +           efi-disk-image))))))

I’d prefer:

  (define* (find-image #:optional (system (%current-system)))
    …)

In your case, you’d need to make this call:

  (find-image "i586-gnu")

(Beware of the triplet/system type distinction!)

Perhaps the other call sites need to be adjusted.

> From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
> Date: Wed, 10 Jun 2020 00:10:28 +0200
> Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.
>
> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
> hurd-vm-disk-image): New procedures.
> (hurd-in-vm-service-type): New variable.
> (<hurd-in-vm-configuration>): New record type.
> * doc/guix.texi (Virtualization Services): Document it.

s/hurd-in-vm/hurd-vm/ in the commit log.

Otherwise LGTM, thank you!

Ludo’.

Message #41 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 17:04:58 +0200

Hey janneke,

> +  (let ((target (%current-target-system)))
> +    (match file-system-type
> +      ("iso9660" iso9660-image)
> +      (_ (cond
> +          ((and target
> +                (hurd-triplet? target))
> +           hurd-disk-image)
> +          (else
> +           efi-disk-image))))))

I think it would be safe to pass a "target" argument. Then the two
actual callers could pass (current-target-system) as target
argument. This is guaranteed to return a correct value, whereas
%current-target-system is not here.

> +@lisp
> +(service hurd-vm-service-type
> +         (hurd-vm-configuration
> +          (disk-size (* 5000 (expt 2 20))) ;5G
> +          (memory-size 1024)))             ;1024MiB

That's really nice! We could really use a (guix units) module or so
where we would put those definitions: "(define MiB (expt 2 20))". Then
we could use:

--8<---------------cut here---------------start------------->8---
(service hurd-vm-service-type
         (hurd-vm-configuration
          (disk-size (* 5 GiB)
          (memory-size (* 1024 MiB)))
--8<---------------cut here---------------end--------------->8---

Well, this is really not a blocking thing.

> +(define* (disk-image os #:key (image-size 'guess) target)
> +  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
> +  (let ((base-image (find-image "ext2")))
> +    (system-image
> +     (image (inherit base-image)
> +            (size image-size)
> +            (operating-system
> +              (with-parameters ((%current-target-system target))
> +                os))))))

Yeah then again, I think it would be nice to use something like the
"hurd-disk-image" as I proposed here:
https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.

This way, no need to call "find-image". You could just write:

--8<---------------cut here---------------start------------->8---
(define (image os #:key (image-size 'guess))
  (image
   (inherit hurd-disk-image)
   (size image-size)
   (operating-system os)))
--8<---------------cut here---------------end--------------->8---

> +  (image       hurd-vm-configuration-image              ;string
> +               (thunked)

Then the thunked field wouldn't be required.

Mostly comments related to the fact that (gnu image) needs some
polishing, your patches are really nice here :)

Thanks,

Mathieu

Message #44 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 17:39:20 +0200

Hi Mathieu!

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>> So, assuming ‘find-image’ is non-monadic, the code above becomes
>> something like:
>>
>>   (system-image
>>     (image (inherit base-image)
>>            (size image-size)
>>            (operating-system
>>             (with-parameters ((%current-target-system "i586-pc-gnu"))
>>               os))))
>
> I would prefer 'target' to be part of the image itself, as I proposed
> here: https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.
>
> There's no way for now, that the image is built without cross-compiling
> for "i586-pc-gnu", so I think it could be part of the "image" record
> itself.
>
> WDYT?

Yes, why not, a ‘target’ field in <image> sounds fine.

In general, I think lowerable objects should not be parameterized by
target/system.  However, <image> is an exception because it needs to
access the operating system record, and still use the desired target, if
any.

Thanks,
Ludo’.

Message #47 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 23:33:38 +0200

Mathieu Othacehe writes:

Hello Mathieu,

>> +  (let ((target (%current-target-system)))
>> +    (match file-system-type
>> +      ("iso9660" iso9660-image)
>> +      (_ (cond
>> +          ((and target
>> +                (hurd-triplet? target))
>> +           hurd-disk-image)
>> +          (else
>> +           efi-disk-image))))))
>
> I think it would be safe to pass a "target" argument. Then the two
> actual callers could pass (current-target-system) as target
> argument. This is guaranteed to return a correct value, whereas
> %current-target-system is not here.

Okay, sure.  I've made that change for now.

>> +@lisp
>> +(service hurd-vm-service-type
>> +         (hurd-vm-configuration
>> +          (disk-size (* 5000 (expt 2 20))) ;5G
>> +          (memory-size 1024)))             ;1024MiB
>
> That's really nice! We could really use a (guix units) module or so
> where we would put those definitions: "(define MiB (expt 2 20))". Then
> we could use:
>
> (service hurd-vm-service-type
>          (hurd-vm-configuration
>           (disk-size (* 5 GiB)
>           (memory-size (* 1024 MiB)))

I like it!

> Well, this is really not a blocking thing.

>> +(define* (disk-image os #:key (image-size 'guess) target)
>> +  "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
>> +  (let ((base-image (find-image "ext2")))
>> +    (system-image
>> +     (image (inherit base-image)
>> +            (size image-size)
>> +            (operating-system
>> +              (with-parameters ((%current-target-system target))
>> +                os))))))
>
> Yeah then again, I think it would be nice to use something like the
> "hurd-disk-image" as I proposed here:
> https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.
>
> This way, no need to call "find-image". You could just write:
>
> (define (image os #:key (image-size 'guess))
>   (image
>    (inherit hurd-disk-image)
>    (size image-size)
>    (operating-system os)))

Oh yes...that's great.  It seems that until we have that, as Ludo
suggested with this:

--8<---------------cut here---------------start------------->8---
    (with-parameters ((%current-target-system target))
      (system-image
       (image (inherit base-image)
              (size disk-size)
              (operating-system os))))))
--8<---------------cut here---------------end--------------->8---

we have now something "that works" for the Hurd.

>> +  (image       hurd-vm-configuration-image              ;string
>> +               (thunked)
>
> Then the thunked field wouldn't be required.
>
> Mostly comments related to the fact that (gnu image) needs some
> polishing, your patches are really nice here :)

Thank you.  I'll be working to prepare a v3 series that works on master
right now; then we can see how to proceed.

Thanks!
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #50 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Fri, 12 Jun 2020 23:33:51 +0200

Ludovic Courtès writes:

Hello,

> Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>>> It’s a volatile VM, due to the use of ‘-snapshot’, right?
>>
>> By default: Yes.  That seemed more ready-to-use.  A stateful VM image
>> would need to an out-of-store, writable copy.  You can actually do that
>> and modify the hurd-vm-configuration.
>
> It’s maybe worth mentioning in the manual.

Ah right, I wanted to do that after your remark...but forgot.  Could be
an unpleasant surprise to lose everything.  I've now added

--8<---------------cut here---------------start------------->8---
Note that by default the VM image is volatile, i.e., once stopped the
contents are lost.  If you want a stateful image instead, override the
configuration's @code{image} and @code{options} without
the @code{--snapshot} flag using something along these lines:

@lisp
(service hurd-vm-service-type
         (hurd-vm-configuration
          (image    (const "/out/of/store/writable/hurd.img"))
          (options '("--device" "rtl8139,netdev=net0"
                     "--netdev"
                     "user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222"))))
@end lisp
--8<---------------cut here---------------end--------------->8---

to the patch.

>>> (The Hurd actually has “sub-Hurds”¹ and “neighborhurds”².  I wonder if
>>> it’s our duty to coin another term… a guesthurd? a visithurd?)
>>>
>>> ¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
>>> ² https://www.gnu.org/software/hurd/hurd/neighborhurd.html
>>
>> Oh, that's cool!  Associating along from the neighborhurd pun, what
>> about a "childhurd" (as a pun on childhood -- only needed while the Hurd
>> is growing up)?
>
> “Childhurd”, LOVE IT!

Heh; ...

>> "herd start childhurd" -- hmm?  In the updated patch, I still have
>> hurd-vm.  If we do our duty and coin "childhurd", should I just
>> s/hurd-vm/childhurd/g ?
>
> Shepherd services can have more than one name, so I propose to have both!

... great, now lemme see where to mention this.  I just put in in the
code

--8<---------------cut here---------------start------------->8---
    (list
     (shepherd-service
      (documentation "Run the Hurd in a Virtual Machine: a Childhurd.")
      (provision '(hurd-vm childhurd))
--8<---------------cut here---------------end--------------->8---

and docs here

--8<---------------cut here---------------start------------->8---
@example
herd start hurd-vm
herd stop childhurd
@end example
--8<---------------cut here---------------end--------------->8---

> Ooh, sadness.  We can’t do that because the image machinery really
> expects an <operating-system>.
>
> What if you move ‘with-parameters’ around (system-image …) instead?

=> \o/ that works!  You're genius ;-)

>>>> +(define %hurd-in-vm-operating-system
>> [..]
>>>> +  (operating-system
>>>> +               (service openssh-service-type
>>>> +                        (openssh-configuration
>>>> +                         (openssh openssh-sans-x)
>> [..]
>>>> +               %base-services/hurd))))
>>>
>>> I understand the need to factorize useful configs, but IMO it doesn’t
>>> belong here.  So I’d just leave it out.  There’s already
>>> ‘%hurd-default-operating-system’ that does the heavy lifting anyway.
>>
>> Sure, removed!  Users will most probably want to add an openssh server
>> using openssh-sans-x; but I guess that's something for a blog post or
>> cookbook then.
>
> Yeah.
>
> Hmm, come to think about it, we need a default value here anyway, so
> after all, we have a good reason to have it here.  (Says the guy who
> changes his mind.)

Heh, it's such a pleasure working with you.  I've put it back again,
without the -in- bit.  (says the guy who was easily convinced we didn't
need it.)

>> From b01b8d2a46a6a04cb8f09d74c06cbbc82878f070 Mon Sep 17 00:00:00 2001
>> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
>> Date: Thu, 11 Jun 2020 22:52:12 +0200
>> Subject: [PATCH v2 1/2] image: Make 'find-image' non-monadic.
>>
>> * gnu/system/image.scm (find-image): Make non-monadic.
>> * gnu/tests/install.scm (run-install): Update caller.
>> * guix/scripts/system.scm (perform-action): Likewise.
>
> [...]
>
>>    "Find and return an image that could match the given FILE-SYSTEM-TYPE.  This
>>  is useful to adapt to interfaces written before the addition of the <image>
>>  record."
>> -  (mlet %store-monad ((target (current-target-system)))
>> -    (mbegin %store-monad
>> -      (return
>> -       (match file-system-type
>> -         ("iso9660" iso9660-image)
>> -         (_ (cond
>> -             ((and target
>> -                   (hurd-triplet? target))
>> -              hurd-disk-image)
>> -             (else
>> -              efi-disk-image))))))))
>> +  (let ((target (%current-target-system)))
>> +    (match file-system-type
>> +      ("iso9660" iso9660-image)
>> +      (_ (cond
>> +          ((and target
>> +                (hurd-triplet? target))
>> +           hurd-disk-image)
>> +          (else
>> +           efi-disk-image))))))
>
> I’d prefer:
>
>   (define* (find-image #:optional (system (%current-system)))
>     …)
>
> In your case, you’d need to make this call:
>
>   (find-image "i586-gnu")
>
> (Beware of the triplet/system type distinction!)
>
> Perhaps the other call sites need to be adjusted.

Then Mathieu writes

> > I would prefer 'target' to be part of the image itself, as I proposed
> > here: https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.
> >
> > There's no way for now, that the image is built without cross-compiling
> > for "i586-pc-gnu", so I think it could be part of the "image" record
> > itself.
> >
> > WDYT?

and Ludo writes

> Yes, why not, a ‘target’ field in <image> sounds fine.

So...I'm sticking with a the 'target' parameter for now; as that was
Mathieu's suggestion and a closer match to the final solution of
moving TARGET into image.

>> From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
>> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
>> Date: Wed, 10 Jun 2020 00:10:28 +0200
>> Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.
>>
>> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
>> hurd-vm-disk-image): New procedures.
>> (hurd-in-vm-service-type): New variable.
>> (<hurd-in-vm-configuration>): New record type.
>> * doc/guix.texi (Virtualization Services): Document it.
>
> s/hurd-in-vm/hurd-vm/ in the commit log.
>
> Otherwise LGTM, thank you!

Thanks!  I'll be sending a v3 series that should be OK (apart from the
image work that's not finished yet), and we can decide what to do.

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #53 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
To: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>,
 Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Fri, 12 Jun 2020 23:42:13 +0200

* gnu/system/image.scm (find-image): Make non-monadic.  Add 'target'
parameter.
* gnu/tests/install.scm (run-install): Update caller,
passing (%current-target-system).
* guix/scripts/system.scm (perform-action): Likewise.
---
 gnu/system/image.scm    | 22 ++++++++++------------
 gnu/tests/install.scm   |  6 ++++--
 guix/scripts/system.scm |  3 ++-
 3 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index a0e6bf31f1..ac55301a60 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -547,20 +548,17 @@ image, depending on IMAGE format."
                              #:grub-mkrescue-environment
                              '(("MKRESCUE_SED_MODE" . "mbr_hfs")))))))
 
-(define (find-image file-system-type)
+(define (find-image file-system-type target)
   "Find and return an image that could match the given FILE-SYSTEM-TYPE.  This
 is useful to adapt to interfaces written before the addition of the <image>
 record."
-  (mlet %store-monad ((target (current-target-system)))
-    (mbegin %store-monad
-      (return
-       (match file-system-type
-         ("iso9660" iso9660-image)
-         (_ (cond
-             ((and target
-                   (hurd-triplet? target))
-              hurd-disk-image)
-             (else
-              efi-disk-image))))))))
+  (match file-system-type
+    ("iso9660" iso9660-image)
+    (_ (cond
+        ((and target
+              (hurd-triplet? target))
+         hurd-disk-image)
+        (else
+         efi-disk-image)))))
 
 ;;; image.scm ends here
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 6bd8c7d3d2..82098290f7 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2017, 2019 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;; Copyright © 2020 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -228,8 +229,9 @@ packages defined in installation-os."
   (mlet* %store-monad ((_      (set-grafting #f))
                        (system (current-system))
                        (target (operating-system-derivation target-os))
-                       (base-image (find-image
-                                    installation-disk-image-file-system-type))
+                       (base-image -> (find-image
+                                       installation-disk-image-file-system-type
+                                       (%current-target-system)))
 
                        ;; Since the installation system has no network access,
                        ;; we cheat a little bit by adding TARGET to its GC
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 3d7aa77cb7..6f3880cc34 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -5,6 +5,7 @@
 ;;; Copyright © 2017, 2019 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;; Copyright © 2018 Ricardo Wurmus <rekado <at> elephly.net>
 ;;; Copyright © 2019 Christopher Baines <mail <at> cbaines.net>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -801,7 +802,7 @@ static checks."
       (check-initrd-modules os)))
 
   (mlet* %store-monad
-      ((image     (find-image file-system-type))
+      ((image ->  (find-image file-system-type (%current-target-system)))
        (sys       (system-derivation-for-action os image action
                                                 #:file-system-type file-system-type
                                                 #:image-size image-size
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #56 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
To: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>,
 Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
Date: Fri, 12 Jun 2020 23:42:14 +0200

* gnu/services/virtualization.scm (hurd-vm-shepherd-service,
hurd-vm-disk-image): New procedures.
(%hurd-vm-operating-system, hurd-vm-service-type): New variable.
(<hurd-vm-configuration>): New record type.
* doc/guix.texi (Virtualization Services): Document it.
---
 doc/guix.texi                   |  83 +++++++++++++++++++++
 gnu/services/virtualization.scm | 126 ++++++++++++++++++++++++++++++--
 2 files changed, 202 insertions(+), 7 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 15e077a41c..ed36d5014b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24583,6 +24583,89 @@ Return true if @var{obj} is a platform object.
 Return the name of @var{platform}---a string such as @code{"arm"}.
 @end deffn
 
+
+@subsubheading The Hurd in a Virtual Machine
+
+@cindex @code{hurd}
+@cindex the Hurd
+
+Service @code{hurd-vm} provides support for running GNU/Hurd in a
+virtual machine (VM), a so-called ``Childhurd''.  The virtual machine is
+a Shepherd service that can be controlled with commands such as:
+
+@example
+herd start hurd-vm
+herd stop childhurd
+@end example
+
+The given GNU/Hurd operating system configuration is cross-compiled.
+
+@defvr {Scheme Variable} hurd-vm-service-type
+This is the type of the Hurd in a Virtual Machine service.  Its value
+must be a @code{hurd-vm-configuration} object, which specifies the
+operating system (@pxref{operating-system Reference}) and the disk size
+for the Hurd Virtual Machine, the QEMU package to use as well as the
+options for running it.
+
+For example:
+
+@lisp
+(service hurd-vm-service-type
+         (hurd-vm-configuration
+          (disk-size (* 5000 (expt 2 20))) ;5G
+          (memory-size 1024)))             ;1024MiB
+@end lisp
+
+would create a disk image big enough to build GNU <at> tie{}Hello, with some
+extra memory.
+@end defvr
+
+@deftp {Data Type} hurd-vm-configuration
+The data type representing the configuration for
+@code{hurd-vm-service-type}.
+
+@table @asis
+@item @code{os} (default: @var{%hurd-vm-operating-system})
+The operating system to instantiate.  This default is bare-bones with a
+permissive OpenSSH secure shell daemon listening on port 2222
+(@pxref{Networking Services, @code{openssh-service-type}}).
+
+@item @code{qemu} (default: @code{qemu-minimal})
+The QEMU package to use.
+
+@item @code{image} (default: @var{hurd-vm-disk-image})
+The procedure used to build the disk-image built from this
+configuration.
+
+@item @code{disk-size} (default: @code{'guess})
+The size of the disk image.
+
+@item @code{memory-size} (default: @code{512})
+The memory size of the Virtual Machine in mebibytes.
+
+@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
+      @code{"--netdev"} @
+      @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
+      @code{"--snapshot"} @
+      @code{"--hda")})
+The extra options for running QEMU.
+@end table
+@end deftp
+
+Note that by default the VM image is volatile, i.e., once stopped the
+contents are lost.  If you want a stateful image instead, override the
+configuration's @code{image} and @code{options} without
+the @code{--snapshot} flag using something along these lines:
+
+@lisp
+(service hurd-vm-service-type
+         (hurd-vm-configuration
+          (image    (const "/out/of/store/writable/hurd.img"))
+          (options '("--device" "rtl8139,netdev=net0"
+                     "--netdev"
+                     "user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222"))))
+@end lisp
+
 @node Version Control Services
 @subsection Version Control Services
 
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 989e439d5d..dc1e91b0bd 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Ryan Moe <ryan.moe <at> gmail.com>
 ;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -18,24 +19,41 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services virtualization)
-  #:use-module (gnu services)
-  #:use-module (gnu services configuration)
+  #:use-module (gnu bootloader)
+  #:use-module (gnu bootloader grub)
+  #:use-module (gnu image)
+  #:use-module (gnu packages admin)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu packages virtualization)
   #:use-module (gnu services base)
+  #:use-module (gnu services configuration)
   #:use-module (gnu services dbus)
   #:use-module (gnu services shepherd)
-  #:use-module (gnu system shadow)
+  #:use-module (gnu services ssh)
+  #:use-module (gnu services)
   #:use-module (gnu system file-systems)
-  #:use-module (gnu packages admin)
-  #:use-module (gnu packages virtualization)
-  #:use-module (guix records)
+  #:use-module (gnu system hurd)
+  #:use-module (gnu system image)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system)
+  #:use-module (guix derivations)
   #:use-module (guix gexp)
+  #:use-module (guix monads)
   #:use-module (guix packages)
+  #:use-module (guix records)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
 
-  #:export (libvirt-configuration
+  #:export (%hurd-vm-operating-system
+            hurd-vm-configuration
+            hurd-vm-service-type
+
+            libvirt-configuration
             libvirt-service-type
             virtlog-configuration
             virtlog-service-type
@@ -773,3 +791,97 @@ given QEMU package."
                  "This service supports transparent emulation of binaries
 compiled for other architectures using QEMU and the @code{binfmt_misc}
 functionality of the kernel Linux.")))
+
+
+;;;
+;;; The Hurd in VM service.
+;;;
+
+(define %hurd-vm-operating-system
+  (operating-system
+    (inherit %hurd-default-operating-system)
+    (host-name "guixydevel")
+    (timezone "Europe/Amsterdam")
+    (bootloader (bootloader-configuration
+                 (bootloader grub-minimal-bootloader)
+                 (target "/dev/vda")
+                 (timeout 0)))
+    (services (cons*
+               (service openssh-service-type
+                        (openssh-configuration
+                         (openssh openssh-sans-x)
+                         (use-pam? #f)
+                         (port-number 2222)
+                         (permit-root-login #t)
+                         (allow-empty-passwords? #t)
+                         (password-authentication? #t)))
+               %base-services/hurd))))
+
+(define-record-type* <hurd-vm-configuration>
+  hurd-vm-configuration make-hurd-vm-configuration
+  hurd-vm-configuration?
+  (os          hurd-vm-configuration-os                 ;<operating-system>
+               (default %hurd-vm-operating-system))
+  (qemu        hurd-vm-configuration-qemu               ;<package>
+               (default qemu-minimal))
+  (image       hurd-vm-configuration-image              ;string
+               (thunked)
+               (default (hurd-vm-disk-image this-record)))
+  (disk-size   hurd-vm-configuration-disk-size          ;number or 'guess
+               (default 'guess))
+  (memory-size hurd-vm-configuration-memory-size        ;number
+               (default 512))
+  (options     hurd-vm-configuration-options            ;list of string
+               (default
+                 `("--device" "rtl8139,netdev=net0"
+                   "--netdev" ,(string-append
+                                "user,id=net0"
+                                ",hostfwd=tcp:127.0.0.1:20022-:2222"
+                                ",hostfwd=tcp:127.0.0.1:25900-:5900")
+                   "--snapshot"
+                   "--hda"))))
+
+(define (hurd-vm-disk-image config)
+  "Return a disk-image for the Hurd according to CONFIG."
+  (let ((os (hurd-vm-configuration-os config))
+        (disk-size (hurd-vm-configuration-disk-size config))
+        (target (and (not (%current-target-system)) "i586-pc-gnu"))
+        (base-image (find-image "ext2" (%current-target-system))))
+    (with-parameters ((%current-target-system target))
+      (system-image
+       (image (inherit base-image)
+              (size disk-size)
+              (operating-system os))))))
+
+(define (hurd-vm-shepherd-service config)
+  "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
+
+  (let ((image       (hurd-vm-configuration-image config))
+        (qemu        (hurd-vm-configuration-qemu config))
+        (memory-size (hurd-vm-configuration-memory-size config))
+        (options     (hurd-vm-configuration-options config)))
+
+    (define vm-command
+      #~(list
+         (string-append #$qemu "/bin/qemu-system-i386")
+         #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+         "-m" (number->string #$memory-size)
+         #$@options
+         #+image))
+
+    (list
+     (shepherd-service
+      (documentation "Run the Hurd in a Virtual Machine: a Childhurd.")
+      (provision '(hurd-vm childhurd))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor #$vm-command))
+      (stop  #~(make-kill-destructor))))))
+
+(define hurd-vm-service-type
+  (service-type
+   (name 'hurd-vm)
+   (extensions (list (service-extension shepherd-root-service-type
+                                        hurd-vm-shepherd-service)))
+   (default-value (hurd-vm-configuration))
+   (description
+    "Provide a Virtual Machine running the GNU/Hurd.")))
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #59 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Diego Nicola Barbato <dnbarbato <at> posteo.de>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41785 <at> debbugs.gnu.org,
 Marius Bakke <marius <at> gnu.org>
Subject: Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
Date: Sat, 13 Jun 2020 09:30:39 +0200

Diego Nicola Barbato writes:

Hi,

> Jan Nieuwenhuizen <janneke <at> gnu.org> writes:
>
> [...]
>
>> and the interwebs told me that to start the VM, you have to add "-cpu
>> host"; so I started it using
>>
>> /gnu/store/k2b7nx34cwyi6yk49wgy4hg9mrwcmll5-run-vm.sh -cpu host -m
>> 2G -device rtl8139,netdev=net0 -netdev
>> user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:25900-:25900
>>
>>
>> and trying to "ssh -p 20022 localhost" from inside the bare-bones VM now
>> prints
>>
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> key_exchange_identification: read: Connection reset by peer
>> Connection reset by 127.0.0.1 port 20022
>>
>> ...something networky with QEMU.  Ideas?
>
> I've recently had intermittent issues with 'guix system vm' and hostfwd.
> Does it help if you remove the '-nic user,model=virtio-net-pci' option
> from (a copy of) the run-vm.sh script so that there's only one netdev?

That's helpful!  I now noticed that the QEMU command above I used was
the one I have been using to start the Hurd, networking wise.

After running

--8<---------------cut here---------------start------------->8---
./run-vm.sh -cpu host -m 2G -net nic -net user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:25900,hostfwd=tcp:127.0.0.1:20022-:20022
--8<---------------cut here---------------end--------------->8---

I noticed that instead of two network interfaces, there is now just one,
as expected; much better.

No luck ssh'ing into the second VM, directly or indirectly.  I'm not
exploring this further as we don't have an actually need for this
vm-in-vm thing.

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #62 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Sat, 13 Jun 2020 12:56:50 +0200

Hey Jan,

>                         (system (current-system))
>                         (target (operating-system-derivation target-os))
> -                       (base-image (find-image
> -                                    installation-disk-image-file-system-type))
> +                       (base-image -> (find-image
> +                                       installation-disk-image-file-system-type
> +                                       (%current-target-system)))

This should be:

--8<---------------cut here---------------start------------->8---
(target (current-target-system))
...
(base-image -> (find-image
                installation-disk-image-file-system-type
                target))
--8<---------------cut here---------------end--------------->8---

> +      ((image ->  (find-image file-system-type (%current-target-system)))
>         (sys       (system-derivation-for-action os image action
>                                                  #:file-system-type file-system-type
>                                                  #:image-size image-size

Same here, otherwise this look good to me.

Thanks,

Mathieu

Message #65 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
Date: Sat, 13 Jun 2020 14:49:23 +0200

[Message part 1 (text/plain, inline)]

Hey!

> +(define (hurd-vm-disk-image config)
> +  "Return a disk-image for the Hurd according to CONFIG."
> +  (let ((os (hurd-vm-configuration-os config))
> +        (disk-size (hurd-vm-configuration-disk-size config))
> +        (target (and (not (%current-target-system)) "i586-pc-gnu"))
> +        (base-image (find-image "ext2" (%current-target-system))))
> +    (with-parameters ((%current-target-system target))
> +      (system-image
> +       (image (inherit base-image)
> +              (size disk-size)
> +              (operating-system os))))))

With the attached patch, you could write:

--8<---------------cut here---------------start------------->8---
(define (hurd-vm-disk-image config)
  "Return a disk-image for the Hurd according to CONFIG."
  (let ((os (hurd-vm-configuration-os config))
        (disk-size (hurd-vm-configuration-disk-size config)))
    (system-image
     (image
      (inherit hurd-disk-image)
      (size disk-size)
      (operating-system os)))))
--8<---------------cut here---------------end--------------->8---

WDYT?

Mathieu

[0001-image-Add-target-support.patch (text/x-diff, inline)]

From dbcfd86a74903cb0fe77843518625436d749ed09 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Sat, 13 Jun 2020 14:01:18 +0200
Subject: [PATCH] image: Add 'target' support.

* gnu/image.scm (<image>)[target]: New field,
(image-target): new public method.
* gnu/system/image.scm (hurd-disk-image): Set "i586-pc-gnu" as image 'target'
field,
(maybe-with-target): new procedure,
(system-image): honor image 'target' field using the above procedure.
---
 gnu/image.scm        |  3 ++
 gnu/system/image.scm | 66 +++++++++++++++++++++++++++-----------------
 2 files changed, 43 insertions(+), 26 deletions(-)

diff --git a/gnu/image.scm b/gnu/image.scm
index 0a92d168e9..19b466527b 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -33,6 +33,7 @@
             image
             image-name
             image-format
+            image-target
             image-size
             image-operating-system
             image-partitions
@@ -67,6 +68,8 @@
   image make-image
   image?
   (format             image-format) ;symbol
+  (target             image-target
+                      (default #f))
   (size               image-size  ;size in bytes as integer
                       (default 'guess))
   (operating-system   image-operating-system  ;<operating-system>
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 6c4573509d..7b45fdfea7 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -103,6 +103,7 @@
 (define hurd-disk-image
   (image
    (format 'disk-image)
+   (target "i586-pc-gnu")
    (partitions
     (list (partition
            (size 'guess)
@@ -518,6 +519,14 @@ it can be used for bootloading."
                             (type root-file-system-type))
                           file-systems-to-keep)))))
 
+(define-syntax-rule (maybe-with-target image exp ...)
+  (let ((target (image-target image)))
+    (if target
+        (with-parameters ((%current-target-system target))
+          exp ...)
+        (begin
+          exp ...))))
+
 (define* (system-image image)
   "Return the derivation of IMAGE.  It can be a raw disk-image or an ISO9660
 image, depending on IMAGE format."
@@ -529,32 +538,33 @@ image, depending on IMAGE format."
          (bootcfg (operating-system-bootcfg os))
          (bootloader (bootloader-configuration-bootloader
                       (operating-system-bootloader os))))
-    (case (image-format image)
-      ((disk-image)
-       (system-disk-image image*
-                          #:bootcfg bootcfg
-                          #:bootloader bootloader
-                          #:register-closures? register-closures?
-                          #:inputs `(("system" ,os)
-                                     ("bootcfg" ,bootcfg))))
-      ((iso9660)
-       (system-iso9660-image
-        image*
-        #:bootcfg bootcfg
-        #:bootloader bootloader
-        #:register-closures? register-closures?
-        #:inputs `(("system" ,os)
-                   ("bootcfg" ,bootcfg))
-        ;; Make sure to use a mode that does no imply
-        ;; HFS+ tree creation that may fail with:
-        ;;
-        ;; "libisofs: FAILURE : Too much files to mangle,
-        ;; cannot guarantee unique file names"
-        ;;
-        ;; This happens if some limits are exceeded, see:
-        ;; https://lists.gnu.org/archive/html/grub-devel/2020-06/msg00048.html
-        #:grub-mkrescue-environment
-        '(("MKRESCUE_SED_MODE" . "mbr_only")))))))
+    (maybe-with-target image
+      (case (image-format image)
+        ((disk-image)
+         (system-disk-image image*
+                            #:bootcfg bootcfg
+                            #:bootloader bootloader
+                            #:register-closures? register-closures?
+                            #:inputs `(("system" ,os)
+                                       ("bootcfg" ,bootcfg))))
+        ((iso9660)
+         (system-iso9660-image
+          image*
+          #:bootcfg bootcfg
+          #:bootloader bootloader
+          #:register-closures? register-closures?
+          #:inputs `(("system" ,os)
+                     ("bootcfg" ,bootcfg))
+          ;; Make sure to use a mode that does no imply
+          ;; HFS+ tree creation that may fail with:
+          ;;
+          ;; "libisofs: FAILURE : Too much files to mangle,
+          ;; cannot guarantee unique file names"
+          ;;
+          ;; This happens if some limits are exceeded, see:
+          ;; https://lists.gnu.org/archive/html/grub-devel/2020-06/msg00048.html
+          #:grub-mkrescue-environment
+          '(("MKRESCUE_SED_MODE" . "mbr_only"))))))))
 
 (define (find-image file-system-type)
   "Find and return an image that could match the given FILE-SYSTEM-TYPE.  This
@@ -572,4 +582,8 @@ record."
              (else
               efi-disk-image))))))))
 
+;;; Local Variables:
+;;; eval: (put 'maybe-with-target 'scheme-indent-function 1)
+;;; End:
+
 ;;; image.scm ends here
-- 
2.24.0

Message #68 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Sat, 13 Jun 2020 15:05:11 +0200

Mathieu Othacehe writes:

Hey Mathieu,

>>                         (system (current-system))
>>                         (target (operating-system-derivation target-os))
>> -                       (base-image (find-image
>> -                                    installation-disk-image-file-system-type))
>> +                       (base-image -> (find-image
>> +                                       installation-disk-image-file-system-type
>> +                                       (%current-target-system)))
>
> This should be:
>
> (target (current-target-system))
> ...
> (base-image -> (find-image
>                 installation-disk-image-file-system-type
>                 target))

Ah, right.  Done and moved this

>>                         (target (operating-system-derivation target-os))

down.

>> +      ((image ->  (find-image file-system-type (%current-target-system)))
>>         (sys       (system-derivation-for-action os image action
>>                                                  #:file-system-type file-system-type
>>                                                  #:image-size image-size
>
> Same here, otherwise this look good to me.

Thanks!  Pushed to master as 7ca533c7237622d70b423033c4506217d9ce4014

(probably/hopefully soon to be revisited by your image-catalog feature :-)

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #71 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
Date: Sat, 13 Jun 2020 15:10:18 +0200

Mathieu Othacehe writes:

Hey Mathieu!

>> +(define (hurd-vm-disk-image config)
>> +  "Return a disk-image for the Hurd according to CONFIG."
>> +  (let ((os (hurd-vm-configuration-os config))
>> +        (disk-size (hurd-vm-configuration-disk-size config))
>> +        (target (and (not (%current-target-system)) "i586-pc-gnu"))
>> +        (base-image (find-image "ext2" (%current-target-system))))
>> +    (with-parameters ((%current-target-system target))
>> +      (system-image
>> +       (image (inherit base-image)
>> +              (size disk-size)
>> +              (operating-system os))))))
>
> With the attached patch, you could write:
>
> (define (hurd-vm-disk-image config)
>   "Return a disk-image for the Hurd according to CONFIG."
>   (let ((os (hurd-vm-configuration-os config))
>         (disk-size (hurd-vm-configuration-disk-size config)))
>     (system-image
>      (image
>       (inherit hurd-disk-image)
>       (size disk-size)
>       (operating-system os)))))
>
> WDYT?

I like it a lot!  Ludo already said adding target to image would be a
good idea.  Go for it!

> From dbcfd86a74903cb0fe77843518625436d749ed09 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe <at> gnu.org>
> Date: Sat, 13 Jun 2020 14:01:18 +0200
> Subject: [PATCH] image: Add 'target' support.

This is already one step towards the image catalog, looks good to me!

Note that I found https://bugs.gnu.org/41835 so I had to revert that
patch in order to test...less than great; but I didn't want to push
a revert without communicating about it first.

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #74 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
Date: Sat, 13 Jun 2020 16:35:04 +0200

Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

> +(define-syntax-rule (maybe-with-target image exp ...)
> +  (let ((target (image-target image)))
> +    (if target
> +        (with-parameters ((%current-target-system target))
> +          exp ...)
> +        (begin
> +          exp ...))))

I think you don’t need the ‘if’ here.  But then maybe you don’t need the
macro either since you can always write:

  (with-parameters ((%current-target-system (image-target image)))
    …)

Anyway, all this looks great to me! :-)

Ludo’.

Message #77 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, "Jan \(janneke\) Nieuwenhuizen" <janneke <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
Date: Sat, 13 Jun 2020 17:01:46 +0200

Hey Ludo,

> I think you don’t need the ‘if’ here.  But then maybe you don’t need the
> macro either since you can always write:

Thanks for the quick review :)

You're right it's not needed, I just removed it.

Mathieu

Message #80 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: [PATCH v4] services: Add 'hurd-in-vm service-type'.
Date: Sun, 14 Jun 2020 14:10:45 +0200

[Message part 1 (text/plain, inline)]

Jan Nieuwenhuizen writes:

Hello,

> we have now something "that works" for the Hurd.

Hmm...that was too fast; apparently I fooled myself last Friday.

Anyway, I found it!  Using this

--8<---------------cut here---------------start------------->8---
(define (hurd-vm-disk-image config)
  "Return a disk-image for the Hurd according to CONFIG."
  (with-parameters ((%current-target-system "i586-pc-gnu"))
    (let ((os (hurd-vm-configuration-os config))
          (disk-size (hurd-vm-configuration-disk-size config)))
      (system-image
       (image
        (inherit hurd-disk-image)
        (size disk-size)
        (operating-system os))))))
--8<---------------cut here---------------end--------------->8---

almost worked...but it goes wrong cross-building the services.

In gnu/services/shepherd.scm's scm->go, we were bitten by

--8<---------------cut here---------------start------------->8---
(define (scm->go file)
  ;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
  (let ((target (%current-target-system)))
    (with-extensions (list shepherd)
--8<---------------cut here---------------end--------------->8---

...but we now have let-system; so using

--8<---------------cut here---------------start------------->8---
diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm
index 77c4d0a8be..e14ceca231 100644
--- a/gnu/services/shepherd.scm
+++ b/gnu/services/shepherd.scm
@@ -266,8 +266,7 @@ stored."
 (define (scm->go file)
   "Compile FILE, which contains code to be loaded by shepherd's config file,
 and return the resulting '.go' file."
-  ;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
-  (let ((target (%current-target-system)))
+  (let-system (system target)
     (with-extensions (list shepherd)
       (computed-file (string-append (basename (scheme-file-name file) ".scm")
                                     ".go")
--8<---------------cut here---------------end--------------->8---

produces a working hurd-vm service!

New patch attached, I intend to push to master once rebased, all the
tests pass, etc. ;-)

Phew, thank you!

Greetings,
Janneke

[v4-0001-services-Add-hurd-vm-service-type.patch (text/x-patch, inline)]

From ef424afa4bc83a3e7fc7ff2ac831de02c2022c73 Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
Date: Wed, 10 Jun 2020 00:10:28 +0200
Subject: [PATCH v4] services: Add 'hurd-vm service-type'.

* gnu/services/virtualization.scm (hurd-vm-shepherd-service,
hurd-vm-disk-image): New procedures.
(%hurd-vm-operating-system, hurd-vm-service-type): New variables.
(<hurd-vm-configuration>): New record type.
* doc/guix.texi (Virtualization Services): Document it.
* gnu/services/shepherd.scm (scm->go): Use let-system, remove FIXME.  Fixes
fixes cross-building of shepherd modules for the Hurd image.
---
 doc/guix.texi                   |  83 +++++++++++++++++++++
 gnu/services/shepherd.scm       |   3 +-
 gnu/services/virtualization.scm | 125 ++++++++++++++++++++++++++++++--
 3 files changed, 202 insertions(+), 9 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 15e077a41c..ed36d5014b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24583,6 +24583,89 @@ Return true if @var{obj} is a platform object.
 Return the name of @var{platform}---a string such as @code{"arm"}.
 @end deffn
 
+
+@subsubheading The Hurd in a Virtual Machine
+
+@cindex @code{hurd}
+@cindex the Hurd
+
+Service @code{hurd-vm} provides support for running GNU/Hurd in a
+virtual machine (VM), a so-called ``Childhurd''.  The virtual machine is
+a Shepherd service that can be controlled with commands such as:
+
+@example
+herd start hurd-vm
+herd stop childhurd
+@end example
+
+The given GNU/Hurd operating system configuration is cross-compiled.
+
+@defvr {Scheme Variable} hurd-vm-service-type
+This is the type of the Hurd in a Virtual Machine service.  Its value
+must be a @code{hurd-vm-configuration} object, which specifies the
+operating system (@pxref{operating-system Reference}) and the disk size
+for the Hurd Virtual Machine, the QEMU package to use as well as the
+options for running it.
+
+For example:
+
+@lisp
+(service hurd-vm-service-type
+         (hurd-vm-configuration
+          (disk-size (* 5000 (expt 2 20))) ;5G
+          (memory-size 1024)))             ;1024MiB
+@end lisp
+
+would create a disk image big enough to build GNU <at> tie{}Hello, with some
+extra memory.
+@end defvr
+
+@deftp {Data Type} hurd-vm-configuration
+The data type representing the configuration for
+@code{hurd-vm-service-type}.
+
+@table @asis
+@item @code{os} (default: @var{%hurd-vm-operating-system})
+The operating system to instantiate.  This default is bare-bones with a
+permissive OpenSSH secure shell daemon listening on port 2222
+(@pxref{Networking Services, @code{openssh-service-type}}).
+
+@item @code{qemu} (default: @code{qemu-minimal})
+The QEMU package to use.
+
+@item @code{image} (default: @var{hurd-vm-disk-image})
+The procedure used to build the disk-image built from this
+configuration.
+
+@item @code{disk-size} (default: @code{'guess})
+The size of the disk image.
+
+@item @code{memory-size} (default: @code{512})
+The memory size of the Virtual Machine in mebibytes.
+
+@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
+      @code{"--netdev"} @
+      @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
+      @code{"--snapshot"} @
+      @code{"--hda")})
+The extra options for running QEMU.
+@end table
+@end deftp
+
+Note that by default the VM image is volatile, i.e., once stopped the
+contents are lost.  If you want a stateful image instead, override the
+configuration's @code{image} and @code{options} without
+the @code{--snapshot} flag using something along these lines:
+
+@lisp
+(service hurd-vm-service-type
+         (hurd-vm-configuration
+          (image    (const "/out/of/store/writable/hurd.img"))
+          (options '("--device" "rtl8139,netdev=net0"
+                     "--netdev"
+                     "user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222"))))
+@end lisp
+
 @node Version Control Services
 @subsection Version Control Services
 
diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm
index 77c4d0a8be..e14ceca231 100644
--- a/gnu/services/shepherd.scm
+++ b/gnu/services/shepherd.scm
@@ -266,8 +266,7 @@ stored."
 (define (scm->go file)
   "Compile FILE, which contains code to be loaded by shepherd's config file,
 and return the resulting '.go' file."
-  ;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
-  (let ((target (%current-target-system)))
+  (let-system (system target)
     (with-extensions (list shepherd)
       (computed-file (string-append (basename (scheme-file-name file) ".scm")
                                     ".go")
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 989e439d5d..98b94f7464 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Ryan Moe <ryan.moe <at> gmail.com>
 ;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -18,24 +19,41 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services virtualization)
-  #:use-module (gnu services)
-  #:use-module (gnu services configuration)
+  #:use-module (gnu bootloader)
+  #:use-module (gnu bootloader grub)
+  #:use-module (gnu image)
+  #:use-module (gnu packages admin)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu packages virtualization)
   #:use-module (gnu services base)
+  #:use-module (gnu services configuration)
   #:use-module (gnu services dbus)
   #:use-module (gnu services shepherd)
-  #:use-module (gnu system shadow)
+  #:use-module (gnu services ssh)
+  #:use-module (gnu services)
   #:use-module (gnu system file-systems)
-  #:use-module (gnu packages admin)
-  #:use-module (gnu packages virtualization)
-  #:use-module (guix records)
+  #:use-module (gnu system hurd)
+  #:use-module (gnu system image)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system)
+  #:use-module (guix derivations)
   #:use-module (guix gexp)
+  #:use-module (guix monads)
   #:use-module (guix packages)
+  #:use-module (guix records)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
 
-  #:export (libvirt-configuration
+  #:export (%hurd-vm-operating-system
+            hurd-vm-configuration
+            hurd-vm-service-type
+
+            libvirt-configuration
             libvirt-service-type
             virtlog-configuration
             virtlog-service-type
@@ -773,3 +791,96 @@ given QEMU package."
                  "This service supports transparent emulation of binaries
 compiled for other architectures using QEMU and the @code{binfmt_misc}
 functionality of the kernel Linux.")))
+
+
+;;;
+;;; The Hurd in VM service: a Childhurd.
+;;;
+
+(define %hurd-vm-operating-system
+  (operating-system
+    (inherit %hurd-default-operating-system)
+    (host-name "childhurd")
+    (timezone "Europe/Amsterdam")
+    (bootloader (bootloader-configuration
+                 (bootloader grub-minimal-bootloader)
+                 (target "/dev/vda")
+                 (timeout 0)))
+    (services (cons*
+               (service openssh-service-type
+                        (openssh-configuration
+                         (openssh openssh-sans-x)
+                         (use-pam? #f)
+                         (port-number 2222)
+                         (permit-root-login #t)
+                         (allow-empty-passwords? #t)
+                         (password-authentication? #t)))
+               %base-services/hurd))))
+
+(define-record-type* <hurd-vm-configuration>
+  hurd-vm-configuration make-hurd-vm-configuration
+  hurd-vm-configuration?
+  (os          hurd-vm-configuration-os                 ;<operating-system>
+               (default %hurd-vm-operating-system))
+  (qemu        hurd-vm-configuration-qemu               ;<package>
+               (default qemu-minimal))
+  (image       hurd-vm-configuration-image              ;string
+               (thunked)
+               (default (hurd-vm-disk-image this-record)))
+  (disk-size   hurd-vm-configuration-disk-size          ;number or 'guess
+               (default 'guess))
+  (memory-size hurd-vm-configuration-memory-size        ;number
+               (default 512))
+  (options     hurd-vm-configuration-options            ;list of string
+               (default
+                 `("--device" "rtl8139,netdev=net0"
+                   "--netdev" ,(string-append
+                                "user,id=net0"
+                                ",hostfwd=tcp:127.0.0.1:20022-:2222"
+                                ",hostfwd=tcp:127.0.0.1:25900-:5900")
+                   "--snapshot"
+                   "--hda"))))
+
+(define (hurd-vm-disk-image config)
+  "Return a disk-image for the Hurd according to CONFIG."
+  (with-parameters ((%current-target-system "i586-pc-gnu"))
+    (let ((os (hurd-vm-configuration-os config))
+          (disk-size (hurd-vm-configuration-disk-size config)))
+      (system-image
+       (image
+        (inherit hurd-disk-image)
+        (size disk-size)
+        (operating-system os))))))
+
+(define (hurd-vm-shepherd-service config)
+  "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
+
+  (let ((image       (hurd-vm-configuration-image config))
+        (qemu        (hurd-vm-configuration-qemu config))
+        (memory-size (hurd-vm-configuration-memory-size config))
+        (options     (hurd-vm-configuration-options config)))
+
+    (define vm-command
+      #~(list
+         (string-append #$qemu "/bin/qemu-system-i386")
+         #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
+         "-m" (number->string #$memory-size)
+         #$@options
+         #+image))
+
+    (list
+     (shepherd-service
+      (documentation "Run the Hurd in a Virtual Machine: a Childhurd.")
+      (provision '(hurd-vm childhurd))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor #$vm-command))
+      (stop  #~(make-kill-destructor))))))
+
+(define hurd-vm-service-type
+  (service-type
+   (name 'hurd-vm)
+   (extensions (list (service-extension shepherd-root-service-type
+                                        hurd-vm-shepherd-service)))
+   (default-value (hurd-vm-configuration))
+   (description
+    "Provide a Virtual Machine running the GNU/Hurd.")))
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[Message part 3 (text/plain, inline)]

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #83 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Sun, 14 Jun 2020 14:37:58 +0200

[Message part 1 (text/plain, inline)]

Hey!

>>> +      ((image ->  (find-image file-system-type (%current-target-system)))
>>>         (sys       (system-derivation-for-action os image action
>>>                                                  #:file-system-type file-system-type
>>>                                                  #:image-size image-size
>>
>> Same here, otherwise this look good to me.

I missed something, "target" variable is shadowing the "target"
argument. The attached patch should fix the issue, I'm testing it.

Thanks,

Mathieu

[0001-scripts-system-Fix-init-command.patch (text/x-diff, inline)]

From 760e46d2ff57fab2b67656b85d800572bbc56f4b Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Sun, 14 Jun 2020 14:16:37 +0200
Subject: [PATCH] scripts: system: Fix "init" command.

This is a follow-up of 7ca533c7237622d70b423033c4506217d9ce4014. The
introduced "target" variable is shadowing the target argument.

* guix/scripts/system.scm (perform-action): Rename "target" variable to
"target*".
---
 guix/scripts/system.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 6769a602b1..212b49f008 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -802,8 +802,8 @@ static checks."
       (check-initrd-modules os)))
 
   (mlet* %store-monad
-      ((target    (current-target-system))
-       (image ->  (find-image file-system-type target))
+      ((target*   (current-target-system))
+       (image ->  (find-image file-system-type target*))
        (sys       (system-derivation-for-action os image action
                                                 #:file-system-type file-system-type
                                                 #:image-size image-size
-- 
2.26.2

Message #86 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
Date: Sun, 14 Jun 2020 14:44:38 +0200

Hey janneke!

> (define (hurd-vm-disk-image config)
>   "Return a disk-image for the Hurd according to CONFIG."
>   (with-parameters ((%current-target-system "i586-pc-gnu"))
>     (let ((os (hurd-vm-configuration-os config))
>           (disk-size (hurd-vm-configuration-disk-size config)))
>       (system-image
>        (image
>         (inherit hurd-disk-image)
>         (size disk-size)
>         (operating-system os))))))

Nice! I'm not sure why we still need the "with-parameters" call because,
it's done first thing in "system-image". I'd like to understand this
before proceeding, do you have a branch where I could test it?

> -  ;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
> -  (let ((target (%current-target-system)))
> +  (let-system (system target)
>      (with-extensions (list shepherd)
>        (computed-file (string-append (basename (scheme-file-name file) ".scm")
>                                      ".go")
>
> produces a working hurd-vm service!

Good catch!

Thanks,

Mathieu

Message #89 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Sun, 14 Jun 2020 15:12:11 +0200

Mathieu Othacehe writes:

Hi Mathieu,

>>>> +      ((image ->  (find-image file-system-type (%current-target-system)))
>>>>         (sys       (system-derivation-for-action os image action
>>>>                                                  #:file-system-type file-system-type
>>>>                                                  #:image-size image-size
>>>
>>> Same here, otherwise this look good to me.
>
> I missed something, "target" variable is shadowing the "target"
> argument. The attached patch should fix the issue, I'm testing it.

Oops!

> From 760e46d2ff57fab2b67656b85d800572bbc56f4b Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe <at> gnu.org>
> Date: Sun, 14 Jun 2020 14:16:37 +0200
> Subject: [PATCH] scripts: system: Fix "init" command.
>
> This is a follow-up of 7ca533c7237622d70b423033c4506217d9ce4014. The
> introduced "target" variable is shadowing the target argument.
>
> * guix/scripts/system.scm (perform-action): Rename "target" variable to
> "target*".
> ---
>  guix/scripts/system.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
> index 6769a602b1..212b49f008 100644
> --- a/guix/scripts/system.scm
> +++ b/guix/scripts/system.scm
> @@ -802,8 +802,8 @@ static checks."
>        (check-initrd-modules os)))
>  
>    (mlet* %store-monad
> -      ((target    (current-target-system))
> -       (image ->  (find-image file-system-type target))
> +      ((target*   (current-target-system))
> +       (image ->  (find-image file-system-type target*))
>         (sys       (system-derivation-for-action os image action
>                                                  #:file-system-type file-system-type
>                                                  #:image-size image-size

Yeah, that looks right!  I didn't notice either.  I could have been more
careful/suspicious, as in the other, similar case in
gnu/tests/install.scm I moved

         (target (operating-system-derivation target-os))

down to have mlet* do the shadowing properly.

Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #92 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
Date: Sun, 14 Jun 2020 15:18:18 +0200

Mathieu Othacehe writes:

Hey Mathieu,

>> (define (hurd-vm-disk-image config)
>>   "Return a disk-image for the Hurd according to CONFIG."
>>   (with-parameters ((%current-target-system "i586-pc-gnu"))
>>     (let ((os (hurd-vm-configuration-os config))
>>           (disk-size (hurd-vm-configuration-disk-size config)))
>>       (system-image
>>        (image
>>         (inherit hurd-disk-image)
>>         (size disk-size)
>>         (operating-system os))))))
>
> Nice! I'm not sure why we still need the "with-parameters" call because,
> it's done first thing in "system-image". I'd like to understand this
> before proceeding, do you have a branch where I could test it?

It's on my gitlab master...almost ready to push ;-)

Initially I did not have this "with-parameters", but then in
gnu/system.scm:

--8<---------------cut here---------------start------------->8---
(define (hurd-multiboot-modules os)
...
         (libc (if target
                   (with-parameters ((%current-target-system #f))
                     ;; TODO: cross-libc has extra patches for the Hurd;
                     ;; remove in next rebuild cycle
                     (cross-libc target))
                   glibc))
--8<---------------cut here---------------end--------------->8---

we take the ELSE branch here -- and that does not work.  AIUI, this is
really a temporary kludge until the next rebuild cycle we can move some
hurd-specific glibc patches from cross-libc to glibc-proper.

...but it's tricky to see how this all plays together, not entirely
clear to me anyway...

>> -  ;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
>> -  (let ((target (%current-target-system)))
>> +  (let-system (system target)
>>      (with-extensions (list shepherd)
>>        (computed-file (string-append (basename (scheme-file-name file) ".scm")
>>                                      ".go")
>>
>> produces a working hurd-vm service!
>
> Good catch!

:-)
Almost there...

Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #95 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Sun, 14 Jun 2020 15:32:09 +0200

Jan Nieuwenhuizen writes:

Hello again,

>>>>> +      ((image ->  (find-image file-system-type (%current-target-system)))
>>>>>         (sys       (system-derivation-for-action os image action
>>>>>                                                  #:file-system-type file-system-type
>>>>>                                                  #:image-size image-size
>>>>
>>>> Same here, otherwise this look good to me.
>>
>> I missed something, "target" variable is shadowing the "target"
>> argument. The attached patch should fix the issue, I'm testing it.
>
> Oops!
>
>> From 760e46d2ff57fab2b67656b85d800572bbc56f4b Mon Sep 17 00:00:00 2001
>> From: Mathieu Othacehe <othacehe <at> gnu.org>
>> Date: Sun, 14 Jun 2020 14:16:37 +0200
>> Subject: [PATCH] scripts: system: Fix "init" command.

Just to confirm that this fixes

    make check-system TESTS="installed-extlinux-os"

failing early, where it now possibly fails for lack of disk
space...retrying...

Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #98 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 41785 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
Date: Sun, 14 Jun 2020 17:44:06 +0200

> Just to confirm that this fixes
>
>     make check-system TESTS="installed-extlinux-os"

Ok, thanks for checking. I pushed this one.

> failing early, where it now possibly fails for lack of disk
> space...retrying...

Yup, that's another issue I'm working on!

Thanks,

Mathieu

Message #101 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
Date: Sun, 14 Jun 2020 17:52:19 +0200

> (define (hurd-multiboot-modules os)
> ...
>          (libc (if target
>                    (with-parameters ((%current-target-system #f))
>                      ;; TODO: cross-libc has extra patches for the Hurd;
>                      ;; remove in next rebuild cycle
>                      (cross-libc target))
>                    glibc))
>
> we take the ELSE branch here -- and that does not work.  AIUI, this is
> really a temporary kludge until the next rebuild cycle we can move some
> hurd-specific glibc patches from cross-libc to glibc-proper.

Oh, I know why! In "system-image", there are a few calls, in the first
let, that are outside the "with-parameters", in particular "bootcfg"
which triggers the problem you are describing above.

Maybe something as naive as:

--8<---------------cut here---------------start------------->8---
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1bda25fd7f..8a3c4e22d9 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -525,14 +525,14 @@ it can be used for bootloading."
 image, depending on IMAGE format."
   (define substitutable? (image-substitutable? image))
 
-  (let* ((os (operating-system-for-image image))
-         (image* (image-with-os image os))
-         (target (image-target image))
-         (register-closures? (has-guix-service-type? os))
-         (bootcfg (operating-system-bootcfg os))
-         (bootloader (bootloader-configuration-bootloader
-                      (operating-system-bootloader os))))
-    (with-parameters ((%current-target-system target))
+  (with-parameters ((%current-target-system target))
+    (let* ((os (operating-system-for-image image))
+           (image* (image-with-os image os))
+           (target (image-target image))
+           (register-closures? (has-guix-service-type? os))
+           (bootcfg (operating-system-bootcfg os))
+           (bootloader (bootloader-configuration-bootloader
+                        (operating-system-bootloader os))))
--8<---------------cut here---------------end--------------->8---

would do the trick.

Thanks,

Mathieu

Message #104 received at 41785 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785 <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
Date: Sun, 14 Jun 2020 18:22:52 +0200

> +  (with-parameters ((%current-target-system target))
> +    (let* ((os (operating-system-for-image image))
> +           (image* (image-with-os image os))
> +           (target (image-target image))
> +           (register-closures? (has-guix-service-type? os))
> +           (bootcfg (operating-system-bootcfg os))
> +           (bootloader (bootloader-configuration-bootloader
> +                        (operating-system-bootloader os))))
>
> would do the trick.

I went ahead and pushed a variant of this as
c9f6e2e5bdff186583bdc360832b57f4c56e3427.

Thanks,

Mathieu

Message #109 received at 41785-done <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 41785-done <at> debbugs.gnu.org
Subject: Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
Date: Sun, 14 Jun 2020 18:42:10 +0200

Mathieu Othacehe writes:

>> (define (hurd-multiboot-modules os)
>> ...
>>          (libc (if target
>>                    (with-parameters ((%current-target-system #f))
>>                      ;; TODO: cross-libc has extra patches for the Hurd;
>>                      ;; remove in next rebuild cycle
>>                      (cross-libc target))
>>                    glibc))
>>
>> we take the ELSE branch here -- and that does not work.  AIUI, this is
>> really a temporary kludge until the next rebuild cycle we can move some
>> hurd-specific glibc patches from cross-libc to glibc-proper.
>
> Oh, I know why! In "system-image", there are a few calls, in the first
> let, that are outside the "with-parameters", in particular "bootcfg"
> which triggers the problem you are describing above.

Ah, yes that makes sense.

> Maybe something as naive as:
>
> diff --git a/gnu/system/image.scm b/gnu/system/image.scm

[..]
> -  (let* ((os (operating-system-for-image image))
[..]
> -    (with-parameters ((%current-target-system target))
> +  (with-parameters ((%current-target-system target))
> +    (let* ((os (operating-system-for-image image))
>
> would do the trick.

It does!

> I went ahead and pushed a variant of this as
> c9f6e2e5bdff186583bdc360832b57f4c56e3427.

Woohoo!  I remove the with-parameters in hurd-vm-disk-image and pushed
to master 5e9cf93364d87c70f8bfad915417cd75d21c0fed

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.