GNU bug report logs - #43155
[PATCH] hydra//build-machines: Update childhurd-net-options for secret-service.

Previous Next

To reply to this bug, email your comments to 43155 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: guix-patches <at> gnu.org
Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
 secret-service.
Date: Tue, 01 Sep 2020 16:46:43 +0200

[Message part 1 (text/plain, inline)]

Hi!

With bug https://bugs.gnu.org/43106 just closed we now have a nice way
to inject secrets into the Childhurds.

Using the attached patch, which needs a fresh pull and reconfigure on
berlin (at least the nodes 101,102 that run Childhurds), we can create a
tree of childhurd secrets like so

--8<---------------cut here---------------start------------->8---
/etc/childhurd/etc/guix/signing-key.pub
/etc/childhurd/etc/guix/signing-key.sec
/etc/childhurd/etc/ssh/ssh_host_ed25519_key
/etc/childhurd/etc/ssh/ssh_host_ecdsa_key
/etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
/etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
--8<---------------cut here---------------end--------------->8---

...and then we should be able to start offloading builds for the Hurd.
(I guess we then also need to add a cuirass jobs for the Hurd?)

Janneke

[0001-hydra-build-machines-Update-childhurd-net-options-fo.patch (text/x-patch, inline)]

From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
Date: Tue, 1 Sep 2020 16:31:42 +0200
Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
 secret-service.
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
[childhurd-net-options]: Include secret-service local QEMU forwarding.
Use variables from (gnu services virtualization).
---
 hydra/modules/sysadmin/build-machines.scm | 31 ++++++++++++++++-------
 1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/hydra/modules/sysadmin/build-machines.scm b/hydra/modules/sysadmin/build-machines.scm
index b4afcbe..0385b6a 100644
--- a/hydra/modules/sysadmin/build-machines.scm
+++ b/hydra/modules/sysadmin/build-machines.scm
@@ -120,15 +120,28 @@ EMULATED-ARCHITECTURES, unless it's empty."
 
   (define (childhurd-net-options . config)
     "Expose SSH and VNC ports on 0.0.0.0; for first Childhurd VM those
-are 10022 and 15900."
-    (let ((id 0))
-      (define (qemu-vm-port base)
-        (number->string (+ base (* 1000 id))))
-      `("--device" "rtl8139,netdev=net0"
-        "--netdev" ,(string-append
-                     "user,id=net0"
-                     ",hostfwd=tcp:0.0.0.0:" (qemu-vm-port 10022) "-:2222"
-                     ",hostfwd=tcp:0.0.0.0:" (qemu-vm-port 15900) "-:5900"))))
+are 10022 and 15900.  Keep secret-service port local."
+    `("--device" "rtl8139,netdev=net0"
+      "--netdev" ,(string-append
+                   "user,id=net0"
+                   ",hostfwd=tcp:127.0.0.1:"
+                   (number->string (hurd-vm-port
+                                    config
+                                    (@@ (gnu services virtualization)
+                                        %hurd-vm-secrets-port)))
+                   "-:1004"
+                   ",hostfwd=tcp:0.0.0.0:"
+                   (number->string (hurd-vm-port
+                                    config
+                                    (@@ (gnu services virtualization)
+                                        %hurd-vm-ssh-port)))
+                   "-:2222"
+                   ",hostfwd=tcp:0.0.0.0:"
+                   (number->string (hurd-vm-port
+                                    config
+                                    (@@ (gnu services virtualization)
+                                        %hurd-vm-vnc-port)))
+                   "-:5900")))
 
   (define sysadmins
     (list (sysadmin (name "ludo")
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[Message part 3 (text/plain, inline)]

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #8 received at 43155 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 43155 <at> debbugs.gnu.org
Subject: Re: [bug#43155] [PATCH] hydra//build-machines: Update
 childhurd-net-options for secret-service.
Date: Tue, 01 Sep 2020 23:19:44 +0200

Hi!

Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:

> With bug https://bugs.gnu.org/43106 just closed we now have a nice way
> to inject secrets into the Childhurds.
>
> Using the attached patch, which needs a fresh pull and reconfigure on
> berlin (at least the nodes 101,102 that run Childhurds), we can create a
> tree of childhurd secrets like so
>
> /etc/childhurd/etc/guix/signing-key.pub
> /etc/childhurd/etc/guix/signing-key.sec
> /etc/childhurd/etc/ssh/ssh_host_ed25519_key
> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
> /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
>
> ...and then we should be able to start offloading builds for the Hurd.

Yup!  Probably we’ll create /etc/childhurd/HOST for each VM, so we also
need to adjust <hurd-vm-configuration> accordingly, right?

(I realize that the current code will silently keep going if we forget
to put the secret files in place; IOW, the service config doesn’t show
the files we intended to push as secrets.  Oh well, we’ll see that
later.)

> (I guess we then also need to add a cuirass jobs for the Hurd?)

Yes, or maybe just change ‘systems’ in the Cuirass specs for
‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
which doesn’t sound like a great idea for now.  Perhaps we can simply
add a separate jobset pulling from ‘master’ but building only for
i586-gnu and only the “core” package set?

>>From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
> Date: Tue, 1 Sep 2020 16:31:42 +0200
> Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
>  secret-service.
> Content-Transfer-Encoding: 8bit
> Content-Type: text/plain; charset=UTF-8
>
> * hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
> [childhurd-net-options]: Include secret-service local QEMU forwarding.
> Use variables from (gnu services virtualization).

LGTM, thanks!

Ludo’.

Message #11 received at 43155 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 43155 <at> debbugs.gnu.org
Subject: Re: [bug#43155] [PATCH] hydra//build-machines: Update
 childhurd-net-options for secret-service.
Date: Wed, 02 Sep 2020 07:58:13 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès writes:

Hi!

> Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> With bug https://bugs.gnu.org/43106 just closed we now have a nice way
>> to inject secrets into the Childhurds.
>>
>> Using the attached patch, which needs a fresh pull and reconfigure on
>> berlin (at least the nodes 101,102 that run Childhurds), we can create a
>> tree of childhurd secrets like so
>>
>> /etc/childhurd/etc/guix/signing-key.pub
>> /etc/childhurd/etc/guix/signing-key.sec
>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key
>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
>>
>> ...and then we should be able to start offloading builds for the Hurd.
>
> Yup!  Probably we’ll create /etc/childhurd/HOST for each VM, so we also
> need to adjust <hurd-vm-configuration> accordingly, right?

Yes, we can add something like

      (secret-root (format #f "/etc/childhurd/~a" id))

to the

    (service hurd-vm-service-type
        (hurd-vm-configuration
          ...

(i'm a bit curious, though, why we would want to differentiate between
childhurds, they can be all identical?)

> (I realize that the current code will silently keep going if we forget
> to put the secret files in place; IOW, the service config doesn’t show
> the files we intended to push as secrets.  Oh well, we’ll see that
> later.)

Yes, I guess that's a feature -- "you" can start it once, then do
something like

    mkdir -p /etc/childhurd/etc
    scp -r childhurd:/etc/guix /etc/childhurd/etc
    scp -r childhurd:/etc/ssh /etc/childhurd/etc

>> (I guess we then also need to add a cuirass jobs for the Hurd?)
>
> Yes, or maybe just change ‘systems’ in the Cuirass specs for
> ‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
> which doesn’t sound like a great idea for now.

I agree, not much sense in that yet.

> Perhaps we can simply add a separate jobset pulling from ‘master’ but
> building only for i586-gnu and only the “core” package set?

Hmm, why can't I find the definition of "core"?.  Anyway, It would be a
great first step to build (everything needef for) "hello", after that we
want to have/try "guile-3.0" and possibly "guix".

>>>From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
>> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
>> Date: Tue, 1 Sep 2020 16:31:42 +0200
>> Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
>>  secret-service.
>> Content-Transfer-Encoding: 8bit
>> Content-Type: text/plain; charset=UTF-8
>>
>> * hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
>> [childhurd-net-options]: Include secret-service local QEMU forwarding.
>> Use variables from (gnu services virtualization).
>
> LGTM, thanks!

Great, pushed to guix-maintenance as 04c0fc1ea110b82d6180bbc1b2f895e55e746cd8

Janneke

...after first pushing this -- Ooopss typo fix

[0001-hydra-build-machines-Oops-typo-in-childhurd-net-opti.patch (text/x-patch, inline)]

From 35dd1de08f1b812a22184e925b089ffc471c52de Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
Date: Wed, 2 Sep 2020 07:52:13 +0200
Subject: [PATCH 1/2] hydra/build-machines: Oops, typo in
 childhurd-net-options.
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
[childhurd-net-options]: Remove stray dot from parameter list.
---
 hydra/modules/sysadmin/build-machines.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hydra/modules/sysadmin/build-machines.scm b/hydra/modules/sysadmin/build-machines.scm
index b4afcbe..0a3e113 100644
--- a/hydra/modules/sysadmin/build-machines.scm
+++ b/hydra/modules/sysadmin/build-machines.scm
@@ -118,7 +118,7 @@ EMULATED-ARCHITECTURES, unless it's empty."
                        (mcron-configuration (jobs (list gc-job))))
               (operating-system-user-services %hurd-vm-operating-system)))))
 
-  (define (childhurd-net-options . config)
+  (define (childhurd-net-options config)
     "Expose SSH and VNC ports on 0.0.0.0; for first Childhurd VM those
 are 10022 and 15900."
     (let ((id 0))
-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[Message part 3 (text/plain, inline)]

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #14 received at 43155 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 43155 <at> debbugs.gnu.org
Subject: Re: [bug#43155] [PATCH] hydra//build-machines: Update
 childhurd-net-options for secret-service.
Date: Wed, 02 Sep 2020 22:08:03 +0200

Hi,

Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:

> Ludovic Courtès writes:
>
> Hi!
>
>> Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:
>>
>>> With bug https://bugs.gnu.org/43106 just closed we now have a nice way
>>> to inject secrets into the Childhurds.
>>>
>>> Using the attached patch, which needs a fresh pull and reconfigure on
>>> berlin (at least the nodes 101,102 that run Childhurds), we can create a
>>> tree of childhurd secrets like so
>>>
>>> /etc/childhurd/etc/guix/signing-key.pub
>>> /etc/childhurd/etc/guix/signing-key.sec
>>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key
>>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
>>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
>>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
>>>
>>> ...and then we should be able to start offloading builds for the Hurd.
>>
>> Yup!  Probably we’ll create /etc/childhurd/HOST for each VM, so we also
>> need to adjust <hurd-vm-configuration> accordingly, right?
>
> Yes, we can add something like
>
>       (secret-root (format #f "/etc/childhurd/~a" id))
>
> to the
>
>     (service hurd-vm-service-type
>         (hurd-vm-configuration
>           ...

Sounds good.

> (i'm a bit curious, though, why we would want to differentiate between
> childhurds, they can be all identical?)

Well, dunno if it really matters for our specific use case, but it seems
“cleaner” to me to give each childhurd its identity.  OTOH, these are
VMs and they run on the same physical machine, so…

>> (I realize that the current code will silently keep going if we forget
>> to put the secret files in place; IOW, the service config doesn’t show
>> the files we intended to push as secrets.  Oh well, we’ll see that
>> later.)
>
> Yes, I guess that's a feature -- "you" can start it once, then do
> something like
>
>     mkdir -p /etc/childhurd/etc
>     scp -r childhurd:/etc/guix /etc/childhurd/etc
>     scp -r childhurd:/etc/ssh /etc/childhurd/etc

Right, that can be convenient.  OTOH, from the perspective of having
declarative OS configs, it’s not great because this aspect of the config
are left out.  But maybe that’s an issue we can have if/when we
generalize ‘secret-service-type’.

>>> (I guess we then also need to add a cuirass jobs for the Hurd?)
>>
>> Yes, or maybe just change ‘systems’ in the Cuirass specs for
>> ‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
>> which doesn’t sound like a great idea for now.
>
> I agree, not much sense in that yet.
>
>> Perhaps we can simply add a separate jobset pulling from ‘master’ but
>> building only for i586-gnu and only the “core” package set?
>
> Hmm, why can't I find the definition of "core"?.  Anyway, It would be a
> great first step to build (everything needef for) "hello", after that we
> want to have/try "guile-3.0" and possibly "guix".

Sure.  The “core” subset is defined in (gnu ci).

>>>>From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
>>> From: "Jan (janneke) Nieuwenhuizen" <janneke <at> gnu.org>
>>> Date: Tue, 1 Sep 2020 16:31:42 +0200
>>> Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
>>>  secret-service.
>>> Content-Transfer-Encoding: 8bit
>>> Content-Type: text/plain; charset=UTF-8
>>>
>>> * hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
>>> [childhurd-net-options]: Include secret-service local QEMU forwarding.
>>> Use variables from (gnu services virtualization).
>>
>> LGTM, thanks!
>
> Great, pushed to guix-maintenance as 04c0fc1ea110b82d6180bbc1b2f895e55e746cd8

Thanks!

Ludo’.

Message #17 received at 43155 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 43155 <at> debbugs.gnu.org
Subject: Re: [bug#43155] [PATCH] hydra//build-machines: Update
 childhurd-net-options for secret-service.
Date: Thu, 03 Sep 2020 12:19:49 +0200

Ludovic Courtès writes:

Hi,

> Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> Ludovic Courtès writes:
>> Yes, we can add something like
>>
>>       (secret-root (format #f "/etc/childhurd/~a" id))
>>
>> to the
>>
>>     (service hurd-vm-service-type
>>         (hurd-vm-configuration
>>           ...
>
> Sounds good.
>
>> (i'm a bit curious, though, why we would want to differentiate between
>> childhurds, they can be all identical?)
>
> Well, dunno if it really matters for our specific use case, but it seems
> “cleaner” to me to give each childhurd its identity.  OTOH, these are
> VMs and they run on the same physical machine, so…

Right...

>>> (I realize that the current code will silently keep going if we forget
>>> to put the secret files in place; IOW, the service config doesn’t show
>>> the files we intended to push as secrets.  Oh well, we’ll see that
>>> later.)
>>
>> Yes, I guess that's a feature -- "you" can start it once, then do
>> something like
>>
>>     mkdir -p /etc/childhurd/etc
>>     scp -r childhurd:/etc/guix /etc/childhurd/etc
>>     scp -r childhurd:/etc/ssh /etc/childhurd/etc
>
> Right, that can be convenient.  OTOH, from the perspective of having
> declarative OS configs, it’s not great because this aspect of the config
> are left out.  But maybe that’s an issue we can have if/when we
> generalize ‘secret-service-type’.

Ah, I see -- it could lead to "silent" failure/differences if
/etc/childhurd somehow disappears -- isn't re-created upon new install.
It makes sense to at least be less than silent, "fail early" is always
good.

>>>> (I guess we then also need to add a cuirass jobs for the Hurd?)
>>>
>>> Yes, or maybe just change ‘systems’ in the Cuirass specs for
>>> ‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
>>> which doesn’t sound like a great idea for now.
>>
>> I agree, not much sense in that yet.
>>
>>> Perhaps we can simply add a separate jobset pulling from ‘master’ but
>>> building only for i586-gnu and only the “core” package set?
>>
>> Hmm, why can't I find the definition of "core"?.  Anyway, It would be a
>> great first step to build (everything needef for) "hello", after that we
>> want to have/try "guile-3.0" and possibly "guix".
>
> Sure.  The “core” subset is defined in (gnu ci).

As discussed on IRC that could get an update.  Would you like to do
that, seems like an easy edit but I'm a bit unsure about the choices and
consequences there?

I think once the offloading works we'll want to try building guix; and
it could be nice if as many dependencies that "just happen to build" are
actually available.  It's waay to early to try to build everything but
we may want something in between.  Or add "guix" to core-packages,
maybe?  Just wondering out loud here...

Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.