GNU bug report logs - #41839
[PATCH 0/2] ci: Build Guix System images.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41839 in the body.
You can then email your comments to 41839 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH 0/2] ci: Build Guix System images.
Date: Sat, 13 Jun 2020 21:40:01 +0200

Hello,

Now that we have a new shiny new "image" record that contains all the required
informations to build or cross-build a Guix System image, it would be nice if
Cuirass could provide associated substitutes.

This serie allows to build a list of Guix System images in the CI. For now
this list only contains the Hurd barebones Guix System image.

Thanks,

Mathieu

Mathieu Othacehe (2):
  image: Move hurd image definition to a dedicated file.
  ci: Build Guix System images.

 gnu/ci.scm                 | 69 +++++++++++++++++-----------------
 gnu/image.scm              |  2 +
 gnu/local.mk               |  2 +
 gnu/system/image.scm       | 23 +++---------
 gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 121 insertions(+), 51 deletions(-)
 create mode 100644 gnu/system/images/hurd.scm

-- 
2.26.2

Message #8 received at 41839 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 41839 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH 2/2] ci: Build Guix System images.
Date: Sat, 13 Jun 2020 21:41:40 +0200

Build a list of Guix System images. For now, this list only contains the Hurd
barebones Guix System image.

* gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
(%u-boot-systems): remove unused variable,
(%guix-system-images): new variable
(qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
new "%guix-system-images" variable,
(system-test-jobs): adapt accordingly,
(hydra-jobs): ditto.
---
 gnu/ci.scm | 69 +++++++++++++++++++++++++++---------------------------
 1 file changed, 35 insertions(+), 34 deletions(-)

diff --git a/gnu/ci.scm b/gnu/ci.scm
index fa67168e22..2d49009810 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -53,6 +53,7 @@
   #:use-module (gnu system image)
   #:use-module (gnu system vm)
   #:use-module (gnu system install)
+  #:use-module (gnu system images hurd)
   #:use-module (gnu tests)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
@@ -185,18 +186,21 @@ SYSTEM."
               (remove (either from-32-to-64? same? pointless?)
                       %cross-targets)))
 
-(define %guixsd-supported-systems
-  '("x86_64-linux" "i686-linux" "armhf-linux"))
+;; Architectures that are able to build or cross-build Guix System images.
+;; This does not mean that other architectures are not supported, only that
+;; they are often not fast enough to support Guix System images building.
+(define %guix-system-supported-systems
+  '("x86_64-linux" "i686-linux"))
 
-(define %u-boot-systems
-  '("armhf-linux"))
+(define %guix-system-images
+  (list hurd-barebones-disk-image))
 
-(define (qemu-jobs store system)
-  "Return a list of jobs that build QEMU images for SYSTEM."
+(define (image-jobs store system)
+  "Return a list of jobs that build images for SYSTEM."
   (define (->alist drv)
     `((derivation . ,(derivation-file-name drv))
-      (description . "Stand-alone QEMU image of the GNU system")
-      (long-description . "This is a demo stand-alone QEMU image of the GNU
+      (description . "Stand-alone image of the GNU system")
+      (long-description . "This is a demo stand-alone image of the GNU
 system.")
       (license . ,(license-name gpl3+))
       (max-silent-time . 600)
@@ -211,29 +215,30 @@ system.")
                    (parameterize ((%graft? #f))
                      (->alist drv))))))
 
+  (define (build-image image)
+    (run-with-store store
+      (mbegin %store-monad
+        (set-guile-for-build (default-guile))
+        (lower-object (system-image image)))))
+
   (define MiB
     (expt 2 20))
 
-  (if (member system %guixsd-supported-systems)
-      (list (->job 'usb-image
-                   (run-with-store store
-                     (mbegin %store-monad
-                       (set-guile-for-build (default-guile))
-                       (lower-object
-                        (system-image
-                         (image
-                          (inherit efi-disk-image)
-                          (size (* 1500 MiB))
-                          (operating-system installation-os)))))))
-            (->job 'iso9660-image
-                   (run-with-store store
-                     (mbegin %store-monad
-                       (set-guile-for-build (default-guile))
-                       (lower-object
-                        (system-image
-                         (image
-                          (inherit iso9660-image)
-                          (operating-system installation-os))))))))
+  (if (member system %guix-system-supported-systems)
+      `(,(->job 'usb-image
+                (build-image
+                 (image
+                  (inherit efi-disk-image)
+                  (size (* 1500 MiB))
+                  (operating-system installation-os))))
+        ,(->job 'iso9660-image
+                (build-image
+                 (image
+                  (inherit iso9660-image)
+                  (operating-system installation-os))))
+        ,@(map (lambda (image)
+                 (->job (image-name image) (build-image image)))
+               %guix-system-images))
       '()))
 
 (define channel-build-system
@@ -305,11 +310,7 @@ system.")
                                 "." system))))
       (cons name (test->thunk test))))
 
-  (if (and (member system %guixsd-supported-systems)
-
-           ;; XXX: Our build farm has too few ARMv7 machines and they are very
-           ;; slow, so skip system tests there.
-           (not (string=? system "armhf-linux")))
+  (if (member system %guix-system-supported-systems)
       ;; Override the value of 'current-guix' used by system tests.  Using a
       ;; channel instance makes tests that rely on 'current-guix' less
       ;; expensive.  It also makes sure we get a valid Guix package when this
@@ -486,7 +487,7 @@ Return #f if no such checkout is found."
                                   (package->job store package
                                                 system))))
                        (append (filter-map job all)
-                               (qemu-jobs store system)
+                               (image-jobs store system)
                                (system-test-jobs store system
                                                  #:source source
                                                  #:commit commit)
-- 
2.26.2

Message #11 received at 41839 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 41839 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH 1/2] image: Move hurd image definition to a dedicated file.
Date: Sat, 13 Jun 2020 21:41:39 +0200

This moves hurd-disk-image to a dedicated file. It also defines a default
operating-system so that the image can be built standalone.

* gnu/system/images/hurd.scm: New file,
* gnu/local.mk (GNU_SYSTEM_MODULES): add it.
* gnu/image.scm (<image>)[name]: New field.
* gnu/system/image.scm (root-offset, root-label): Export it,
(hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
file above,
(find-image): adapt to avoid loop dependency.
---
 gnu/image.scm              |  2 +
 gnu/local.mk               |  2 +
 gnu/system/image.scm       | 23 +++---------
 gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 86 insertions(+), 17 deletions(-)
 create mode 100644 gnu/system/images/hurd.scm

diff --git a/gnu/image.scm b/gnu/image.scm
index 19b466527b..dc66f2c533 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -67,6 +67,8 @@
 (define-record-type* <image>
   image make-image
   image?
+  (name               image-name ;symbol
+                      (default #f))
   (format             image-format) ;symbol
   (target             image-target
                       (default #f))
diff --git a/gnu/local.mk b/gnu/local.mk
index 583274235b..eeeb276478 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -629,6 +629,8 @@ GNU_SYSTEM_MODULES =				\
   %D%/system/uuid.scm				\
   %D%/system/vm.scm				\
 						\
+  %D%/system/images/hurd.scm			\
+						\
   %D%/machine.scm				\
   %D%/machine/digital-ocean.scm			\
   %D%/machine/ssh.scm				\
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1bda25fd7f..cb8fe18bcc 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -53,10 +53,12 @@
   #:use-module (srfi srfi-35)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
-  #:export (esp-partition
+  #:export (root-offset
+            root-label
+
+            esp-partition
             root-partition
 
-            hurd-disk-image
             efi-disk-image
             iso9660-image
 
@@ -101,20 +103,6 @@
                      (list #:make-device-nodes
                            make-hurd-device-nodes)))))
 
-(define hurd-disk-image
-  (image
-   (format 'disk-image)
-   (target "i586-pc-gnu")
-   (partitions
-    (list (partition
-           (size 'guess)
-           (offset root-offset)
-           (label root-label)
-           (file-system "ext2")
-           (file-system-options '("-o" "hurd" "-O" "ext_attr"))
-           (flags '(boot))
-           (initializer hurd-initialize-root-partition))))))
-
 (define efi-disk-image
   (image
    (format 'disk-image)
@@ -569,7 +557,8 @@ addition of the <image> record."
     (_ (cond
         ((and target
               (hurd-triplet? target))
-         hurd-disk-image)
+         (module-ref (resolve-interface '(gnu system images hurd))
+                     'hurd-disk-image))
         (else
          efi-disk-image)))))
 
diff --git a/gnu/system/images/hurd.scm b/gnu/system/images/hurd.scm
new file mode 100644
index 0000000000..4417f03cc8
--- /dev/null
+++ b/gnu/system/images/hurd.scm
@@ -0,0 +1,76 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu system images hurd)
+  #:use-module (guix gexp)
+  #:use-module (gnu bootloader)
+  #:use-module (gnu bootloader grub)
+  #:use-module (gnu image)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu services)
+  #:use-module (gnu services ssh)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system hurd)
+  #:use-module (gnu system image)
+  #:export (hurd-barebones-os
+            hurd-disk-image
+            hurd-barebones-disk-image))
+
+(define hurd-barebones-os
+  (operating-system
+    (inherit %hurd-default-operating-system)
+    (bootloader (bootloader-configuration
+                 (bootloader grub-minimal-bootloader)
+                 (target "/dev/sdX")))
+    (file-systems (cons (file-system
+                          (device (file-system-label "my-root"))
+                          (mount-point "/")
+                          (type "ext2"))
+                        %base-file-systems))
+    (host-name "guixygnu")
+    (timezone "Europe/Amsterdam")
+    (packages (cons openssh-sans-x %base-packages/hurd))
+    (services (cons (service openssh-service-type
+                             (openssh-configuration
+                              (openssh openssh-sans-x)
+                              (use-pam? #f)
+                              (port-number 2222)
+                              (permit-root-login #t)
+                              (allow-empty-passwords? #t)
+                              (password-authentication? #t)))
+               %base-services/hurd))))
+
+(define hurd-disk-image
+  (image
+   (format 'disk-image)
+   (target "i586-pc-gnu")
+   (partitions
+    (list (partition
+           (size 'guess)
+           (offset root-offset)
+           (label root-label)
+           (file-system "ext2")
+           (flags '(boot))
+           (initializer (gexp initialize-root-partition)))))))
+
+(define hurd-barebones-disk-image
+  (image
+   (inherit hurd-disk-image)
+   (name 'hurd-barebones-disk-image)
+   (operating-system hurd-barebones-os)))
-- 
2.26.2

Message #14 received at 41839 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41839 <at> debbugs.gnu.org
Subject: Re: [bug#41839] [PATCH 1/2] image: Move hurd image definition to a
 dedicated file.
Date: Tue, 23 Jun 2020 23:47:18 +0200

Hi!

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> This moves hurd-disk-image to a dedicated file. It also defines a default
> operating-system so that the image can be built standalone.
>
> * gnu/system/images/hurd.scm: New file,
> * gnu/local.mk (GNU_SYSTEM_MODULES): add it.
> * gnu/image.scm (<image>)[name]: New field.
> * gnu/system/image.scm (root-offset, root-label): Export it,
> (hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
> file above,
> (find-image): adapt to avoid loop dependency.

LGTM!

It would be good to check by running “make as-derivation” if gnu/image/*
gets included.

Ludo’.

Message #17 received at 41839 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 41839 <at> debbugs.gnu.org
Subject: Re: [bug#41839] [PATCH 2/2] ci: Build Guix System images.
Date: Tue, 23 Jun 2020 23:48:56 +0200

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> Build a list of Guix System images. For now, this list only contains the Hurd
> barebones Guix System image.
>
> * gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
> (%u-boot-systems): remove unused variable,
> (%guix-system-images): new variable
> (qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
> new "%guix-system-images" variable,
> (system-test-jobs): adapt accordingly,
> (hydra-jobs): ditto.

Cool.

> +  (if (member system %guix-system-supported-systems)
> +      `(,(->job 'usb-image
> +                (build-image
> +                 (image
> +                  (inherit efi-disk-image)
> +                  (size (* 1500 MiB))
> +                  (operating-system installation-os))))
> +        ,(->job 'iso9660-image
> +                (build-image
> +                 (image
> +                  (inherit iso9660-image)
> +                  (operating-system installation-os))))
> +        ,@(map (lambda (image)
> +                 (->job (image-name image) (build-image image)))
> +               %guix-system-images))
>        '()))

I think the effect is that we’ll also cross-build GNU/Hurd images from
i686-linux, right?  It would be nice to avoid that and only cross-build
from x86_64.

Otherwise LGTM, thank you!

Ludo’.

Message #20 received at 41839 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 41839 <at> debbugs.gnu.org
Subject: Re: [bug#41839] [PATCH 1/2] image: Move hurd image definition to a
 dedicated file.
Date: Thu, 25 Jun 2020 11:54:46 +0200

Hey,

> It would be good to check by running “make as-derivation” if gnu/image/*
> gets included.

Yep, its included. Thanks for reviewing!

Mathieu

Message #25 received at 41839-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 41839-done <at> debbugs.gnu.org
Subject: Re: [bug#41839] [PATCH 2/2] ci: Build Guix System images.
Date: Thu, 25 Jun 2020 12:02:30 +0200

Hey,

> I think the effect is that we’ll also cross-build GNU/Hurd images from
> i686-linux, right?  It would be nice to avoid that and only cross-build
> from x86_64.

Yes, I fixed that (in two tries actually :p). Now working on adding this
image to the website.

Thanks,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.