GNU bug report logs - #64708
[PATCH 0/8] Fix cross build and native build for the Hurd.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 64708 in the body.
You can then email your comments to 64708 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: guix-patches <at> gnu.org
Subject: [PATCH 0/8] Fix cross build and native build for the Hurd.
Date: Tue, 18 Jul 2023 15:11:52 +0200

Hi!

So...updating mig using git-fetch created a circular dependency because
mig-boot0 re-used mig's source, and we cannot use git in commencement.

This patch series now uses a hack by Ludo (thanks!) as suggested on IRC
<https://logs.guix.gnu.org/guix/2023-07-17.log#233051> to not directly use
generated cgit tarballs from savannah.

The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
updated so that native builds work again.  Because upstream does not produce
release tarballs we now also need autoconf and automake in commencement.

Note that building natively won't get very far, as for many packages (guile,
coreutils, grep, findutils, diffutils, gettext-minimal, m4, openssl, python,
...) the `check' phase will fail.  See the `hurd-team' branch for updates on
this.

Now that mig-boot0 uses its own source definition, we can safely re-apply the
mig update.

Greetings,
Janneke

Janneke Nieuwenhuizen (6):
  gnu: commencement: Add autoconf-boot0.
  gnu: commencement: Add automake-boot0.
  gnu: commencement: gnumach-headers-boot0: Update to 1.8+git20221224.
  gnu: commencement: mig-boot0: Update to 1.8+git20230520.
  gnu: commencement: hurd-headers-boot0: Update to 0.9.git20230216.
  gnu: commencement: hurd-minimal-boot0: Update to 0.9.git20230216.

Josselin Poiret (1):
  gnu: mig: Update to 1.8+git20230520.

Ludovic Courtès (1):
  gnu: commencement: Add git-fetch-from-tarball utility.

 gnu/packages/commencement.scm | 200 ++++++++++++++++++++++++++--------
 gnu/packages/hurd.scm         |  20 ++--
 2 files changed, 160 insertions(+), 60 deletions(-)


base-commit: 94f83088d237ecd032c411bb94e2d647a2931990
-- 
2.40.1

Message #8 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Subject: [PATCH 1/8] gnu: commencement: Add autoconf-boot0.
Date: Tue, 18 Jul 2023 15:14:57 +0200

* gnu/packages/commencement.scm (autoconf-boot0): New variable.
---
 gnu/packages/commencement.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 9bd0c113c2..3ccd51ac52 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -33,6 +33,7 @@
 (define-module (gnu packages commencement)
   #:use-module (gnu packages)
   #:use-module (gnu packages bootstrap)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages c)
@@ -2602,6 +2603,15 @@ (define with-boot0
   (package-with-explicit-inputs %boot0-inputs
                                 %bootstrap-guile))
 
+(define autoconf-boot0
+  (with-boot0
+   (package
+     (inherit autoconf)
+     (name "autoconf-boot0")
+     (native-inputs (list m4-boot0 perl-boot0))
+     (inputs '())
+     (arguments (list #:tests? #f)))))
+
 (define gnumach-headers-boot0
   (with-boot0
    (package
-- 
2.40.1

Message #11 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Subject: [PATCH 2/8] gnu: commencement: Add automake-boot0.
Date: Tue, 18 Jul 2023 15:14:58 +0200

* gnu/packages/commencement.scm (automake-boot0): New variable.
---
 gnu/packages/commencement.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 3ccd51ac52..d0603e62c8 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2612,6 +2612,19 @@ (define autoconf-boot0
      (inputs '())
      (arguments (list #:tests? #f)))))
 
+(define automake-boot0
+  (with-boot0
+   (package
+     (inherit automake)
+     (name "automake-boot0")
+     (source (origin
+               (inherit (package-source automake))
+               (patches '())))        ;test are skipped anyway
+     (native-inputs (list autoconf-boot0 m4-boot0 perl-boot0))
+     (inputs '())
+     (arguments
+      (list #:tests? #f)))))
+
 (define gnumach-headers-boot0
   (with-boot0
    (package
-- 
2.40.1

Message #14 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Subject: [PATCH 4/8] gnu: commencement: gnumach-headers-boot0: Update to
 1.8+git20221224.
Date: Tue, 18 Jul 2023 15:15:00 +0200

* gnu/packages/commencement.scm (gnumach-headers-boot0)[name]: New field; give
proper name.
[source]: Update to 1.8+git20221224, using git-fetch-from-tarball.
[native-inputs]: Add autoconf-boot0, automake-boot0, texinfo-boot0.
[arguments]: Add 'patch-compat' phase to allow building hurd-minimal etc, with
our out-of-date bootstsrap binaries.
---
 gnu/packages/commencement.scm | 46 ++++++++++++++++++++++++++---------
 1 file changed, 34 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 1d69c2a80f..b15a71feb3 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2660,18 +2660,40 @@ (define gnumach-headers-boot0
   (with-boot0
    (package
      (inherit gnumach-headers)
-     (version "1.8-116-g28b53508")
-     (source (bootstrap-origin
-              (origin
-                (method url-fetch)
-                (uri (list (string-append "mirror://gnu/guix/mirror/gnumach-"
-                                          version ".tar.gz")
-                           (string-append "https://lilypond.org/janneke/hurd/"
-                                          "gnumach-" version ".tar.gz")))
-                (sha256
-                 (base32
-                  "006i0zgwy81vxarpfm12vip4q6i5mgmi5mmy5ldvxp5hx9h3l0zg")))))
-     (native-inputs '()))))
+     (name "gnumach-headers-boot0")
+     (version "1.8+git20221224")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
+                  "gnumach-" version ".tar.gz"))
+            (sha256
+             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32
+          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
+     (native-inputs (list autoconf-boot0 automake-boot0 texinfo-boot0))
+     (arguments
+      (substitute-keyword-arguments (package-arguments gnumach-headers)
+        ((#:phases phases)
+         #~(modify-phases #$phases
+             (add-after 'unpack 'patch-compat
+               (lambda _
+                 (substitute* '("include/device/device_types.h"
+                                "include/mach_debug/slab_info.h"
+                                "include/mach_debug/vm_info.h")
+                   (("rpc_vm_size_t") "unsigned int")
+                   (("rpc_vm_offset_t") "unsigned int")
+                   (("rpc_long_natural_t") "unsigned long")
+                   (("long_natural_t") "unsigned long")))))))))))
 
 (define mig-boot0
   (let* ((mig (package
-- 
2.40.1

Message #17 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Subject: [PATCH 5/8] gnu: commencement: mig-boot0: Update to 1.8+git20230520.
Date: Tue, 18 Jul 2023 15:15:01 +0200

* gnu/packages/commencement.scm (mig-boot0): Remove redundant
outer let.
[name]: New field, give proper name.
[version]: New field
[source]: Update to 1.8+git20230520, using git-fetch-from-tarball.
[native-inputs]: Add autoconf-boot0, automake-boot0.
[inputs]: Add gnumach-headers-boot0.
---
 gnu/packages/commencement.scm | 42 ++++++++++++++++++++++++++---------
 1 file changed, 31 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index b15a71feb3..05a6badea0 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2696,17 +2696,37 @@ (define gnumach-headers-boot0
                    (("long_natural_t") "unsigned long")))))))))))
 
 (define mig-boot0
-  (let* ((mig (package
-                 (inherit (package-with-bootstrap-guile mig))
-                 (native-inputs `(("bison" ,bison-boot0)
-                                  ("flex" ,flex-boot0)))
-                 (inputs `(("flex" ,flex-boot0)))
-                 (arguments
-                  ;; TODO: On next rebuild cycle, reuse phases from 'mig'.
-                  `(#:configure-flags
-                    `(,(string-append "LDFLAGS=-Wl,-rpath="
-                                      (assoc-ref %build-inputs "flex") "/lib/")))))))
-    (with-boot0 mig)))
+  (with-boot0
+   (package
+     (inherit mig)
+     (name "mig-boot0")
+     (version "1.8+git20230520")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/mig.git/snapshot/"
+                  "mig-" version ".tar.gz"))
+            (sha256
+             (base32 "1l1vfm4wap5yxylv91wssgpy7fnq22wp3akgd5nv995kychfa9jy")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/mig.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32 "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))
+     (native-inputs (list autoconf-boot0 automake-boot0 bison-boot0 flex-boot0
+                          gnumach-headers-boot0))
+     (inputs (list flex-boot0 gnumach-headers-boot0))
+     (arguments
+      (list
+       #:configure-flags
+       #~(list (string-append "LDFLAGS=-Wl,-rpath="
+                              #$(this-package-native-input "flex")
+                              "/lib/")))))))
 
 (define hurd-version-boot0 "0.9-229-ga1efcee8")
 (define hurd-source-boot0
-- 
2.40.1

Message #20 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Subject: [PATCH 6/8] gnu: commencement: hurd-headers-boot0: Update to
 0.9.git20230216.
Date: Tue, 18 Jul 2023 15:15:02 +0200

* gnu/packages/commencement.scm (hurd-headers-boot0): Remove redundant outer
let and package-with-bootstrap-guile.
[name]: New field; give proper name.
[version]: New field.
[source]: Update to 0.9.git20230216, using git-fetch-from-tarball.
[inputs]: Add autoconf-boot0, automake-boot0, gnumach-headers-boot0.
---
 gnu/packages/commencement.scm | 31 +++++++++++++++++++++++++------
 1 file changed, 25 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 05a6badea0..86246c3b08 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2742,12 +2742,31 @@ (define hurd-source-boot0
         "0bq2q2jisxcy0kgcm6rz0z2fddwxxm7azsama7li28a2m08kdpzy")))))
 
 (define hurd-headers-boot0
-  (let ((hurd-headers (package (inherit hurd-headers)
-                               (version hurd-version-boot0)
-                               (source hurd-source-boot0)
-                               (native-inputs `(("mig" ,mig-boot0)))
-                               (inputs '()))))
-    (with-boot0 (package-with-bootstrap-guile hurd-headers))))
+  (with-boot0
+   (package
+     (inherit hurd-headers)
+     (name "hurd-headers-boot0")
+     (version "0.9.git20230216")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/hurd.git/snapshot/"
+                  "hurd-v" version ".tar.gz"))
+            (sha256
+             (base32 "1f75nlkcl00dqnnrbrj1frvzs2qibfpygj3gwywqi85aldjl48y7")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/hurd.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32 "0jm1dnqkx4kdwmby0z5w0yqp9m5qp4hbxd4jxlyhiqm8nkw9mkvv"))))
+     (native-inputs
+      (list autoconf-boot0 automake-boot0 mig-boot0))
+     (inputs '()))))
 
 (define hurd-minimal-boot0
   (let ((hurd-minimal (package (inherit hurd-minimal)
-- 
2.40.1

Message #23 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 3/8] gnu: commencement: Add git-fetch-from-tarball utility.
Date: Tue, 18 Jul 2023 15:14:59 +0200

From: Ludovic Courtès <ludo <at> gnu.org>

* gnu/packages/commencement.scm (git-fetch-from-tarball): New procedure.
---
 gnu/packages/commencement.scm | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index d0603e62c8..1d69c2a80f 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -57,7 +57,10 @@ (define-module (gnu packages commencement)
   #:use-module (gnu packages xml)
   #:use-module (guix gexp)
   #:use-module (guix packages)
+  #:use-module ((guix store) #:select (%store-monad))
+  #:use-module (guix monads)
   #:use-module (guix download)
+  #:use-module ((guix git-download) #:select (git-reference git-file-name))
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
   #:use-module ((guix licenses) #:prefix license:)
@@ -90,6 +93,34 @@ (define-module (gnu packages commencement)
 ;;;
 ;;; Code:
 
+(define* (git-fetch-from-tarball tarball)
+  (lambda* (url hash-algo hash
+                #:optional name
+                #:key (system (%current-system))
+                (guile %bootstrap-guile))
+    (mlet %store-monad ((guile (package->derivation guile system)))
+      (gexp->derivation
+       (or name "git-checkout")
+       (with-imported-modules '((guix build utils))
+         #~(begin
+             (use-modules (guix build utils)
+                          (ice-9 ftw)
+                          (ice-9 match))
+             (setenv "PATH"
+                     #+(file-append %bootstrap-coreutils&co "/bin"))
+             (invoke (string-append #+tar "/bin/tar")
+                     "xf" #$tarball)
+             (match (scandir ".")
+               (("." ".." directory)
+                (copy-recursively directory #$output)))))
+       #:recursive? #t
+       #:hash-algo hash-algo
+       #:hash hash
+       #:system system
+       #:guile-for-build guile
+       #:graft? #f
+       #:local-build? #t))))
+
 (define bootar
   (package
     (name "bootar")
-- 
2.40.1

Message #26 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Subject: [PATCH 7/8] gnu: commencement: hurd-minimal-boot0: Update to
 0.9.git20230216.
Date: Tue, 18 Jul 2023 15:15:03 +0200

* gnu/packages/commencement.scm (hurd-minimal-boot0): Remove redundant let
and package-with-bootstrap-guile.
[name]: New field; give proper name.
[version]: New field.
[source]: Update to 0.9.git20230216, using hurd-headers-boot0' source.
[inputs]: Add autoconf-boot0, automake-boot0, gnumach-headers-boot0.
[native-inputs]: Add autoconf-boot0, automake-boot0, and
gnumach-headers-boot0.
* gnu/packages/commencement.scm (hurd-version-boot0, hurd-source-boot0):
Remove.
---
 gnu/packages/commencement.scm | 27 ++++++++-------------------
 1 file changed, 8 insertions(+), 19 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 86246c3b08..7e988129a5 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2728,19 +2728,6 @@ (define mig-boot0
                               #$(this-package-native-input "flex")
                               "/lib/")))))))
 
-(define hurd-version-boot0 "0.9-229-ga1efcee8")
-(define hurd-source-boot0
-  (let ((version hurd-version-boot0))
-    (origin
-      (method url-fetch)
-      (uri (list (string-append "mirror://gnu/guix/mirror/hurd-v"
-                                version ".tar.gz")
-                 (string-append "https://lilypond.org/janneke/hurd/"
-                                "hurd-v" version ".tar.gz")))
-      (sha256
-       (base32
-        "0bq2q2jisxcy0kgcm6rz0z2fddwxxm7azsama7li28a2m08kdpzy")))))
-
 (define hurd-headers-boot0
   (with-boot0
    (package
@@ -2769,12 +2756,14 @@ (define hurd-headers-boot0
      (inputs '()))))
 
 (define hurd-minimal-boot0
-  (let ((hurd-minimal (package (inherit hurd-minimal)
-                               (version hurd-version-boot0)
-                               (source hurd-source-boot0)
-                               (native-inputs `(("mig" ,mig-boot0)))
-                               (inputs '()))))
-    (with-boot0 (package-with-bootstrap-guile hurd-minimal))))
+  (with-boot0
+   (package
+     (inherit hurd-minimal)
+     (name "hurd-minimal-boot0")
+     (source (package-source hurd-headers-boot0))
+     (native-inputs
+      (list autoconf-boot0 automake-boot0 gnumach-headers-boot0 mig-boot0))
+     (inputs (list gnumach-headers-boot0)))))
 
 (define/system-dependent hurd-core-headers-boot0
   ;; Return the Hurd and Mach headers as well as initial Hurd libraries for
-- 
2.40.1

Message #29 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: 64708 <at> debbugs.gnu.org
Cc: Josselin Poiret <dev <at> jpoiret.xyz>
Subject: [PATCH 8/8] gnu: mig: Update to 1.8+git20230520.
Date: Tue, 18 Jul 2023 15:15:04 +0200

From: Josselin Poiret <dev <at> jpoiret.xyz>

Now that mig-boot0 has been updated not to use mig's source, we can
safely update mig.

* gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.
---
 gnu/packages/hurd.scm | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/hurd.scm b/gnu/packages/hurd.scm
index 3785c47750..4064a9e2e8 100644
--- a/gnu/packages/hurd.scm
+++ b/gnu/packages/hurd.scm
@@ -91,24 +91,20 @@ (define-public gnumach-headers
 (define-public mig
   (package
     (name "mig")
-    (version "1.8+git20220827")
+    (version "1.8+git20230520")
     (source (origin
-              (method url-fetch)
-              ;; XXX: Version 2.35 of glibc can only be built with an
-              ;; unreleased version of MiG:
-              ;; <https://lists.gnu.org/archive/html/bug-hurd/2023-03/msg00025.html>.
-              ;; It cannot be fetched from Git though, as the extra dependency
-              ;; on Autoconf/Automake would complicate bootstrapping.
-              (uri (string-append "mirror://gnu/guix/mirror/mig-"
-                                  version ".tar.gz"))
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://git.savannah.gnu.org/git/hurd/mig.git")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
               (sha256
                (base32
-                "163d37s9lscd6zxyfng421m9nl857464mgjj90xsrcl5ykbng5p2"))
-              (patches (search-patches "mig-cpu.h-generation.patch"))))
+                "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))
     (build-system gnu-build-system)
     ;; Flex is needed both at build and run time.
     (inputs (list gnumach-headers flex))
-    (native-inputs (list flex bison))
+    (native-inputs (list autoconf automake flex bison))
     (arguments
      (list #:tests? #f
            #:phases
-- 
2.40.1

Message #32 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:13:58 +0200

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> From: Ludovic Courtès <ludo <at> gnu.org>
>
> * gnu/packages/commencement.scm (git-fetch-from-tarball): New procedure.

I guess this one is very weird so it deserves an explanation.

The problems we’re trying to solve are:

  1. We cannot use ‘git-fetch’ at this point because that’d introduce a
     circular dependency: <https://issues.guix.gnu.org/63331>.

  2. Downloading a cgit-generated tarball is unsafe: the tarball might
     be modified when cgit is updated or whatever, leading to a hash
     mismatch that we wouldn’t be able to resolve (well,
     disarchive.guix.gnu.org might be able to add an entry for that
     tarball, but then there’s currently no fallback in case of hash
     mismatch: <https://issues.guix.gnu.org/28659>).

So we need to obtain our Git checkouts by some other mean.  There are
further complications:

  3. We could use (web client), but without (gnutls) since it’s
     unavailable at this stage.

  4. We cannot use (guix swh) as is because Guile-JSON doesn’t build
     with Guile 2.0, which is what we have here.

The trick below is the only way I could think of to reconcile these
issues: it essentially stacks two fixed-output derivation.  The first
one downloads that auto-generated tarball (bad!), and the second one
extracts the contents of that tarball using ‘tar’ from
‘%bootstrap-coreutils&co’.  The output of the second one is identical to
what ‘git-fetch’ would give us (or ‘swh-download’).

That makes it more robust.  It also allows ‘guix lint -c archival’ to do
its work and similarly (well, almost) for ‘build-package-metadata.scm’,
the program that builds <https://guix.gnu.org/sources.json>, which SWH
periodically ingests.

Pfeww, long story.

> +(define* (git-fetch-from-tarball tarball)

Maybe add this docstring:

    "Return an <origin> method equivalent to 'git-fetch', except that it
  fetches the checkout from TARBALL, a tarball containing said
  checkout.

  The purpose of this procedure is to work around bootstrapping issues:
  'git-fetch' depends on Git, which is much higher in the dependency
  graph."

> +             (setenv "PATH"
> +                     #+(file-append %bootstrap-coreutils&co "/bin"))
> +             (invoke (string-append #+tar "/bin/tar")
> +                     "xf" #$tarball)

This should be: (invoke "tar" "xf" #$tarball).
Otherwise we’d get a cyclic dependency with ‘tar’.

(I didn’t notice because there’s a bug where ‘-s’ is ignored here:

  ./pre-inst-env guix build -e '(@@ (gnu packages commencement) mig-boot0)' \
    -s i586-gnu -Sd

)

OK with these changes!

Ludo’.

Message #35 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:16:37 +0200

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> +     (name "gnumach-headers-boot0")
> +     (version "1.8+git20221224")
> +     (source
> +      (origin
> +        (method
> +         (git-fetch-from-tarball
> +          (origin
> +            (method url-fetch)
> +            (uri (string-append
> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
> +                  "gnumach-" version ".tar.gz"))
> +            (sha256
> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
> +        (uri (git-reference
> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
> +              (commit version)))
> +        (file-name (git-file-name name version))
> +        (sha256
> +         (base32
> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))

Since this is the same revision as in hurd.scm, we can reduce
duplication like so:

  (origin
    (inherit (package-source gnumach-headers))
    (method (git-fetch-from-tarball …)))

Otherwise LGTM.

Message #38 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:17:20 +0200

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> +     (source
> +      (origin
> +        (method
> +         (git-fetch-from-tarball
> +          (origin
> +            (method url-fetch)
> +            (uri (string-append
> +                  "https://git.savannah.gnu.org/cgit/hurd/mig.git/snapshot/"
> +                  "mig-" version ".tar.gz"))
> +            (sha256
> +             (base32 "1l1vfm4wap5yxylv91wssgpy7fnq22wp3akgd5nv995kychfa9jy")))))
> +        (uri (git-reference
> +              (url "https://git.savannah.gnu.org/git/hurd/mig.git")
> +              (commit version)))
> +        (file-name (git-file-name name version))
> +        (sha256
> +         (base32 "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))

Likewise: (origin (inherit (package-source mig)) …)

Message #41 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org, Josselin Poiret <dev <at> jpoiret.xyz>
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:20:34 +0200

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> From: Josselin Poiret <dev <at> jpoiret.xyz>
>
> Now that mig-boot0 has been updated not to use mig's source, we can
> safely update mig.
>
> * gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.

This one could be avoided, at least for now, due to
4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
on ftp.gnu.org.  (We could also avoid the corresponding change in
‘mig-boot0’.)

But really, either way is fine, as long as both cross-compilation and
native compilation work after that.  :-)

Ludo’.

Message #44 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:23:30 +0200

Hi,

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

> So...updating mig using git-fetch created a circular dependency because
> mig-boot0 re-used mig's source, and we cannot use git in commencement.
>
> This patch series now uses a hack by Ludo (thanks!) as suggested on IRC
> <https://logs.guix.gnu.org/guix/2023-07-17.log#233051> to not directly use
> generated cgit tarballs from savannah.
>
> The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
> updated so that native builds work again.  Because upstream does not produce
> release tarballs we now also need autoconf and automake in commencement.

I sent minor suggestions, but overall LGTM!

As of 4e11cac6be68a674c16965dc74e7da54cd3168df, we have
cross-compilation to i586-pc-gnu working again.  After this series, we
should still have that + ‘-s i586-gnu’ (native compilation) working
(modulo test failures fixed by <https://issues.guix.gnu.org/64711>, of
course).  I haven’t checked whether this is the case but that’s the
idea, right?  :-)

Thanks again for this!

Ludo’.

Message #47 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:31:19 +0200

Ludovic Courtès writes:

Hi!

> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> +     (name "gnumach-headers-boot0")
>> +     (version "1.8+git20221224")
>> +     (source
>> +      (origin
>> +        (method
>> +         (git-fetch-from-tarball
>> +          (origin
>> +            (method url-fetch)
>> +            (uri (string-append
>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>> +                  "gnumach-" version ".tar.gz"))
>> +            (sha256
>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>> +        (uri (git-reference
>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>> +              (commit version)))
>> +        (file-name (git-file-name name version))
>> +        (sha256
>> +         (base32
>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>
> Since this is the same revision as in hurd.scm, we can reduce
> duplication like so:
>
>   (origin
>     (inherit (package-source gnumach-headers))
>     (method (git-fetch-from-tarball …)))

Eh...no?  Because gnumach-headers uses method git-fetch now!

> Otherwise LGTM.

Thanks!

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #50 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org, Josselin Poiret <dev <at> jpoiret.xyz>
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Tue, 18 Jul 2023 23:38:45 +0200

Ludovic Courtès writes:

Hi!

> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> From: Josselin Poiret <dev <at> jpoiret.xyz>
>>
>> Now that mig-boot0 has been updated not to use mig's source, we can
>> safely update mig.
>>
>> * gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.
>
> This one could be avoided, at least for now, due to
> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
> on ftp.gnu.org.  (We could also avoid the corresponding change in
> ‘mig-boot0’.)

I believe we'd rather re-instate this change to build mig fram git
so that we can do --with-commit to track all the neat things that
upstream is doing, right Josselin?

> But really, either way is fine, as long as both cross-compilation and
> native compilation work after that.  :-)

Sure!

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #53 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Wed, 19 Jul 2023 00:21:03 +0200

Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:

>> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>>
>>> +     (name "gnumach-headers-boot0")
>>> +     (version "1.8+git20221224")
>>> +     (source
>>> +      (origin
>>> +        (method
>>> +         (git-fetch-from-tarball
>>> +          (origin
>>> +            (method url-fetch)
>>> +            (uri (string-append
>>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>>> +                  "gnumach-" version ".tar.gz"))
>>> +            (sha256
>>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>>> +        (uri (git-reference
>>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>>> +              (commit version)))
>>> +        (file-name (git-file-name name version))
>>> +        (sha256
>>> +         (base32
>>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>>
>> Since this is the same revision as in hurd.scm, we can reduce
>> duplication like so:
>>
>>   (origin
>>     (inherit (package-source gnumach-headers))
>>     (method (git-fetch-from-tarball …)))
>
> Eh...no?  Because gnumach-headers uses method git-fetch now!

That’s the whole point: here we override the method with one that’s
close to the graph root, but the <git-reference> and hash are the same
as the one from ‘gnumach-headers’.  Inheriting makes the intent
explicit.

Which is pretty cool no?

Ludo’.

Message #56 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Wed, 19 Jul 2023 07:00:06 +0200

Ludovic Courtès writes:

> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>>> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>>>
>>>> +     (name "gnumach-headers-boot0")
>>>> +     (version "1.8+git20221224")
>>>> +     (source
>>>> +      (origin
>>>> +        (method
>>>> +         (git-fetch-from-tarball
>>>> +          (origin
>>>> +            (method url-fetch)
>>>> +            (uri (string-append
>>>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>>>> +                  "gnumach-" version ".tar.gz"))
>>>> +            (sha256
>>>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>>>> +        (uri (git-reference
>>>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>>>> +              (commit version)))
>>>> +        (file-name (git-file-name name version))
>>>> +        (sha256
>>>> +         (base32
>>>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>>>
>>> Since this is the same revision as in hurd.scm, we can reduce
>>> duplication like so:
>>>
>>>   (origin
>>>     (inherit (package-source gnumach-headers))
>>>     (method (git-fetch-from-tarball …)))

Ah, I didn't really read this properly; pretty cool!

>> Eh...no?  Because gnumach-headers uses method git-fetch now!
>
> That’s the whole point: here we override the method with one that’s
> close to the graph root, but the <git-reference> and hash are the same
> as the one from ‘gnumach-headers’.  Inheriting makes the intent
> explicit.

I still wonder a bit if it's a good idea to have commencement packages /
this commencement package inherit from hurd.scm; it means a world
rebuild whenever hurd.scm is updated (unless you put something back like
the verbose origin declaration above).

I'll take with your suggestion and go for the less code duplication
option right now, we can change it again when the need arises.

> Which is pretty cool no?

Yeah!  Sorry for the confusion :)

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #61 received at 64708-done <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64708-done <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Wed, 19 Jul 2023 07:53:24 +0200

Ludovic Courtès writes:

Hello,

> Janneke Nieuwenhuizen <janneke <at> gnu.org> skribis:
>
>> The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
>> updated so that native builds work again.  Because upstream does not produce
>> release tarballs we now also need autoconf and automake in commencement.
>
> I sent minor suggestions, but overall LGTM!
>
> As of 4e11cac6be68a674c16965dc74e7da54cd3168df, we have
> cross-compilation to i586-pc-gnu working again.

That's right, someone beat me to it ;-) (thanks!!)

> After this series, we should still have that + ‘-s i586-gnu’ (native
> compilation) working (modulo test failures fixed by
> <https://issues.guix.gnu.org/64711>, of course).  I haven’t checked
> whether this is the case but that’s the idea, right?  :-)

Certainly!

> Thanks again for this!

That's always good to hear.

Pushed to master as 8831d3ba9f799b7342ab466d625097f1982513a9

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Message #64 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Josselin Poiret <dev <at> jpoiret.xyz>
To: Janneke Nieuwenhuizen <janneke <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Wed, 19 Jul 2023 11:08:03 +0200

[Message part 1 (text/plain, inline)]

Hi Janneke and Ludo,

Janneke Nieuwenhuizen <janneke <at> gnu.org> writes:

> Ludovic Courtès writes:
>> This one could be avoided, at least for now, due to
>> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
>> on ftp.gnu.org.  (We could also avoid the corresponding change in
>> ‘mig-boot0’.)
>
> I believe we'd rather re-instate this change to build mig fram git
> so that we can do --with-commit to track all the neat things that
> upstream is doing, right Josselin?

Yes, that was the point, we don't want to upload a new tarball to the
ftp all the time (esp. because I can't do it myself :) ).

Best,
-- 
Josselin Poiret

[signature.asc (application/pgp-signature, inline)]

Message #67 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Josselin Poiret <dev <at> jpoiret.xyz>
To: Ludovic Courtès <ludo <at> gnu.org>, Janneke Nieuwenhuizen
 <janneke <at> gnu.org>
Cc: 64708 <at> debbugs.gnu.org
Subject: Re: [bug#64708] [PATCH 0/8] Fix cross build and native build for
 the Hurd.
Date: Wed, 19 Jul 2023 11:11:06 +0200

[Message part 1 (text/plain, inline)]

Hi Ludo and Janneke.

Ludovic Courtès <ludo <at> gnu.org> writes:

> The trick below is the only way I could think of to reconcile these
> issues: it essentially stacks two fixed-output derivation.  The first
> one downloads that auto-generated tarball (bad!), and the second one
> extracts the contents of that tarball using ‘tar’ from
> ‘%bootstrap-coreutils&co’.  The output of the second one is identical to
> what ‘git-fetch’ would give us (or ‘swh-download’).
>
> That makes it more robust.  It also allows ‘guix lint -c archival’ to do
> its work and similarly (well, almost) for ‘build-package-metadata.scm’,
> the program that builds <https://guix.gnu.org/sources.json>, which SWH
> periodically ingests.

I don't 100% understand what this resolves though, since this still
depends on the unstable tarball in the end!  In both cases, we'd rely on
CI to have the stable copy, or SWH providing it, since we wouldn't be
able to rebuild it.

Best,
-- 
Josselin Poiret

[signature.asc (application/pgp-signature, inline)]

Message #70 received at 64708 <at> debbugs.gnu.org (full text, mbox):

From: Janneke Nieuwenhuizen <janneke <at> gnu.org>
To: Josselin Poiret <dev <at> jpoiret.xyz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 64708 <at> debbugs.gnu.org
Subject: Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the
 Hurd.
Date: Wed, 19 Jul 2023 11:32:12 +0200

Josselin Poiret writes:

Hi Josselin and Ludo,

> Janneke Nieuwenhuizen <janneke <at> gnu.org> writes:
>
>> Ludovic Courtès writes:
>>> This one could be avoided, at least for now, due to
>>> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
>>> on ftp.gnu.org.  (We could also avoid the corresponding change in
>>> ‘mig-boot0’.)
>>
>> I believe we'd rather re-instate this change to build mig fram git
>> so that we can do --with-commit to track all the neat things that
>> upstream is doing, right Josselin?
>
> Yes, that was the point, we don't want to upload a new tarball to the
> ftp all the time (esp. because I can't do it myself :) ).

So, that's what I've done, I think we're OK!

Thanks for the confirmation,
Janneke

-- 
Janneke Nieuwenhuizen <janneke <at> gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.