Package: guix-patches;
Reported by: Pierre Langlois <pierre.langlois <at> gmx.com>
Date: Sun, 29 Aug 2021 23:32:01 UTC
Severity: normal
Tags: patch
Done: Pierre Langlois <pierre.langlois <at> gmx.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 50261 in the body.
You can then email your comments to 50261 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
guix-patches <at> gnu.org:bug#50261; Package guix-patches.
(Sun, 29 Aug 2021 23:32:01 GMT) Full text and rfc822 format available.Pierre Langlois <pierre.langlois <at> gmx.com>:guix-patches <at> gnu.org.
(Sun, 29 Aug 2021 23:32:01 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Pierre Langlois <pierre.langlois <at> gmx.com> To: guix-patches <at> gnu.org Subject: [PATCH] gnu: node: Enable cross-compilation. Date: Sun, 29 Aug 2021 23:06:00 +0100
[Message part 1 (text/plain, inline)]
Hi Guix! While working on tree-sitter I realized our node packages didn't cross-compile [0] so I thought I'd give it a go, here's a patch! Note that fundamentally, because of how the V8 JavaScript engine bootstraps itself, cross-compiling only works for platforms with the same bitness. So a 32-bit binary cannot be produced on a 64-bit system, for example on x86_64: --8<---------------cut here---------------start------------->8--- # Build OK. $ guix build --target=aarch64-linux-gnu node # Will fail... $ guix build --target=arm-linux-gnueabihf node # ... unless the host is 32-bit. $ guix build --system=i686-linux --target=arm-linux-gnueabihf node --8<---------------cut here---------------end--------------->8--- I'm not sure there's any way to express this restriction in the package definition, it would be good to fail in a useful way, any thoughts? I suspect a mechanism for this isn't really worth the effort, I don't know of any other packages like this. As a drive-by, the esbuild package didn't build on 32-bit systems so I attached another patch for that. It was due to the build system using the "-race" option to detect race conditions in the tests, and this mechanism only works on 64-bit platforms. Thanks, Pierre [0]: https://lists.gnu.org/archive/html/guix-patches/2021-08/msg00646.html
[signature.asc (application/pgp-signature, inline)]
[0001-gnu-node-Enable-cross-compilation.patch (text/x-patch, inline)]
From 8b4cc4573d438eb108b8d1a9bcdf99f3af98b514 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois <at> gmx.com>
Date: Wed, 11 Aug 2021 16:43:45 +0100
Subject: [PATCH 1/2] gnu: node: Enable cross-compilation.
Node runs parts of itself on the host for bootstraping therefore for
cross-compiling support we need to fidle with the rpath in the build system,
as well as duplicating some of the dependencies as native-inputs and inputs.
* gnu/pakcages/node.scm (node)[arguments]: Refer to /bin/sh and /bin/env
directly instead of using (which). Add new 'set-bootstrap-host-rpath phase
to correctly set the rpath for binaries that are meant to run on the host.
Pass --cross-compiling and --dest-cpu to configure script if needed. Set the
CC_host, CXX_host, CC, CCX and PKG_CONFIG variable for cross-compilation.
Refer to the host python. Do not return #t from any phases.
[native-inputs]: Add c-ares, http-parser, icu4c, libuv, nghttp2, openssl and
zlib. Remove which.
[inputs]: Add bash and coreutils.
(llhttp-bootstrap)[arguments]: Refer to esbuild via (or native-inputs inputs).
(node-lts)[arguments]: Add new 'set-bootstrap-host-rpath phase to correctly
set the rpath for host binaries. Pass --cross-compiling and --dest-cpu to
configure script if needed. Set the CC_host, CXX_host, CCX and PKG_CONFIG
variable for cross-compilation. Refer to the host python. Do not return #t
from any phases. Refer to /bin/sh and /bin/env directly instead of
using (which). Do not return #t from any phases.
[native-inputs]: Hardcode native inputs instead of inheriting them from node.
[inputs]: Add bash and coreutils.
---
gnu/packages/node.scm | 188 ++++++++++++++++++++++++++++++++++--------
1 file changed, 153 insertions(+), 35 deletions(-)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 36c45e9c7a..2dc23b881c 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -6,7 +6,7 @@
;;; Copyright © 2017 Mike Gerwitz <mtg <at> gnu.org>
;;; Copyright © 2018 Tobias Geerinckx-Rice <me <at> tobias.gr>
;;; Copyright © 2018, 2019, 2020, 2021 Marius Bakke <marius <at> gnu.org>
-;;; Copyright © 2020 Pierre Langlois <pierre.langlois <at> gmx.com>
+;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois <at> gmx.com>
;;; Copyright © 2020 Ricardo Wurmus <rekado <at> elephly.net>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune <at> gmail.com>
;;;
@@ -38,6 +38,7 @@
#:use-module (gnu packages)
#:use-module (gnu packages adns)
#:use-module (gnu packages base)
+ #:use-module (gnu packages bash)
#:use-module (gnu packages compression)
#:use-module (gnu packages gcc)
#:use-module (gnu packages icu4c)
@@ -48,7 +49,9 @@
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
#:use-module (gnu packages tls)
- #:use-module (gnu packages web))
+ #:use-module (gnu packages web)
+ #:use-module (ice-9 match)
+ #:use-module (srfi srfi-26))
(define-public node
(package
@@ -111,14 +114,14 @@
"test/parallel/test-stdio-closed.js"
"test/sequential/test-child-process-emfile.js")
(("'/bin/sh'")
- (string-append "'" (which "sh") "'")))
+ (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
;; Fix hardcoded /usr/bin/env references.
(substitute* '("test/parallel/test-child-process-default-options.js"
"test/parallel/test-child-process-env.js"
"test/parallel/test-child-process-exec-env.js")
(("'/usr/bin/env'")
- (string-append "'" (which "env") "'")))
+ (string-append "'" (assoc-ref inputs "coreutils") "/bin/env'")))
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
@@ -155,23 +158,65 @@
;; TODO: Regenerate certs instead.
(for-each delete-file
'("test/parallel/test-tls-passphrase.js"
- "test/parallel/test-tls-server-verify.js"))
- #t))
+ "test/parallel/test-tls-server-verify.js"))))
+ (add-before 'configure 'set-bootstrap-host-rpath
+ (lambda* (#:key native-inputs inputs #:allow-other-keys)
+ (let* ((inputs (or native-inputs inputs))
+ (c-ares (assoc-ref inputs "c-ares"))
+ (http-parser (assoc-ref inputs "http-parser"))
+ (icu4c (assoc-ref inputs "icu4c"))
+ (nghttp2 (assoc-ref inputs "nghttp2"))
+ (openssl (assoc-ref inputs "openssl"))
+ (libuv (assoc-ref inputs "libuv"))
+ (zlib (assoc-ref inputs "zlib")))
+ (substitute* "deps/v8/gypfiles/v8.gyp"
+ (("'target_name': 'torque'," target)
+ (string-append target
+ "'ldflags': ['-Wl,-rpath="
+ c-ares "/lib:"
+ http-parser "/lib:"
+ icu4c "/lib:"
+ nghttp2 "/lib:"
+ openssl "/lib:"
+ libuv "/lib:"
+ zlib "/lib"
+ "'],"))))))
(replace 'configure
;; Node's configure script is actually a python script, so we can't
;; run it with bash.
- (lambda* (#:key outputs (configure-flags '()) inputs
+ (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
#:allow-other-keys)
(let* ((prefix (assoc-ref outputs "out"))
- (flags (cons (string-append "--prefix=" prefix)
- configure-flags)))
+ (flags (cons*
+ (string-append "--prefix=" prefix)
+ ,@(if (%current-target-system)
+ `("--cross-compiling"
+ ,(string-append "--dest-cpu="
+ (match (%current-target-system)
+ ((? (cut string-prefix? "arm" <>))
+ "arm")
+ ((? (cut string-prefix? "aarch64" <>))
+ "arm64")
+ ((? (cut string-prefix? "i686" <>))
+ "ia32")
+ ((? (cut string-prefix? "x86_64" <>))
+ "x64")
+ ((? (cut string-prefix? "powerpc64" <>))
+ "ppc64")
+ (_ "unsupported"))))
+ '())
+ configure-flags)))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
;; Node's configure script expects the CC environment variable to
;; be set.
- (setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc"))
+ (setenv "CC_host" "gcc")
+ (setenv "CXX_host" "g++")
+ (setenv "CC" ,(cc-for-target))
+ (setenv "CXX" ,(cxx-for-target))
+ (setenv "PKG_CONFIG" ,(pkg-config-for-target))
(apply invoke
- (string-append (assoc-ref inputs "python")
+ (string-append (assoc-ref (or native-inputs inputs) "python")
"/bin/python")
"configure" flags))))
(add-after 'patch-shebangs 'patch-npm-shebang
@@ -181,29 +226,37 @@
(npm (string-append bindir "/npm"))
(target (readlink npm)))
(with-directory-excursion bindir
- (patch-shebang target (list bindir))
- #t))))
+ (patch-shebang target (list bindir))))))
(add-after 'install 'patch-node-shebang
(lambda* (#:key outputs #:allow-other-keys)
(let* ((bindir (string-append (assoc-ref outputs "out")
"/bin"))
(npx (readlink (string-append bindir "/npx"))))
(with-directory-excursion bindir
- (patch-shebang npx (list bindir))
- #t)))))))
+ (patch-shebang npx (list bindir)))))))))
(native-inputs
- `(("python" ,python-2)
+ `(;; Runtime dependencies for binaries used as a bootstrap.
+ ("c-ares" ,c-ares)
+ ("http-parser" ,http-parser)
+ ("icu4c" ,icu4c)
+ ("libuv" ,libuv)
+ ("nghttp2" ,nghttp2 "lib")
+ ("openssl" ,openssl)
+ ("zlib" ,zlib)
+ ;; Regular build-time dependencies.
("perl" ,perl)
("pkg-config" ,pkg-config)
("procps" ,procps)
- ("util-linux" ,util-linux)
- ("which" ,which)))
+ ("python" ,python-2)
+ ("util-linux" ,util-linux)))
(native-search-paths
(list (search-path-specification
(variable "NODE_PATH")
(files '("lib/node_modules")))))
(inputs
- `(("c-ares" ,c-ares)
+ `(("bash" ,bash)
+ ("coreutils" ,coreutils)
+ ("c-ares" ,c-ares)
("http-parser" ,http-parser)
("icu4c" ,icu4c)
("libuv" ,libuv)
@@ -551,9 +604,10 @@ parser definition into a C output.")
#:phases
(modify-phases %standard-phases
(replace 'configure
- (lambda* (#:key inputs #:allow-other-keys)
- (let ((esbuild (string-append (assoc-ref inputs "esbuild")
- "/bin/esbuild")))
+ (lambda* (#:key native-inputs inputs #:allow-other-keys)
+ (let ((esbuild (string-append
+ (assoc-ref (or native-inputs inputs) "esbuild")
+ "/bin/esbuild")))
(invoke esbuild
"--platform=node"
"--outfile=bin/generate.js"
@@ -625,21 +679,72 @@ source files.")
"--with-intl=system-icu"))
((#:phases phases)
`(modify-phases ,phases
+ (replace 'set-bootstrap-host-rpath
+ (lambda* (#:key native-inputs inputs #:allow-other-keys)
+ (let* ((inputs (or native-inputs inputs))
+ (c-ares (assoc-ref inputs "c-ares"))
+ (google-brotli (assoc-ref inputs "google-brotli"))
+ (icu4c (assoc-ref inputs "icu4c"))
+ (nghttp2 (assoc-ref inputs "nghttp2"))
+ (openssl (assoc-ref inputs "openssl"))
+ (libuv (assoc-ref inputs "libuv"))
+ (zlib (assoc-ref inputs "zlib"))
+ (host-binaries '("torque"
+ "bytecode_builtins_list_generator"
+ "gen-regexp-special-case"
+ "node_mksnapshot"
+ "mksnapshot")))
+ (substitute* '("node.gyp" "tools/v8_gypfiles/v8.gyp")
+ (((string-append "'target_name': '("
+ (string-join host-binaries "|")
+ ")',")
+ target)
+ (string-append target
+ "'ldflags': ['-Wl,-rpath="
+ c-ares "/lib:"
+ google-brotli "/lib:"
+ icu4c "/lib:"
+ nghttp2 "/lib:"
+ openssl "/lib:"
+ libuv "/lib:"
+ zlib "/lib"
+ "'],"))))))
(replace 'configure
;; Node's configure script is actually a python script, so we can't
;; run it with bash.
- (lambda* (#:key outputs (configure-flags '()) inputs
+ (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
#:allow-other-keys)
(let* ((prefix (assoc-ref outputs "out"))
- (flags (cons (string-append "--prefix=" prefix)
- configure-flags)))
+ (flags (cons*
+ (string-append "--prefix=" prefix)
+ ,@(if (%current-target-system)
+ `("--cross-compiling"
+ ,(string-append "--dest-cpu="
+ (match (%current-target-system)
+ ((? (cut string-prefix? "arm" <>))
+ "arm")
+ ((? (cut string-prefix? "aarch64" <>))
+ "arm64")
+ ((? (cut string-prefix? "i686" <>))
+ "ia32")
+ ((? (cut string-prefix? "x86_64" <>))
+ "x64")
+ ((? (cut string-prefix? "powerpc64" <>))
+ "ppc64")
+ (_ "unsupported"))))
+ '())
+ configure-flags)))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
;; Node's configure script expects the CC environment variable to
;; be set.
+ (setenv "CC_host" "gcc")
+ (setenv "CXX_host" "g++")
(setenv "CC" ,(cc-for-target))
+ (setenv "CXX" ,(cxx-for-target))
+ (setenv "PKG_CONFIG" ,(pkg-config-for-target))
(apply invoke
- (string-append (assoc-ref inputs "python")
+ (string-append (assoc-ref (or native-inputs inputs) "python")
"/bin/python3")
"configure" flags))))
(replace 'patch-files
@@ -652,14 +757,14 @@ source files.")
"test/parallel/test-stdio-closed.js"
"test/sequential/test-child-process-emfile.js")
(("'/bin/sh'")
- (string-append "'" (which "sh") "'")))
+ (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
;; Fix hardcoded /usr/bin/env references.
(substitute* '("test/parallel/test-child-process-default-options.js"
"test/parallel/test-child-process-env.js"
"test/parallel/test-child-process-exec-env.js")
(("'/usr/bin/env'")
- (string-append "'" (which "env") "'")))
+ (string-append "'" (assoc-ref inputs "coreutils") "/bin/env'")))
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
@@ -707,20 +812,33 @@ source files.")
(copy-file (string-append llhttp "/src/http.c")
"deps/llhttp/src/http.c")
(copy-file (string-append llhttp "/include/llhttp.h")
- "deps/llhttp/include/llhttp.h"))
- #t))))))
+ "deps/llhttp/include/llhttp.h"))))))))
+ (native-inputs
+ `(;; Runtime dependencies for binaries used as a bootstrap.
+ ("c-ares" ,c-ares)
+ ("google-brotli" ,google-brotli)
+ ("icu4c" ,icu4c-67)
+ ("libuv" ,libuv-for-node)
+ ("nghttp2" ,nghttp2 "lib")
+ ("openssl" ,openssl)
+ ("zlib" ,zlib)
+ ;; Regular build-time dependencies.
+ ("perl" ,perl)
+ ("pkg-config" ,pkg-config)
+ ("procps" ,procps)
+ ("python" ,python)
+ ("util-linux" ,util-linux)))
(inputs
- `(("c-ares" ,c-ares)
+ `(("bash" ,bash)
+ ("coreutils" ,coreutils)
+ ("c-ares" ,c-ares)
("icu4c" ,icu4c-67)
("libuv" ,libuv-for-node)
("llhttp" ,llhttp-bootstrap)
("google-brotli" ,google-brotli)
("nghttp2" ,nghttp2 "lib")
("openssl" ,openssl)
- ("zlib" ,zlib)))
- (native-inputs
- (alist-replace "python" (list python-3)
- (package-native-inputs node)))))
+ ("zlib" ,zlib)))))
(define-public libnode
(package/inherit node
--
2.33.0
[0002-gnu-esbuild-Disable-race-detector-on-32-bit-targets.patch (text/x-patch, inline)]
From 17cd8ccb2255d4bb392a94468abeb785e134ff1e Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois <at> gmx.com>
Date: Mon, 30 Aug 2021 00:25:58 +0100
Subject: [PATCH 2/2] gnu: esbuild: Disable race detector on 32-bit targets.
* gnu/packages/web.scm (esbuild)[arguments]: Set the ESBUILD_RACE
variable to an empty string to remove the -race option.
---
gnu/packages/web.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index f0ac9ccee2..5817d2dd95 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -1672,13 +1672,17 @@ used to validate and fix HTML data.")
#t))))
(build-system go-build-system)
(arguments
- '(#:import-path "github.com/evanw/esbuild/cmd/esbuild"
+ `(#:import-path "github.com/evanw/esbuild/cmd/esbuild"
#:unpack-path "github.com/evanw/esbuild"
#:phases
(modify-phases %standard-phases
(replace 'check
(lambda* (#:key tests? unpack-path #:allow-other-keys)
(when tests?
+ ;; The "Go Race Detector" is only supported on 64-bit
+ ;; platforms, this variable disables it.
+ (unless ,(target-64bit?)
+ (setenv "ESBUILD_RACE" ""))
(with-directory-excursion (string-append "src/" unpack-path)
(invoke "make" "test-go")))
#t)))))
--
2.33.0
guix-patches <at> gnu.org:bug#50261; Package guix-patches.
(Tue, 31 Aug 2021 13:43:01 GMT) Full text and rfc822 format available.Message #8 received at 50261 <at> debbugs.gnu.org (full text, mbox):
From: Mathieu Othacehe <othacehe <at> gnu.org> To: Pierre Langlois <pierre.langlois <at> gmx.com> Cc: 50261 <at> debbugs.gnu.org Subject: Re: bug#50261: [PATCH] gnu: node: Enable cross-compilation. Date: Tue, 31 Aug 2021 15:41:39 +0200
Hello Pierre, > While working on tree-sitter I realized our node packages didn't > cross-compile [0] so I thought I'd give it a go, here's a patch! Nice! > I'm not sure there's any way to express this restriction in the package > definition, it would be good to fail in a useful way, any thoughts? I > suspect a mechanism for this isn't really worth the effort, I don't know > of any other packages like this. Yeah that's a strange limitation, I don't think we have something similar elsewhere. A restriction that compares the host and target bitness is maybe not worth it. > - (string-append (assoc-ref inputs "python") > + (string-append (assoc-ref (or native-inputs inputs) "python") You have several lines such as this one that are longer than the 78 columns limit. Other than that, it looks cool, and you can go ahead. Thanks, Mathieu
Pierre Langlois <pierre.langlois <at> gmx.com>:Pierre Langlois <pierre.langlois <at> gmx.com>:Message #13 received at 50261-done <at> debbugs.gnu.org (full text, mbox):
From: Pierre Langlois <pierre.langlois <at> gmx.com> To: Mathieu Othacehe <othacehe <at> gnu.org> Cc: 50261-done <at> debbugs.gnu.org, Pierre Langlois <pierre.langlois <at> gmx.com> Subject: Re: bug#50261: [PATCH] gnu: node: Enable cross-compilation. Date: Thu, 02 Sep 2021 20:51:49 +0100
[Message part 1 (text/plain, inline)]
Hi Mathieu,
Mathieu Othacehe <othacehe <at> gnu.org> writes:
> Hello Pierre,
>
>> While working on tree-sitter I realized our node packages didn't
>> cross-compile [0] so I thought I'd give it a go, here's a patch!
>
> Nice!
>
>> I'm not sure there's any way to express this restriction in the package
>> definition, it would be good to fail in a useful way, any thoughts? I
>> suspect a mechanism for this isn't really worth the effort, I don't know
>> of any other packages like this.
>
> Yeah that's a strange limitation, I don't think we have something
> similar elsewhere. A restriction that compares the host and target
> bitness is maybe not worth it.
>
>> - (string-append (assoc-ref inputs "python")
>> + (string-append (assoc-ref (or native-inputs inputs) "python")
>
> You have several lines such as this one that are longer than the 78
> columns limit.
I did my best to fix those by reworking the configure flags as such:
--8<---------------cut here---------------start------------->8---
(let* ((prefix (assoc-ref outputs "out"))
(xflags ,(if (%current-target-system)
`'("--cross-compiling"
,(string-append
"--dest-cpu="
(match (%current-target-system)
((? (cut string-prefix? "arm" <>))
"arm")
((? (cut string-prefix? "aarch64" <>))
"arm64")
((? (cut string-prefix? "i686" <>))
"ia32")
((? (cut string-prefix? "x86_64" <>))
"x64")
((? (cut string-prefix? "powerpc64" <>))
"ppc64")
(_ "unsupported"))))
''()))
(flags (cons (string-append "--prefix=" prefix)
(append xflags configure-flags))))
--8<---------------cut here---------------end--------------->8---
It's still one column over, but I figured it was OK, I couldn't find
another way to write it without sacrificing readability, hope that's OK
:-).
> Other than that, it looks cool, and you can go ahead.
Thanks for the review! Pushed with
9f7c4f380fdd86d81c805b72e4d05e9e658d3dc2 and
5ee38c467298091e98fa12be45facdcc63a59a87.
Pierre
[signature.asc (application/pgp-signature, inline)]
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Fri, 01 Oct 2021 11:24:07 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.