GNU bug report logs - #77231
[PATCH javascript-team 0/6] Improve importer and build-system.

Previous Next

To reply to this bug, email your comments to 77231 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: guix-patches <at> gnu.org
Cc: Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH javascript-team 0/6] Improve importer and build-system.
Date: Mon, 24 Mar 2025 08:18:53 +0100

Like in python, there are some development dependencies we want to
avoid since they are not contributing to the build at all.  This patch
series adds options to ignore them.

This change requires to rebuild node, so I prefer to do that on CI.

Nicolas Graves (6):
  import: npm-binary: Fix import.
  import: npm-binary: Improve npm-package->package-sexp.
  import: npm-binary: Improve-style.
  build-system/node: Add phase 'delete-unwanted-dev-dependencies.
  import: npm-binary: Handle vector of licenses.
  import: npm-binary: Filter out npm-ignored-inputs.

 guix/build/node-build-system.scm |  39 +++++++-
 guix/import/npm-binary.scm       | 160 ++++++++++++++++++-------------
 tests/npm-binary.scm             | 157 +++++++++++++++++-------------
 3 files changed, 215 insertions(+), 141 deletions(-)

-- 
2.48.1

Message #8 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH 2/6] import: npm-binary: Improve npm-package->package-sexp.
Date: Mon, 24 Mar 2025 08:29:13 +0100

* guix/import/npm-binary.scm (npm-package->package-sexp): Use record
matching.
---
 guix/import/npm-binary.scm | 121 ++++++++++++++++++-------------------
 1 file changed, 58 insertions(+), 63 deletions(-)

diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm
index c43b84f3d5..f095651c34 100644
--- a/guix/import/npm-binary.scm
+++ b/guix/import/npm-binary.scm
@@ -196,69 +196,64 @@ (define resolve-spec
       (($ <versioned-package> name version)
        (resolve-package name (string->semver-range version)))))
 
-  (if (package-revision? npm-package)
-      (let ((name (package-revision-name npm-package))
-            (version (package-revision-version npm-package))
-            (home-page (package-revision-home-page npm-package))
-            (dependencies (package-revision-dependencies npm-package))
-            (dev-dependencies (package-revision-dev-dependencies npm-package))
-            (peer-dependencies (package-revision-peer-dependencies npm-package))
-            (license (package-revision-license npm-package))
-            (description (package-revision-description npm-package))
-            (dist (package-revision-dist npm-package)))
-        (let* ((name (npm-name->name name))
-               (url (dist-tarball dist))
-               (home-page (if (string? home-page)
-                              home-page
-                              (string-append %default-page "/" (uri-encode name))))
-               (synopsis description)
-               (resolved-deps (map resolve-spec
-                                   (append dependencies peer-dependencies)))
-               (peer-names (map versioned-package-name peer-dependencies))
-               ;; lset-difference for treating peer-dependencies as dependencies,
-               ;; which leads to dependency cycles.  lset-union for treating them as
-               ;; (ignored) dev-dependencies, which leads to broken packages.
-               (dev-names
-                (lset-union string=
-                            (map versioned-package-name dev-dependencies)
-                            peer-names))
-               (extra-phases
-                (match dev-names
-                  (() '())
-                  ((dev-names ...)
-                   `((add-after 'patch-dependencies 'delete-dev-dependencies
-                       (lambda _
-                         (modify-json
-                          (delete-dependencies '(,@(reverse dev-names)))))))))))
-          (values
-           `(package
-              (name ,name)
-              (version ,(semver->string (package-revision-version npm-package)))
-              (source (origin
-                        (method url-fetch)
-                        (uri ,url)
-                        (sha256 (base32 ,(hash-url url)))))
-              (build-system node-build-system)
-              (arguments
-               (list
-                #:tests? #f
-                #:phases
-                #~(modify-phases %standard-phases
-                    (delete 'build)
-                    ,@extra-phases)))
-              ,@(match dependencies
-                  (() '())
-                  ((dependencies ...)
-                   `((inputs
-                      (list ,@(map package-revision->symbol resolved-deps))))))
-              (home-page ,home-page)
-              (synopsis ,synopsis)
-              (description ,description)
-              (license ,license))
-           (map (match-lambda (($ <package-revision> name version)
-                               (list name (semver->string version))))
-                resolved-deps))))
-      (values #f '())))
+  (match npm-package
+    (($ <package-revision>
+        name version home-page dependencies dev-dependencies
+        peer-dependencies license description dist)
+     (let* ((name (npm-name->name name))
+            (url (dist-tarball dist))
+            (home-page (if (string? home-page)
+                           home-page
+                           (string-append %default-page "/" (uri-encode name))))
+            (synopsis description)
+            (resolved-deps (map resolve-spec
+                                (append dependencies peer-dependencies)))
+            (peer-names (map versioned-package-name peer-dependencies))
+            ;; lset-difference for treating peer-dependencies as dependencies,
+            ;; which leads to dependency cycles.  lset-union for treating them as
+            ;; (ignored) dev-dependencies, which leads to broken packages.
+            (dev-names
+             (lset-union string=
+                         (map versioned-package-name dev-dependencies)
+                         peer-names))
+            (extra-phases
+             (match dev-names
+               (() '())
+               ((dev-names ...)
+                `((add-after 'patch-dependencies 'delete-dev-dependencies
+                    (lambda _
+                      (modify-json
+                       (delete-dependencies '(,@(reverse dev-names)))))))))))
+       (values
+        `(package
+           (name ,name)
+           (version ,(semver->string (package-revision-version npm-package)))
+           (source (origin
+                     (method url-fetch)
+                     (uri ,url)
+                     (sha256 (base32 ,(hash-url url)))))
+           (build-system node-build-system)
+           (arguments
+            (list
+             #:tests? #f
+             #:phases
+             #~(modify-phases %standard-phases
+                 (delete 'build)
+                 ,@extra-phases)))
+           ,@(match dependencies
+               (() '())
+               ((dependencies ...)
+                `((inputs
+                   (list ,@(map package-revision->symbol resolved-deps))))))
+           (home-page ,home-page)
+           (synopsis ,synopsis)
+           (description ,description)
+           (license ,license))
+        (map (match-lambda (($ <package-revision> name version)
+                            (list name (semver->string version))))
+             resolved-deps))))
+    (_
+     (values #f '()))))
 
 
 ;;;
-- 
2.48.1

Message #11 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH 3/6] import: npm-binary: Improve-style.
Date: Mon, 24 Mar 2025 08:29:14 +0100

* guix/import/binary.scm (resolve-package): Use and-let*.
---
 guix/import/npm-binary.scm | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm
index f095651c34..60d7c07a8e 100644
--- a/guix/import/npm-binary.scm
+++ b/guix/import/npm-binary.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2019, 2020 Timothy Sample <samplet <at> ngyro.com>
 ;;; Copyright © 2021 Lars-Dominik Braun <lars <at> 6xq.net>
 ;;; Copyright © 2020, 2023, 2024 Jelle Licht <jlicht <at> fsfe.org>
+;;; Copyright © 2025 Nicolas Graves <ngraves <at> ngraves.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -33,6 +34,7 @@ (define-module (guix import npm-binary)
   #:use-module (ice-9 regex)
   #:use-module (json)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-2)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-41)
   #:use-module (srfi srfi-9)
@@ -158,11 +160,11 @@ (define* (semver-latest svs #:optional (svr *semver-range-any*))
         (sort svs semver>?)))
 
 (define* (resolve-package name #:optional (svr *semver-range-any*))
-  (let ((meta (lookup-meta-package* name)))
-    (and meta
-         (let* ((version (semver-latest (or (meta-package-versions meta) '()) svr))
-                (pkg (meta-package-package meta version)))
-           pkg))))
+  (and-let*
+      ((meta (lookup-meta-package* name))
+       (version (semver-latest (or (meta-package-versions meta) '()) svr))
+       (pkg (meta-package-package meta version)))
+    pkg))
 
 
 ;;;
-- 
2.48.1

Message #14 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH 4/6] build-system/node: Add phase
 'delete-unwanted-dev-dependencies.
Date: Mon, 24 Mar 2025 08:29:15 +0100

* guix/build/node-build-system.scm (npm-ignored-inputs): New variable.
(delete-dependencies): Extend procedure to accept a filtering
procedure.
(delete-unwanted-dev-dependencies): Define new phase to ignore all
npm-ignored-inputs systematically.
(%standard-phases): Add it here.
---
 guix/build/node-build-system.scm | 39 ++++++++++++++++++++++++++++----
 1 file changed, 35 insertions(+), 4 deletions(-)

diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 05940bc997..50e023b3ca 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -29,6 +29,7 @@ (define-module (guix build node-build-system)
   #:use-module (ice-9 match)
   #:use-module (json)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-71)
   #:export (%standard-phases
             delete-dependencies
@@ -37,9 +38,21 @@ (define-module (guix build node-build-system)
             modify-json
             modify-json-fields
             node-build
+            npm-ignored-inputs
             replace-fields
             with-atomic-json-file-replacement))
 
+(define npm-ignored-inputs
+  (list
+   (list "aud" "nsp" ; passed end-of-life
+         "covert" ; code coverage
+         "auto-changelog" "npmignore" "evalmd" ; development tools
+         "eclint" "eslint" "prettier-standard" "standard" ; lint
+         "in-publish" "np" "safe-publish-latest") ; upload integration tools
+   ;; Second value is a list of prefixes to ignore
+   ;; Handy for personal configs and extensions of ignored inputs
+   (list "@ljharb/" "eslint-")))
+
 (define* (assoc-ref* alist key #:optional default)
   "Like assoc-ref, but return DEFAULT instead of #f if no value exists."
   (match (assoc key alist)
@@ -100,7 +113,7 @@ (define* (modify-json #:key (file "package.json") #:rest all-arguments)
           modifications))
       file)))
 
-(define (delete-dependencies dependencies-to-remove)
+(define (delete-dependencies predicate-or-dependencies)
   "Rewrite 'package.json' to allow the build to proceed without packages
 listed in 'dependencies-to-remove', a list of strings naming npm packages.
 
@@ -114,9 +127,13 @@ (define (delete-dependencies dependencies-to-remove)
           dependency-key
           (lambda (dependencies)
             (remove
-              (lambda (dependency)
-                (member (car dependency) dependencies-to-remove))
-              dependencies))))
+             (match predicate-or-dependencies
+               ((? procedure? predicate)
+                predicate)
+               ((? list? dependencies-to-remove)
+                (lambda (dependency)
+                  (member (car dependency) dependencies-to-remove))))
+             dependencies))))
       pkg-meta
       (list
         "devDependencies"
@@ -399,11 +416,25 @@ (define scripts
                                   "echo Guix: avoiding node-gyp rebuild"))
           out)))))
 
+(define delete-unwanted-dev-dependencies
+  (lambda* args
+    (modify-json
+     (delete-dependencies
+      (lambda (input)
+        (match npm-ignored-inputs
+          (((ignored ...) (prefixes ...) . ())
+           (or (member (car input) ignored)
+               (any (cut string-prefix? <> (car input))
+                    prefixes)))
+          (_ #f)))))))
+
 (define %standard-phases
   (modify-phases gnu:%standard-phases
     (add-after 'unpack 'set-home set-home)
     (add-before 'configure 'patch-dependencies patch-dependencies)
     (add-after 'patch-dependencies 'delete-lockfiles delete-lockfiles)
+    (add-after 'patch-dependencies 'delete-unwanted-dev-dependencies
+      delete-unwanted-dev-dependencies)
     (replace 'configure configure)
     (replace 'build build)
     (replace 'check check)
-- 
2.48.1

Message #17 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH 1/6] import: npm-binary: Fix import.
Date: Mon, 24 Mar 2025 08:29:12 +0100

This is a follow-up to b65f5b731c538227b51645d6d81b0cbb731bb6e4.

* guix/import/npm-binary.scm (npm-package->package-sexp): Use modify-json.
* tests/npm-binary.scm (npm-binary->guix-package test): Use modify-json.
---
 guix/import/npm-binary.scm | 3 ++-
 tests/npm-binary.scm       | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm
index 92ac7d338e..c43b84f3d5 100644
--- a/guix/import/npm-binary.scm
+++ b/guix/import/npm-binary.scm
@@ -228,7 +228,8 @@ (define resolve-spec
                   ((dev-names ...)
                    `((add-after 'patch-dependencies 'delete-dev-dependencies
                        (lambda _
-                         (delete-dependencies '(,@(reverse dev-names))))))))))
+                         (modify-json
+                          (delete-dependencies '(,@(reverse dev-names)))))))))))
           (values
            `(package
               (name ,name)
diff --git a/tests/npm-binary.scm b/tests/npm-binary.scm
index cf85e572b3..0cc2864546 100755
--- a/tests/npm-binary.scm
+++ b/tests/npm-binary.scm
@@ -133,7 +133,8 @@ (define have-guile-semver?
                              (delete 'build)
                              (add-after 'patch-dependencies 'delete-dev-dependencies
                                (lambda _
-                                 (delete-dependencies '("node-megabuilder"))))))))
+                                 (modify-json
+                                  (delete-dependencies '("node-megabuilder")))))))))
               (inputs (list node-bar-0.1.2))
               (home-page "https://github.com/quartz/foo")
               (synopsis "General purpose utilities to foo your bars")
-- 
2.48.1

Message #20 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH 5/6] import: npm-binary: Handle vector of licenses.
Date: Mon, 24 Mar 2025 08:29:16 +0100

* guix/import/npm-binary.scm (<package-revision>)[license]: Handle the
case where a vector of licenses is used.

* tests/npm-binary.scm (foo-json): Redefine as a procedure with
license keyword.
  (test-source-hash): Redefine with direct reference to test-source.
  (foo-sexp): Redefine as a procedure with license keyword.
  (npm-binary->guix-package test): Use foo-json and foo-sexp.
  (npm-binary->guix-package with multiple licenses): Add test.
---
 guix/import/npm-binary.scm |  16 +++-
 tests/npm-binary.scm       | 158 +++++++++++++++++++++----------------
 2 files changed, 102 insertions(+), 72 deletions(-)

diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm
index 60d7c07a8e..01079c2814 100644
--- a/guix/import/npm-binary.scm
+++ b/guix/import/npm-binary.scm
@@ -105,7 +105,17 @@ (define-json-mapping <package-revision> make-package-revision package-revision?
               (match (assoc "type" alist)
                 ((_ . (? string? type))
                  (spdx-string->license type))
-                (_ #f)))))
+                (_ #f)))
+             ((? vector? vector)
+              (match (filter-map
+                      (match-lambda
+                        ((? string? str) (spdx-string->license str))
+                        (_ #f))
+                      (vector->list vector))
+                ((license rest ...)
+                 (cons* license rest))
+                ((license)
+                 license)))))
   (description package-revision-description             ;string
                "description" empty-or-string)
   (dist package-revision-dist "dist" json->dist))       ;dist
@@ -250,7 +260,9 @@ (define resolve-spec
            (home-page ,home-page)
            (synopsis ,synopsis)
            (description ,description)
-           (license ,license))
+           (license ,(if (list? license)
+                         `(list ,@license)
+                         license)))
         (map (match-lambda (($ <package-revision> name version)
                             (list name (semver->string version))))
              resolved-deps))))
diff --git a/tests/npm-binary.scm b/tests/npm-binary.scm
index 0cc2864546..b1c6174020 100755
--- a/tests/npm-binary.scm
+++ b/tests/npm-binary.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2024 Jelle Licht <jlicht <at> fsfe.org>
+;;; Copyright © 2025 Nicolas Graves <ngraves <at> ngraves.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -24,42 +25,35 @@ (define-module (test-npm-binary)
   #:use-module (srfi srfi-64)
   #:use-module (ice-9 iconv)
   #:use-module (ice-9 match)
+  #:use-module (json)
   #:export (run-test))
 
-(define foo-json
-  "{
-  \"name\": \"foo\",
-  \"dist-tags\": {
-    \"latest\": \"1.2.3\",
-    \"next\": \"2.0.1-beta4\"
-  },
-  \"description\": \"General purpose utilities to foo your bars\",
-  \"homepage\": \"https://github.com/quartz/foo\",
-  \"repository\": \"quartz/foo\",
-  \"versions\": {
-    \"1.2.3\": {
-      \"name\": \"foo\",
-      \"description\": \"General purpose utilities to foo your bars\",
-      \"version\": \"1.2.3\",
-      \"author\": \"Jelle Licht <jlicht <at> fsfe.org>\",
-      \"devDependencies\": {
-        \"node-megabuilder\": \"^0.0.2\"
-      },
-      \"dependencies\": {
-        \"bar\": \"^0.1.0\"
-      },
-      \"repository\": {
-        \"url\": \"quartz/foo\"
-      },
-      \"homepage\": \"https://github.com/quartz/foo\",
-      \"license\": \"MIT\",
-      \"dist\": {
-        \"tarball\": \"https://registry.npmjs.org/foo/-/foo-1.2.3.tgz\"
-      }
-    }
-  }
-}")
+(define* (foo-json #:key (license "MIT"))
+  "Create a JSON description of an example foo npm package, optionally using a
+different @var{license}."
+  (scm->json-string
+   `((name . "foo")
+     (dist-tags . ((latest . "1.2.3")
+                   (next . "2.0.1-beta4")))
+     (description . "General purpose utilities to foo your bars")
+     (homepage . "https://github.com/quartz/foo")
+     (repository . "quartz/foo")
+     (versions
+      . ((1.2.3
+          . ((name . "foo")
+             (description . "General purpose utilities to foo your bars")
+             (version . "1.2.3")
+             (author . "Jelle Licht <jlicht <at> fsfe.org>")
+             (devDependencies . ((node-megabuilder . "^0.0.2")))
+             (dependencies . ((bar . "^0.1.0")))
+             (repository . ((url . "quartz/foo")))
+             (homepage . "https://github.com/quartz/foo")
+             (license . ,license)
+             (dist
+              . ((tarball
+                  . "https://registry.npmjs.org/foo/-/foo-1.2.3.tgz"))))))))))
 
+;; Dependency JSON for the bar package
 (define bar-json
   "{
   \"name\": \"bar\",
@@ -87,61 +81,85 @@ (define bar-json
   }
 }")
 
-(define test-source-hash
-  "")
-
 (define test-source
   "Empty file\n")
 
+(define test-source-hash
+  (bytevector->nix-base32-string
+   (gcrypt-sha256 (string->bytevector test-source "utf-8"))))
+
 (define have-guile-semver?
   (false-if-exception (resolve-interface '(semver))))
 
+(define* (foo-sexp #:key (license 'license:expat))
+  `(package
+     (name "node-foo")
+     (version "1.2.3")
+     (source (origin
+               (method url-fetch)
+               (uri "https://registry.npmjs.org/foo/-/foo-1.2.3.tgz")
+               (sha256
+                (base32 "1n0h7zg9zzv4f7yn2gp0mq1v107im7pi6qq4k6q86rixz71ijklh"))))
+     (build-system node-build-system)
+     (arguments
+      (list #:tests? #f
+            #:phases
+            (gexp (modify-phases %standard-phases
+                    (delete 'build)
+                    (add-after 'patch-dependencies 'delete-dev-dependencies
+                      (lambda _
+                        (modify-json
+                         (delete-dependencies '("node-megabuilder")))))))))
+     (inputs (list node-bar-0.1.2))
+     (home-page "https://github.com/quartz/foo")
+     (synopsis "General purpose utilities to foo your bars")
+     (description "General purpose utilities to foo your bars")
+     (license ,license)))
+
 (test-begin "npm")
 
 (unless have-guile-semver? (test-skip 1))
-(test-assert "npm-binary->guix-package"
+(test-assert "npm-binary->guix-package base case"
   (mock ((guix http-client) http-fetch
          (lambda* (url #:rest _)
            (match url
              ("https://registry.npmjs.org/foo"
-              (values (open-input-string foo-json)
-                      (string-length foo-json)))
+              (let ((json-foo (foo-json)))
+                (values (open-input-string json-foo)
+                        (string-length json-foo))))
              ("https://registry.npmjs.org/bar"
               (values (open-input-string bar-json)
                       (string-length bar-json)))
              ("https://registry.npmjs.org/foo/-/foo-1.2.3.tgz"
-              (set! test-source-hash
-                    (bytevector->nix-base32-string
-                     (gcrypt-sha256 (string->bytevector test-source "utf-8"))))
               (values (open-input-string test-source)
                       (string-length test-source))))))
-        (match (npm-binary->guix-package "foo")
-          (`(package
-              (name "node-foo")
-              (version "1.2.3")
-              (source (origin
-                        (method url-fetch)
-                        (uri "https://registry.npmjs.org/foo/-/foo-1.2.3.tgz")
-                        (sha256
-                         (base32
-                          ,test-source-hash))))
-              (build-system node-build-system)
-              (arguments
-               (list #:tests? #f
-                     #:phases
-                     (gexp (modify-phases %standard-phases
-                             (delete 'build)
-                             (add-after 'patch-dependencies 'delete-dev-dependencies
-                               (lambda _
-                                 (modify-json
-                                  (delete-dependencies '("node-megabuilder")))))))))
-              (inputs (list node-bar-0.1.2))
-              (home-page "https://github.com/quartz/foo")
-              (synopsis "General purpose utilities to foo your bars")
-              (description "General purpose utilities to foo your bars")
-              (license license:expat))
-           #t)
-          (x
-           (pk 'fail x #f)))))
+        (let ((sexp-foo (foo-sexp)))
+          (match (npm-binary->guix-package "foo")
+            (sexp-foo
+             #t)
+            (x
+             (pk 'fail x #f))))))
+
+(test-assert "npm-binary->guix-package with multiple licenses"
+  (mock ((guix http-client) http-fetch
+         (lambda* (url #:rest _)
+           (match url
+             ("https://registry.npmjs.org/foo"
+              (let ((json-foo (foo-json #:license #("MIT" "Apache2.0"))))
+                (values (open-input-string json-foo)
+                        (string-length json-foo))))
+             ("https://registry.npmjs.org/bar"
+              (values (open-input-string bar-json)
+                      (string-length bar-json)))
+             ("https://registry.npmjs.org/foo/-/foo-1.2.3.tgz"
+              (values (open-input-string test-source)
+                      (string-length test-source))))))
+        (let ((sexp-foo (foo-sexp
+                         #:license '(list license:expat license:asl2.0))))
+          (match (npm-binary->guix-package "foo")
+            (sexp-foo
+             #t)
+            (x
+             (pk 'fail x #f))))))
 
 (test-end "npm")
-- 
2.48.1

Message #23 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: [PATCH 6/6] import: npm-binary: Filter out npm-ignored-inputs.
Date: Mon, 24 Mar 2025 08:29:17 +0100

* guix/import/npm-binary.scm (npm-keep-input?): Add variable.
(npm-package->package-sexp): Use npm-keep-input? to filter out
unwanted inputs.
---
 guix/import/npm-binary.scm | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm
index 01079c2814..fed063f55d 100644
--- a/guix/import/npm-binary.scm
+++ b/guix/import/npm-binary.scm
@@ -24,6 +24,7 @@ (define-module (guix import npm-binary)
   #:use-module (gcrypt hash)
   #:use-module (gnu packages)
   #:use-module (guix base32)
+  #:use-module ((guix build node-build-system) #:select (npm-ignored-inputs))
   #:use-module (guix http-client)
   #:use-module (guix import json)
   #:use-module (guix import utils)
@@ -201,6 +202,16 @@ (define (package-revision->symbol package)
          (name (npm-name->name npm-name)))
     (name+version->symbol name version)))
 
+(define (npm-keep-input? input)
+  "If INPUT is not among `npm-ignored-inputs', return it.  Else return #f."
+  (match npm-ignored-inputs
+    (((ignored ...) (prefixes ...) . ())
+     (let ((name (versioned-package-name input)))
+       (and (not (or (member name ignored)
+                     (any (cut string-prefix? <> name) prefixes)))
+            input)))
+    (_ #f)))
+
 (define (npm-package->package-sexp npm-package)
   "Return the `package' s-expression for an NPM-PACKAGE."
   (define resolve-spec
@@ -218,6 +229,9 @@ (define resolve-spec
                            home-page
                            (string-append %default-page "/" (uri-encode name))))
             (synopsis description)
+            (dependencies (filter-map npm-keep-input? dependencies))
+            (dev-dependencies (filter-map npm-keep-input? dev-dependencies))
+            (peer-dependencies (filter-map npm-keep-input? peer-dependencies))
             (resolved-deps (map resolve-spec
                                 (append dependencies peer-dependencies)))
             (peer-names (map versioned-package-name peer-dependencies))
-- 
2.48.1

Message #26 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Jelle Licht <jlicht <at> fsfe.org>
To: 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org, Nicolas Graves <ngraves <at> ngraves.fr>
Subject: Re: [bug#77231] [PATCH 4/6] build-system/node: Add phase
 'delete-unwanted-dev-dependencies.
Date: Mon, 24 Mar 2025 22:24:11 +0100

Thanks for working all of this though, I really appreciate it all! I've
applied patches 1, 2, 3, and 5 locally, and after doing some sanity
checks over night, will be pushing it to the javascript-team

W.r.t patches 4 and 6;
A hardcoded list of ignored inputs as part of the build system seems
like it will tightly couple package expressions to a particular version
of guix. Changing it in the future (by adding or removing things) will
also be annoying for packages in guix, as well as generated package
expressions 'in the wilds'.

A similar argument was better framed in the discussion around
issues.guix.gnu.org/51838 (warning: long read). 

The idea of having the importer be able to effectively ignore a large
part of the huge dependency graph if we know it won't be useful to us is
a cool feature to have. Ideally it'd somehow be part of the generated
output as well though, so the generated package expressions run on more
revisions of guix.

Tangentially, how did you run into issues with this particular list of
deps?  Are they not often encountered as devDependencies (and
subsequently explicitly removed in the package expressions generated by
the importer)? Or is the point to minimise (generated) code duplication
between packages to remove these ne'er-needed-deps?

Thanks again!
- Jelle


Nicolas Graves via Guix-patches via <guix-patches <at> gnu.org> writes:

> * guix/build/node-build-system.scm (npm-ignored-inputs): New variable.
> (delete-dependencies): Extend procedure to accept a filtering
> procedure.
> (delete-unwanted-dev-dependencies): Define new phase to ignore all
> npm-ignored-inputs systematically.
> (%standard-phases): Add it here.
> ---
>  guix/build/node-build-system.scm | 39 ++++++++++++++++++++++++++++----
>  1 file changed, 35 insertions(+), 4 deletions(-)
>
> diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
> index 05940bc997..50e023b3ca 100644
> --- a/guix/build/node-build-system.scm
> +++ b/guix/build/node-build-system.scm
> @@ -29,6 +29,7 @@ (define-module (guix build node-build-system)
>    #:use-module (ice-9 match)
>    #:use-module (json)
>    #:use-module (srfi srfi-1)
> +  #:use-module (srfi srfi-26)
>    #:use-module (srfi srfi-71)
>    #:export (%standard-phases
>              delete-dependencies
> @@ -37,9 +38,21 @@ (define-module (guix build node-build-system)
>              modify-json
>              modify-json-fields
>              node-build
> +            npm-ignored-inputs
>              replace-fields
>              with-atomic-json-file-replacement))
>  
> +(define npm-ignored-inputs
> +  (list
> +   (list "aud" "nsp" ; passed end-of-life
> +         "covert" ; code coverage
> +         "auto-changelog" "npmignore" "evalmd" ; development tools
> +         "eclint" "eslint" "prettier-standard" "standard" ; lint
> +         "in-publish" "np" "safe-publish-latest") ; upload integration tools
> +   ;; Second value is a list of prefixes to ignore
> +   ;; Handy for personal configs and extensions of ignored inputs
> +   (list "@ljharb/" "eslint-")))
> +
>  (define* (assoc-ref* alist key #:optional default)
>    "Like assoc-ref, but return DEFAULT instead of #f if no value exists."
>    (match (assoc key alist)
> @@ -100,7 +113,7 @@ (define* (modify-json #:key (file "package.json") #:rest all-arguments)
>            modifications))
>        file)))
>  
> -(define (delete-dependencies dependencies-to-remove)
> +(define (delete-dependencies predicate-or-dependencies)
>    "Rewrite 'package.json' to allow the build to proceed without packages
>  listed in 'dependencies-to-remove', a list of strings naming npm packages.
>  
> @@ -114,9 +127,13 @@ (define (delete-dependencies dependencies-to-remove)
>            dependency-key
>            (lambda (dependencies)
>              (remove
> -              (lambda (dependency)
> -                (member (car dependency) dependencies-to-remove))
> -              dependencies))))
> +             (match predicate-or-dependencies
> +               ((? procedure? predicate)
> +                predicate)
> +               ((? list? dependencies-to-remove)
> +                (lambda (dependency)
> +                  (member (car dependency) dependencies-to-remove))))
> +             dependencies))))
>        pkg-meta
>        (list
>          "devDependencies"
> @@ -399,11 +416,25 @@ (define scripts
>                                    "echo Guix: avoiding node-gyp rebuild"))
>            out)))))
>  
> +(define delete-unwanted-dev-dependencies
> +  (lambda* args
> +    (modify-json
> +     (delete-dependencies
> +      (lambda (input)
> +        (match npm-ignored-inputs
> +          (((ignored ...) (prefixes ...) . ())
> +           (or (member (car input) ignored)
> +               (any (cut string-prefix? <> (car input))
> +                    prefixes)))
> +          (_ #f)))))))
> +
>  (define %standard-phases
>    (modify-phases gnu:%standard-phases
>      (add-after 'unpack 'set-home set-home)
>      (add-before 'configure 'patch-dependencies patch-dependencies)
>      (add-after 'patch-dependencies 'delete-lockfiles delete-lockfiles)
> +    (add-after 'patch-dependencies 'delete-unwanted-dev-dependencies
> +      delete-unwanted-dev-dependencies)
>      (replace 'configure configure)
>      (replace 'build build)
>      (replace 'check check)
> -- 
> 2.48.1

Message #29 received at 77231 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: Jelle Licht <jlicht <at> fsfe.org>, 77231 <at> debbugs.gnu.org
Cc: jelle.licht <at> fsfe.org
Subject: Re: [bug#77231] [PATCH 4/6] build-system/node: Add phase
 'delete-unwanted-dev-dependencies.
Date: Tue, 25 Mar 2025 08:19:23 +0100

On 2025-03-24 22:24, Jelle Licht wrote:

> Thanks for working all of this though, I really appreciate it all! I've
> applied patches 1, 2, 3, and 5 locally, and after doing some sanity
> checks over night, will be pushing it to the javascript-team
>
> W.r.t patches 4 and 6;
> A hardcoded list of ignored inputs as part of the build system seems
> like it will tightly couple package expressions to a particular version
> of guix. Changing it in the future (by adding or removing things) will
> also be annoying for packages in guix, as well as generated package
> expressions 'in the wilds'.
>
> A similar argument was better framed in the discussion around
> issues.guix.gnu.org/51838 (warning: long read).

I did the same for python in 68315.  Wasn't aware of that discussion
indeed.

A good general (not Node specific) altenative IMO would be to have
"development-inputs" that are NOT injected in the build/derivation
neither evaluated (when <package> is compiled) but whose names could be
made available as an argument to the build-system.  That makes it
uncessary to record them at the build-system level (although adds more
package boilerplate).

Python basically has the same issue with a lot of false/unrequired
native-inputs.  I think I heard Nix solves this with a concept similar
to what I describe.

> The idea of having the importer be able to effectively ignore a large
> part of the huge dependency graph if we know it won't be useful to us is
> a cool feature to have. Ideally it'd somehow be part of the generated
> output as well though, so the generated package expressions run on more
> revisions of guix.

Do you mean we should do that as a snippet or in an earlier phase?

> Tangentially, how did you run into issues with this particular list of
> deps?  Are they not often encountered as devDependencies (and
> subsequently explicitly removed in the package expressions generated by
> the importer)? Or is the point to minimise (generated) code duplication
> between packages to remove these ne'er-needed-deps?

The latter, but it felt needed for some reason, the sheer count of lines
and readability is consequential.

>
> Thanks again!
> - Jelle

-- 
Best regards,
Nicolas Graves

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.