Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 62375) by debbugs.gnu.org; 28 Mar 2023 15:49:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 28 11:49:47 2023
Received: from localhost ([127.0.0.1]:50545 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1phBZz-0004xb-Fv
for submit <at> debbugs.gnu.org; Tue, 28 Mar 2023 11:49:47 -0400
Received: from eggs.gnu.org ([209.51.188.92]:60690)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1phBZv-0004xJ-DB
for 62375 <at> debbugs.gnu.org; Tue, 28 Mar 2023 11:49:46 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1phBZo-0000YO-5k; Tue, 28 Mar 2023 11:49:36 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
From; bh=Y7DrnRzb/9ppGjPYdH6NEeWKbHZWIVoRlJrTBs/fx64=; b=PGZCHVKIfx490nGTK5MJ
yJ7tnozcP8374GdjYaqdURLHjxU64jwOx6p/hFzfpejf4yZ8NbdTDJUNUZPtLzNKCl9jOw2hllNTW
uHpB4kbjg5PlmQpBEiEQo4Nl32gxeXR4+3zZ99P5SlnUrCwv7IpypXd+SyVVs5WTOd3gPuszy4WHi
4H8t8ez2xcvHPc//SkEXlBpFZed1vjC9+vRcZ2g29GKG5ExUGuuzllbGZ/y849QX3NkhWz4arh50q
bOQzHjMGGfsNvxmMgoRq0szFLtroGwRJJGZLqZZci+D/rghwJusDKC2zmPPtxZBur1g6joKujkv2/
+D8UWahn7S6vNA==;
Received: from [193.50.110.81] (helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1phBZn-0000zv-Nn; Tue, 28 Mar 2023 11:49:35 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: jlicht@HIDDEN
Subject: Re: [PATCH] import: Add binary npm importer.
References: <a41fe3ee10c48d36d3c84cf9f22ace8ca5bcd7c3.1679484420.git.jlicht@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Octidi 8 Germinal an 231 de la =?utf-8?Q?R=C3=A9volu?=
=?utf-8?Q?tion=2C?= jour de la Jonquille
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Tue, 28 Mar 2023 17:49:32 +0200
In-Reply-To: <a41fe3ee10c48d36d3c84cf9f22ace8ca5bcd7c3.1679484420.git.jlicht@HIDDEN>
(jlicht@HIDDEN's message of "Wed, 22 Mar 2023 12:27:16 +0100")
Message-ID: <87tty4g9kj.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 62375
Cc: Timothy Sample <samplet@HIDDEN>, Josselin Poiret <dev@HIDDEN>,
Tobias Geerinckx-Rice <me@HIDDEN>,
Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
Christopher Baines <mail@HIDDEN>, Lars-Dominik Braun <lars@HIDDEN>,
62375 <at> debbugs.gnu.org, Ricardo Wurmus <rekado@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hello Jelle!
jlicht@HIDDEN skribis:
> From: Jelle Licht <jlicht@HIDDEN>
>
> * guix/scripts/import.scm: (importers): Add "npm-binary".
> * guix/import/npm-binary.scm: New file.
> * guix/scripts/import/npm-binary.scm: New file.
> * Makefile.am: Add them.
>
> Co-authored-by: Timothy Sample <samplet@HIDDEN>
> Co-authored-by: Lars-Dominik Braun <lars@HIDDEN>
Woohoo! I think it=E2=80=99ll be useful to many, even if we know it=E2=80=
=99s far from
=E2=80=9Cideal=E2=80=9D by our standards.
We=E2=80=99ll need doc under =E2=80=9CInvoking guix import=E2=80=9D and tes=
ts in
=E2=80=98tests/npm-binary.scm=E2=80=99, similar to what=E2=80=99s done for =
the other importers.
The doc should clarify why it=E2=80=99s called that way and what the limita=
tions
are.
For tests, I=E2=80=99d recommend mocking the npmjs.org HTTP servers using
=E2=80=98with-http-server=E2=80=99 as in =E2=80=98tests/cpan.scm=E2=80=99.
Also, please add docstrings to top-level procedures.
> +;; TODO: Support other registries
> +(define *registry* "https://registry.npmjs.org")
> +(define *default-page* "https://www.npmjs.com/package")
The convention currently is more like =E2=80=98%registry=E2=80=99.
> +(define (http-error-code arglist)
> + (match arglist
> + (('http-error _ _ _ (code)) code)
> + (_ #f)))
Unused. :-)
> +(define (hash-url url)
> + "Downloads the resource at URL and computes the base32 hash for it."
> + (call-with-temporary-output-file
> + (lambda (temp port)
> + (begin ((@ (guix import utils) url-fetch) url temp)
> + (guix-hash-url temp)))))
Maybe something more like: (port-sha256 (http-fetch =E2=80=A6)) ?
> +(define (npm-package->package-sexp npm-package)
> + "Return the `package' s-expression for an NPM-PACKAGE."
> + (define (new-or-existing-inputs resolved-deps)
> + (map package-revision->input resolved-deps))
> +
> + (match npm-package
> + (($ <package-revision> name version home-page dependencies dev-depen=
dencies peer-dependencies license description dist)
Please use =E2=80=98match-record=E2=80=99 instead and keep lines below 80 c=
hars. :-)
> + (let* ((name (npm-name->name name))
> + (url (dist-tarball dist))
> + (home-page (if (string? home-page)
> + home-page
> + (string-append *default-page* "/" (uri-encode=
name))))
> + (synopsis description)
> + (resolved-deps (map (match-lambda (($ <versioned-package> na=
me version)
> + (resolve-package name (st=
ring->semver-range version)))) (append dependencies peer-dependencies)))
Likewise.
That=E2=80=99s it! Could you send an updated patch?
Thanks,
Ludo=E2=80=99.
guix-patches@HIDDEN:bug#62375; Package guix-patches.
Full text available.
Received: (at 62375) by debbugs.gnu.org; 22 Mar 2023 11:27:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 22 07:27:24 2023
Received: from localhost ([127.0.0.1]:33662 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pewcl-0004EO-1x
for submit <at> debbugs.gnu.org; Wed, 22 Mar 2023 07:27:24 -0400
Received: from mail1.fsfe.org ([217.69.89.151]:60960)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <jlicht@HIDDEN>) id 1pewcg-0004EC-M6
for 62375 <at> debbugs.gnu.org; Wed, 22 Mar 2023 07:27:21 -0400
From: jlicht@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501;
t=1679484437;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding;
bh=h9KQB9dGnO55Ur+khT2vGPB9YbpYuFP3tY1BOgTunuQ=;
b=M+MQMrd0RbNvFreCoU3zE+WmC+swantH8PoViOvuSHdhQekn3jv8nnH5XEmBMlzvPH5Nde
yxIvGCQNnaxPUj2sVbZKmVFj4SPEzK6xqhUMPNh+BXcGHp3fFRZKkvBkJZho5TlFcdRuDz
L+zYQCryg7I80/7N6DncjWhYLyj6i6M=
To: 62375 <at> debbugs.gnu.org
Subject: [PATCH] import: Add binary npm importer.
Date: Wed, 22 Mar 2023 12:27:16 +0100
Message-Id: <a41fe3ee10c48d36d3c84cf9f22ace8ca5bcd7c3.1679484420.git.jlicht@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: 3.0 (+++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: From: Jelle Licht <jlicht@HIDDEN> *
guix/scripts/import.scm:
(importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. *
guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them.
Content analysis details: (3.0 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
3.0 MANY_TO_CC Sent to 10+ recipients
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
X-Debbugs-Envelope-To: 62375
Cc: Timothy Sample <samplet@HIDDEN>, Josselin Poiret <dev@HIDDEN>,
Tobias Geerinckx-Rice <me@HIDDEN>,
Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
=?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>,
Christopher Baines <mail@HIDDEN>, Lars-Dominik Braun <lars@HIDDEN>,
Ricardo Wurmus <rekado@HIDDEN>, Jelle Licht <jlicht@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: From: Jelle Licht <jlicht@HIDDEN> * guix/scripts/import.scm:
(importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. *
guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them.
Content analysis details: (2.0 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
3.0 MANY_TO_CC Sent to 10+ recipients
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
-1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list
manager
From: Jelle Licht <jlicht@HIDDEN>
* guix/scripts/import.scm: (importers): Add "npm-binary".
* guix/import/npm-binary.scm: New file.
* guix/scripts/import/npm-binary.scm: New file.
* Makefile.am: Add them.
Co-authored-by: Timothy Sample <samplet@HIDDEN>
Co-authored-by: Lars-Dominik Braun <lars@HIDDEN>
---
Makefile.am | 2 +
guix/import/npm-binary.scm | 269 +++++++++++++++++++++++++++++
guix/scripts/import.scm | 2 +-
guix/scripts/import/npm-binary.scm | 113 ++++++++++++
4 files changed, 385 insertions(+), 1 deletion(-)
create mode 100644 guix/import/npm-binary.scm
create mode 100644 guix/scripts/import/npm-binary.scm
diff --git a/Makefile.am b/Makefile.am
index 23b939b674..52def58ae2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -288,6 +288,7 @@ MODULES = \
guix/import/kde.scm \
guix/import/launchpad.scm \
guix/import/minetest.scm \
+ guix/import/npm-binary.scm \
guix/import/opam.scm \
guix/import/print.scm \
guix/import/pypi.scm \
@@ -339,6 +340,7 @@ MODULES = \
guix/scripts/import/hexpm.scm \
guix/scripts/import/json.scm \
guix/scripts/import/minetest.scm \
+ guix/scripts/import/npm-binary.scm \
guix/scripts/import/opam.scm \
guix/scripts/import/pypi.scm \
guix/scripts/import/stackage.scm \
diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm
new file mode 100644
index 0000000000..f9b54263e4
--- /dev/null
+++ b/guix/import/npm-binary.scm
@@ -0,0 +1,269 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019, 2020 Timothy Sample <samplet@HIDDEN>
+;;; Copyright © 2020, 2023 Jelle Licht <jlicht@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import npm-binary)
+ #:use-module (guix import json)
+ #:use-module (guix import utils)
+ #:use-module (guix memoization)
+ #:use-module ((gnu services configuration) #:select (alist?))
+ #:use-module (guix utils)
+ #:use-module (gnu packages)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 regex)
+ #:use-module (ice-9 receive)
+ #:use-module (json)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-9)
+ #:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-41)
+ #:use-module (web client)
+ #:use-module (web response)
+ #:use-module (web uri)
+ #:export (npm-binary-recursive-import
+ npm-binary->guix-package
+ package-json->guix-package
+ make-versioned-package
+ name+version->symbol))
+
+;; Autoload Guile-Semver so we only have a soft dependency.
+(module-autoload! (current-module)
+ '(semver) '(string->semver semver? semver->string semver=? semver>?))
+(module-autoload! (current-module)
+ '(semver ranges) '(*semver-range-any* string->semver-range semver-range-contains?))
+
+;; Dist-tags
+(define-json-mapping <dist-tags> make-dist-tags dist-tags?
+ json->dist-tags
+ (latest dist-tags-latest "latest" string->semver))
+
+(define-record-type <versioned-package>
+ (make-versioned-package name version)
+ versioned-package?
+ (name versioned-package-name) ;string
+ (version versioned-package-version)) ;string
+
+(define (dependencies->versioned-packages entries)
+ (match entries
+ (((names . versions) ...)
+ (map make-versioned-package names versions))
+ (_ '())))
+
+(define (extract-license license-string)
+ (if (unspecified? license-string)
+ 'unspecified!
+ (spdx-string->license license-string)))
+
+(define-json-mapping <dist> make-dist dist?
+ json->dist
+ (tarball dist-tarball))
+
+(define (empty-or-string s)
+ (if (string? s) s ""))
+
+(define-json-mapping <package-revision> make-package-revision package-revision?
+ json->package-revision
+ (name package-revision-name)
+ (version package-revision-version "version" string->semver) ;semver
+ (home-page package-revision-home-page "homepage") ;string
+ (dependencies package-revision-dependencies "dependencies" ;list of versioned-package
+ dependencies->versioned-packages)
+ (dev-dependencies package-revision-dev-dependencies ;list of versioned-package
+ "devDependencies" dependencies->versioned-packages)
+ (peer-dependencies package-revision-peer-dependencies ;list of versioned-package
+ "peerDependencies" dependencies->versioned-packages)
+ (license package-revision-license "license" ;license | #f
+ (match-lambda
+ ((? unspecified?) #f)
+ ((? string? str) (spdx-string->license str))
+ ((? alist? alist)
+ (match (assoc "type" alist)
+ ((_ . (? string? type))
+ (spdx-string->license type))
+ (_ #f)))))
+ (description package-revision-description ; string
+ "description" empty-or-string)
+ (dist package-revision-dist "dist" json->dist)) ;dist
+
+(define (versions->package-revisions versions)
+ (match versions
+ (((version . package-spec) ...)
+ (map json->package-revision package-spec))
+ (_ '())))
+
+(define (versions->package-versions versions)
+ (match versions
+ (((version . package-spec) ...)
+ (map string->semver versions))
+ (_ '())))
+
+(define-json-mapping <meta-package> make-meta-package meta-package?
+ json->meta-package
+ (name meta-package-name) ;string
+ (description meta-package-description) ;string
+ (dist-tags meta-package-dist-tags "dist-tags" json->dist-tags) ;dist-tags
+ (revisions meta-package-revisions "versions" versions->package-revisions))
+
+;; TODO: Support other registries
+(define *registry* "https://registry.npmjs.org")
+(define *default-page* "https://www.npmjs.com/package")
+
+(define (lookup-meta-package name)
+ (let ((json (json-fetch (string-append *registry* "/" (uri-encode name)))))
+ (and=> json json->meta-package)))
+
+(define lookup-meta-package* (memoize lookup-meta-package))
+
+(define (http-error-code arglist)
+ (match arglist
+ (('http-error _ _ _ (code)) code)
+ (_ #f)))
+
+(define (meta-package-versions meta)
+ (map package-revision-version
+ (meta-package-revisions meta)))
+
+(define (meta-package-latest meta)
+ (and=> (meta-package-dist-tags meta) dist-tags-latest))
+
+(define* (meta-package-package meta #:optional
+ (version (meta-package-latest meta)))
+ (match version
+ ((? semver?) (find (lambda (revision)
+ (semver=? version (package-revision-version revision)))
+ (meta-package-revisions meta)))
+ ((? string?) (meta-package-package meta (string->semver version)))
+ (_ #f)))
+
+(define* (semver-latest svs #:optional (svr *semver-range-any*))
+ (find (cut semver-range-contains? svr <>)
+ (sort svs semver>?)))
+
+(define* (resolve-package name #:optional (svr *semver-range-any*))
+ (let ((meta (lookup-meta-package* name)))
+ (and meta
+ (let* ((version (semver-latest (or (meta-package-versions meta) '()) svr))
+ (pkg (meta-package-package meta version)))
+ pkg))))
+
+
+;;;
+;;; Converting packages
+;;;
+
+(define (hash-url url)
+ "Downloads the resource at URL and computes the base32 hash for it."
+ (call-with-temporary-output-file
+ (lambda (temp port)
+ (begin ((@ (guix import utils) url-fetch) url temp)
+ (guix-hash-url temp)))))
+
+(define (npm-name->name npm-name)
+ "Return a Guix package name for the npm package with name NPM-NAME."
+ (define (clean name)
+ (string-map (lambda (chr) (if (char=? chr #\/) #\- chr))
+ (string-filter (negate (cut char=? <> #\@)) name)))
+ (guix-name "node-" (clean npm-name)))
+
+(define (name+version->symbol name version)
+ (string->symbol (string-append name "-" version)))
+
+(define (package-revision->symbol package)
+ (let* ((npm-name (package-revision-name package))
+ (version (semver->string (package-revision-version package)))
+ (name (npm-name->name npm-name)))
+ (name+version->symbol name version)))
+
+(define (package-revision->input package)
+ "Return the `inputs' entry for PACKAGE."
+ (let* ((npm-name (package-revision-name package))
+ (name (npm-name->name npm-name)))
+ `(,name
+ (,'unquote ,(package-revision->symbol package)))))
+
+(define (npm-package->package-sexp npm-package)
+ "Return the `package' s-expression for an NPM-PACKAGE."
+ (define (new-or-existing-inputs resolved-deps)
+ (map package-revision->input resolved-deps))
+
+ (match npm-package
+ (($ <package-revision> name version home-page dependencies dev-dependencies peer-dependencies license description dist)
+ (let* ((name (npm-name->name name))
+ (url (dist-tarball dist))
+ (home-page (if (string? home-page)
+ home-page
+ (string-append *default-page* "/" (uri-encode name))))
+ (synopsis description)
+ (resolved-deps (map (match-lambda (($ <versioned-package> name version)
+ (resolve-package name (string->semver-range version)))) (append dependencies peer-dependencies)))
+ (peer-names (map versioned-package-name peer-dependencies))
+ ;; lset-difference for treating peer-dependencies as dependencies, which leads to dependency cycles.
+ ;; lset-union for treating them as (ignored) dev-dependencies, which leads to broken packages.
+ (dev-names (lset-union string= (map versioned-package-name dev-dependencies) peer-names))
+ (extra-phases (match dev-names
+ (() '())
+ ((dev-names ...)
+ `((add-after 'patch-dependencies 'delete-dev-dependencies
+ (lambda _
+ (delete-dependencies '(,@(reverse dev-names))))))))))
+ (values
+ `(package
+ (name ,name)
+ (version ,(semver->string (package-revision-version npm-package)))
+ (source (origin
+ (method url-fetch)
+ (uri ,url)
+ (sha256 (base32 ,(hash-url url)))))
+ (build-system node-build-system)
+ (arguments
+ '(#:tests? #f
+ #:phases (modify-phases %standard-phases
+ (delete 'build)
+ ,@extra-phases)))
+ ,@(match dependencies
+ (() '())
+ ((dependencies ...)
+ `((inputs
+ (,'quasiquote ,(map package-revision->input resolved-deps))))))
+ (home-page ,home-page)
+ (synopsis ,synopsis)
+ (description ,description)
+ (license ,license))
+ (map (match-lambda (($ <package-revision> name version)
+ (list name (semver->string version))))
+ resolved-deps))))
+ (_ #f)))
+
+
+;;;
+;;; Interface
+;;;
+
+(define npm-binary->guix-package
+ (lambda* (name #:key (version *semver-range-any*) #:allow-other-keys)
+ (let* ((svr (match version
+ ((? string?) (string->semver-range version))
+ (_ version)))
+ (pkg (resolve-package name svr)))
+ (and=> pkg npm-package->package-sexp))))
+
+(define* (npm-binary-recursive-import package-name #:key version)
+ (recursive-import package-name
+ #:repo->guix-package (memoize npm-binary->guix-package)
+ #:version version
+ #:guix-name npm-name->name))
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index f84a964a53..dccf6488b2 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -47,7 +47,7 @@ (define %standard-import-options '())
(define importers '("gnu" "pypi" "cpan" "hackage" "stackage" "egg" "elpa"
"gem" "go" "cran" "crate" "texlive" "json" "opam"
- "minetest" "elm" "hexpm"))
+ "minetest" "elm" "hexpm" "npm-binary"))
(define (resolve-importer name)
(let ((module (resolve-interface
diff --git a/guix/scripts/import/npm-binary.scm b/guix/scripts/import/npm-binary.scm
new file mode 100644
index 0000000000..825c43bbc3
--- /dev/null
+++ b/guix/scripts/import/npm-binary.scm
@@ -0,0 +1,113 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2014 David Thompson <davet@HIDDEN>
+;;; Copyright © 2018 Ricardo Wurmus <rekado@HIDDEN>
+;;; Copyright © 2019 Timothy Sample <samplet@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts import npm-binary)
+ #:use-module (guix ui)
+ #:use-module (guix utils)
+ #:use-module (guix scripts)
+ #:use-module (guix import npm-binary)
+ #:use-module (guix scripts import)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-37)
+ #:use-module (srfi srfi-41)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 format)
+ #:export (guix-import-npm-binary))
+
+
+;;;
+;;; Command-line options.
+;;;
+
+(define %default-options
+ '())
+
+(define (show-help)
+ (display (G_ "Usage: guix import npm-binary PACKAGE-NAME [VERSION]
+Import and convert the NPM package PACKAGE-NAME using the
+`npm-build-system' (but without building the package from source)."))
+ (display (G_ "
+ -h, --help display this help and exit"))
+ (display (G_ "
+ -r, --recursive import packages recursively"))
+ (display (G_ "
+ -V, --version display version information and exit"))
+ (newline)
+ (show-bug-report-information))
+
+(define %options
+ ;; Specification of the command-line options.
+ (cons* (option '(#\h "help") #f #f
+ (lambda args
+ (show-help)
+ (exit 0)))
+ (option '(#\V "version") #f #f
+ (lambda args
+ (show-version-and-exit "guix import npm-binary")))
+ (option '(#\r "recursive") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'recursive #t result)))
+ %standard-import-options))
+
+
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-import-npm-binary . args)
+ (define (parse-options)
+ ;; Return the alist of option values.
+ (args-fold* args %options
+ (lambda (opt name arg result)
+ (leave (G_ "~A: unrecognized option~%") name))
+ (lambda (arg result)
+ (alist-cons 'argument arg result))
+ %default-options))
+
+ (let* ((opts (parse-options))
+ (args (filter-map (match-lambda
+ (('argument . value)
+ value)
+ (_ #f))
+ (reverse opts))))
+ (let loop ((args args))
+ (match args
+ ((package-name version)
+ (if (assoc-ref opts 'recursive)
+ ;; Recursive import
+ (map (match-lambda
+ ((and ('package ('name name) ('version version) . rest) pkg)
+ `(define-public ,(name+version->symbol name version)
+ ,pkg))
+ (_ #f))
+ (npm-binary-recursive-import package-name #:version version))
+ ;; Single import
+ (let ((sexp (npm-binary->guix-package package-name #:version version)))
+ (unless sexp
+ (leave (G_ "failed to download meta-data for package '~a@~a'~%")
+ package-name version))
+ sexp)))
+ ((package-name)
+ (loop (list package-name "*")))
+ (()
+ (leave (G_ "too few arguments~%")))
+ ((many ...)
+ (leave (G_ "too many arguments~%")))))))
--
2.39.2
guix-patches@HIDDEN:bug#62375; Package guix-patches.
Full text available.Received: (at submit) by debbugs.gnu.org; 22 Mar 2023 11:25:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 22 07:25:20 2023 Received: from localhost ([127.0.0.1]:33651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pewal-0004AY-Mk for submit <at> debbugs.gnu.org; Wed, 22 Mar 2023 07:25:20 -0400 Received: from lists.gnu.org ([209.51.188.17]:39482) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <jlicht@HIDDEN>) id 1pewai-0004AN-NI for submit <at> debbugs.gnu.org; Wed, 22 Mar 2023 07:25:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <jlicht@HIDDEN>) id 1pewag-00022c-Bj for guix-patches@HIDDEN; Wed, 22 Mar 2023 07:25:14 -0400 Received: from mail1.fsfe.org ([2001:aa8:ffed:f5f3::151]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <jlicht@HIDDEN>) id 1pewae-0006UW-1q; Wed, 22 Mar 2023 07:25:14 -0400 From: jlicht@HIDDEN DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1679484305; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=aHvktX4KrRbyjP0CtnfUSBovzYlZw7d91zc9mKc7tmQ=; b=FLOiOltT2xPz8JO/KAnflzwoMj0XJZ7VoxQh+MBMj7nw/TLrNNQXJ/O27uoGZYQlvibS/f AqO3sBZ9gkc5p2XSGyhb05/5s32iAM6AFG3nYskUnXLghlQA5W31illC+NCAAuRCffKxzr ACnZQn0Clg2u9zFmDpVDjDhPFhHkLsw= To: guix-patches@HIDDEN Subject: [PATCH 0/1] npm binary importer Date: Wed, 22 Mar 2023 12:25:04 +0100 Message-Id: <cover.1679484068.git.jlicht@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2001:aa8:ffed:f5f3::151; envelope-from=jlicht@HIDDEN; helo=mail1.fsfe.org X-Spam_score_int: -70 X-Spam_score: -7.1 X-Spam_bar: ------- X-Spam_report: (-7.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: dev@HIDDEN, me@HIDDEN, zimon.toutoune@HIDDEN, othacehe@HIDDEN, ludo@HIDDEN, mail@HIDDEN, rekado@HIDDEN, Jelle Licht <jlicht@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.3 (--) From: Jelle Licht <jlicht@HIDDEN> Folks, Here a revised patch to add the npm binary importer. To give some context, 'binary' here refers to the fact that this downloads archives straight from the npm registry at https://registry.npmjs.org. Some of these downloaded archives may not contain the original sources, so unless properly vetted the output of this importer does not allow one to easily generate package expressions for inclusion in guix's collection of packages. It should work as-is for most simple NPM packages. As noted in an inline comment somewhere, the way both npm and our very own node-build-system treats peer dependencies may require some manual intervention from time to time to hook in the right dependency at the right spot. The upside here is that when this happens, it's either trivial for a human to spot and fix, or wholly incompatible with our current approach so not fixable without writing a custom build system to deal with dependency cycles :-). Please test and review, let us get this merged as it might be useful to many people for building out their personal channels and/or package expressions. Special thanks to Timothy Sample and Lars-Dominik Braun for involvement in realising this, way too long ago. Jelle Licht (1): import: Add binary npm importer. Makefile.am | 2 + guix/import/npm-binary.scm | 269 +++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/npm-binary.scm | 113 ++++++++++++ 4 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 guix/import/npm-binary.scm create mode 100644 guix/scripts/import/npm-binary.scm -- 2.39.2
jlicht@HIDDEN:guix-patches@HIDDEN.
Full text available.guix-patches@HIDDEN:bug#62375; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.