GNU bug report logs - #37525
[PATCH 0/3] Recursive option for crate importer

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37525 in the body.
You can then email your comments to 37525 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: guix-patches <at> gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH 0/3] Recursive option for crate importer
Date: Thu, 26 Sep 2019 14:28:12 -0400

This add recursive functionalty for the crate importer. Currently it doesn't respect the versioning. To do that we will need to add semver functionality as well so that we can look up the correct version of dependencies. 

Martin Becze (3):
  added recusive import functionality to the crate importer
  updated the crate import script to accept recursive option
  updated docs for import crate

 doc/guix.texi                 |  7 +++++++
 guix/import/crate.scm         | 29 +++++++++++++++++++----------
 guix/scripts/import/crate.scm | 35 +++++++++++++++++++++++++----------
 3 files changed, 51 insertions(+), 20 deletions(-)

-- 
2.23.0

Message #8 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH 1/3] added recusive import functionality to the crate importer
Date: Thu, 26 Sep 2019 14:31:01 -0400

---
 guix/import/crate.scm | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index fd1974eae8..8dc014d232 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -40,6 +40,7 @@
   #:use-module (srfi srfi-26)
   #:export (crate->guix-package
             guix-package->crate-name
+            crate-recursive-import
             %crate-updater))
 
 
@@ -218,16 +219,24 @@ latest version of CRATE-NAME."
               (cargo-development-inputs
                (sort (map crate-dependency-id dev-dep-crates)
                      string-ci<?)))
-         (make-crate-sexp #:name crate-name
-                          #:version (crate-version-number version*)
-                          #:cargo-inputs cargo-inputs
-                          #:cargo-development-inputs cargo-development-inputs
-                          #:home-page (or (crate-home-page crate)
-                                          (crate-repository crate))
-                          #:synopsis (crate-description crate)
-                          #:description (crate-description crate)
-                          #:license (and=> (crate-version-license version*)
-                                           string->license)))))
+         (values
+          (make-crate-sexp #:name crate-name
+                           #:version (crate-version-number version*)
+                           #:cargo-inputs cargo-inputs
+                           #:cargo-development-inputs cargo-development-inputs
+                           #:home-page (or (crate-home-page crate)
+                                           (crate-repository crate))
+                           #:synopsis (crate-description crate)
+                           #:description (crate-description crate)
+                           #:license (and=> (crate-version-license version*)
+                                            string->license))
+          (append cargo-inputs cargo-development-inputs)))))
+
+(define (crate-recursive-import crate-name)
+  (recursive-import crate-name #f
+                    #:repo->guix-package (lambda (name repo)
+                                           (crate->guix-package name))
+                    #:guix-name crate-name->package-name))
 
 (define (guix-package->crate-name package)
   "Return the crate name of PACKAGE."
-- 
2.23.0

Message #11 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH 2/3] updated the crate import script to accept recursive option
Date: Thu, 26 Sep 2019 14:31:02 -0400

---
 guix/scripts/import/crate.scm | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)

diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 7ae8638911..19c8277d14 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -28,6 +28,7 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-37)
+  #:use-module (srfi srfi-41)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
   #:export (guix-import-crate))
@@ -46,6 +47,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
   (display (G_ "
   -h, --help             display this help and exit"))
   (display (G_ "
+  -r, --recursive        import packages recursively"))
+  (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
   (show-bug-report-information))
@@ -59,6 +62,9 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (option '(#\V "version") #f #f
                  (lambda args
                    (show-version-and-exit "guix import crate")))
+         (option '(#\r "recursive") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'recursive #t result)))
          %standard-import-options))
 
 
@@ -79,22 +85,31 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
 
   (let* ((opts (parse-options))
          (args (filter-map (match-lambda
-                            (('argument . value)
-                             value)
-                            (_ #f))
+                             (('argument . value)
+                              value)
+                             (_ #f))
                            (reverse opts))))
     (match args
       ((spec)
        (define-values (name version)
          (package-name->name+version spec))
 
-       (let ((sexp (crate->guix-package name version)))
-         (unless sexp
-           (leave (G_ "failed to download meta-data for package '~a'~%")
-                  (if version
-                      (string-append name "@" version)
-                      name)))
-         sexp))
+       (if (assoc-ref opts 'recursive)
+           (map (match-lambda
+                  ((and ('package ('name name) . rest) pkg)
+                   `(define-public ,(string->symbol name)
+                      ,pkg))
+                  (_ #f))
+                (reverse
+                 (stream->list
+                  (crate-recursive-import name))))
+           (let ((sexp (crate->guix-package name version)))
+             (unless sexp
+               (leave (G_ "failed to download meta-data for package '~a'~%")
+                      (if version
+                          (string-append name "@" version)
+                          name)))
+             sexp)))
       (()
        (leave (G_ "too few arguments~%")))
       ((many ...)
-- 
2.23.0

Message #14 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH 3/3] updated docs for import crate
Date: Thu, 26 Sep 2019 14:31:03 -0400

---
 doc/guix.texi | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 14c4514b31..3b3645b854 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9054,6 +9054,13 @@ in Guix.
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}, as in this example:
 
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+@end table
+
 @example
 guix import crate blake2-rfc
 @end example
-- 
2.23.0

Message #17 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Subject: previous patch had mistake in docs
Date: Tue, 01 Oct 2019 13:17:25 -0700

Hiy yall, the previous patch had mistake in docs. a new patch with
change log incoming!

Message #20 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH v2 3/3] updated docs for import crate
Date: Tue,  1 Oct 2019 16:54:58 -0400

* doc/guix.texi update docs
---

This fixes a bug in the prevous patch. And adds better commits to the git log!

 doc/guix.texi | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 14c4514b31..a49c9d2b26 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9054,8 +9054,14 @@ in Guix.
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}, as in this example:
 
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+
 @example
-guix import crate blake2-rfc
+guix import crate -r blake2-rfc
 @end example
 
 The crate importer also allows you to specify a version string:
-- 
2.23.0

Message #23 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH v2 1/3] added recusive import functionality to the crate
 importer
Date: Tue,  1 Oct 2019 16:54:56 -0400

* /guix/import/crate.scm (crate-recursive-import crate-name): added recusive import proc
* /guix/import/crate.scm (crate->guix-package): return inputs for use by the recurive importer
---
 guix/import/crate.scm | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index fd1974eae8..8dc014d232 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -40,6 +40,7 @@
   #:use-module (srfi srfi-26)
   #:export (crate->guix-package
             guix-package->crate-name
+            crate-recursive-import
             %crate-updater))
 
 
@@ -218,16 +219,24 @@ latest version of CRATE-NAME."
               (cargo-development-inputs
                (sort (map crate-dependency-id dev-dep-crates)
                      string-ci<?)))
-         (make-crate-sexp #:name crate-name
-                          #:version (crate-version-number version*)
-                          #:cargo-inputs cargo-inputs
-                          #:cargo-development-inputs cargo-development-inputs
-                          #:home-page (or (crate-home-page crate)
-                                          (crate-repository crate))
-                          #:synopsis (crate-description crate)
-                          #:description (crate-description crate)
-                          #:license (and=> (crate-version-license version*)
-                                           string->license)))))
+         (values
+          (make-crate-sexp #:name crate-name
+                           #:version (crate-version-number version*)
+                           #:cargo-inputs cargo-inputs
+                           #:cargo-development-inputs cargo-development-inputs
+                           #:home-page (or (crate-home-page crate)
+                                           (crate-repository crate))
+                           #:synopsis (crate-description crate)
+                           #:description (crate-description crate)
+                           #:license (and=> (crate-version-license version*)
+                                            string->license))
+          (append cargo-inputs cargo-development-inputs)))))
+
+(define (crate-recursive-import crate-name)
+  (recursive-import crate-name #f
+                    #:repo->guix-package (lambda (name repo)
+                                           (crate->guix-package name))
+                    #:guix-name crate-name->package-name))
 
 (define (guix-package->crate-name package)
   "Return the crate name of PACKAGE."
-- 
2.23.0

Message #26 received at 37525 <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: 37525 <at> debbugs.gnu.org
Cc: Martin Becze <mjbecze <at> riseup.net>
Subject: [PATCH v2 2/3] updated the crate import script to accept recursive
 option
Date: Tue,  1 Oct 2019 16:54:57 -0400

* guix/scripts/import/crate.scm (show-help, guix-import-crate): added recursive option
---
 guix/scripts/import/crate.scm | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)

diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 7ae8638911..19c8277d14 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -28,6 +28,7 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-37)
+  #:use-module (srfi srfi-41)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
   #:export (guix-import-crate))
@@ -46,6 +47,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
   (display (G_ "
   -h, --help             display this help and exit"))
   (display (G_ "
+  -r, --recursive        import packages recursively"))
+  (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
   (show-bug-report-information))
@@ -59,6 +62,9 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (option '(#\V "version") #f #f
                  (lambda args
                    (show-version-and-exit "guix import crate")))
+         (option '(#\r "recursive") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'recursive #t result)))
          %standard-import-options))
 
 
@@ -79,22 +85,31 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
 
   (let* ((opts (parse-options))
          (args (filter-map (match-lambda
-                            (('argument . value)
-                             value)
-                            (_ #f))
+                             (('argument . value)
+                              value)
+                             (_ #f))
                            (reverse opts))))
     (match args
       ((spec)
        (define-values (name version)
          (package-name->name+version spec))
 
-       (let ((sexp (crate->guix-package name version)))
-         (unless sexp
-           (leave (G_ "failed to download meta-data for package '~a'~%")
-                  (if version
-                      (string-append name "@" version)
-                      name)))
-         sexp))
+       (if (assoc-ref opts 'recursive)
+           (map (match-lambda
+                  ((and ('package ('name name) . rest) pkg)
+                   `(define-public ,(string->symbol name)
+                      ,pkg))
+                  (_ #f))
+                (reverse
+                 (stream->list
+                  (crate-recursive-import name))))
+           (let ((sexp (crate->guix-package name version)))
+             (unless sexp
+               (leave (G_ "failed to download meta-data for package '~a'~%")
+                      (if version
+                          (string-append name "@" version)
+                          name)))
+             sexp)))
       (()
        (leave (G_ "too few arguments~%")))
       ((many ...)
-- 
2.23.0

Message #31 received at 37525-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Martin Becze <mjbecze <at> riseup.net>
Cc: 37525-done <at> debbugs.gnu.org
Subject: Re: [bug#37525] [PATCH v2 1/3] added recusive import functionality to
 the crate importer
Date: Tue, 01 Oct 2019 23:33:07 +0200

Hi,

Applied!

Note that I merged patches #2 and #3, because we usually document
features as we add them, not separately.  I had to fix up the Texinfo
markup so that ‘--recursive’ is described under ‘crate’.

Anyway it seems to work well, thanks for the patches!

Ludo’.

Message #34 received at 37525-done <at> debbugs.gnu.org (full text, mbox):

From: Martin Becze <mjbecze <at> riseup.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 37525-done <at> debbugs.gnu.org
Subject: Re: [bug#37525] [PATCH v2 1/3] added recusive import functionality to
 the crate importer
Date: Tue, 01 Oct 2019 14:52:47 -0700

On 2019-10-01 21:33, Ludovic Courtès wrote:
> Hi,
> 
> Applied!
> 
> Note that I merged patches #2 and #3, because we usually document
> features as we add them, not separately.  I had to fix up the Texinfo
> markup so that ‘--recursive’ is described under ‘crate’.
> 
> Anyway it seems to work well, thanks for the patches!
> 
> Ludo’.

Awesome thanks! 

> features as we add them, not separately.

Do you mean in the same commit that makes the changes to the code? or
when you apply patches to master?

Message #37 received at 37525-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Martin Becze <mjbecze <at> riseup.net>
Cc: 37525-done <at> debbugs.gnu.org
Subject: Re: [bug#37525] [PATCH v2 1/3] added recusive import functionality to
 the crate importer
Date: Wed, 02 Oct 2019 16:28:04 +0200

Martin Becze <mjbecze <at> riseup.net> skribis:

> On 2019-10-01 21:33, Ludovic Courtès wrote:

[...]

>> features as we add them, not separately.
>
> Do you mean in the same commit that makes the changes to the code? or
> when you apply patches to master?

I mean that when we add a user-visible change, such as a new
command-line option, we should document it in the same commit that
actually adds the feature.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.