GNU bug report logs - #35935
[PATCH] guix: import: simplify recursive import

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35935 in the body.
You can then email your comments to 35935 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: guix-patches <at> gnu.org
Cc: Robert Vollmert <rob <at> vllmrt.net>
Subject: [PATCH] guix: import: simplify recursive import
Date: Mon, 27 May 2019 22:07:31 +0200

This simplifies the logic of recursive-import, intending no
major functional changes. The package import function is no
longer called twice per package. Failed imports now make it
to the package stream as '() instead of #f.

* guix/import/utils.scm: Simplify recursive-import.
---
 guix/import/utils.scm | 86 ++++++++++++++++---------------------------
 1 file changed, 32 insertions(+), 54 deletions(-)

diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 516c0cfaa2..ff548b809a 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -378,57 +378,35 @@ separated by PRED."
                            #:allow-other-keys)
   "Generate a stream of package expressions for PACKAGE-NAME and all its
 dependencies."
-  (receive (package . dependencies)
-      (repo->guix-package package-name repo)
-    (if (not package)
-        stream-null
-
-        ;; Generate a lazy stream of package expressions for all unknown
-        ;; dependencies in the graph.
-        (let* ((make-state (lambda (queue done)
-                             (cons queue done)))
-               (next       (match-lambda
-                             (((next . rest) . done) next)))
-               (imported   (match-lambda
-                             ((queue . done) done)))
-               (done?      (match-lambda
-                             ((queue . done)
-                              (zero? (length queue)))))
-               (unknown?   (lambda* (dependency #:optional (done '()))
-                             (and (not (member dependency
-                                               done))
-                                  (null? (find-packages-by-name
-                                          (guix-name dependency))))))
-               (update     (lambda (state new-queue)
-                             (match state
-                               (((head . tail) . done)
-                                (make-state (lset-difference
-                                             equal?
-                                             (lset-union equal? new-queue tail)
-                                             done)
-                                            (cons head done)))))))
-          (stream-cons
-           package
-           (stream-unfold
-            ;; map: produce a stream element
-            (lambda (state)
-              (repo->guix-package (next state) repo))
-
-            ;; predicate
-            (negate done?)
-
-            ;; generator: update the queue
-            (lambda (state)
-              (receive (package . dependencies)
-                  (repo->guix-package (next state) repo)
-                (if package
-                    (update state (filter (cut unknown? <>
-                                               (cons (next state)
-                                                     (imported state)))
-                                          (car dependencies)))
-                    ;; TODO: Try the other archives before giving up
-                    (update state (imported state)))))
-
-            ;; initial state
-            (make-state (filter unknown? (car dependencies))
-                        (list package-name))))))))
+  (define (exists? dependency)
+    (not (null? (find-packages-by-name (guix-name dependency)))))
+  (define initial-state (list #f (list package-name) (list)))
+  (define (step state)
+    (match state
+      ((prev (next . rest) done)
+       (define (handle? dep)
+         (and
+           (not (equal? dep next))
+           (not (member dep done))
+           (not (exists? dep))))
+       (receive (package . dependencies) (repo->guix-package next repo)
+         (list
+           (if package package '()) ;; default #f on failure would interrupt
+           (if package
+             (lset-union equal? rest (filter handle? (car dependencies)))
+             rest)
+           (cons next done))))
+      ((prev '() done)
+       (list #f '() done))))
+
+  ;; Generate a lazy stream of package expressions for all unknown
+  ;; dependencies in the graph.
+  (stream-unfold
+    ;; map: produce a stream element
+    (match-lambda ((latest queue done) latest))
+    ;; predicate
+    (match-lambda ((latest queue done) latest))
+    ;; generator: update the queue
+    step
+    ;; initial state
+    (step initial-state)))
-- 
2.20.1 (Apple Git-117)

Message #8 received at 35935 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: Oleg Pykhalov <go.wigust <at> gmail.com>, 35935 <at> debbugs.gnu.org
Subject: Re: [bug#35935] [PATCH] guix: import: simplify recursive import
Date: Sun, 02 Jun 2019 21:52:45 +0200

Hello Robert,

Robert Vollmert <rob <at> vllmrt.net> skribis:

> This simplifies the logic of recursive-import, intending no
> major functional changes. The package import function is no
> longer called twice per package. Failed imports now make it
> to the package stream as '() instead of #f.
>
> * guix/import/utils.scm: Simplify recursive-import.
                         ^
Minor issue: please make sure to mention the modified entities here
(procedures, variables, etc.); see
<https://www.gnu.org/prep/standards/html_node/Change-Logs.html>.

I think Oleg worked on this part before; Oleg, could you comment
and/or apply?

‘tests/import-utils.scm’ doesn’t seem to be testing this procedure,
perhaps that’s something we should fix eventually.

Thanks,
Ludo’.

Message #11 received at 35935 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: 35935 <at> debbugs.gnu.org
Cc: Robert Vollmert <rob <at> vllmrt.net>
Subject: [PATCH] guix: import: Simplify recursive import.
Date: Sun,  2 Jun 2019 22:19:46 +0200

This simplifies the logic of recursive-import, intending no
major functional changes. The package import function is no
longer called twice per package. Failed imports now make it
to the package stream as '() instead of #f.

* guix/import/utils.scm (recursive-import): Simplify.
---
 guix/import/utils.scm | 86 ++++++++++++++++---------------------------
 1 file changed, 32 insertions(+), 54 deletions(-)

diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 516c0cfaa2..ff548b809a 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -378,57 +378,35 @@ separated by PRED."
                            #:allow-other-keys)
   "Generate a stream of package expressions for PACKAGE-NAME and all its
 dependencies."
-  (receive (package . dependencies)
-      (repo->guix-package package-name repo)
-    (if (not package)
-        stream-null
-
-        ;; Generate a lazy stream of package expressions for all unknown
-        ;; dependencies in the graph.
-        (let* ((make-state (lambda (queue done)
-                             (cons queue done)))
-               (next       (match-lambda
-                             (((next . rest) . done) next)))
-               (imported   (match-lambda
-                             ((queue . done) done)))
-               (done?      (match-lambda
-                             ((queue . done)
-                              (zero? (length queue)))))
-               (unknown?   (lambda* (dependency #:optional (done '()))
-                             (and (not (member dependency
-                                               done))
-                                  (null? (find-packages-by-name
-                                          (guix-name dependency))))))
-               (update     (lambda (state new-queue)
-                             (match state
-                               (((head . tail) . done)
-                                (make-state (lset-difference
-                                             equal?
-                                             (lset-union equal? new-queue tail)
-                                             done)
-                                            (cons head done)))))))
-          (stream-cons
-           package
-           (stream-unfold
-            ;; map: produce a stream element
-            (lambda (state)
-              (repo->guix-package (next state) repo))
-
-            ;; predicate
-            (negate done?)
-
-            ;; generator: update the queue
-            (lambda (state)
-              (receive (package . dependencies)
-                  (repo->guix-package (next state) repo)
-                (if package
-                    (update state (filter (cut unknown? <>
-                                               (cons (next state)
-                                                     (imported state)))
-                                          (car dependencies)))
-                    ;; TODO: Try the other archives before giving up
-                    (update state (imported state)))))
-
-            ;; initial state
-            (make-state (filter unknown? (car dependencies))
-                        (list package-name))))))))
+  (define (exists? dependency)
+    (not (null? (find-packages-by-name (guix-name dependency)))))
+  (define initial-state (list #f (list package-name) (list)))
+  (define (step state)
+    (match state
+      ((prev (next . rest) done)
+       (define (handle? dep)
+         (and
+           (not (equal? dep next))
+           (not (member dep done))
+           (not (exists? dep))))
+       (receive (package . dependencies) (repo->guix-package next repo)
+         (list
+           (if package package '()) ;; default #f on failure would interrupt
+           (if package
+             (lset-union equal? rest (filter handle? (car dependencies)))
+             rest)
+           (cons next done))))
+      ((prev '() done)
+       (list #f '() done))))
+
+  ;; Generate a lazy stream of package expressions for all unknown
+  ;; dependencies in the graph.
+  (stream-unfold
+    ;; map: produce a stream element
+    (match-lambda ((latest queue done) latest))
+    ;; predicate
+    (match-lambda ((latest queue done) latest))
+    ;; generator: update the queue
+    step
+    ;; initial state
+    (step initial-state)))
-- 
2.20.1 (Apple Git-117)

Message #14 received at 35935 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 35935 <at> debbugs.gnu.org
Subject: Re: [bug#35935] [PATCH] guix: import: simplify recursive import
Date: Sun, 2 Jun 2019 22:23:56 +0200

> On 2. Jun 2019, at 21:52, Ludovic Courtès <ludo <at> gnu.org> wrote:
> Robert Vollmert <rob <at> vllmrt.net> skribis:
>> * guix/import/utils.scm: Simplify recursive-import.
>                         ^
> Minor issue: please make sure to mention the modified entities here
> (procedures, variables, etc.); see
> <https://www.gnu.org/prep/standards/html_node/Change-Logs.html>.

Fixed, thanks for the heads up.

> I think Oleg worked on this part before; Oleg, could you comment
> and/or apply?
> 
> ‘tests/import-utils.scm’ doesn’t seem to be testing this procedure,
> perhaps that’s something we should fix eventually.

Agreed. It does seem to be covered somewhat by tests/gem.scm.
I also tested it a bit by hand, with some haskell packages.

Message #17 received at 35935 <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: 35935 <at> debbugs.gnu.org
Subject: Re: [bug#35935] [PATCH] guix: import: simplify recursive import
Date: Mon, 03 Jun 2019 22:56:21 +0300

[Message part 1 (text/plain, inline)]

Hello Robert,

Robert Vollmert <rob <at> vllmrt.net> writes:

[…]

>> I think Oleg worked on this part before; Oleg, could you comment
>> and/or apply?
>> 
>> ‘tests/import-utils.scm’ doesn’t seem to be testing this procedure,
>> perhaps that’s something we should fix eventually.
>
> Agreed. It does seem to be covered somewhat by tests/gem.scm.
> I also tested it a bit by hand, with some haskell packages.

True, ‘tests/gem.scm’ file tests ‘recursive-import’ procedure by
invoking ‘gem-recursive-import’.  I think this test is good enough for
our purpose - use ‘recursive-import’ in other package importers and make
sure it works. Thoughts?

I also added a copyright line if you don't mind:

    ;;; Copyright © 2019 Robert Vollmert <rob <at> vllmrt.net>

Is it OK, Robert?  I tested manually with ‘gem’ and ‘elpa’ recursive
importers and ready to push :-)

Thanks,
Oleg.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 35935 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 35935 <at> debbugs.gnu.org
Subject: Re: [bug#35935] [PATCH] guix: import: simplify recursive import
Date: Mon, 3 Jun 2019 21:59:24 +0200

Hello Oleg,

thanks for having a look.

> On 3. Jun 2019, at 21:56, Oleg Pykhalov <go.wigust <at> gmail.com> wrote:

[…]

> I also added a copyright line if you don't mind:
> 
>    ;;; Copyright © 2019 Robert Vollmert <rob <at> vllmrt.net>
> 
> Is it OK, Robert?  I tested manually with ‘gem’ and ‘elpa’ recursive
> importers and ready to push :-)

Yes, of course. (Should I generally be adding copyright lines when edit
a file?

Cheers
Robert

Message #25 received at 35935-done <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: 35935-done <at> debbugs.gnu.org
Subject: Re: [bug#35935] [PATCH] guix: import: simplify recursive import
Date: Mon, 03 Jun 2019 23:31:00 +0300

[Message part 1 (text/plain, inline)]

Robert Vollmert <rob <at> vllmrt.net> writes:

[…]

>> I also added a copyright line if you don't mind:
>> 
>>    ;;; Copyright © 2019 Robert Vollmert <rob <at> vllmrt.net>
>> 
>> Is it OK, Robert?  I tested manually with ‘gem’ and ‘elpa’ recursive
>> importers and ready to push :-)
>
> Yes, of course. (Should I generally be adding copyright lines when edit
> a file?

Yes.

Pushed as 5b315f3ea93020df52bc11105064a1398687e572 to master.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.