GNU bug report logs - #62036
[PATCH] guix: packages: Consider 'patches' by 'package-direct-sources'.

Previous Next

Package: guix-patches;

Reported by: Simon Tournier <zimon.toutoune <at> gmail.com>

Date: Tue, 7 Mar 2023 17:19:01 UTC

Severity: normal

Tags: patch

Done: Ludovic Courtès <ludo <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 62036 in the body.
You can then email your comments to 62036 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to mail <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, rekado <at> elephly.net, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:
bug#62036; Package guix-patches. (Tue, 07 Mar 2023 17:19:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Simon Tournier <zimon.toutoune <at> gmail.com>:
New bug report received and forwarded. Copy sent to mail <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, rekado <at> elephly.net, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org. (Tue, 07 Mar 2023 17:19:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: Simon Tournier <zimon.toutoune <at> gmail.com>
Subject: [PATCH] guix: packages: Consider 'patches' by
 'package-direct-sources'.
Date: Tue,  7 Mar 2023 18:18:33 +0100
* guix/packages.scm (package-direct-sources): Return 'origin' from 'patches'.
* tests/packages.scm: Test it.
---
 guix/packages.scm  | 10 ++++++++--
 tests/packages.scm | 17 ++++++++++++++++-
 2 files changed, 24 insertions(+), 3 deletions(-)

Hi,

This patch improves some coverage when listing all the origins (fixed outputs)
of some packages.  The procedure 'packages-direct-sources' already lists the
'origin' as inputs but is missing the 'origin' as 'patches'.  For an instance
of such, see the package 'ntp'.

Note that it is not recursive.  If an 'origin' is listed in 'patches' or
'inputs' and that origin also contains 'patches' which is another 'origin'
then it will be missed.  For now, it is not covered because it somehow adds
complexity without an instance (yet) of such case.

Cheers,
simon




diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..0f88564ab4 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2021 Chris Marusich <cmmarusich <at> gmail.com>
 ;;; Copyright © 2022 Maxime Devos <maximedevos <at> telenet.be>
 ;;; Copyright © 2022 jgart <jgart <at> dismail.de>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1239,8 +1240,13 @@ (define-syntax modify-inputs
 
 (define (package-direct-sources package)
   "Return all source origins associated with PACKAGE; including origins in
-PACKAGE's inputs."
-  `(,@(or (and=> (package-source package) list) '())
+PACKAGE's inputs and patches."
+  (define (expand source)
+    (append
+     (list source)
+     (filter origin? (origin-patches source))))
+
+  `(,@(or (and=> (package-source package) expand) '())
     ,@(filter-map (match-lambda
                    ((_ (? origin? orig) _ ...)
                     orig)
diff --git a/tests/packages.scm b/tests/packages.scm
index f58c47817b..27fb918f90 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2018 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;; Copyright © 2021 Maxime Devos <maximedevos <at> telenet.be>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -418,12 +419,15 @@ (define read-at
 (let* ((o (dummy-origin))
        (u (dummy-origin))
        (i (dummy-origin))
+       (j (dummy-origin (patches (list o))))
        (a (dummy-package "a"))
        (b (dummy-package "b" (inputs (list a i))))
        (c (package (inherit b) (source o)))
        (d (dummy-package "d"
             (build-system trivial-build-system)
-            (source u) (inputs (list c)))))
+            (source u) (inputs (list c))))
+       (e (dummy-package "e" (source j)))
+       (f (package (inherit e) (inputs (list u)))))
   (test-assert "package-direct-sources, no source"
     (null? (package-direct-sources a)))
   (test-equal "package-direct-sources, #f source"
@@ -437,6 +441,17 @@ (define read-at
       (and (= (length (pk 's-sources s)) 2)
            (member o s)
            (member i s))))
+  (test-assert "package-direct-sources, with patches"
+    (let ((s (package-direct-sources e)))
+      (and (= (length (pk 's-sources s)) 2)
+           (member o s)
+           (member j s))))
+  (test-assert "package-direct-sources, with patches and inputs"
+    (let ((s (package-direct-sources f)))
+      (and (= (length (pk 's-sources s)) 3)
+           (member o s)
+           (member j s)
+           (member u s))))
   (test-assert "package-transitive-sources"
     (let ((s (package-transitive-sources d)))
       (and (= (length (pk 'd-sources s)) 3)

base-commit: 723fc5df3e964fcecb09c7c6fd48f00f97e2e806
-- 
2.38.1





Information forwarded to guix-patches <at> gnu.org:
bug#62036; Package guix-patches. (Thu, 09 Mar 2023 19:44:01 GMT) Full text and rfc822 format available.

Message #8 received at 62036 <at> debbugs.gnu.org (full text, mbox):

From: Josselin Poiret <dev <at> jpoiret.xyz>
To: Simon Tournier <zimon.toutoune <at> gmail.com>, 62036 <at> debbugs.gnu.org
Cc: Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>,
 ludo <at> gnu.org, Christopher Baines <mail <at> cbaines.net>,
 Ricardo Wurmus <rekado <at> elephly.net>
Subject: Re: [bug#62036] [PATCH] guix: packages: Consider 'patches' by
 'package-direct-sources'.
Date: Thu, 09 Mar 2023 20:43:26 +0100
[Message part 1 (text/plain, inline)]
Hi Simon,

Simon Tournier <zimon.toutoune <at> gmail.com> writes:

>  (define (package-direct-sources package)
>    "Return all source origins associated with PACKAGE; including origins in
> -PACKAGE's inputs."
> -  `(,@(or (and=> (package-source package) list) '())
> +PACKAGE's inputs and patches."
> +  (define (expand source)
> +    (append
> +     (list source)
> +     (filter origin? (origin-patches source))))

* cough * (cons source (filter ...)) * cough *

Other than that, LGTM!  Tests worked fine on my end.  No idea what this
is used for though :p

Best,
-- 
Josselin Poiret
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches <at> gnu.org:
bug#62036; Package guix-patches. (Fri, 10 Mar 2023 12:10:02 GMT) Full text and rfc822 format available.

Message #11 received at 62036 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Josselin Poiret <dev <at> jpoiret.xyz>, 62036 <at> debbugs.gnu.org
Cc: Ricardo Wurmus <rekado <at> elephly.net>, Mathieu Othacehe <othacehe <at> gnu.org>,
 ludo <at> gnu.org, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Christopher Baines <mail <at> cbaines.net>
Subject: Re: [bug#62036] [PATCH] guix: packages: Consider 'patches' by
 'package-direct-sources'.
Date: Fri, 10 Mar 2023 12:33:06 +0100
Hi Josselin,

On jeu., 09 mars 2023 at 20:43, Josselin Poiret via Guix-patches via <guix-patches <at> gnu.org> wrote:

>> +    (append
>> +     (list source)
>> +     (filter origin? (origin-patches source))))
>
> * cough * (cons source (filter ...)) * cough *

Ahah!  Somehow I removed from my mental toolbox cons, car and cdr
because I am spending too much time explaining to non-lispers. :-)

Thanks for showing me the light. ;-)


> Other than that, LGTM!  Tests worked fine on my end.  No idea what this
> is used for though :p

For instance, it can be used to list all the ’origin’ of a package.
Consider the package ’tensorflow’, it reads,

--8<---------------cut here---------------start------------->8---
    (native-inputs
[...]
       ;; The commit hashes and URLs for third-party source code are taken
       ;; from "tensorflow/workspace.bzl".
       ("boringssl-src"
        ,(let ((commit "ee7aa02")
               (revision "1"))
           (origin
             (method git-fetch)
             (uri (git-reference
                   (url "https://boringssl.googlesource.com/boringssl")
--8<---------------cut here---------------end--------------->8---

where some inputs are not packages but just ’origin’.  Therefore, the
procedure allows to get all the ’origin’, the one from the field
’origin’ and also the ones from inputs.

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use(gnu packages machine-learning)
scheme@(guix-user)> ,pp (map origin-uri (package-direct-sources tensorflow))
$1 = (#<<git-reference> url: "https://github.com/tensorflow/tensorflow" commit: "v1.9.0" recursive?: #f>
 #<<git-reference> url: "https://boringssl.googlesource.com/boringssl" commit: "ee7aa02" recursive?: #f>
[...]
--8<---------------cut here---------------end--------------->8---

For some packages as ’ntp’, the patches are also a list of ’origin’,

--8<---------------cut here---------------start------------->8---
     (origin
       (method url-fetch)
       (uri (list (string-append
                   "https://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-"
[...]
       ;; Add an upstream patch to fix build with GCC 10.  Taken from
       ;; <https://bugs.ntp.org/show_bug.cgi?id=3688>.
       (patches (list (origin
                        (method url-fetch)
                        (uri "https://bugs.ntp.org/attachment.cgi?id=1760&action=diff&context=patch&collapsed=&headers=1&format=raw")
[...]
--8<---------------cut here---------------end--------------->8---

and the patch allows to also extract them:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use(gnu packages ntp)
scheme@(guix-user)> ,pp (map origin-uri (package-direct-sources ntp))
$2 = (("https://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-4.2/ntp-4.2.8p15.tar.gz"
  "http://archive.ntp.org/ntp4/ntp-4.2/ntp-4.2.8p15.tar.gz")
 "https://bugs.ntp.org/attachment.cgi?id=1760&action=diff&context=patch&collapsed=&headers=1&format=raw"
 "https://bugs.ntp.org/attachment.cgi?id=1814&action=diff&collapsed=&headers=1&format=raw")
--8<---------------cut here---------------end--------------->8---


This way it improves the coverage with Software Heritage.  All the
source code (origin) is extracted and feed some SWH loader.  The code is
there:

https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/build-package-metadata.scm#n58

Cheers,
simon




Information forwarded to guix-patches <at> gnu.org:
bug#62036; Package guix-patches. (Fri, 10 Mar 2023 12:14:02 GMT) Full text and rfc822 format available.

Message #14 received at 62036 <at> debbugs.gnu.org (full text, mbox):

From: Josselin Poiret <dev <at> jpoiret.xyz>
To: Simon Tournier <zimon.toutoune <at> gmail.com>, 62036 <at> debbugs.gnu.org
Cc: Ricardo Wurmus <rekado <at> elephly.net>, Mathieu Othacehe <othacehe <at> gnu.org>,
 ludo <at> gnu.org, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Christopher Baines <mail <at> cbaines.net>
Subject: Re: [bug#62036] [PATCH] guix: packages: Consider 'patches' by
 'package-direct-sources'.
Date: Fri, 10 Mar 2023 13:12:52 +0100
[Message part 1 (text/plain, inline)]
Hi Simon,

Simon Tournier <zimon.toutoune <at> gmail.com> writes:

> This way it improves the coverage with Software Heritage.  All the
> source code (origin) is extracted and feed some SWH loader.  The code is
> there:
>
> https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/build-package-metadata.scm#n58

Ahh, that's the part I was missing.  Great work!

Best,
-- 
Josselin Poiret
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches <at> gnu.org:
bug#62036; Package guix-patches. (Sat, 11 Mar 2023 16:52:01 GMT) Full text and rfc822 format available.

Message #17 received at 62036 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: 62036 <at> debbugs.gnu.org
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>,
 ludo <at> gnu.org, Christopher Baines <mail <at> cbaines.net>,
 Ricardo Wurmus <rekado <at> elephly.net>
Subject: [PATCH v2] guix: packages: Consider 'patches' by
 'package-direct-sources'.
Date: Sat, 11 Mar 2023 17:51:10 +0100
* guix/packages.scm (package-direct-sources): Return 'origin' from 'patches'.
* tests/packages.scm: Test it.
---
 guix/packages.scm  | 10 ++++++++--
 tests/packages.scm | 17 ++++++++++++++++-
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..7b098e21f0 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2021 Chris Marusich <cmmarusich <at> gmail.com>
 ;;; Copyright © 2022 Maxime Devos <maximedevos <at> telenet.be>
 ;;; Copyright © 2022 jgart <jgart <at> dismail.de>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1239,8 +1240,13 @@ (define-syntax modify-inputs
 
 (define (package-direct-sources package)
   "Return all source origins associated with PACKAGE; including origins in
-PACKAGE's inputs."
-  `(,@(or (and=> (package-source package) list) '())
+PACKAGE's inputs and patches."
+  (define (expand source)
+    (cons
+     source
+     (filter origin? (origin-patches source))))
+
+  `(,@(or (and=> (package-source package) expand) '())
     ,@(filter-map (match-lambda
                    ((_ (? origin? orig) _ ...)
                     orig)
diff --git a/tests/packages.scm b/tests/packages.scm
index f58c47817b..27fb918f90 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2018 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
 ;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;; Copyright © 2021 Maxime Devos <maximedevos <at> telenet.be>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -418,12 +419,15 @@ (define read-at
 (let* ((o (dummy-origin))
        (u (dummy-origin))
        (i (dummy-origin))
+       (j (dummy-origin (patches (list o))))
        (a (dummy-package "a"))
        (b (dummy-package "b" (inputs (list a i))))
        (c (package (inherit b) (source o)))
        (d (dummy-package "d"
             (build-system trivial-build-system)
-            (source u) (inputs (list c)))))
+            (source u) (inputs (list c))))
+       (e (dummy-package "e" (source j)))
+       (f (package (inherit e) (inputs (list u)))))
   (test-assert "package-direct-sources, no source"
     (null? (package-direct-sources a)))
   (test-equal "package-direct-sources, #f source"
@@ -437,6 +441,17 @@ (define read-at
       (and (= (length (pk 's-sources s)) 2)
            (member o s)
            (member i s))))
+  (test-assert "package-direct-sources, with patches"
+    (let ((s (package-direct-sources e)))
+      (and (= (length (pk 's-sources s)) 2)
+           (member o s)
+           (member j s))))
+  (test-assert "package-direct-sources, with patches and inputs"
+    (let ((s (package-direct-sources f)))
+      (and (= (length (pk 's-sources s)) 3)
+           (member o s)
+           (member j s)
+           (member u s))))
   (test-assert "package-transitive-sources"
     (let ((s (package-transitive-sources d)))
       (and (= (length (pk 'd-sources s)) 3)

base-commit: 7376424844e0dc1103a9d86799c2fb9bd4aa35eb
-- 
2.38.1





Reply sent to Ludovic Courtès <ludo <at> gnu.org>:
You have taken responsibility. (Mon, 13 Mar 2023 11:16:02 GMT) Full text and rfc822 format available.

Notification sent to Simon Tournier <zimon.toutoune <at> gmail.com>:
bug acknowledged by developer. (Mon, 13 Mar 2023 11:16:02 GMT) Full text and rfc822 format available.

Message #22 received at 62036-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 62036-done <at> debbugs.gnu.org, Mathieu Othacehe <othacehe <at> gnu.org>,
 Christopher Baines <mail <at> cbaines.net>, Ricardo Wurmus <rekado <at> elephly.net>
Subject: Re: [PATCH v2] guix: packages: Consider 'patches' by
 'package-direct-sources'.
Date: Mon, 13 Mar 2023 12:14:51 +0100
Hello,

Simon Tournier <zimon.toutoune <at> gmail.com> skribis:

> * guix/packages.scm (package-direct-sources): Return 'origin' from 'patches'.
> * tests/packages.scm: Test it.

Applied, thanks!

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 10 Apr 2023 11:24:06 GMT) Full text and rfc822 format available.

This bug report was last modified 353 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.