GNU bug report logs - #63462
[PATCH maintenance] hydra: build-package-metadata: Update fields in sources.json.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 63462 in the body.
You can then email your comments to 63462 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: Simon Tournier <zimon.toutoune <at> gmail.com>
Subject: [PATCH maintenance] hydra: build-package-metadata: Update fields in
 sources.json.
Date: Fri, 12 May 2023 11:45:04 +0200

This format had been discussed in
<https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868>.

* hydra/build-package-metadata.scm (origin->json)[resolve]: Add the fields
'outputHashAlgo' and 'outputHashMode'.
---
 hydra/build-package-metadata.scm | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/hydra/build-package-metadata.scm b/hydra/build-package-metadata.scm
index 1ddb409..d110d4d 100755
--- a/hydra/build-package-metadata.scm
+++ b/hydra/build-package-metadata.scm
@@ -1,7 +1,7 @@
 #!/usr/bin/env -S guix repl --
 !#
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2020, 2021 Simon Tournier <zimon.toutoune <at> gmail.com>
+;;; Copyright © 2020, 2021, 2023 Simon Tournier <zimon.toutoune <at> gmail.com>
 ;;; Copyright © 2023 Ludovic Courtès <ludo <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -160,13 +160,23 @@ superseded packages."
                  (else '()))
          ,@(if (or (eq? url-fetch method)
                    (eq? url-fetch/tarbomb method)
-                   (eq? url-fetch/zipbomb method))
+                   (eq? url-fetch/zipbomb method)
+                   (eq? git-fetch method)
+                   (eq? svn-fetch method)
+                   (eq? svn-multi-fetch method)
+                   (eq? hg-fetch method))
                (let* ((content-hash (origin-hash origin))
                       (hash-value (content-hash-value content-hash))
                       (hash-algorithm (content-hash-algorithm content-hash))
                       (algorithm-string (symbol->string hash-algorithm)))
                  `(("integrity" . ,(string-append algorithm-string "-"
-                                                  (base64-encode hash-value)))))
+                                                  (base64-encode hash-value)))
+                   ("outputHashAlgo" . ,algorithm-string)
+                   ("outputHashMode" . ,(if (or (eq? url-fetch method)
+                                                (eq? url-fetch/tarbomb method)
+                                                (eq? url-fetch/zipbomb method))
+                                            "flat"
+                                            "recursive"))))
                '())
          ,@(if (eq? method git-fetch)
                `(("git_ref" . ,(git-reference-commit uri)))

base-commit: 5a6b8cd7286d1c8906beaa35ebba6eff5bae3a79
-- 
2.38.1

Message #8 received at 63462 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: 63462 <at> debbugs.gnu.org
Subject: Re: bug#63462: [PATCH maintenance] hydra: build-package-metadata:
 Update fields in sources.json.
Date: Fri, 12 May 2023 13:47:40 +0200

Hi,

Simon Tournier <zimon.toutoune <at> gmail.com> skribis:

> This format had been discussed in
> <https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868>.
>
> * hydra/build-package-metadata.scm (origin->json)[resolve]: Add the fields
> 'outputHashAlgo' and 'outputHashMode'.

Neat, thanks for the update!  (I pushed it on your behalf because you’re
not currently authorized in maintenance.git; please email guix-sysadmin
if you’d like to be there.)

>                   `(("integrity" . ,(string-append algorithm-string "-"
> -                                                  (base64-encode hash-value)))))
> +                                                  (base64-encode hash-value)))
> +                   ("outputHashAlgo" . ,algorithm-string)
> +                   ("outputHashMode" . ,(if (or (eq? url-fetch method)
> +                                                (eq? url-fetch/tarbomb method)
> +                                                (eq? url-fetch/zipbomb method))
> +                                            "flat"
> +                                            "recursive"))))

I wish they had opted for something more expressive (and also with a
more meaningful name :-)) than ‘outputHashMode’, similar to what you did
for ‘guix hash -S’.  But anyway, it does the job!

It looks like SWH will be storing nar hashes; will there be endpoints to
query directories by nar hash?

Thanks,
Ludo’.

Message #13 received at 63462 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 63462 <at> debbugs.gnu.org
Subject: Re: [bug#63462] [PATCH maintenance] hydra: build-package-metadata:
 Update fields in sources.json.
Date: Fri, 12 May 2023 14:12:17 +0200

Hi,

On ven., 12 mai 2023 at 13:47, Ludovic Courtès <ludo <at> gnu.org> wrote:

>                               (I pushed it on your behalf because you’re
> not currently authorized in maintenance.git; please email guix-sysadmin
> if you’d like to be there.)

Ok, I will.


> I wish they had opted for something more expressive (and also with a
> more meaningful name :-)) than ‘outputHashMode’, similar to what you did
> for ‘guix hash -S’.  But anyway, it does the job!

Well, I am biased. ;-)  Another poor naming inherited from Nix [1]. :-)

1: https://nixos.org/manual/nix/stable/language/advanced-attributes.html


> It looks like SWH will be storing nar hashes; will there be endpoints to
> query directories by nar hash?

This is still ongoing.  IIUC, it’s the last item of SWH issue #4868 [2].

2: https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868


Cheers,
simon

Message #16 received at 63462 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: 63462 <at> debbugs.gnu.org
Subject: Re: [bug#63462] [PATCH maintenance] hydra: build-package-metadata:
 Update fields in sources.json.
Date: Wed, 24 May 2023 16:32:56 +0200

Hi,

Simon Tournier <zimon.toutoune <at> gmail.com> skribis:

> On ven., 12 mai 2023 at 13:47, Ludovic Courtès <ludo <at> gnu.org> wrote:

[...]

>> I wish they had opted for something more expressive (and also with a
>> more meaningful name :-)) than ‘outputHashMode’, similar to what you did
>> for ‘guix hash -S’.  But anyway, it does the job!
>
> Well, I am biased. ;-)  Another poor naming inherited from Nix [1]. :-)

Yeah.

BTW, I’ve restarted the relevant mcron job and it’s rolling out now:

--8<---------------cut here---------------start------------->8---
$ wget -qO- https://guix.gnu.org/sources.json |jq |grep output[A-Z] | head
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
      "outputHashAlgo": "sha256",
      "outputHashMode": "recursive",
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
--8<---------------cut here---------------end--------------->8---

>> It looks like SWH will be storing nar hashes; will there be endpoints to
>> query directories by nar hash?
>
> This is still ongoing.  IIUC, it’s the last item of SWH issue #4868 [2].
>
> 2: https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868

Nice!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.