GNU bug report logs - #52591
downloading from software archives as default even if version is wrong

Previous Next

To reply to this bug, email your comments to 52591 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <lichengtai <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: downloading from software archives as default even if version is wrong
Date: Fri, 17 Dec 2021 15:40:45 -0800

I was trying to update a package.   I updated the package definition:

---patch--
diff --git a/gnu/packages/mono.scm b/gnu/packages/mono.scm
index 6a7afbafbf..9eeb46ef16 100644
--- a/gnu/packages/mono.scm
+++ b/gnu/packages/mono.scm
@@ -38,7 +38,7 @@ (define-module (gnu packages mono)
 (define-public mono
   (package
     (name "mono")
-    (version "4.4.1.0")
+    (version "4.8.1")
     (source (origin
               (method url-fetch)
               (uri (string-append
--end of patch--

which at this time shall fail as I have not updated the hash checksum.  If I do

 ./pre-inst-env guix build  -k -K   mono

that actually succeeded but it shall have failed.

This log fragment show what happened:

----log fragment--
The following derivations will be built:
   /gnu/store/cdqqdalqwl3m9bhnfyb0v6nfkzvfj0x1-mono-4.8.1.drv
   /gnu/store/qs8jgc33jpwwmzq7ppdiv49qll6q32sl-mono-4.8.1.tar.xz.drv
   /gnu/store/6aih7gpx3p1dpsbdhkri7mk3qqnqkk9x-mono-4.8.1.tar.bz2.drv
building /gnu/store/6aih7gpx3p1dpsbdhkri7mk3qqnqkk9x-mono-4.8.1.tar.bz2.drv...

Starting download of
/gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
From http://download.mono-project.com/sources/mono/mono-4.8.1.tar.bz2...
following redirection to
`https://download.mono-project.com/sources/mono/mono-4.8.1.tar.bz2'...
download failed
"https://download.mono-project.com/sources/mono/mono-4.8.1.tar.bz2"
404 "Not Found"

Starting download of
/gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
From https://ci.guix.gnu.org/file/mono-4.8.1.tar.bz2/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y90dcjc3dkspc
fbnn37cphn...
download failed
"https://ci.guix.gnu.org/file/mono-4.8.1.tar.bz2/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y
90dcjc3dkspcfbnn37cphn" 404 "Not Found"

Starting download of
/gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
From https://tarballs.nixos.org/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y90dcjc3dkspcfbnn37cphn...
download failed
"https://tarballs.nixos.org/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y90dcjc3dkspcfbnn37cph
n" 404 "Not Found"

Starting download of
/gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
From https://archive.softwareheritage.org/api/1/content/sha256:165e7686b5cbb1ab671b4cb2069207999c3d700
44002190b6ec84bb1fdf62b4a/raw/...
downloading from
https://archive.softwareheritage.org/api/1/content/sha256:165e7686b5cbb1ab671b4cb2069
207999c3d70044002190b6ec84bb1fdf62b4a/raw/ ...
............
successfully built
/gnu/store/6aih7gpx3p1dpsbdhkri7mk3qqnqkk9x-mono-4.8.1.tar.bz2.drv
building /gnu/store/qs8jgc33jpwwmzq7ppdiv49qll6q32sl-mono-4.8.1.tar.xz.drv...
mono-4.4.1/
mono-4.4.1/test-driver
mono-4.4.1/support/
mono-4.4.1/support/syslog.c
mono-4.4.1/support/supportw.h
....

--end of log fratgment--


as seen above, somehow the old version was downloaded from a cached
copy at softwareheritage archives, and it proceeds to build.   This
should not proceed but fail for wrong checksum.

Message #8 received at 52591 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Andy Tai <lichengtai <at> gmail.com>, 52591 <at> debbugs.gnu.org
Subject: Re: bug#52591: downloading from software archives as default even
 if version is wrong
Date: Fri, 17 Dec 2021 23:58:20 +0000

Andy Tai schreef op vr 17-12-2021 om 15:40 [-0800]:
> @@ -38,7 +38,7 @@ (define-module (gnu packages mono)
>  (define-public mono
>    (package
>      (name "mono")
> -    (version "4.4.1.0")
> +    (version "4.8.1")
>      (source (origin
>                (method url-fetch)
>                (uri (string-append
> --end of patch--

You forgot to update the sha256 hash. You can use
"guix download https://the-website/mono-4.8.1.tar.bz2" to determine the
hash. Or toggle a single character & look at the hash mismatch line.

Does that work for you?

[...]
> as seen above, somehow the old version was downloaded from a cached
> copy at softwareheritage archives, and it proceeds to build.   This
> should not proceed but fail for wrong checksum.

SWH isn't a cache, its an archive that keeps everything forever.

Guix has no way to determine if upstream is doing unspeakable things to
their tarballs (*) and hence needs to use SWH, or if someone forgot to
change the hash on guix. This has been noticed in the past, without
any ideas on how to somehow teach guix to determine the case.

(*) e.g. in-place modification, which changes the hash

Greetings,
Maxime.

Message #11 received at 52591 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Andy Tai <lichengtai <at> gmail.com>
Cc: 52591 <at> debbugs.gnu.org
Subject: Re: bug#52591: downloading from software archives as default even if
 version is wrong
Date: Fri, 17 Dec 2021 19:06:46 -0500

On Fri, Dec 17, 2021 at 03:40:45PM -0800, Andy Tai wrote:
> as seen above, somehow the old version was downloaded from a cached
> copy at softwareheritage archives, and it proceeds to build.   This
> should not proceed but fail for wrong checksum.

This can happen with the Nix content-addressed archive fallback that we
use, too.

Basically, when the normal URLs fail, Guix queries these
content-addressed archives by content. That is, by their hash.

The version number is considered to be metadata, not content, and so the
download "succeeds".

Since the source origins are named by their hash, this is the correct
behaviour, although there is some room for error, as you've found.

Do you have any ideas about how to improve things?

We taught Guix to respect the GUIX_DOWNLOAD_FALLBACK_TEST [0]
environment variable, which controls how Guix uses these fallback
archives.

I recommend setting it to "none" if you are doing package development in
order to avoid this pitfall. Otherwise, these archives are helpful for
using Guix, since sources do disappear upstream, URLs change, etc, and
we'd like for old versions of Guix to be usable.

[0]
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c4a7aa82e25503133a1bd33148d17968c899a5f5
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=bd61d62182bfda4a695757ec66810b28e8e1a6d0

Message #14 received at 52591 <at> debbugs.gnu.org (full text, mbox):

From: raingloom <raingloom <at> riseup.net>
To: Andy Tai <lichengtai <at> gmail.com>
Cc: 52591 <at> debbugs.gnu.org
Subject: Re: bug#52591: downloading from software archives as default even
 if version is wrong
Date: Sat, 18 Dec 2021 01:07:23 +0100

On Fri, 17 Dec 2021 15:40:45 -0800
Andy Tai <lichengtai <at> gmail.com> wrote:

> I was trying to update a package.   I updated the package definition:
> 
> ---patch--
> diff --git a/gnu/packages/mono.scm b/gnu/packages/mono.scm
> index 6a7afbafbf..9eeb46ef16 100644
> --- a/gnu/packages/mono.scm
> +++ b/gnu/packages/mono.scm
> @@ -38,7 +38,7 @@ (define-module (gnu packages mono)
>  (define-public mono
>    (package
>      (name "mono")
> -    (version "4.4.1.0")
> +    (version "4.8.1")
>      (source (origin
>                (method url-fetch)
>                (uri (string-append
> --end of patch--
> 
> which at this time shall fail as I have not updated the hash
> checksum.  If I do
> 
>  ./pre-inst-env guix build  -k -K   mono
> 
> that actually succeeded but it shall have failed.
> 
> This log fragment show what happened:
> 
> ----log fragment--
> The following derivations will be built:
>    /gnu/store/cdqqdalqwl3m9bhnfyb0v6nfkzvfj0x1-mono-4.8.1.drv
>    /gnu/store/qs8jgc33jpwwmzq7ppdiv49qll6q32sl-mono-4.8.1.tar.xz.drv
>    /gnu/store/6aih7gpx3p1dpsbdhkri7mk3qqnqkk9x-mono-4.8.1.tar.bz2.drv
> building
> /gnu/store/6aih7gpx3p1dpsbdhkri7mk3qqnqkk9x-mono-4.8.1.tar.bz2.drv...
> 
> Starting download of
> /gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
> From
> http://download.mono-project.com/sources/mono/mono-4.8.1.tar.bz2...
> following redirection to
> `https://download.mono-project.com/sources/mono/mono-4.8.1.tar.bz2'...
> download failed
> "https://download.mono-project.com/sources/mono/mono-4.8.1.tar.bz2"
> 404 "Not Found"
> 
> Starting download of
> /gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
> From
> https://ci.guix.gnu.org/file/mono-4.8.1.tar.bz2/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y90dcjc3dkspc
> fbnn37cphn... download failed
> "https://ci.guix.gnu.org/file/mono-4.8.1.tar.bz2/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y
> 90dcjc3dkspcfbnn37cphn" 404 "Not Found"
> 
> Starting download of
> /gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
> From
> https://tarballs.nixos.org/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y90dcjc3dkspcfbnn37cphn...
> download failed
> "https://tarballs.nixos.org/sha256/0jibyvyv2jy8dq5ij0j00iq3v74r0y90dcjc3dkspcfbnn37cph
> n" 404 "Not Found"
> 
> Starting download of
> /gnu/store/l6309vrm95cwpmcwgjj5xgdddp6vfly5-mono-4.8.1.tar.bz2
> From
> https://archive.softwareheritage.org/api/1/content/sha256:165e7686b5cbb1ab671b4cb2069207999c3d700
> 44002190b6ec84bb1fdf62b4a/raw/... downloading from
> https://archive.softwareheritage.org/api/1/content/sha256:165e7686b5cbb1ab671b4cb2069
> 207999c3d70044002190b6ec84bb1fdf62b4a/raw/ ...
> ............
> successfully built
> /gnu/store/6aih7gpx3p1dpsbdhkri7mk3qqnqkk9x-mono-4.8.1.tar.bz2.drv
> building
> /gnu/store/qs8jgc33jpwwmzq7ppdiv49qll6q32sl-mono-4.8.1.tar.xz.drv...
> mono-4.4.1/ mono-4.4.1/test-driver
> mono-4.4.1/support/
> mono-4.4.1/support/syslog.c
> mono-4.4.1/support/supportw.h
> ....
> 
> --end of log fratgment--
> 
> 
> as seen above, somehow the old version was downloaded from a cached
> copy at softwareheritage archives, and it proceeds to build.   This
> should not proceed but fail for wrong checksum.
> 
> 
> 

This is somewhat unfortunate, but also kind of expected behaviour.
Fixed output derivations don't have version information as far as I
know and part of the usefulness of cryptographic checksums on them is
that they can be downloaded from anywhere, even when their original
servers go down.
There isn't really any practical way I can see for testing if the
checksum is correct when the URL is wrong. How should Guix know what
file it should download?
I think the best workflow is to `use guix refresh -u` instead of
manually updating the package, but sadly that doesn't always work,
although I think the git based updater was merged recently, which
should improve your chances of success.

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: Andy Tai <lichengtai <at> gmail.com>, 52591 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#52591: downloading from software archives as default even
 if version is wrong
Date: Sat, 18 Dec 2021 01:06:48 +0100

[Message part 1 (text/plain, inline)]

Maxime Devos 写道：
> You forgot to update the sha256 hash.

That's it, in a nutshell!

> which at this time shall fail as I have not updated the hash 
> checksum.

No, that's the reason it *succeeds*!

Guix is mostly content-addressed, meaning that the hash identifies 
a file.  If you don't change the hash, you're not asking for a new 
file, by definition.  URLs are like directions (hence: ‘locator’), 
not identifiers.

You can easily generate a bogus-but-valid hash by negating the 
first digit, which (in nix-base32 at least), can only be either 0 
or 1.  Guix will then print the correct hash of whatever it found 
at the URI you specified.

OK to close as NOTABUG?

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <lichengtai <at> gmail.com>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: bug-guix <at> gnu.org, 52591 <at> debbugs.gnu.org,
 Maxime Devos <maximedevos <at> telenet.be>
Subject: Re: bug#52591: downloading from software archives as default even if
 version is wrong
Date: Fri, 17 Dec 2021 16:23:11 -0800

On Fri, Dec 17, 2021 at 4:10 PM Tobias Geerinckx-Rice <me <at> tobias.gr> wrote:
> OK to close as NOTABUG?
sure

Message #29 received at 52591 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Andy Tai <lichengtai <at> gmail.com>
Cc: Tobias Geerinckx-Rice <me <at> tobias.gr>, 52591 <at> debbugs.gnu.org
Subject: Re: bug#52591: downloading from software archives as default even if
 version is wrong
Date: Fri, 17 Dec 2021 20:51:46 -0500

On Fri, Dec 17, 2021 at 04:23:11PM -0800, Andy Tai wrote:
> On Fri, Dec 17, 2021 at 4:10 PM Tobias Geerinckx-Rice <me <at> tobias.gr> wrote:
> > OK to close as NOTABUG?
> sure

Wait! :)

Although I agree that this behaviour is not a bug, it's something that
many Guix contributors discover independently and find confusing, or
even distressing.

And we sometimes receive patches that include the wrong source hash or
wrong URL because of similar issues; sometimes these mistakes are even
committed to guix.git.

Maybe there is *something* we can do to improve this experience, if only
a section in the manual or a tweak to the UI.

Any ideas?

Message #32 received at 52591 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Leo Famulari <leo <at> famulari.name>
Cc: Andy Tai <lichengtai <at> gmail.com>, 52591 <at> debbugs.gnu.org
Subject: Re: bug#52591: downloading from software archives as default even
 if version is wrong
Date: Sat, 18 Dec 2021 02:57:28 +0100

[Message part 1 (text/plain, inline)]

Leo Famulari 写道：
> Maybe there is *something* we can do to improve this experience, 
> if only
> a section in the manual or a tweak to the UI.

OK!  Let's keep it open for discussion.

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.