GNU bug report logs - #32245
Guix does not search sources locally by hash

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32245 in the body.
You can then email your comments to 32245 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
To: <bug-guix <at> gnu.org>
Subject: Guix does not search sources locally by hash
Date: Sun, 22 Jul 2018 11:56:07 +0200

[Message part 1 (text/plain, inline)]

[Note: This concrete package is nonfree/Yoghurt-ware, but it's just
used here for showing the general problem, so I think it is OK to post
it here].

I renamed a package definition from "eclipse-java" to "java-eclipse".
Now upstream seams no longer be availabe:

$> ./pre-inst-env  guix build java-eclipse
[..]
@ build-started /gnu/store/lqfw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv - x86_64-linux /var/log/guix/drvs/lq//fw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv.bz2
guile: warning: failed to install locale
warning: failed to install locale: Invalid argument

Starting download of /gnu/store/dcjzplbvc27amcdgd9kffmyhskdw80d7-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz
From https://www.eclipse.org/downloads/download.php?r=1&nf=1&file=/technology/epp/downloads/release/oxygen/3a/java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz...

ERROR: download failed "https://www.eclipse.org/downloads/download.php?r=1&nf=1&file=/technology/epp/downloads/release/oxygen/3a/java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz" 404 "Not Found"

Starting download of /gnu/store/dcjzplbvc27amcdgd9kffmyhskdw80d7-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz
From http://mirror.hydra.gnu.org/file/java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz/sha256/0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim...
ERROR: download failed "http://mirror.hydra.gnu.org/file/java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz/sha256/0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim" 404 "Not Found"

Starting download of /gnu/store/dcjzplbvc27amcdgd9kffmyhskdw80d7-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz
From http://tarballs.nixos.org/sha256/0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim...
ERROR: download failed "http://tarballs.nixos.org/sha256/0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim" 404 "Not Found"

[...]

So, what you see here is that Guix is looking for the sources JUST by
hash on remote sites (Hydra, Nix).

But it does NOT search locally. Because I already have:

guix hash /gnu/store/56h6snwcawpzk6rhcwgk442wsx6k86q2-eclipse-java-oxygen-3a-linux-gtk-x86_64.tar.gz
0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim


If Guix searches remotely by hash, I would find it obvious that Guix
searches beforehand locally by hash. But it does not.

Björn

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 32245 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
Cc: 32245 <at> debbugs.gnu.org
Subject: Re: bug#32245: Guix does not search sources locally by hash
Date: Mon, 23 Jul 2018 10:41:14 +0200

Hi,

Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de> skribis:

> $> ./pre-inst-env  guix build java-eclipse
> [..]
> @ build-started /gnu/store/lqfw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv - x86_64-linux /var/log/guix/drvs/lq//fw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv.bz2

[...]

> But it does NOT search locally. Because I already have:
>
> guix hash /gnu/store/56h6snwcawpzk6rhcwgk442wsx6k86q2-eclipse-java-oxygen-3a-linux-gtk-x86_64.tar.gz
> 0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim

It does, but the file name matters.  In this case what you have is
“eclipse-java-…” but you’re trying to build “java-eclipse-…”.

So you first need to add the file to the store under its expected name.
You can do that for instance with:

  guix download file://…/java-eclipse-…

HTH!

Ludo’.

Message #11 received at 32245 <at> debbugs.gnu.org (full text, mbox):

From: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 32245 <at> debbugs.gnu.org
Subject: Re: bug#32245: Guix does not search sources locally by hash
Date: Mon, 23 Jul 2018 11:50:53 +0200

[Message part 1 (text/plain, inline)]

On Mon, 23 Jul 2018 10:41:14 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hi,
> 
> Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de> skribis:
> 
> > $> ./pre-inst-env  guix build java-eclipse
> > [..]
> > @
> > build-started /gnu/store/lqfw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv
> > -
> > x86_64-linux /var/log/guix/drvs/lq//fw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv.bz2
> 
> [...]
> 
> > But it does NOT search locally. Because I already have:
> >
> > guix
> > hash /gnu/store/56h6snwcawpzk6rhcwgk442wsx6k86q2-eclipse-java-oxygen-3a-linux-gtk-x86_64.tar.gz
> > 0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim
> 
> It does, but the file name matters.  In this case what you have is
> “eclipse-java-…” but you’re trying to build “java-eclipse-…”.
> 
> So you first need to add the file to the store under its expected
> name. You can do that for instance with:
> 
>   guix download file://…/java-eclipse-…
> 
> HTH!

Not sure yet. I havent't tried it yet, but I'm sure your solution
helps with the specific problem here. (Also updating the URL helps even
better :-))

What I feel strange here is:

* When the file is already on the local disk, it has to match a certain
file name criteria. Quite hard criteria.
* But when asking outside repositories over the network, we just "take
anything that has the hashsum".

Is that wanted behaviour? 

Björn

[Message part 2 (application/pgp-signature, inline)]

Message #14 received at 32245 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
Cc: 32245 <at> debbugs.gnu.org
Subject: Re: bug#32245: Guix does not search sources locally by hash
Date: Mon, 23 Jul 2018 16:50:49 +0200

Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de> skribis:

> On Mon, 23 Jul 2018 10:41:14 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Hi,
>> 
>> Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de> skribis:
>> 
>> > $> ./pre-inst-env  guix build java-eclipse
>> > [..]
>> > @
>> > build-started /gnu/store/lqfw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv
>> > -
>> > x86_64-linux /var/log/guix/drvs/lq//fw971srfifgql68ibdgh58vi4d8fq3-java-eclipse-oxygen-3a-linux-gtk-x86_64.tar.gz.drv.bz2
>> 
>> [...]
>> 
>> > But it does NOT search locally. Because I already have:
>> >
>> > guix
>> > hash /gnu/store/56h6snwcawpzk6rhcwgk442wsx6k86q2-eclipse-java-oxygen-3a-linux-gtk-x86_64.tar.gz
>> > 0m7y7jfm059w01x9j5b5qkinjjmhkyygpjabhjf19fg2smxmwcim
>> 
>> It does, but the file name matters.  In this case what you have is
>> “eclipse-java-…” but you’re trying to build “java-eclipse-…”.
>> 
>> So you first need to add the file to the store under its expected
>> name. You can do that for instance with:
>> 
>>   guix download file://…/java-eclipse-…
>> 
>> HTH!
>
> Not sure yet. I havent't tried it yet, but I'm sure your solution
> helps with the specific problem here. (Also updating the URL helps even
> better :-))
>
> What I feel strange here is:
>
> * When the file is already on the local disk, it has to match a certain
> file name criteria. Quite hard criteria.
> * But when asking outside repositories over the network, we just "take
> anything that has the hashsum".
>
> Is that wanted behaviour? 

It’s not optimal, as you pointed out.  But it’s not implemented in these
terms.  The ‘add-to-store’ RPC returns a file name that’s a function of
the content and the file name.  So the way ‘add-to-store’ works is:

  1. compute store file name = g(content, filename)
     (See ‘fixed-output-path’ in (guix store).)

  2. if store file name exists, return it, else create it.

Since /var/guix/db/db.sqlite contains the content hash of each store
item, we could in theory perform a lookup by hash.  To do that we’d need
to add a new RPC and use it in the correct places.  It’d be a nice
addition.

Ludo’.

Message #19 received at 32245-done <at> debbugs.gnu.org (full text, mbox):

From: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 32245-done <at> debbugs.gnu.org
Subject: Re: bug#32245: Guix does not search sources locally by hash
Date: Mon, 22 Oct 2018 09:00:39 +0200

[Message part 1 (text/plain, inline)]

I'm closing this one as I think it has very low priority.

Björn

[Message part 2 (application/pgp-signature, inline)]

Message #22 received at 32245 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 32245 <at> debbugs.gnu.org, 
 Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 32245-done <at> debbugs.gnu.org
Subject: Re: bug#32245: Guix does not search sources locally by hash
Date: Mon, 22 Oct 2018 10:06:09 +0200

Hello Björn,

It might be better to just lower the priority, or make it wishlist.


Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de> ezt írta (időpont:
2018. okt. 22., H, 9:01):
>
> I'm closing this one as I think it has very low priority.
>
> Björn

g_bor

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.