GNU bug report logs - #31085
Unexpected behaviour when running `guix build lilypond'

Previous Next

Package: guix;

Reported by: Diego Nicola Barbato <dnbarbato <at> posteo.de>

Date: Fri, 6 Apr 2018 20:23:02 UTC

Severity: normal

Done: ludo <at> gnu.org (Ludovic Courtès)

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 31085 in the body.
You can then email your comments to 31085 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#31085; Package guix. (Fri, 06 Apr 2018 20:23:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Diego Nicola Barbato <dnbarbato <at> posteo.de>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Fri, 06 Apr 2018 20:23:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Diego Nicola Barbato <dnbarbato <at> posteo.de>
To: bug-guix <at> gnu.org
Subject: Unexpected behaviour when running `guix build lilypond'
Date: Fri, 06 Apr 2018 22:22:07 +0200
Hello Guix

I have experienced some unexpected behaviour when running `guix build
lilypond':
After verifying that there was a substitute available with `guix weather
-m m.scm' (Where m.scm evaluates to a manifest containing only lilypond)
I ran `guix build lilypond --dry-run' which claimed that a substitute
would be downloaded.  I then ran `guix build lilypond' which proceeded
to build lilypond from source (instead of downloading the substitute).

Upon explaining this on IRC it was suggested that I try running `guix
build --no-grafts lilypond' (which actually downloaded the substitute)
then deleting the locally built lilypond with `guix gc --delete
/gnu/store/...' and finally running `guix build lilypond' again (which,
this time, grafted the substituted lilypond instead of building it from
source again).  While this fixed the issue I was told that this
behaviour was indeed unexpected.

I run GuixSD (commit: ae81bf4f995466a4650e2756d2763b8a163d2f63) on an
x86-64 machine.

Greetings

Diego




Information forwarded to bug-guix <at> gnu.org:
bug#31085; Package guix. (Thu, 19 Apr 2018 15:39:02 GMT) Full text and rfc822 format available.

Message #8 received at 31085 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Diego Nicola Barbato <dnbarbato <at> posteo.de>
Cc: 31085 <at> debbugs.gnu.org
Subject: Re: bug#31085: Unexpected behaviour when running `guix build lilypond'
Date: Thu, 19 Apr 2018 17:38:07 +0200
Hello,

Diego Nicola Barbato <dnbarbato <at> posteo.de> skribis:

> I have experienced some unexpected behaviour when running `guix build
> lilypond':
> After verifying that there was a substitute available with `guix weather
> -m m.scm' (Where m.scm evaluates to a manifest containing only lilypond)
> I ran `guix build lilypond --dry-run' which claimed that a substitute
> would be downloaded.  I then ran `guix build lilypond' which proceeded
> to build lilypond from source (instead of downloading the substitute).
>
> Upon explaining this on IRC it was suggested that I try running `guix
> build --no-grafts lilypond' (which actually downloaded the substitute)
> then deleting the locally built lilypond with `guix gc --delete
> /gnu/store/...' and finally running `guix build lilypond' again (which,
> this time, grafted the substituted lilypond instead of building it from
> source again).  While this fixed the issue I was told that this
> behaviour was indeed unexpected.

We’d have to see if this is still reproducible, but I have a plausible
explanation.

Substitute info is cached locally.  guix-daemon caches it under
/var/guix/substitute/cache, but ‘guix weather’ caches it under
~/.cache/guix/substitute (that’s because it needs fine-grain control
over substitute info and thus cannot simply use the
‘substitutable-path-info’ daemon RPC.)  Each cached entry has a
time-to-live (TTL).

What could have happened is that /var/guix/substitute/ had a
not-expired-yet entry saying that there’s no substitute for LilyPond
(which is why ‘guix build’ ended up building from source), whereas ‘guix
weather’ was run at a point when there was a substitute.

If that happens again, I’d suggest capturing immediately the two cached
entries so we can see whether this explanation holds.

Thanks,
Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#31085; Package guix. (Fri, 20 Apr 2018 21:21:01 GMT) Full text and rfc822 format available.

Message #11 received at 31085 <at> debbugs.gnu.org (full text, mbox):

From: Diego Nicola Barbato <dnbarbato <at> posteo.de>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 31085 <at> debbugs.gnu.org
Subject: Re: bug#31085: Unexpected behaviour when running `guix build lilypond'
Date: Fri, 20 Apr 2018 23:20:01 +0200
Hello

ludo <at> gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Diego Nicola Barbato <dnbarbato <at> posteo.de> skribis:
>
>> I have experienced some unexpected behaviour when running `guix build
>> lilypond':
>> After verifying that there was a substitute available with `guix weather
>> -m m.scm' (Where m.scm evaluates to a manifest containing only lilypond)
>> I ran `guix build lilypond --dry-run' which claimed that a substitute
>> would be downloaded.  I then ran `guix build lilypond' which proceeded
>> to build lilypond from source (instead of downloading the substitute).
>>
>> Upon explaining this on IRC it was suggested that I try running `guix
>> build --no-grafts lilypond' (which actually downloaded the substitute)
>> then deleting the locally built lilypond with `guix gc --delete
>> /gnu/store/...' and finally running `guix build lilypond' again (which,
>> this time, grafted the substituted lilypond instead of building it from
>> source again).  While this fixed the issue I was told that this
>> behaviour was indeed unexpected.
>
> We’d have to see if this is still reproducible, but I have a plausible
> explanation.

I can consistently reproduce this in a VM with the following steps:

First I run:

 $ qemu-system-x86_64 -enable-kvm -snapshot -m 4G $(guix system vm-image bare-bones.scm --image-size=8G)

(Where bare-bones.scm is the bare-bones.tmpl after replacing /dev/sdX
with /dev/sda.)

Then after logging in as root:

 # guix pull --commit=872bda5de52a8f0514230ebc4e9680aab74f509a
 # guix build --dry-run lilypond

Which returns:

52.1 MB would be downloaded:
   /gnu/store/0amx7bcbs518lkqwfh2azmqrp2yqib0g-lilypond-2.19.80
   /gnu/store/7b5ykfl6jbrdl8j7xp630fga4as3234z-ghostscript-9.22
   /gnu/store/j4vj7h3wyb532g2j0axzjj43z2a0dg81-python-2.7.14
   /gnu/store/k2ak44m0miind785x22mmpbcwi0mq7hq-freetype-2.8.1
   /gnu/store/mkhfqx7m7pniyic0kh0lnafmajymn4dr-guile-1.8.8
   /gnu/store/pwbx5fhjrq9crr1c0d2x08ch0l6vr3cv-pango-1.40.14
   /gnu/store/qm8ri32n0rkh749v3jb3x8s8ksjl7yd3-fontconfig-2.12.6
   /gnu/store/sm37m59gq3smxxz8gs4jikn50qg0g7xh-glib-2.54.2

Then:

 # guix build lilypond

Which, after downloading the dependencies, starts to build lilypond from
source.

> Substitute info is cached locally.  guix-daemon caches it under
> /var/guix/substitute/cache, but ‘guix weather’ caches it under
> ~/.cache/guix/substitute (that’s because it needs fine-grain control
> over substitute info and thus cannot simply use the
> ‘substitutable-path-info’ daemon RPC.)  Each cached entry has a
> time-to-live (TTL).
>
> What could have happened is that /var/guix/substitute/ had a
> not-expired-yet entry saying that there’s no substitute for LilyPond
> (which is why ‘guix build’ ended up building from source), whereas ‘guix
> weather’ was run at a point when there was a substitute.

The unexpected behaviour was that `guix build lilypond' started to build
lilypond from source after `guix build --dry-run lilypond' had claimed
that nothing would be built and that it would download a substitute.  I
assume that `guix build --dry-run' is not affected by the substitute
info used by `guix weather'.  I only ran `guix weather' in order to
double check that there were indeed substitutes available (Sorry for the
red herring).

Greetings

Diego




Reply sent to ludo <at> gnu.org (Ludovic Courtès):
You have taken responsibility. (Mon, 23 Apr 2018 13:27:01 GMT) Full text and rfc822 format available.

Notification sent to Diego Nicola Barbato <dnbarbato <at> posteo.de>:
bug acknowledged by developer. (Mon, 23 Apr 2018 13:27:02 GMT) Full text and rfc822 format available.

Message #16 received at 31085-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Diego Nicola Barbato <dnbarbato <at> posteo.de>
Cc: 31085-done <at> debbugs.gnu.org
Subject: Re: bug#31085: Unexpected behaviour when running `guix build lilypond'
Date: Mon, 23 Apr 2018 15:26:24 +0200
Hi,

Diego Nicola Barbato <dnbarbato <at> posteo.de> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> Hello,
>>
>> Diego Nicola Barbato <dnbarbato <at> posteo.de> skribis:
>>
>>> I have experienced some unexpected behaviour when running `guix build
>>> lilypond':
>>> After verifying that there was a substitute available with `guix weather
>>> -m m.scm' (Where m.scm evaluates to a manifest containing only lilypond)
>>> I ran `guix build lilypond --dry-run' which claimed that a substitute
>>> would be downloaded.  I then ran `guix build lilypond' which proceeded
>>> to build lilypond from source (instead of downloading the substitute).
>>>
>>> Upon explaining this on IRC it was suggested that I try running `guix
>>> build --no-grafts lilypond' (which actually downloaded the substitute)
>>> then deleting the locally built lilypond with `guix gc --delete
>>> /gnu/store/...' and finally running `guix build lilypond' again (which,
>>> this time, grafted the substituted lilypond instead of building it from
>>> source again).  While this fixed the issue I was told that this
>>> behaviour was indeed unexpected.
>>
>> We’d have to see if this is still reproducible, but I have a plausible
>> explanation.
>
> I can consistently reproduce this in a VM with the following steps:
>
> First I run:
>
>  $ qemu-system-x86_64 -enable-kvm -snapshot -m 4G $(guix system vm-image bare-bones.scm --image-size=8G)
>
> (Where bare-bones.scm is the bare-bones.tmpl after replacing /dev/sdX
> with /dev/sda.)
>
> Then after logging in as root:
>
>  # guix pull --commit=872bda5de52a8f0514230ebc4e9680aab74f509a
>  # guix build --dry-run lilypond
>
> Which returns:
>
> 52.1 MB would be downloaded:
>    /gnu/store/0amx7bcbs518lkqwfh2azmqrp2yqib0g-lilypond-2.19.80
>    /gnu/store/7b5ykfl6jbrdl8j7xp630fga4as3234z-ghostscript-9.22
>    /gnu/store/j4vj7h3wyb532g2j0axzjj43z2a0dg81-python-2.7.14
>    /gnu/store/k2ak44m0miind785x22mmpbcwi0mq7hq-freetype-2.8.1
>    /gnu/store/mkhfqx7m7pniyic0kh0lnafmajymn4dr-guile-1.8.8
>    /gnu/store/pwbx5fhjrq9crr1c0d2x08ch0l6vr3cv-pango-1.40.14
>    /gnu/store/qm8ri32n0rkh749v3jb3x8s8ksjl7yd3-fontconfig-2.12.6
>    /gnu/store/sm37m59gq3smxxz8gs4jikn50qg0g7xh-glib-2.54.2
>
> Then:
>
>  # guix build lilypond
>
> Which, after downloading the dependencies, starts to build lilypond from
> source.

I could reproduce it.  This is fixed by commit
5e5d6613a3837586ccab51cd988b44c7df99253b.

The story is quite surprising: the ‘font-tex-gyre’ packages uses
‘url-fetch/zipbomb’.  ‘url-fetch/zipbomb’ refers to unzip to do its
job.  With grafts enabled, ‘url-fetch/zipbomb’ would use a grafted
unzip, leading to a different /gnu/store/…-tg-2.005otf.zip than when
grafts are disabled.

Consequently, the base lilypond.drv would already depend on whether or
not grafts are enabled.  When grafts are enabled, you would end up
having to build a lilypond that’s different from what “guix build
lilypond --no-grafts” builds because of this.

I hope this explanation makes any sense but anyway, it’s fixed now.  :-)

Thanks,
Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Tue, 22 May 2018 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 5 years and 312 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.