GNU bug report logs - #38100
'--with-input', '--with-git-url' etc. cause unnecessary rebuilds

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 38100 in the body.
You can then email your comments to 38100 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: bug-Guix <at> gnu.org
Subject: ‘--with-input’ causes unintended rebuilds
Date: Thu, 07 Nov 2019 13:35:15 +0100

Hello,

Consider this example:

--8<---------------cut here---------------start------------->8---
$ guix build glib -nd
/gnu/store/9zz9hvzaz06f40a4cbvhskb183x676w4-glib-2.60.6.drv
$ guix build glib --with-input=inkscape=libreoffice -nd
/gnu/store/15f9jkpakmsaz8i2a0gy4kir1zyk29vi-glib-2.60.6.drv
$ guix describe
Generacio 114	Nov 02 2019 11:32:51	(nuna)
  guix ab1c063
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: ab1c063ab08e069fbe62919828fa634a2e222bbf
--8<---------------cut here---------------end--------------->8---

Since GLib does not depend on Inkscape, the ‘--with-input’ flag should
have no effect: we should get the same glib derivation.  However, we’re
not.

If we diff the ‘glibc-2.60.6-guile-builder’ files of each derivation, we
see that the second one has a duplicate entry:

  (define %build-inputs
    `(…
      ("python" . "/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4")
      ("python" . "/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4")
      …))

whereas the first one doesn’t have this duplicate entry.  IOW, the two
derivations are functionally equivalent but are not bit-identical.

Indeed, evaluating:

  (bag-transitive-inputs
   (package->bag ((package-input-rewriting '()) glib)))

shows that we have two “python” packages there that are not ‘eq?’.

To be continued…

Ludo’.

Message #8 received at 38100 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 38100 <at> debbugs.gnu.org
Subject: Re: bug#38100: ‘--with-input’ causes
 unintended rebuilds
Date: Fri, 08 Nov 2019 22:06:32 +0100

Hi,

Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:

> Indeed, evaluating:
>
>   (bag-transitive-inputs
>    (package->bag ((package-input-rewriting '()) glib)))
>
> shows that we have two “python” packages there that are not ‘eq?’.

The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
‘python-build-system’ and thus has ‘python’ as an implicit input.

‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
that implicit ‘python’ untouched.

Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
equality, we end up with two “python” packages that are not ‘eq?’ but
are functionally equivalent: the one produced by
‘package-input-rewriting’, and the implicit dependency of
‘python-libxml2’.  QED.

(This is essentially the same as <https://bugs.gnu.org/30155>.)

I’m not sure how to address it.

Ludo’.

Message #19 received at 38100-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 38100-done <at> debbugs.gnu.org
Subject: Re: bug#38100: ‘--with-input’ causes
 unintended rebuilds
Date: Sun, 27 Sep 2020 23:46:22 +0200

Hey there!

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:
>
>> Indeed, evaluating:
>>
>>   (bag-transitive-inputs
>>    (package->bag ((package-input-rewriting '()) glib)))
>>
>> shows that we have two “python” packages there that are not ‘eq?’.
>
> The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
> ‘python-build-system’ and thus has ‘python’ as an implicit input.
>
> ‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
> that implicit ‘python’ untouched.
>
> Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
> equality, we end up with two “python” packages that are not ‘eq?’ but
> are functionally equivalent: the one produced by
> ‘package-input-rewriting’, and the implicit dependency of
> ‘python-libxml2’.  QED.
>
> (This is essentially the same as <https://bugs.gnu.org/30155>.)

Good news, this is fixed by 2bf6f962b91123b0474c0f7123cd17efe7f09a66,
which introduces package rewriting including implicit inputs!

Before getting there, this issue did get on my nerves for a while.  Here
are several ways to address this issue that I thought of:

  1. Have ‘package-input-rewriting/spec’ traverse implicit inputs, at
     least optionally.  We wouldn’t end up with an
     equivalent-but-not-eq? ‘python’ in the example above.  It does
     change the semantics though, and it may be nice to keep a “shallow”
     replacement option.  That’s what
     2bf6f962b91123b0474c0f7123cd17efe7f09a66 does.

  2. Do (delete-duplicates input-drvs) in ‘bag->derivation’.  That seems
     wise, but it’s unfortunately impossible on ‘master’ because of
     <https://issues.guix.gnu.org/43508>.

  3. ‘package-input-rewriting/spec’ preserves eq?-ness for packages not
     transformed; in the example above, the transformation result would
     be eq? to ‘glib’ because ‘--with-input=libreoffice=inkscape’ had no
     effect.  Tricky to implement efficiently, perhaps not worth it.

I think #2 might still be worth investigating, but it may have
undesirable implications too.  #3 is hardly doable.

All in all, I’m glad that #1 addresses the issue, because it’s also
something we wanted anyway.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.