GNU bug report logs - #44835
gnu/ci.go: Embeds build path, breaking reproducible builds

Previous Next

Package: guix;

Reported by: Vagrant Cascadian <vagrant <at> reproducible-builds.org>

Date: Tue, 24 Nov 2020 04:21:01 UTC

Severity: normal

Done: Vagrant Cascadian <vagrant <at> reproducible-builds.org>

To reply to this bug, email your comments to 44835 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#44835; Package guix. (Tue, 24 Nov 2020 04:21:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Vagrant Cascadian <vagrant <at> reproducible-builds.org>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Tue, 24 Nov 2020 04:21:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vagrant Cascadian <vagrant <at> reproducible-builds.org>
To: bug-guix <at> gnu.org
Subject: gnu/ci.go: Embeds build path, breaking reproducible builds
Date: Mon, 23 Nov 2020 20:19:54 -0800
[Message part 1 (text/plain, inline)]
Something in gnu/ci.scm is embedding the build path when compiled into
gnu/ci.go, as can be seen:

  https://tests.reproducible-builds.org/debian/rb-pkg/experimental/amd64/diffoscope-results/guix.html

  ./usr/lib/x86_64-linux-gnu/guile/3.0/site-ccache/gnu/ci.go
  strings --all --bytes=8 {}
  Offset 85, 15 lines modified	Offset 85, 15 lines modified
  ...
  92 	/build/1st/guix-1.2.0~rc2/gnu/ci.scm	92 	/build/2/guix-1.2.0~rc2/2nd/gnu/ci.scm

While guix builds of guix are typically built with a consistent build
path, it would be nice to fix this issue for other environments where
the build path may vary between builds.


My *wild* guess is it maybe has something to do with the use of
canonicalize-path:

  (define (find-current-checkout arguments)
  "Find the first checkout of ARGUMENTS that provided the current file.
Return #f if no such checkout is found."
  (let ((current-root
         (canonicalize-path
          (string-append (dirname (current-filename)) "/.."))))
    (find (lambda (argument)
            (and=> (assq-ref argument 'file-name)
                   (lambda (name)
                     (string=? name current-root)))) arguments)))

Either directly or indirectly... does canonicalize-path resolve at build
time? run time? somewhere in-between? Or is this a red herring, and
something else entirely is responsible? I'll let a competent schemer
ponder this! :)


Thanks for all your help!


live well,
  vagrant
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#44835; Package guix. (Thu, 26 Nov 2020 21:40:02 GMT) Full text and rfc822 format available.

Message #8 received at 44835 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Vagrant Cascadian <vagrant <at> reproducible-builds.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 44835 <at> debbugs.gnu.org
Subject: Re: bug#44835: gnu/ci.go: Embeds build path, breaking reproducible
 builds
Date: Thu, 26 Nov 2020 22:39:08 +0100
[Message part 1 (text/plain, inline)]
Hi!

Vagrant Cascadian <vagrant <at> reproducible-builds.org> skribis:

> My *wild* guess is it maybe has something to do with the use of
> canonicalize-path:
>
>   (define (find-current-checkout arguments)
>   "Find the first checkout of ARGUMENTS that provided the current file.
> Return #f if no such checkout is found."
>   (let ((current-root
>          (canonicalize-path
>           (string-append (dirname (current-filename)) "/.."))))
>     (find (lambda (argument)
>             (and=> (assq-ref argument 'file-name)
>                    (lambda (name)
>                      (string=? name current-root)))) arguments)))

‘canonicalize-path’ is called at run time, so that’s fine.  However,
‘current-filename’ is a macro that captures the source file name at
build time, so it’s the likely culprit here.

I was going to go with something like:

[Message part 2 (text/x-patch, inline)]
diff --git a/gnu/ci.scm b/gnu/ci.scm
index 5548d9560e..0bacfbe025 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -488,7 +488,8 @@ valid."
 Return #f if no such checkout is found."
   (let ((current-root
          (canonicalize-path
-          (string-append (dirname (current-filename)) "/.."))))
+          (string-append (dirname (search-path %load-path "gnu/ci.scm"))
+                         "/.."))))
     (find (lambda (argument)
             (and=> (assq-ref argument 'file-name)
                    (lambda (name)
[Message part 3 (text/plain, inline)]
… but I don’t think we can assume that the checkout is in the
‘%load-path’ when this code is executed.  WDYT, Mathieu?

Looking at f71b0a0012d46bd30ead1a14ed58fd59647415e2, which introduced
this, there might be other options too.

Thanks,
Ludo’.

Information forwarded to bug-guix <at> gnu.org:
bug#44835; Package guix. (Fri, 27 Nov 2020 20:26:02 GMT) Full text and rfc822 format available.

Message #11 received at 44835 <at> debbugs.gnu.org (full text, mbox):

From: Vagrant Cascadian <vagrant <at> reproducible-builds.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 44835 <at> debbugs.gnu.org
Subject: Re: bug#44835: gnu/ci.go: Embeds build path,
 breaking reproducible builds
Date: Fri, 27 Nov 2020 12:25:05 -0800
[Message part 1 (text/plain, inline)]
On 2020-11-26, Ludovic Courtès wrote:
> Vagrant Cascadian <vagrant <at> reproducible-builds.org> skribis:
>> My *wild* guess is it maybe has something to do with the use of
>> canonicalize-path:
>>
>>   (define (find-current-checkout arguments)
>>   "Find the first checkout of ARGUMENTS that provided the current file.
>> Return #f if no such checkout is found."
>>   (let ((current-root
>>          (canonicalize-path
>>           (string-append (dirname (current-filename)) "/.."))))
>>     (find (lambda (argument)
>>             (and=> (assq-ref argument 'file-name)
>>                    (lambda (name)
>>                      (string=? name current-root)))) arguments)))
>
> ‘canonicalize-path’ is called at run time, so that’s fine.  However,
> ‘current-filename’ is a macro that captures the source file name at
> build time, so it’s the likely culprit here.
>
> I was going to go with something like:
>
> diff --git a/gnu/ci.scm b/gnu/ci.scm
> index 5548d9560e..0bacfbe025 100644
> --- a/gnu/ci.scm
> +++ b/gnu/ci.scm
> @@ -488,7 +488,8 @@ valid."
>  Return #f if no such checkout is found."
>    (let ((current-root
>           (canonicalize-path
> -          (string-append (dirname (current-filename)) "/.."))))
> +          (string-append (dirname (search-path %load-path "gnu/ci.scm"))
> +                         "/.."))))
>      (find (lambda (argument)
>              (and=> (assq-ref argument 'file-name)
>                     (lambda (name)
>
> … but I don’t think we can assume that the checkout is in the
> ‘%load-path’ when this code is executed.  WDYT, Mathieu?
>
> Looking at f71b0a0012d46bd30ead1a14ed58fd59647415e2, which introduced
> this, there might be other options too.

This does resolve the reproducibility issues after some "quick" testing;
no idea if it breaks anything, though I didn't see any new test suite
failures (though my builds are patched to skip many tests)...


live well,
  vagrant
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#44835; Package guix. (Tue, 01 Dec 2020 09:42:02 GMT) Full text and rfc822 format available.

Message #14 received at 44835 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 44835 <at> debbugs.gnu.org, Vagrant Cascadian <vagrant <at> reproducible-builds.org>
Subject: Re: bug#44835: gnu/ci.go: Embeds build path, breaking reproducible
 builds
Date: Tue, 01 Dec 2020 10:41:00 +0100
Hey,

> … but I don’t think we can assume that the checkout is in the
> ‘%load-path’ when this code is executed.  WDYT, Mathieu?

Cuirass happens to add checkouts to the %load-path just before loading
this file.

I tested your path, it works fine. I think it is acceptable as this (gnu
ci) interface needs a big rework anyway.

I can apply your patch if it's ok for you.

Thanks,

Mathieu




Information forwarded to bug-guix <at> gnu.org:
bug#44835; Package guix. (Thu, 03 Dec 2020 13:24:02 GMT) Full text and rfc822 format available.

Message #17 received at 44835 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 44835 <at> debbugs.gnu.org, Vagrant Cascadian <vagrant <at> reproducible-builds.org>
Subject: Re: bug#44835: gnu/ci.go: Embeds build path, breaking reproducible
 builds
Date: Thu, 03 Dec 2020 14:23:52 +0100
Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>> … but I don’t think we can assume that the checkout is in the
>> ‘%load-path’ when this code is executed.  WDYT, Mathieu?
>
> Cuirass happens to add checkouts to the %load-path just before loading
> this file.

Is that systematic?  Isn’t it only when ‘load_path_inputs’ is non-empty?

> I tested your path, it works fine. I think it is acceptable as this (gnu
> ci) interface needs a big rework anyway.
>
> I can apply your patch if it's ok for you.

Sure if you’re confident you can go ahead.  :-)

Thanks,
Ludo’.




Reply sent to Vagrant Cascadian <vagrant <at> reproducible-builds.org>:
You have taken responsibility. (Thu, 07 Mar 2024 21:48:02 GMT) Full text and rfc822 format available.

Notification sent to Vagrant Cascadian <vagrant <at> reproducible-builds.org>:
bug acknowledged by developer. (Thu, 07 Mar 2024 21:48:02 GMT) Full text and rfc822 format available.

Message #22 received at 44835-done <at> debbugs.gnu.org (full text, mbox):

From: Vagrant Cascadian <vagrant <at> reproducible-builds.org>
To: Ludovic Courtès <ludo <at> gnu.org>, Mathieu Othacehe
 <othacehe <at> gnu.org>
Cc: 44835-done <at> debbugs.gnu.org
Subject: Re: bug#44835: gnu/ci.go: Embeds build path, breaking reproducible
 builds
Date: Thu, 07 Mar 2024 13:46:01 -0800
[Message part 1 (text/plain, inline)]
On 2020-12-03, Ludovic Courtès wrote:
> Mathieu Othacehe <othacehe <at> gnu.org> skribis:
>
>>> … but I don’t think we can assume that the checkout is in the
>>> ‘%load-path’ when this code is executed.  WDYT, Mathieu?
>>
>> Cuirass happens to add checkouts to the %load-path just before loading
>> this file.
>
> Is that systematic?  Isn’t it only when ‘load_path_inputs’ is non-empty?
>
>> I tested your path, it works fine. I think it is acceptable as this (gnu
>> ci) interface needs a big rework anyway.
>>
>> I can apply your patch if it's ok for you.
>
> Sure if you’re confident you can go ahead.  :-)

This looks to have been fixed some time ago in:

  76bea3f8bcd951ded88dfb7f8cad5bc3e5a1701f ci: Remove hydra support.

live well,
  vagrant
[signature.asc (application/pgp-signature, inline)]

This bug report was last modified 20 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.