GNU bug report logs - #30104
ant-bootstrap@1.7.1 does not build reproducibly on core-updates

Previous Next

Package: guix;

Reported by: Chris Marusich <cmmarusich <at> gmail.com>

Date: Sun, 14 Jan 2018 04:17:03 UTC

Severity: normal

Done: Gábor Boskovits <boskovits <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30104 in the body.
You can then email your comments to 30104 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Sun, 14 Jan 2018 04:17:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Chris Marusich <cmmarusich <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Sun, 14 Jan 2018 04:17:03 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Chris Marusich <cmmarusich <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: ant-bootstrap <at> 1.7.1 does not build reproducibly on core-updates
Date: Sat, 13 Jan 2018 20:16:38 -0800
[Message part 1 (text/plain, inline)]
Hi,

At commit 1b321229f4653c5daa873813e24910789c0b2918 (i.e., the current
tip of the core-updates branch), ant-bootstrap <at> 1.7.1 does not build
reproducibly.  This package is defined in gnu/packages/java.scm, but it
is not exported (i.e., it is used privately within the module).  Note
that according to 'guix refresh', currently 215 packages depend on this
package.

To verify that it doesn't build reproducibly, run this command:

  guix build --rounds=2 -e '(@@ (gnu packages java) ant-bootstrap)'

Here's the last bit of output I got:

--8<---------------cut here---------------start------------->8---
output ‘/gnu/store/cl2pxkxrasazj7hb7ykjh895vfj76jfy-ant-bootstrap-1.7.1’ of ‘/gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv’ differs from previous round
@ build-failed /gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv - 1 output ‘/gnu/store/cl2pxkxrasazj7hb7ykjh895vfj76jfy-ant-bootstrap-1.7.1’ of ‘/gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv’ differs from previous round
guix build: error: build failed: build of `/gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv' failed
--8<---------------cut here---------------end--------------->8---

-- 
Chris
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Sun, 14 Jan 2018 09:36:01 GMT) Full text and rfc822 format available.

Message #8 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: Chris Marusich <cmmarusich <at> gmail.com>
Cc: 30104 <at> debbugs.gnu.org
Subject: Re: bug#30104: ant-bootstrap <at> 1.7.1 does not build reproducibly on
 core-updates
Date: Sun, 14 Jan 2018 10:35:35 +0100
[Message part 1 (text/plain, inline)]
Could you please attach a diffoscope output to this?

I've found this in the notes database, does not sound too promising:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/ant.html.

2018-01-14 5:16 GMT+01:00 Chris Marusich <cmmarusich <at> gmail.com>:

> Hi,
>
> At commit 1b321229f4653c5daa873813e24910789c0b2918 (i.e., the current
> tip of the core-updates branch), ant-bootstrap <at> 1.7.1 does not build
> reproducibly.  This package is defined in gnu/packages/java.scm, but it
> is not exported (i.e., it is used privately within the module).  Note
> that according to 'guix refresh', currently 215 packages depend on this
> package.
>
> To verify that it doesn't build reproducibly, run this command:
>
>   guix build --rounds=2 -e '(@@ (gnu packages java) ant-bootstrap)'
>
> Here's the last bit of output I got:
>
> --8<---------------cut here---------------start------------->8---
> output ‘/gnu/store/cl2pxkxrasazj7hb7ykjh895vfj76jfy-ant-bootstrap-1.7.1’
> of ‘/gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv’
> differs from previous round
> @ build-failed /gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv
> - 1 output ‘/gnu/store/cl2pxkxrasazj7hb7ykjh895vfj76jfy-ant-bootstrap-1.7.1’
> of ‘/gnu/store/9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv’
> differs from previous round
> guix build: error: build failed: build of `/gnu/store/
> 9p8yw4cjcb2529gz9l6x4d559xb1qvql-ant-bootstrap-1.7.1.drv' failed
> --8<---------------cut here---------------end--------------->8---
>
> --
> Chris
>
[Message part 2 (text/html, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Sun, 14 Jan 2018 16:41:02 GMT) Full text and rfc822 format available.

Message #11 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: 30104 <at> debbugs.gnu.org, Chris Marusich <cmmarusich <at> gmail.com>
Subject: Re: bug#30104: ant-bootstrap <at> 1.7.1 does not build reproducibly on
 core-updates
Date: Sun, 14 Jan 2018 17:40:11 +0100
Gábor Boskovits <boskovits <at> gmail.com> writes:

> Could you please attach a diffoscope output to this?
>
> I've found this in the notes database, does not sound too promising:
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/ant.html.

We are not using GCJ.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net






Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Thu, 18 Jan 2018 08:59:02 GMT) Full text and rfc822 format available.

Message #14 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 30104 <at> debbugs.gnu.org
Subject: Difference seems to be caused by timestamp
Date: Thu, 18 Jan 2018 09:58:07 +0100
[Message part 1 (text/plain, inline)]
This is due to the modification times of files included in the jar.
No other difference is listed in the diffoscope output.

How do we deal with this in other packages?
That fix should be migrated to apply for this package also.
[Message part 2 (text/html, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Thu, 18 Jan 2018 09:59:01 GMT) Full text and rfc822 format available.

Message #17 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 30104 <at> debbugs.gnu.org
Subject: Re: bug#30104: Difference seems to be caused by timestamp
Date: Thu, 18 Jan 2018 10:58:12 +0100
[Message part 1 (text/plain, inline)]
Ok, we have this in guix/build/ant-builde-system.scm,
it is called strip-jar-timestamps.

This should be done also on ant-bootstrap.

2018-01-18 9:58 GMT+01:00 Gábor Boskovits <boskovits <at> gmail.com>:

> This is due to the modification times of files included in the jar.
> No other difference is listed in the diffoscope output.
>
> How do we deal with this in other packages?
> That fix should be migrated to apply for this package also.
>
[Message part 2 (text/html, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Mon, 11 Jun 2018 11:54:01 GMT) Full text and rfc822 format available.

Message #20 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 30104 <at> debbugs.gnu.org
Cc: Gábor Boskovits <boskovits <at> gmail.com>
Subject: [PATCH] gnu: ant-bootstrap: Make build reproducible.
Date: Mon, 11 Jun 2018 13:52:45 +0200
* gnu/packages/java.scm (ant-bootstrap)[arguments]<#:phases>[strip-jar-timestamps]:
New phase to reset jar file timestamps.
<#:imported-modules>: Import guix build syscalls, gnu-build-system-modules.
<#:modules>: Import guix build: gnu-build-system, utils, syscalls.
[native-inputs]: Add unzip, zip.
---
 gnu/packages/java.scm | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 678cbee18..4fdce65af 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -212,7 +212,13 @@ JNI.")
                 "1cg0lga887qz5iizh6mlkxp01lciymrhmp7wzxpl6zpnldxmzrjx"))))
     (build-system gnu-build-system)
     (arguments
-     `(#:tests? #f ; no "check" target
+     `(#:imported-modules ((guix build syscalls)
+                           ,@%gnu-build-system-modules)
+       #:modules ((srfi srfi-1)
+                  (guix build gnu-build-system)
+                  (guix build utils)
+                  (guix build syscalls))
+       #:tests? #f ; no "check" target
        #:phases
        (modify-phases %standard-phases
          (delete 'bootstrap)
@@ -254,10 +260,43 @@ JNI.")
              (zero? (system* "bash" "bootstrap.sh"
                              (string-append "-Ddist.dir="
                                             (assoc-ref %outputs "out"))))))
+         (add-after 'build 'strip-jar-timestamps ;based on ant-build-system
+           (lambda* (#:key outputs #:allow-other-keys)
+             (define (repack-archive jar)
+               (let* ((dir (mkdtemp! "jar-contents.XXXXXX"))
+                      (manifest (string-append dir "/META-INF/MANIFESTS.MF")))
+                 (and (with-directory-excursion dir
+                        (invoke "unzip" jar))
+                      (delete-file jar)
+                      ;; XXX: copied from (gnu build install)
+                      (for-each (lambda (file)
+                                  (let ((s (lstat file)))
+                                    (unless (eq? (stat:type s) 'symlink)
+                                      (utime file  0 0 0 0))))
+                                (find-files dir #:directories? #t))
+                      ;; It is important that the manifes appears first.
+                      (with-directory-excursion dir
+                        (let* ((files (find-files "." ".*" #:directories? #t))
+                               ;; To ensure that the reference scanner can
+                               ;; detect all store references in the jars
+                               ;; we disable compression with the "-0" option.
+                               (command (if (file-exists? manifest)
+                                            `("zip" "-0" "-X" ,jar ,manifest
+                                              ,@files)
+                                            `("zip" "-0" "-X" ,jar ,@files))))
+                          (apply invoke command)))
+                      (utime jar 0 0)
+                      #t)))
+             (every repack-archive
+                    (find-files
+                     (string-append (assoc-ref %outputs "out") "/lib")
+                     "\\.jar$"))))
          (delete 'install))))
     (native-inputs
      `(("jikes" ,jikes)
-       ("jamvm" ,jamvm-1-bootstrap)))
+       ("jamvm" ,jamvm-1-bootstrap)
+       ("unzip" ,unzip)
+       ("zip", zip)))
     (home-page "http://ant.apache.org")
     (synopsis "Build tool for Java")
     (description
-- 
2.17.1





Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Tue, 12 Jun 2018 13:17:01 GMT) Full text and rfc822 format available.

Message #23 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 30104 <at> debbugs.gnu.org,
 Julien Lepiller <julien <at> lepiller.eu>
Subject: Re: bug#30104: [PATCH] gnu: ant-bootstrap: Make build reproducible.
Date: Tue, 12 Jun 2018 15:15:47 +0200
Hello!

Gábor Boskovits <boskovits <at> gmail.com> skribis:

> * gnu/packages/java.scm (ant-bootstrap)[arguments]<#:phases>[strip-jar-timestamps]:
> New phase to reset jar file timestamps.
> <#:imported-modules>: Import guix build syscalls, gnu-build-system-modules.
> <#:modules>: Import guix build: gnu-build-system, utils, syscalls.
> [native-inputs]: Add unzip, zip.

Great that you’re tackling this issue!

I have very minor comments:

> +         (add-after 'build 'strip-jar-timestamps ;based on ant-build-system
> +           (lambda* (#:key outputs #:allow-other-keys)
> +             (define (repack-archive jar)
> +               (let* ((dir (mkdtemp! "jar-contents.XXXXXX"))
> +                      (manifest (string-append dir "/META-INF/MANIFESTS.MF")))
> +                 (and (with-directory-excursion dir
> +                        (invoke "unzip" jar))
> +                      (delete-file jar)

‘invoke’ throws an exception upon failure, and the return value of
‘delete-file’ is unspecified, so you shouldn’t rely on it.

Thus, it should be written without ‘and’:

  (with-directory-excursion
    (invoke "unzip" jar))
  (delete-file jar)
  …

> +                      ;; It is important that the manifes appears first.
                                                            ^
Typo.

> +                      (utime jar 0 0)

I think this is unnecessary because guix-daemon resets timestamps on all
the file upon build completion.

> +             (every repack-archive
> +                    (find-files
> +                     (string-append (assoc-ref %outputs "out") "/lib")
> +                     "\\.jar$"))))

Use ‘for-each’ instead of ‘every’ (because the return value of
‘repack-archive’ is undefined/unused), and add a trailing #t to denote
success.

If Ricardo and Julien don’t have anything to add, I think you can go
ahead and push with these changes.

Thank you!

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#30104; Package guix. (Tue, 12 Jun 2018 13:23:01 GMT) Full text and rfc822 format available.

Message #26 received at 30104 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 30104 <at> debbugs.gnu.org,
 Gábor Boskovits <boskovits <at> gmail.com>,
 Julien Lepiller <julien <at> lepiller.eu>
Subject: Re: bug#30104: [PATCH] gnu: ant-bootstrap: Make build reproducible.
Date: Tue, 12 Jun 2018 15:21:52 +0200
Ludovic Courtès <ludo <at> gnu.org> writes:

> Hello!
>
> Gábor Boskovits <boskovits <at> gmail.com> skribis:
>
>> * gnu/packages/java.scm (ant-bootstrap)[arguments]<#:phases>[strip-jar-timestamps]:
>> New phase to reset jar file timestamps.
>> <#:imported-modules>: Import guix build syscalls, gnu-build-system-modules.
>> <#:modules>: Import guix build: gnu-build-system, utils, syscalls.
>> [native-inputs]: Add unzip, zip.
>
> Great that you’re tackling this issue!

I concur.  Thanks Gábor!

> If Ricardo and Julien don’t have anything to add, I think you can go
> ahead and push with these changes.

I have nothing to add.  It looks good to me with these changes.

Thanks!

-- 
Ricardo





Reply sent to Gábor Boskovits <boskovits <at> gmail.com>:
You have taken responsibility. (Mon, 18 Jun 2018 08:40:01 GMT) Full text and rfc822 format available.

Notification sent to Chris Marusich <cmmarusich <at> gmail.com>:
bug acknowledged by developer. (Mon, 18 Jun 2018 08:40:02 GMT) Full text and rfc822 format available.

Message #31 received at 30104-done <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 30104-done <at> debbugs.gnu.org
Subject: ant-bootstrap does not build reproducibly
Date: Mon, 18 Jun 2018 10:38:56 +0200
[Message part 1 (text/plain, inline)]
Fixed on staging, commit cded3a759356ff66b7df668bcdbdfa0daf96f4c5.
[Message part 2 (text/html, inline)]

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 16 Jul 2018 11:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 5 years and 257 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.