GNU bug report logs - #57593
Racket 8.6 is not reproducible

Previous Next

Package: guix;

Reported by: Ludovic Courtès <ludo <at> gnu.org>

Date: Mon, 5 Sep 2022 10:05:02 UTC

Severity: normal

To reply to this bug, email your comments to 57593 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to philip <at> philipmcgrath.com, bug-guix <at> gnu.org:
bug#57593; Package guix. (Mon, 05 Sep 2022 10:05:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ludovic Courtès <ludo <at> gnu.org>:
New bug report received and forwarded. Copy sent to philip <at> philipmcgrath.com, bug-guix <at> gnu.org. (Mon, 05 Sep 2022 10:05:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: bug-guix <at> gnu.org
Subject: Racket 8.6 is not reproducible
Date: Mon, 05 Sep 2022 12:04:02 +0200
Hey,

Just noticed that Racket does not build in a reproducible fashion:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 227  Sep 04 2022 23:39:52    (current)
  guix aae98c2
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: aae98c297214f87eb45302863adb021078c41a6f
$ guix challenge racket | head -40
/gnu/store/av255rh362283i1zaiq9rz4rpli69j59-racket-8.6 contents differ:
  local hash: 1w4dnkvpbbrgfasyq8x1cbqw58jzqsny17ms5l1fb1h6iid38bs1
  https://ci.guix.gnu.org/nar/lzip/av255rh362283i1zaiq9rz4rpli69j59-racket-8.6: 1lnxklizpnc599w7n2svb1jaw595wranm9aagd2928fcbiaavbr6
  differing files:
    /lib/racket/pkgs/2d-doc/scribblings/compiled/2d_scrbl.dep
    /lib/racket/pkgs/2d-doc/scribblings/compiled/2d_scrbl.zo
    /lib/racket/pkgs/algol60/compiled/algol60_rkt.dep
    /lib/racket/pkgs/algol60/compiled/algol60_scrbl.dep
    /lib/racket/pkgs/algol60/compiled/algol60_scrbl.zo
    /lib/racket/pkgs/algol60/compiled/cfg-parser_rkt.dep
    /lib/racket/pkgs/algol60/compiled/compile_rkt.dep
    /lib/racket/pkgs/algol60/compiled/parse_rkt.dep
    /lib/racket/pkgs/algol60/compiled/parse_rkt.zo
    /lib/racket/pkgs/algol60/compiled/simplify_rkt.dep
    /lib/racket/pkgs/algol60/compiled/tool_rkt.dep
    /lib/racket/pkgs/algol60/lang/compiled/reader_rkt.dep
    /lib/racket/pkgs/algol60/tests/compiled/export_rkt.dep
    /lib/racket/pkgs/algol60/tests/compiled/export_rkt.zo
    /lib/racket/pkgs/algol60/tests/compiled/syncheck-test_rkt.dep
    /lib/racket/pkgs/algol60/tests/compiled/test_rkt.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/awk_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/awk_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/cmdline_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/cmdline_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/cml_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/cml_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/common_rkt.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/common_rkt.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/compat_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/compat_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/compile_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/compile_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/contract-label_rkt.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/contract_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/contract_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/etc_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/etc_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/file_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/file_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/for_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/for_scrbl.zo
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/include_scrbl.dep
    /lib/racket/pkgs/compatibility-doc/mzlib/scribblings/compiled/include_scrbl.zo
--8<---------------cut here---------------end--------------->8---

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#57593; Package guix. (Wed, 07 Sep 2022 17:39:02 GMT) Full text and rfc822 format available.

Message #8 received at 57593 <at> debbugs.gnu.org (full text, mbox):

From: "Philip McGrath" <philip <at> philipmcgrath.com>
To: Ludovic Courtès <ludo <at> gnu.org>, 57593 <at> debbugs.gnu.org
Subject: Re: bug#57593: Racket 8.6 is not reproducible
Date: Wed, 07 Sep 2022 13:36:33 -0400
Hi,

On Mon, Sep 5, 2022, at 6:04 AM, Ludovic Courtès wrote:
> Hey,
>
> Just noticed that Racket does not build in a reproducible fashion:
>
> --8<---------------cut here---------------start------------->8---
> $ guix describe
> Generation 227  Sep 04 2022 23:39:52    (current)
>   guix aae98c2
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: master
>     commit: aae98c297214f87eb45302863adb021078c41a6f
> $ guix challenge racket | head -40
> /gnu/store/av255rh362283i1zaiq9rz4rpli69j59-racket-8.6 contents differ:
>   local hash: 1w4dnkvpbbrgfasyq8x1cbqw58jzqsny17ms5l1fb1h6iid38bs1
>   
> https://ci.guix.gnu.org/nar/lzip/av255rh362283i1zaiq9rz4rpli69j59-racket-8.6: 
> 1lnxklizpnc599w7n2svb1jaw595wranm9aagd2928fcbiaavbr6
>   differing files:
>     /lib/racket/pkgs/2d-doc/scribblings/compiled/2d_scrbl.dep
>     /lib/racket/pkgs/2d-doc/scribblings/compiled/2d_scrbl.zo

Unfortunately, this is a known issue upstream.

One part of the problem is the way that Chez Scheme uses "type 4" (random) UUIDs to implement 'gensym'. There's some upstream discussion at <https://github.com/cisco/ChezScheme/issues/585>, including an idea from me about "type 5" UUIDs as part of a possible solution.

That may well be the only issue with the core compiler for Racket CS. I know effort has been taken to make Racket's front-end deterministic, and Kent Dybvig says that:

> Chez Scheme compiles are reproducible modulo the globally unique
> names of generated symbols (gensyms), assuming macros don't do
> things like introduce time stamps or random expressions into their output.
> Because of differences in gensym names, the files produced by multiple
> runs are not byte-by-byte identical but should compare equal via
> the system primitive $fasl-file-equal? See, for example, the use
> of $fasl-file-equal? in the s/Mf-base checkboot target.

In the output you sent, the ".zo" files contain Chez machine code with a Racket-specific wrapper, and the ".dep" files contain hashes of dependencies (so they differ when the ".zo" files they track differ).

Debian reports at <https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/racket.html>, discussing Racket 8.2+dfsg1-2:

> Build path captured in html documentation. docindex.sqlite
> contains content in arbitrary order. strings in binaries in
> arbitrary order.

Those issues don't appear to affect Guix: some might be caused by Debian conflating "build" and "install" for Racket packages, and, in any case, they seem related to the "scribble" and/or "racket-index" packages, not the core compiler.

IIRC, there were some reproducibility issues with the Racket BC bytecode compiler. If we can build reproducibly with CS but not BC, it might work out to disable compilation for the bootstrapping BC compilers, or maybe to configure them to use the machine-independent format (basically, a serialized form of what the front-end would hand off to the back-end), at the cost of some performance.

I hope to work on these issues, but I plan to work on a 'racket-build-system' first. Among other things, I hope it will be easier to investigate issues when Racket packages can be built independently.

-Philip




This bug report was last modified 1 year and 203 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.