GNU bug report logs - #43721
Package 'dune' (ocaml) is not Reproducible

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43721 in the body.
You can then email your comments to 43721 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: Package 'dune' (ocaml) is not Reproducible
Date: Wed, 30 Sep 2020 13:56:21 +0200

Dear,

Using Guix 04a459a, the classical:

   guix build dune
   guix build dune --no-grafts --check -K

return:

--8<---------------cut here---------------start------------->8---
guix build: error: derivation `/gnu/store/kg7mnb1b09n5zddgvc4hs8ira3i4jzz2-dune-1.11.3.drv' may not be deterministic: output `/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3' differs from ‘/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check’
--8<---------------cut here---------------end--------------->8---

then:

   diff -r --no-dereference /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3{,-check}
   
returns a large output (truncated here):

--8<---------------cut here---------------start------------->8---
Binary files /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator.cma and /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator.cma differ
Binary files /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt and /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt differ
Binary files /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti and /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti differ

[...]

Binary files /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/_stdune/stdune__User_error.cmti and /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/_stdune/stdune__User_error.cmti differ
Binary files /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/_stdune/stdune__User_message.cmti and /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/_stdune/stdune__User_message.cmti differ
Binary files /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/_stdune/stdune__User_warning.cmti and /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/_stdune/stdune__User_warning.cmti differ
--8<---------------cut here---------------end--------------->8---


Is it expected?  Does it comes from timestamp?  Is it avoidable?


I have not tried (yet) with the patch [1] updating ’dune’.

[1] <http://issues.guix.gnu.org/issue/40091>

All the best,
simon

Message #8 received at 43721 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 43721 <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Wed, 30 Sep 2020 14:41:48 +0200

[Message part 1 (text/plain, inline)]

Zimoun,

Thanks for the report!

zimoun 写道：
> diff -r --no-dereference 
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3{,-check}

Don't use ‘diff’ for this, use something like diffoscope:

 $ diffoscope --exclude-directory-metadata --html dune.html $old 
 $new
 -> https://www.tobias.gr/dune.html

I think the problem is confined to the first screenful: one header 
byte differs, and then $old has an extra byte that shifts 
everything to the right causing these huge diffs.

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 43721 <at> debbugs.gnu.org (full text, mbox):

From: Julien Lepiller <julien <at> lepiller.eu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 43721 <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Wed, 30 Sep 2020 14:42:32 +0200

Le Wed, 30 Sep 2020 13:56:21 +0200,
zimoun <zimon.toutoune <at> gmail.com> a écrit :

> Dear,
> 
> Using Guix 04a459a, the classical:
> 
>    guix build dune
>    guix build dune --no-grafts --check -K
> 
> return:
> 
> --8<---------------cut here---------------start------------->8---
> guix build: error: derivation
> `/gnu/store/kg7mnb1b09n5zddgvc4hs8ira3i4jzz2-dune-1.11.3.drv' may not
> be deterministic: output
> `/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3' differs
> from ‘/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check’
> --8<---------------cut here---------------end--------------->8---
> 
> then:
> 
>    diff -r --no-dereference
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3{,-check} 
> returns a large output (truncated here):
> 
> --8<---------------cut here---------------start------------->8---
> Binary files
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator.cma
> and
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator.cma
> differ Binary files
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt
> and
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt
> differ Binary files
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti
> and
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti
> differ
> 
> [...]
> 
> Binary files
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/_stdune/stdune__User_error.cmti
> and
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/_stdune/stdune__User_error.cmti
> differ Binary files
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/_stdune/stdune__User_message.cmti
> and
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/_stdune/stdune__User_message.cmti
> differ Binary files
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/_stdune/stdune__User_warning.cmti
> and
> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/_stdune/stdune__User_warning.cmti
> differ --8<---------------cut
> here---------------end--------------->8---
> 
> 
> Is it expected?  Does it comes from timestamp?  Is it avoidable?
> 
> 
> I have not tried (yet) with the patch [1] updating ’dune’.
> 
> [1] <http://issues.guix.gnu.org/issue/40091>
> 
> All the best,
> simon
> 
> 
> 

The difference seems to be bigger than a simple timestamp, but it's all
on binary files, and diffoscope is not very useful for that. Maybe file
system ordering? I'm investigating.

Message #14 received at 43721 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Julien Lepiller <julien <at> lepiller.eu>
Cc: 43721 <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Tue, 13 Oct 2020 14:17:10 +0200

Dear,

On Wed, 30 Sep 2020 at 14:42, Julien Lepiller <julien <at> lepiller.eu> wrote:

>> --8<---------------cut here---------------start------------->8---
>> Binary files
>> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator.cma
>> and
>> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator.cma
>> differ Binary files
>> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt
>> and
>> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt
>> differ Binary files
>> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti
>> and
>> /gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti
>> differ

[...]

> The difference seems to be bigger than a simple timestamp, but it's all
> on binary files, and diffoscope is not very useful for that. Maybe file
> system ordering? I'm investigating.

Is it not related to your detailed analysis:

   <http://issues.guix.gnu.org/43745#51>

BTW, the update of “dune“ #43745 seems fixing the issue.


All the best,
simon

Message #17 received at 43721 <at> debbugs.gnu.org (full text, mbox):

From: Julien Lepiller <julien <at> lepiller.eu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 43721 <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Tue, 13 Oct 2020 11:29:39 -0400

Le 13 octobre 2020 08:17:10 GMT-04:00, zimoun <zimon.toutoune <at> gmail.com> a écrit :
>Dear,
>
>On Wed, 30 Sep 2020 at 14:42, Julien Lepiller <julien <at> lepiller.eu>
>wrote:
>
>>> --8<---------------cut here---------------start------------->8---
>>> Binary files
>>>
>/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator.cma
>>> and
>>>
>/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator.cma
>>> differ Binary files
>>>
>/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt
>>> and
>>>
>/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__Extract_obj.cmt
>>> differ Binary files
>>>
>/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti
>>> and
>>>
>/gnu/store/1wwdmzcjhrpal92sz2zwzhyqmbc3w7ri-dune-1.11.3-check/lib/ocaml/site-lib/dune/configurator/configurator__V1.cmti
>>> differ
>
>[...]
>
>> The difference seems to be bigger than a simple timestamp, but it's
>all
>> on binary files, and diffoscope is not very useful for that. Maybe
>file
>> system ordering? I'm investigating.
>
>Is it not related to your detailed analysis:
>
>   <http://issues.guix.gnu.org/43745#51>

Yes, the differences look very similar between these two issues. I've posted a bug report at https://github.com/ocaml/dune/issues/3863. Hopefully this will result in a good fix. As a workaround, building with one thread seems to be effective, but we need to change the dune-build-system for that.

>
>BTW, the update of “dune“ #43745 seems fixing the issue.

Yes indeed :) nice side-effect.

>
>
>All the best,
>simon

-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.

Message #20 received at 43721 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Julien Lepiller <julien <at> lepiller.eu>
Cc: 43721 <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Tue, 13 Oct 2020 17:41:32 +0200

On Tue, 13 Oct 2020 at 17:29, Julien Lepiller <julien <at> lepiller.eu> wrote:

> Yes, the differences look very similar between these two issues. I've posted a bug report at https://github.com/ocaml/dune/issues/3863. Hopefully this will result in a good fix. As a workaround, building with one thread seems to be effective, but we need to change the dune-build-system for that.

Which means core-updates, right?

Message #23 received at 43721 <at> debbugs.gnu.org (full text, mbox):

From: Julien Lepiller <julien <at> lepiller.eu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 43721 <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Tue, 13 Oct 2020 11:54:13 -0400

Le 13 octobre 2020 11:41:32 GMT-04:00, zimoun <zimon.toutoune <at> gmail.com> a écrit :
>On Tue, 13 Oct 2020 at 17:29, Julien Lepiller <julien <at> lepiller.eu>
>wrote:
>
>> Yes, the differences look very similar between these two issues. I've
>posted a bug report at https://github.com/ocaml/dune/issues/3863.
>Hopefully this will result in a good fix. As a workaround, building
>with one thread seems to be effective, but we need to change the
>dune-build-system for that.
>
>Which means core-updates, right?

We don't have more than 300 dependents, so I think it should be master.

Message #28 received at 43721-done <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Julien Lepiller <julien <at> lepiller.eu>
Cc: 43721-done <at> debbugs.gnu.org
Subject: Re: bug#43721: Package 'dune' (ocaml) is not Reproducible
Date: Wed, 28 Oct 2020 19:54:31 +0100

Hi,

On Tue, 13 Oct 2020 at 11:29, Julien Lepiller <julien <at> lepiller.eu> wrote:

>>BTW, the update of “dune“ #43745 seems fixing the issue.
>
> Yes indeed :) nice side-effect.

So closing!

Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.