GNU bug report logs - #70410
Fwd: [PATCH] gotools: Workaround non-reproduceability of automake

Previous Next

To reply to this bug, email your comments to 70410 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Gallager <egall <at> gwmail.gwu.edu>
To: bug-automake <at> gnu.org
Subject: Fwd: [PATCH] gotools: Workaround non-reproduceability of automake
Date: Mon, 15 Apr 2024 20:03:36 -0400

GCC developers have recently found a source of non-determinism in
automake; this is bad for reproducible builds:

---------- Forwarded message ---------
From: Jakub Jelinek <jakub <at> redhat.com>
Date: Mon, Apr 15, 2024 at 8:43 AM
Subject: [PATCH] gotools: Workaround non-reproduceability of automake
To: Ian Lance Taylor <iant <at> golang.org>, Mark Wielaard <mjw <at> redhat.com>
Cc: <gcc-patches <at> gcc.gnu.org>


Hi!

The regen bot recently flagged a difference in gotools/Makefile.in.
Trying it locally, it seems pretty random
for i in `seq 20`; do
PATH=~/automake-1.15.1/bin:~/autoconf-2.69/bin:$PATH automake; echo -n
`git diff Makefile.in | wc -l`" "; done; echo; for i in `seq 20`; do
PATH=~/automake-1.15.1/bin:~/autoconf-2.69/bin:$PATH setarch x86_64 -R
automake; echo -n `git diff Makefile.in | wc -l`" "; done; echo;
14 14 14 0 0 0 14 0 14 0 14 14 14 14 0 14 14 0 0 0
14 0 14 0 0 14 14 14 0 14 14 0 0 14 14 14 0 0 0 14
The 14 line git diff is
diff --git a/gotools/Makefile.in b/gotools/Makefile.in
index 36c2ec2abd3..f40883c39be 100644
--- a/gotools/Makefile.in
+++ b/gotools/Makefile.in
@@ -704,8 +704,8 @@ distclean-generic:
 maintainer-clean-generic:
        @echo "This command is intended for maintainers to use"
        @echo "it deletes files that may require special tools to rebuild."
-@NATIVE_FALSE <at> install-exec-local:
 @NATIVE_FALSE <at> uninstall-local:
+@NATIVE_FALSE <at> install-exec-local:
 clean: clean-am

 clean-am: clean-binPROGRAMS clean-generic clean-noinstPROGRAMS \
so whether it is
@NATIVE_FALSE <at> install-exec-local:
@NATIVE_FALSE <at> uninstall-local:
or
@NATIVE_FALSE <at> uninstall-local:
@NATIVE_FALSE <at> install-exec-local:
depends on some hash table traversal or what.

I'm not familiar with automake/m4 enough to debug that, so I'm
instead offering a workaround, with this patch the order is deterministic.

2024-04-15  Jakub Jelinek  <jakub <at> redhat.com>

        * Makefile.am (install-exec-local, uninstall-local): Add goals
        on the else branch of if NATIVE to ensure reproducibility.
        * Makefile.in: Regenerate.

--- gotools/Makefile.am.jj      2023-11-04 09:02:35.802995614 +0100
+++ gotools/Makefile.am 2024-04-15 14:30:03.647171636 +0200
@@ -366,5 +366,7 @@ else
 # only do this if such a compiler is available.  We also need to get
 # the right values for GOARCH and GOOS in the default build context in
 # the go/build package.  Figure this out later.
+install-exec-local:
+uninstall-local:

 endif
--- gotools/Makefile.in.jj      2023-11-03 17:08:46.000439645 +0100
+++ gotools/Makefile.in 2024-04-15 14:31:32.270040117 +0200
@@ -704,8 +704,6 @@ distclean-generic:
 maintainer-clean-generic:
        @echo "This command is intended for maintainers to use"
        @echo "it deletes files that may require special tools to rebuild."
-@NATIVE_FALSE <at> install-exec-local:
-@NATIVE_FALSE <at> uninstall-local:
 clean: clean-am

 clean-am: clean-binPROGRAMS clean-generic clean-noinstPROGRAMS \
@@ -1035,6 +1033,8 @@ mostlyclean-local:
 # only do this if such a compiler is available.  We also need to get
 # the right values for GOARCH and GOOS in the default build context in
 # the go/build package.  Figure this out later.
+@NATIVE_FALSE <at> install-exec-local:
+@NATIVE_FALSE <at> uninstall-local:

 # Tell versions [3.59,3.63) of GNU make to not export all variables.
 # Otherwise a system limit (for SysV at least) may be exceeded.

        Jakub

Message #8 received at 70410 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: egall <at> gwmail.gwu.edu
Cc: 70410 <at> debbugs.gnu.org
Subject: Re: bug#70410: Fwd: [PATCH] gotools: Workaround non-reproduceability
 of automake
Date: Wed, 17 Apr 2024 09:55:51 -0600

    so whether it is
    @NATIVE_FALSE <at> install-exec-local:
    @NATIVE_FALSE <at> uninstall-local:
    or
    @NATIVE_FALSE <at> uninstall-local:
    @NATIVE_FALSE <at> install-exec-local:
    depends on some hash table traversal or what.

Thanks for the report. Any chance of a Makefile.am that can reproduce
the problem? I wonder if it's related to the @NATIVE_FALSE@ somehow.

Looking at automake.in, it's not obvious to me where a list is failed to
be sorted. Those -local targets aren't generated by automake itself, so
far as I can see. --thanks, karl.

Message #11 received at 70410 <at> debbugs.gnu.org (full text, mbox):

From: Eric Gallager <egall <at> gwmail.gwu.edu>
To: Karl Berry <karl <at> freefriends.org>
Cc: 70410 <at> debbugs.gnu.org
Subject: Re: bug#70410: Fwd: [PATCH] gotools: Workaround non-reproduceability
 of automake
Date: Wed, 17 Apr 2024 13:12:49 -0400

Makefile.am in question is from the gotools subdirectory of GCC:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gotools/Makefile.am;h=80b21847117fb1b685a677725826f4caba4e759e;hb=HEAD
Note that the original reporter, Jakub Jelinek, has said that this
might potentially be due to the use of an old version, so this might
not actually be a current bug after all:
https://gcc.gnu.org/pipermail/gcc-patches/2024-April/649576.html

On Wed, Apr 17, 2024 at 11:56 AM Karl Berry <karl <at> freefriends.org> wrote:
>
>     so whether it is
>     @NATIVE_FALSE <at> install-exec-local:
>     @NATIVE_FALSE <at> uninstall-local:
>     or
>     @NATIVE_FALSE <at> uninstall-local:
>     @NATIVE_FALSE <at> install-exec-local:
>     depends on some hash table traversal or what.
>
> Thanks for the report. Any chance of a Makefile.am that can reproduce
> the problem? I wonder if it's related to the @NATIVE_FALSE@ somehow.
>
> Looking at automake.in, it's not obvious to me where a list is failed to
> be sorted. Those -local targets aren't generated by automake itself, so
> far as I can see. --thanks, karl.
>

Message #14 received at 70410 <at> debbugs.gnu.org (full text, mbox):

From: Nick Bowler <nbowler <at> draconx.ca>
To: Karl Berry <karl <at> freefriends.org>
Cc: egall <at> gwmail.gwu.edu, 70410 <at> debbugs.gnu.org
Subject: Re: bug#70410: Fwd: [PATCH] gotools: Workaround non-reproduceability
 of automake
Date: Wed, 17 Apr 2024 13:26:59 -0400

On 2024-04-17 11:55, Karl Berry wrote:
>     so whether it is
>     @NATIVE_FALSE <at> install-exec-local:
>     @NATIVE_FALSE <at> uninstall-local:
>     or
>     @NATIVE_FALSE <at> uninstall-local:
>     @NATIVE_FALSE <at> install-exec-local:
>     depends on some hash table traversal or what.
> 
> Thanks for the report. Any chance of a Makefile.am that can reproduce
> the problem? I wonder if it's related to the @NATIVE_FALSE@ somehow.
> 
> Looking at automake.in, it's not obvious to me where a list is failed to
> be sorted. Those -local targets aren't generated by automake itself, so
> far as I can see. --thanks, karl.

Code like the above is generated by Automake when -hook or -local
targets are defined within the "true" side of an Automake conditional,
but not in the "false" side, for example:

  % cat >configure.ac <<'EOF'
AC_INIT([test], [0])
AM_INIT_AUTOMAKE([foreign])
AM_CONDITIONAL([NATIVE], [:])
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
EOF
  % cat >Makefile.am <<'EOF'
if NATIVE
install-data-local: ;
install-exec-local: ;
uninstall-local: ;
dist-hook: ;
all-local: ;
endif
EOF
  % autoreconf -is
  % grep NATIVE_FALSE Makefile.in
@NATIVE_FALSE <at> dist-hook:
@NATIVE_FALSE <at> all-local:
@NATIVE_FALSE <at> install-data-local:
@NATIVE_FALSE <at> install-exec-local:
@NATIVE_FALSE <at> uninstall-local:

With an older Automake (1.12.6), the order of these generated targets is
easily seen to be inconsistent.  But with a current version of Automake
(1.16.5) the order seems to be always the same.  So I wonder if this is
reporting an issue that was long ago already fixed in Automake...

Cheers,
  Nick

Message #17 received at 70410 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: nbowler <at> draconx.ca
Cc: egall <at> gwmail.gwu.edu, 70410 <at> debbugs.gnu.org
Subject: Re: bug#70410: Fwd: [PATCH] gotools: Workaround non-reproduceability
 of automake
Date: Thu, 18 Apr 2024 15:04:52 -0600

Per https://gcc.gnu.org/pipermail/gcc-patches/2024-April/649576.html
this was fixed in https://bugs.gnu.org/46744. As I understand it.
Closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.