GNU bug report logs - #26471
Automake 1.15 generates a recheck target that depends on all, breaking parallel builds

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: automake; Reported by: Seth Fowler <seth.fowler@HIDDEN>; Keywords: moreinfo; dated Thu, 13 Apr 2017 05:16:02 UTC; Maintainer for automake is bug-automake@HIDDEN.
Added tag(s) moreinfo. Request was from Mathieu Lirzin <mthl@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 26471 <at> debbugs.gnu.org:


Received: (at 26471) by debbugs.gnu.org; 13 Apr 2017 13:52:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 13 09:52:55 2017
Received: from localhost ([127.0.0.1]:45274 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cyfB9-0003cn-3x
	for submit <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:52:55 -0400
Received: from eggs.gnu.org ([208.118.235.92]:56899)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mthl@HIDDEN>) id 1cyfB7-0003cW-Jt
 for 26471 <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:52:54 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <mthl@HIDDEN>) id 1cyfAy-0002Xv-BH
 for 26471 <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:52:48 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60577)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <mthl@HIDDEN>)
 id 1cyfAy-0002Xq-7K; Thu, 13 Apr 2017 09:52:44 -0400
Received: from mek33-4-82-236-46-88.fbx.proxad.net ([82.236.46.88]:46750
 helo=godel)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <mthl@HIDDEN>)
 id 1cyfAx-0008Cc-La; Thu, 13 Apr 2017 09:52:44 -0400
From: Mathieu Lirzin <mthl@HIDDEN>
To: Seth Fowler <seth.fowler@HIDDEN>
Subject: Re: bug#26471: Automake 1.15 generates a recheck target that depends
 on all, breaking parallel builds
References: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN>
Date: Thu, 13 Apr 2017 15:52:41 +0200
In-Reply-To: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN> (Seth Fowler's
 message of "Wed, 12 Apr 2017 21:54:00 -0700")
Message-ID: <87wpaojtxy.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26471
Cc: 26471 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Hi Seth,

Seth Fowler <seth.fowler@HIDDEN> writes:

> I recently ran into what appears to be a bug in automake and I thought it=
 would be a good idea to let you know.
>
> We had noticed that running =E2=80=9Cmake recheck -j8=E2=80=9D if some so=
urce files
> were dirty would cause random build failures. The symptom was that the
> same file was getting built more than once by different make
> processes, which led to the resulting objects being corrupted. We=E2=80=
=99d
> see messages like this:
>
>     libtool:   error: =E2=80=98foo.lo=E2=80=99 is not a valid libtool obj=
ect
>
> I dug into this a little and the root cause of the problem seems to be
> that unlike the other top-level targets generated by automake (check,
> install, etc.), recheck depends on =E2=80=9Call=E2=80=9D instead of =E2=
=80=9Call-am=E2=80=9D. =E2=80=9Call=E2=80=9D
> doesn=E2=80=99t declare very many dependencies - it just launches a new c=
opy
> of make that builds =E2=80=9Call-am=E2=80=9D. Because recheck also depend=
s on other
> targets (e.g. everything in check_PROGRAMS) which might depend on some
> of the same things as =E2=80=9Call-am=E2=80=9D, in a parallel build the o=
riginal make
> process and the make process spawned by =E2=80=9Call=E2=80=9D can end up =
attempting to
> build the same targets.
>
> I fixed this for our project by copying the generated recheck rule
> into our Makefile.am and replacing =E2=80=9Call=E2=80=9D with =E2=80=9Cal=
l-am=E2=80=9D. After doing
> this, I could no longer reproduce the problem with =E2=80=9Cmake recheck =
-j8=E2=80=9D.
>
> It seems like that change should be made in automake itself. Or maybe I j=
ust missed something - I=E2=80=99m far from an automake guru. =3D)

Seems like a bug.

In order to properly fix this in Automake, we need to have a test
that allow us to reproduce the issue.  Ideally you would provide
this test yourself by taking inspiration from:

  https://git.savannah.gnu.org/cgit/automake.git/tree/t/libtool2.sh?h=3Dmic=
ro

If you are not comfortable with doing that, a tarball or a link to a
repository containing a minimal automake project that allow us to
reproduce the bug manually is good enough.

Thanks for the report.

--=20
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37




Information forwarded to bug-automake@HIDDEN:
bug#26471; Package automake. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 13 Apr 2017 05:15:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 13 01:15:33 2017
Received: from localhost ([127.0.0.1]:44953 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cyX6S-0002Ux-Mm
	for submit <at> debbugs.gnu.org; Thu, 13 Apr 2017 01:15:33 -0400
Received: from eggs.gnu.org ([208.118.235.92]:42777)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <seth.fowler@HIDDEN>) id 1cyWlu-0001sP-DS
 for submit <at> debbugs.gnu.org; Thu, 13 Apr 2017 00:54:18 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <seth.fowler@HIDDEN>) id 1cyWlo-00061p-73
 for submit <at> debbugs.gnu.org; Thu, 13 Apr 2017 00:54:13 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:45573)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <seth.fowler@HIDDEN>) id 1cyWlo-00061k-3U
 for submit <at> debbugs.gnu.org; Thu, 13 Apr 2017 00:54:12 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:33746)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <seth.fowler@HIDDEN>) id 1cyWlm-0008Db-OA
 for bug-automake@HIDDEN; Thu, 13 Apr 2017 00:54:11 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <seth.fowler@HIDDEN>) id 1cyWlj-0005zT-LN
 for bug-automake@HIDDEN; Thu, 13 Apr 2017 00:54:10 -0400
Received: from pv33p36im-asmtp002.me.com ([17.142.213.63]:53545)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <seth.fowler@HIDDEN>) id 1cyWlj-0005xq-DJ
 for bug-automake@HIDDEN; Thu, 13 Apr 2017 00:54:07 -0400
Received: from process-dkim-sign-daemon.pv33p36im-asmtp002.me.com by
 pv33p36im-asmtp002.me.com
 (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016))
 id <0OOB00L00ZY2NJ00@HIDDEN> for bug-automake@HIDDEN; Thu,
 13 Apr 2017 04:54:01 +0000 (GMT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=me.com; s=4d515a;
 t=1492059241; bh=A16rH0FtWmYVC4Y83iVuJNinHBhTnzHVNmYux3oW68I=;
 h=From:Content-type:MIME-version:Subject:Message-id:Date:To;
 b=Pb5s42PJAb/3N1DoKdsMS94E6zXnpajb1BEFuGOfAx2cZhFAvFhQGZUMk2Viyr+fN
 emj/O1NsXX9SR0dbaIpR+65Oz8HnQIdGNrYEZnbAIfH7lWO5Lfxt+Zt+0sYd8kzBE3
 u4oXJi4xzRAq4krH+3kpdGLRlvYqpIR6BuHkVZnlKr6gwWOavY/ruMpvn9wHjGDGP7
 p8RSpxFOkDn8o9pdc5+O33qhe2bg7zO4Q+0bR+DPbZMi/fU3l/l8drvMOKjBZZ12hI
 fRgb6uGMS7SAHKtJvRx1ZAei9uX9y1UmEquC8AvpbEMDMu2A2YNI+0akTMAHkHKfA3
 11c7HQGnpfaew==
Received: from icloud.com ([127.0.0.1]) by pv33p36im-asmtp002.me.com
 (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016))
 with ESMTPSA id <0OOC00EIG0A0IT40@HIDDEN> for
 bug-automake@HIDDEN; Thu, 13 Apr 2017 04:54:01 +0000 (GMT)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,,
 definitions=2017-04-13_04:,, signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0
 clxscore=1034 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.0.1-1701120000 definitions=main-1704130038
From: Seth Fowler <seth.fowler@HIDDEN>
Content-type: text/plain; charset=utf-8
Content-transfer-encoding: quoted-printable
MIME-version: 1.0 (Mac OS X Mail 10.2 \(3259\))
Subject: Automake 1.15 generates a recheck target that depends on all,
 breaking parallel builds
Message-id: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN>
Date: Wed, 12 Apr 2017 21:54:00 -0700
To: bug-automake@HIDDEN
X-Mailer: Apple Mail (2.3259)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.0 (----)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Thu, 13 Apr 2017 01:15:31 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.0 (----)

Hi folks,

I recently ran into what appears to be a bug in automake and I thought =
it would be a good idea to let you know.

We had noticed that running =E2=80=9Cmake recheck -j8=E2=80=9D if some =
source files were dirty would cause random build failures. The symptom =
was that the same file was getting built more than once by different =
make processes, which led to the resulting objects being corrupted. =
We=E2=80=99d see messages like this:

    libtool:   error: =E2=80=98foo.lo=E2=80=99 is not a valid libtool =
object

I dug into this a little and the root cause of the problem seems to be =
that unlike the other top-level targets generated by automake (check, =
install, etc.), recheck depends on =E2=80=9Call=E2=80=9D instead of =
=E2=80=9Call-am=E2=80=9D. =E2=80=9Call=E2=80=9D doesn=E2=80=99t declare =
very many dependencies - it just launches a new copy of make that builds =
=E2=80=9Call-am=E2=80=9D. Because recheck also depends on other targets =
(e.g. everything in check_PROGRAMS) which might depend on some of the =
same things as =E2=80=9Call-am=E2=80=9D, in a parallel build the =
original make process and the make process spawned by =E2=80=9Call=E2=80=9D=
 can end up attempting to build the same targets.

I fixed this for our project by copying the generated recheck rule into =
our Makefile.am and replacing =E2=80=9Call=E2=80=9D with =E2=80=9Call-am=E2=
=80=9D. After doing this, I could no longer reproduce the problem with =
=E2=80=9Cmake recheck -j8=E2=80=9D.

It seems like that change should be made in automake itself. Or maybe I =
just missed something - I=E2=80=99m far from an automake guru. =3D)

Thanks,
- Seth=




Acknowledgement sent to Seth Fowler <seth.fowler@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-automake@HIDDEN. Full text available.
Report forwarded to bug-automake@HIDDEN:
bug#26471; Package automake. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 13 Apr 2017 14:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.