GNU logs - #26471, boring messages

Message sent to bug-automake@HIDDEN:

Subject: bug#26471: Automake 1.15 generates a recheck target that depends on all, breaking parallel builds
Resent-From: Seth Fowler <seth.fowler@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-automake@HIDDEN
Resent-Date: Thu, 13 Apr 2017 05:16:02 +0000
Resent-Message-ID: <handler.26471.B.14920605339611 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 26471 <at> debbugs.gnu.org
From: Seth Fowler <seth.fowler@HIDDEN>
Content-type: text/plain; charset=utf-8
Content-transfer-encoding: quoted-printable
MIME-version: 1.0 (Mac OS X Mail 10.2 \(3259\))
Message-id: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN>
Date: Wed, 12 Apr 2017 21:54:00 -0700
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hi folks,

I recently ran into what appears to be a bug in automake and I thought =
it would be a good idea to let you know.

We had noticed that running =E2=80=9Cmake recheck -j8=E2=80=9D if some =
source files were dirty would cause random build failures. The symptom =
was that the same file was getting built more than once by different =
make processes, which led to the resulting objects being corrupted. =
We=E2=80=99d see messages like this:

    libtool:   error: =E2=80=98foo.lo=E2=80=99 is not a valid libtool =
object

I dug into this a little and the root cause of the problem seems to be =
that unlike the other top-level targets generated by automake (check, =
install, etc.), recheck depends on =E2=80=9Call=E2=80=9D instead of =
=E2=80=9Call-am=E2=80=9D. =E2=80=9Call=E2=80=9D doesn=E2=80=99t declare =
very many dependencies - it just launches a new copy of make that builds =
=E2=80=9Call-am=E2=80=9D. Because recheck also depends on other targets =
(e.g. everything in check_PROGRAMS) which might depend on some of the =
same things as =E2=80=9Call-am=E2=80=9D, in a parallel build the =
original make process and the make process spawned by =E2=80=9Call=E2=80=9D=
 can end up attempting to build the same targets.

I fixed this for our project by copying the generated recheck rule into =
our Makefile.am and replacing =E2=80=9Call=E2=80=9D with =E2=80=9Call-am=E2=
=80=9D. After doing this, I could no longer reproduce the problem with =
=E2=80=9Cmake recheck -j8=E2=80=9D.

It seems like that change should be made in automake itself. Or maybe I =
just missed something - I=E2=80=99m far from an automake guru. =3D)

Thanks,
- Seth=

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Seth Fowler <seth.fowler@HIDDEN>
Subject: bug#26471: Acknowledgement (Automake 1.15 generates a recheck
 target that depends on all, breaking parallel builds)
Message-ID: <handler.26471.B.14920605339611.ack <at> debbugs.gnu.org>
References: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN>
X-Gnu-PR-Message: ack 26471
X-Gnu-PR-Package: automake
Reply-To: 26471 <at> debbugs.gnu.org
Date: Thu, 13 Apr 2017 05:16:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-automake@HIDDEN

If you wish to submit further information on this problem, please
send it to 26471 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
26471: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D26471
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-automake@HIDDEN:

Subject: bug#26471: Automake 1.15 generates a recheck target that depends on all, breaking parallel builds
Resent-From: Mathieu Lirzin <mthl@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-automake@HIDDEN
Resent-Date: Thu, 13 Apr 2017 13:53:01 +0000
Resent-Message-ID: <handler.26471.B26471.149209157513941 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Seth Fowler <seth.fowler@HIDDEN>
Cc: 26471 <at> debbugs.gnu.org
From: Mathieu Lirzin <mthl@HIDDEN>
References: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN>
Date: Thu, 13 Apr 2017 15:52:41 +0200
In-Reply-To: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN> (Seth Fowler's
 message of "Wed, 12 Apr 2017 21:54:00 -0700")
Message-ID: <87wpaojtxy.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hi Seth,

Seth Fowler <seth.fowler@HIDDEN> writes:

> I recently ran into what appears to be a bug in automake and I thought it=
 would be a good idea to let you know.
>
> We had noticed that running =E2=80=9Cmake recheck -j8=E2=80=9D if some so=
urce files
> were dirty would cause random build failures. The symptom was that the
> same file was getting built more than once by different make
> processes, which led to the resulting objects being corrupted. We=E2=80=
=99d
> see messages like this:
>
>     libtool:   error: =E2=80=98foo.lo=E2=80=99 is not a valid libtool obj=
ect
>
> I dug into this a little and the root cause of the problem seems to be
> that unlike the other top-level targets generated by automake (check,
> install, etc.), recheck depends on =E2=80=9Call=E2=80=9D instead of =E2=
=80=9Call-am=E2=80=9D. =E2=80=9Call=E2=80=9D
> doesn=E2=80=99t declare very many dependencies - it just launches a new c=
opy
> of make that builds =E2=80=9Call-am=E2=80=9D. Because recheck also depend=
s on other
> targets (e.g. everything in check_PROGRAMS) which might depend on some
> of the same things as =E2=80=9Call-am=E2=80=9D, in a parallel build the o=
riginal make
> process and the make process spawned by =E2=80=9Call=E2=80=9D can end up =
attempting to
> build the same targets.
>
> I fixed this for our project by copying the generated recheck rule
> into our Makefile.am and replacing =E2=80=9Call=E2=80=9D with =E2=80=9Cal=
l-am=E2=80=9D. After doing
> this, I could no longer reproduce the problem with =E2=80=9Cmake recheck =
-j8=E2=80=9D.
>
> It seems like that change should be made in automake itself. Or maybe I j=
ust missed something - I=E2=80=99m far from an automake guru. =3D)

Seems like a bug.

In order to properly fix this in Automake, we need to have a test
that allow us to reproduce the issue.  Ideally you would provide
this test yourself by taking inspiration from:

  https://git.savannah.gnu.org/cgit/automake.git/tree/t/libtool2.sh?h=3Dmic=
ro

If you are not comfortable with doing that, a tarball or a link to a
repository containing a minimal automake project that allow us to
reproduce the bug manually is good enough.

Thanks for the report.

--=20
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 13 Apr 2017 13:54:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 13 09:54:59 2017
Received: from localhost ([127.0.0.1]:45278 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cyfD9-0003fq-IH
	for submit <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:54:59 -0400
Received: from eggs.gnu.org ([208.118.235.92]:57570)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mthl@HIDDEN>) id 1cyfD8-0003fb-IZ
 for control <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:54:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <mthl@HIDDEN>) id 1cyfCy-00038i-MF
 for control <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:54:53 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60601)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <mthl@HIDDEN>)
 id 1cyfCy-00038e-J5
 for control <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:54:48 -0400
Received: from mek33-4-82-236-46-88.fbx.proxad.net ([82.236.46.88]:46760
 helo=godel)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <mthl@HIDDEN>) id 1cyfCx-0008JP-W2
 for control <at> debbugs.gnu.org; Thu, 13 Apr 2017 09:54:48 -0400
Date: Thu, 13 Apr 2017 15:54:46 +0200
Message-Id: <87vaq8jtuh.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: Mathieu Lirzin <mthl@HIDDEN>
Subject: control message for bug #26471
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

tags 26471 moreinfo

Message sent to bug-automake@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#26471: Automake 1.15 generates a recheck target that depends on all, breaking parallel builds
Resent-From: Mathieu Lirzin <mthl@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-automake@HIDDEN
Resent-Date: Fri, 19 Jan 2018 00:38:02 +0000
Resent-Message-ID: <handler.26471.B26471.15163222323784 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 26471
X-GNU-PR-Package: automake
X-GNU-PR-Keywords: moreinfo
To: Seth Fowler <seth.fowler@HIDDEN>
Cc: 26471 <at> debbugs.gnu.org
Received: via spool by 26471-submit <at> debbugs.gnu.org id=B26471.15163222323784
          (code B ref 26471); Fri, 19 Jan 2018 00:38:02 +0000
Received: (at 26471) by debbugs.gnu.org; 19 Jan 2018 00:37:12 +0000
Received: from localhost ([127.0.0.1]:33869 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ecKgC-0000yy-3m
	for submit <at> debbugs.gnu.org; Thu, 18 Jan 2018 19:37:12 -0500
Received: from eggs.gnu.org ([208.118.235.92]:53109)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mthl@HIDDEN>) id 1ecKg9-0000yh-OF
 for 26471 <at> debbugs.gnu.org; Thu, 18 Jan 2018 19:37:10 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <mthl@HIDDEN>) id 1ecKg3-00077l-Ls
 for 26471 <at> debbugs.gnu.org; Thu, 18 Jan 2018 19:37:04 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37851)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <mthl@HIDDEN>)
 id 1ecKg3-00077f-I7; Thu, 18 Jan 2018 19:37:03 -0500
Received: from [2a01:e35:2ec2:e580:7d5f:f616:fc6f:3970] (port=55768 helo=godel)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <mthl@HIDDEN>)
 id 1ecKg3-0006O8-2n; Thu, 18 Jan 2018 19:37:03 -0500
From: Mathieu Lirzin <mthl@HIDDEN>
References: <7CB98401-195E-4DF1-AAEF-444E19C60586@HIDDEN>
 <87wpaojtxy.fsf@HIDDEN>
Date: Fri, 19 Jan 2018 01:37:00 +0100
In-Reply-To: <87wpaojtxy.fsf@HIDDEN> (Mathieu Lirzin's message of "Thu, 13
 Apr 2017 15:52:41 +0200")
Message-ID: <878tcu65w3.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Hi Seth,

Mathieu Lirzin <mthl@HIDDEN> writes:

> Seth Fowler <seth.fowler@HIDDEN> writes:
>
>> I recently ran into what appears to be a bug in automake and I thought i=
t would be a good idea to let you know.
>>
>> We had noticed that running =E2=80=9Cmake recheck -j8=E2=80=9D if some s=
ource files
>> were dirty would cause random build failures. The symptom was that the
>> same file was getting built more than once by different make
>> processes, which led to the resulting objects being corrupted. We=E2=80=
=99d
>> see messages like this:
>>
>>     libtool:   error: =E2=80=98foo.lo=E2=80=99 is not a valid libtool ob=
ject
>>
>> I dug into this a little and the root cause of the problem seems to be
>> that unlike the other top-level targets generated by automake (check,
>> install, etc.), recheck depends on =E2=80=9Call=E2=80=9D instead of =E2=
=80=9Call-am=E2=80=9D. =E2=80=9Call=E2=80=9D
>> doesn=E2=80=99t declare very many dependencies - it just launches a new =
copy
>> of make that builds =E2=80=9Call-am=E2=80=9D. Because recheck also depen=
ds on other
>> targets (e.g. everything in check_PROGRAMS) which might depend on some
>> of the same things as =E2=80=9Call-am=E2=80=9D, in a parallel build the =
original make
>> process and the make process spawned by =E2=80=9Call=E2=80=9D can end up=
 attempting to
>> build the same targets.
>>
>> I fixed this for our project by copying the generated recheck rule
>> into our Makefile.am and replacing =E2=80=9Call=E2=80=9D with =E2=80=9Ca=
ll-am=E2=80=9D. After doing
>> this, I could no longer reproduce the problem with =E2=80=9Cmake recheck=
 -j8=E2=80=9D.
>>
>> It seems like that change should be made in automake itself. Or maybe I =
just missed something - I=E2=80=99m far from an automake guru. =3D)
>
> Seems like a bug.

Any news?  Would be great if you could provide a way to reproduce this
issue.

Thanks.

--=20
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.