Received: (at 68860) by debbugs.gnu.org; 1 Feb 2024 22:25:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 01 17:25:35 2024
Received: from localhost ([127.0.0.1]:43299 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rVfV0-0005TR-Pe
for submit <at> debbugs.gnu.org; Thu, 01 Feb 2024 17:25:35 -0500
Received: from frenzy.freefriends.org ([198.99.81.75]:45494
helo=freefriends.org) by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <karl@HIDDEN>) id 1rVfUx-0005TI-Qt
for 68860 <at> debbugs.gnu.org; Thu, 01 Feb 2024 17:25:33 -0500
X-Envelope-From: karl@HIDDEN
Received: from freefriends.org (localhost [127.0.0.1])
by freefriends.org (8.14.7/8.14.7) with ESMTP id 411MPK3J015010
(version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Thu, 1 Feb 2024 15:25:20 -0700
Received: (from apache@localhost)
by freefriends.org (8.14.7/8.14.7/Submit) id 411MPKig015007;
Thu, 1 Feb 2024 15:25:20 -0700
Date: Thu, 1 Feb 2024 15:25:20 -0700
Message-Id: <202402012225.411MPKig015007@HIDDEN>
From: Karl Berry <karl@HIDDEN>
To: trojkan@HIDDEN
Subject: Re: bug#68860: race condition with make recheck
In-Reply-To: <f40fd580-f3ed-4bca-86ec-f3ea5b125d1f@HIDDEN>
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 68860
Cc: 68860 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi Peter,
The problem seems to be that both $(TESTS) and check_LIBRARIES depend on
libfoo.a and trigger compilation of foo.cc.
Thanks much for the report and analysis. What you wrote looks sensible
to me.
My understanding of parallel make is a bit hazy,
Me too :(. If anyone else here has a chance to look into this, that
would be great.
One idea would to change the rule for recheck to
It looks plausible. Another possibility that comes to mind is to make
the recheck target more parallel to all, i.e., with a recheck-am
target. I', not sure.
Please find attached a trimmed down example of the problem.
Thanks again. Will ponder. --best, karl.
bug-automake@HIDDEN:bug#68860; Package automake.
Full text available.Received: (at submit) by debbugs.gnu.org; 1 Feb 2024 01:12:04 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 31 20:12:04 2024 Received: from localhost ([127.0.0.1]:39354 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rVLcZ-00051f-Rz for submit <at> debbugs.gnu.org; Wed, 31 Jan 2024 20:12:04 -0500 Received: from lists.gnu.org ([2001:470:142::17]:49826) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <trojkan@HIDDEN>) id 1rVLcX-00051B-4U for submit <at> debbugs.gnu.org; Wed, 31 Jan 2024 20:12:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <trojkan@HIDDEN>) id 1rVLcH-0005CG-VR for bug-automake@HIDDEN; Wed, 31 Jan 2024 20:11:45 -0500 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <trojkan@HIDDEN>) id 1rVLcG-0006gx-9H for bug-automake@HIDDEN; Wed, 31 Jan 2024 20:11:45 -0500 Received: by mail-ot1-x333.google.com with SMTP id 46e09a7af769-6e12d0af927so343400a34.0 for <bug-automake@HIDDEN>; Wed, 31 Jan 2024 17:11:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706749902; x=1707354702; darn=gnu.org; h=to:subject:from:content-language:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=kf58ojmFZ79+ra5mr+gbElDrHTtxajO/0zGA/e4b3bk=; b=heq8/AqQFlHyCnJKA2OdTVFzhUXnK3mR4UIEnQPc5fggIcIcKXO5vE+djLacY2Fkoe o/jaCfFvtA2hlCFomYYay8PzeRZMiwfLSTVXqTQ8qHHT1KkP6ew+hYLlZMIThtkGzLVN +J1VsKrVGLYL46uQb8VnY9Y1dDjJNsRmKaZyVMAOyr3DUF03cKQN5ou7aGhc8kMMYMhZ vke4Up5C1HWgVKStABuP05ATWbWxkSIMBTfHFDxeNPvR1etw1tiotslgqwZjeXUvqsQR V6jbn8/r7hG+N8YgLOvJqXe79is+NZKUpfjcay5KCI25j2JLn93E+7hzIGqkLpoOJQz7 HYSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706749902; x=1707354702; h=to:subject:from:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kf58ojmFZ79+ra5mr+gbElDrHTtxajO/0zGA/e4b3bk=; b=S6vJ/I0OPPFauFCiL0fPZJOQUOq/4aA8Czq1TaIUannlm5kquiinYD3UYw7q6gKALj 5ubXz6bthNNdomtaBQnIhHBhilOZCz9UHkrhCyMZfd3YLtyXqNZtnuQFF1uhG/NMsu3m sOXALM8bHYdgN7p1hkrMYFFQHSK2/RLYJQsBI1I31HXtjQPXB1zZXzhS7w9iaqWg24jv hb0RF2ow48C6A1LPjq2U494ZZ+GJQjj/OxiLzsxtAzkiY4kanncbJD+/3gHM27NC6CIp AF2l3Q4gCvVBLrniQyiyecGGJfoe4Vfgt/LIUtRGdmh/2MNySZ2VrYD1FeBLMU8cQ103 cKwg== X-Gm-Message-State: AOJu0YztCuR9W3zApTlaPBVqRQRw8+lpw0HTLLvrsS36o2098IxjWvpY PQjicA3Ls/UxKiCbs7q47vgD4T6ReXP8vfC8BVvkNSUieXhKTyxwWuNuYFnc X-Google-Smtp-Source: AGHT+IHpA4tKCCfYELafclM/NygRxBGuNXhBnSY3WY+knN7V2S+4LNSu8WPoU2hYZ3Z892sGZ5eHPQ== X-Received: by 2002:a05:6830:448a:b0:6e0:d3be:39cb with SMTP id r10-20020a056830448a00b006e0d3be39cbmr4031830otv.24.1706749902524; Wed, 31 Jan 2024 17:11:42 -0800 (PST) Received: from [192.168.2.71] (110-174-219-126.static.tpgi.com.au. [110.174.219.126]) by smtp.gmail.com with ESMTPSA id r25-20020aa78b99000000b006dbda9a4e6bsm10529155pfd.44.2024.01.31.17.11.41 for <bug-automake@HIDDEN> (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 31 Jan 2024 17:11:42 -0800 (PST) Content-Type: multipart/mixed; boundary="------------5fGjAxjJN9VvfUqcnJVupBXE" Message-ID: <f40fd580-f3ed-4bca-86ec-f3ea5b125d1f@HIDDEN> Date: Thu, 1 Feb 2024 11:11:37 +1000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US From: Peter Johansson <trojkan@HIDDEN> Subject: race condition with make recheck To: bug-automake@HIDDEN Received-SPF: pass client-ip=2607:f8b0:4864:20::333; envelope-from=trojkan@HIDDEN; helo=mail-ot1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.0 (/) This is a multi-part message in MIME format. --------------5fGjAxjJN9VvfUqcnJVupBXE Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi automakers, I think I've found a race condition with 'make recheck' that results in a source file being compiled twice in parallel and resulting in a failure such as mv: cannot stat '.deps/foo.Tpo': No such file or directory In my trimmed down example my Makefile.am looks like: lib_LIBRARIES = libfoo.a libfoo_a_SOURCES = foo.cc check_LIBRARIES = libtest.a libtest_a_SOURCES = test.cc TESTS = one.test two.test TEST_EXTENSIONS = .test AM_DEFAULT_SOURCE_EXT = .cc EXTRA_PROGRAMS = $(TESTS) libtest_a_LIBADD = libfoo.a LDADD = libtest.a libfoo.a The problem seems to be that both $(TESTS) and check_LIBRARIES depend on libfoo.a and trigger compilation of foo.cc. I haven't managed to get the same problem with 'make check', so I thought comparing the generated rules for check: and recheck: would be useful. recheck: all $(check_LIBRARIES) <long rule running failed TESTS> all: config.h $(MAKE) $(AM_MAKEFLAGS) all-am ... check-am: all-am $(MAKE) $(AM_MAKEFLAGS) $(check_LIBRARIES) $(MAKE) $(AM_MAKEFLAGS) check-TESTS check: check-am I can see how the "check-am: all-am" works as firewall against the race condition. OTOH, in the rule for recheck, 'all' triggers a sub-process that will build libfoo.a and in the meantime the main process will build $(check_LIBRARIES) which trigger the building of libfoo.a. My understanding of parallel make is a bit hazy, but I guess the main process and sub-process are only talking wrt how many workers they employ and are not talking about which rules to work on. I suppose this is not by design or that I'm doing something illegal by having check_LIBRARIES depend stuff that is built within 'make all'. I'm not sure what the best way to fix this would be. One idea would to change the rule for recheck to recheck: all $(MAKE) $(AM_MAKEFLAGS) $(check_LIBRARIES) <long rule running failed TESTS> but personally I don't fancy these sub-processes because it feels like they are the core of the problem for these sort of race conditions. I have tested with automake 1.16.5 (ubuntu) and 1.16i. Please find attached a trimmed down example of the problem. Best Regards, Peter --------------5fGjAxjJN9VvfUqcnJVupBXE Content-Type: application/x-shellscript; name="automake.sh" Content-Disposition: attachment; filename="automake.sh" Content-Transfer-Encoding: base64 IyEvYmluL3NoCgpzZXQgLWUKc2V0IC14Cgpta2RpciAtcCBhdXRvbWFrZQpjZCBhdXRvbWFr ZQoKY2F0ID4gY29uZmlndXJlLmFjPDxFT0YKQUNfSU5JVChbZm9vXSwgWzJdKQpBQ19DT05G SUdfU1JDRElSKFtjb25maWcuaC5pbl0pCkFDX0NPTkZJR19IRUFERVJTKFtjb25maWcuaF0p CkFNX0lOSVRfQVVUT01BS0UoW2ZvcmVpZ24gc2lsZW50LXJ1bGVzIHBhcmFsbGVsLXRlc3Rz XSkKQUNfTEFORyhbQysrXSkKQUNfUFJPR19DWFgKQUNfUFJPR19DWFhDUFAKQUNfUFJPR19S QU5MSUIKQUNfQ09ORklHX0ZJTEVTKFtNYWtlZmlsZV0pCkFDX09VVFBVVApFT0YKCmNhdCA+ IE1ha2VmaWxlLmFtIDw8RU9GCmxpYl9MSUJSQVJJRVMgPSBsaWJmb28uYQpsaWJmb29fYV9T T1VSQ0VTID0gZm9vLmNjCgpjaGVja19MSUJSQVJJRVMgPSBsaWJ0ZXN0LmEKbGlidGVzdF9h X1NPVVJDRVMgPSB0ZXN0LmNjCgpURVNUUyA9IG9uZS50ZXN0IHR3by50ZXN0ClRFU1RfRVhU RU5TSU9OUyA9IC50ZXN0CkFNX0RFRkFVTFRfU09VUkNFX0VYVCA9IC5jYwpFWFRSQV9QUk9H UkFNUyA9IFwkKFRFU1RTKQoKbGlidGVzdF9hX0xJQkFERCA9IGxpYmZvby5hCkxEQUREID0g bGlidGVzdC5hIGxpYmZvby5hCkVPRgoKY2F0ID4gZm9vLmggPDxFT0YKI2lmbmRlZiBteV9m b29faAojZGVmaW5lIG15X2Zvb19oCiNpbmNsdWRlIDxzdHJpbmc+CnN0ZDo6c3RyaW5nIGZv byh2b2lkKTsKI2VuZGlmCkVPRgoKY2F0ID4gZm9vLmNjIDw8RU9GCiNpbmNsdWRlIDxjb25m aWcuaD4KI2luY2x1ZGUgImZvby5oIgpzdGQ6OnN0cmluZyBmb28odm9pZCkgeyByZXR1cm4g ImZ1IjsgfQoKRU9GCgpjYXQgPiBvbmUuY2MgPDxFT0YKI2luY2x1ZGUgPGNvbmZpZy5oPgoj aW5jbHVkZSAiZm9vLmgiCiNpbmNsdWRlICJ0ZXN0LmgiCiNpbmNsdWRlIDxpb3N0cmVhbT4K I2luY2x1ZGUgPHN0cmluZz4KaW50IG1haW4oaW50IGFyZ2MsIGNoYXIqIGFyZ3ZbXSkKewoJ aW5pdChhcmdjLCBhcmd2KTsKCXN0ZDo6Y291dCA8PCAiSGVsbG8gV29ybGQhXG4iIDw8IGZv bygpIDw8ICJcbiI7CglyZXR1cm4gMDsKfQpFT0YKCmNhdCA+IHR3by5jYyA8PEVPRgojaW5j bHVkZSA8Y29uZmlnLmg+CiNpbmNsdWRlICJmb28uaCIKI2luY2x1ZGUgInRlc3QuaCIKI2lu Y2x1ZGUgPGlvc3RyZWFtPgojaW5jbHVkZSA8c3RyaW5nPgppbnQgbWFpbihpbnQgYXJnYywg Y2hhciogYXJndltdKQp7Cglpbml0KGFyZ2MsIGFyZ3YpOwoJc3RkOjpzdHJpbmcgc3RyID0g Zm9vKCk7CglpZiAoc3RyICE9ICJmb28iKSB7CgkgICBzdGQ6OmNlcnIgPDwgImVycm9yOiBm b28oKTogIiA8PCBzdHIgPDwgIlxuIjsKCSAgIHJldHVybiAxOwoJfQoJcmV0dXJuIDA7Cn0K RU9GCgpjYXQgPiB0ZXN0LmggPDxFT0YKI2lmbmRlZiBteV90ZXN0X2gKI2RlZmluZSBteV90 ZXN0X2gKdm9pZCBpbml0KGludCBhcmdjLCBjaGFyKiBhcmd2W10pOwojZW5kaWYKRU9GCgpj YXQgPiB0ZXN0LmNjIDw8RU9GCiNpbmNsdWRlIDxjb25maWcuaD4KI2luY2x1ZGUgInRlc3Qu aCIKI2luY2x1ZGUgImZvby5oIgojaW5jbHVkZSA8aW9zdHJlYW0+CnZvaWQgaW5pdChpbnQg YXJnYywgY2hhciogYXJndltdKQp7CglzdGQ6OmNvdXQgPDwgImluaXQgdGVzdCBmb3IgIiA8 PCBmb28oKSA8PCAiXG4iOwp9CkVPRgoKCmF1dG9yZWNvbmYgLWl2ZgouL2NvbmZpZ3VyZSAt LWVuYWJsZS1zaWxlbnQtcnVsZXMKbWFrZSBhbGwgLWo4Cm1ha2UgY2hlY2sgLWo4IHx8IGVj aG8gZmFpbGVkIGFzIGV4cGVjdGVkCgpzZWQgJ3MsImZ1IiwiZm9vIiwnIDwgZm9vLmNjID4g Zm9vLmNjLXQgJiYgbXYgZm9vLmNjLXQgZm9vLmNjCgptYWtlIHJlY2hlY2sgLWoxNgo= --------------5fGjAxjJN9VvfUqcnJVupBXE--
Peter Johansson <trojkan@HIDDEN>:bug-automake@HIDDEN.
Full text available.bug-automake@HIDDEN:bug#68860; Package automake.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.