GNU bug report logs - #13771
GNU make specific stuff cannot be used in RHS of "TESTS ="

Message received at 13771 <at> debbugs.gnu.org:

Received: (at 13771) by debbugs.gnu.org; 8 Jun 2013 09:54:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 08 05:54:16 2013
Received: from localhost ([127.0.0.1]:57658 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UlFqi-0001rb-E3
	for submit <at> debbugs.gnu.org; Sat, 08 Jun 2013 05:54:16 -0400
Received: from mail-we0-f173.google.com ([74.125.82.173]:44345)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <stefano.lattarini@HIDDEN>) id 1UlFqf-0001r3-S8
	for 13771 <at> debbugs.gnu.org; Sat, 08 Jun 2013 05:54:14 -0400
Received: by mail-we0-f173.google.com with SMTP id x54so2805054wes.18
	for <13771 <at> debbugs.gnu.org>; Sat, 08 Jun 2013 02:54:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=message-id:date:from:mime-version:to:cc:subject:references
	:in-reply-to:content-type:content-transfer-encoding;
	bh=2j5B+YE6zK61sB3QNeXZCHdhEn49E2zoe03NwPeosNE=;
	b=lsGFqSzPGTkHzX+sPLE5Y1oWmPKty5BcQbU6H9XFzyeeCdBYyzi4LS6xXABUsuSmYp
	CfqBmtvXT/C2Ci1/GvCwresoxBiIkFJZkGQWPeBBIabx2VW6xykWtx53mt9qG4nVxS5I
	CHnRPN3/6vuEFxuJVAB6p8odj3o/YhAX2IVZckL02H7jt7a1v15yr+Hp8aiy7ds0N5R6
	DlQMlTmktDj4HcDKpOuer+oApVgaiZEuMx/y2g1mggW+2eijyQMtLD49rTISyzkgM2SQ
	2cwdOXdjLAXPnzmsgnTZ05Q3paAVKBZ+Fut/rRzanm7sj2ffp1cHkjujHaGtZPtr3RRU
	FFeg==
X-Received: by 10.180.198.175 with SMTP id jd15mr700505wic.28.1370685242925;
	Sat, 08 Jun 2013 02:54:02 -0700 (PDT)
Received: from [192.168.178.20]
	(host93-95-dynamic.6-79-r.retail.telecomitalia.it. [79.6.95.93])
	by mx.google.com with ESMTPSA id o14sm1529569wiv.3.2013.06.08.02.54.01
	for <multiple recipients>
	(version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
	Sat, 08 Jun 2013 02:54:02 -0700 (PDT)
Message-ID: <51B2FF38.7050801@HIDDEN>
Date: Sat, 08 Jun 2013 11:54:00 +0200
From: Stefano Lattarini <stefano.lattarini@HIDDEN>
MIME-Version: 1.0
To: Daiki Ueno <ueno@HIDDEN>
Subject: Re: bug#13771: make functions cannot be used in RHS of "TESTS ="
References: <m3r4kbcp3y.fsf-ueno-ueno@HIDDEN> <51360D58.7010102@HIDDEN>
In-Reply-To: <51360D58.7010102@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.1 (/)
X-Debbugs-Envelope-To: 13771
Cc: 13771 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.6 (--)

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13771#8>

Resurrecting an old thread to correct some wrong information
(given by myself, sorry about that).

On 03/05/2013 04:20 PM, Stefano Lattarini wrote:
>
> Hi Daiki, sorry for the delay.
> 
> On 02/20/2013 10:27 AM, Daiki Ueno wrote:
>> Hi,
>>
>> I noticed that some modules are using make functions to assign values to
>> TESTS.  For example, I found the following in my GNOME jhbuild checkout:
>>
>> TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
>> TESTS = $(sort $(tests))
>>
>> These cause warnings like:
>>
>> Makefile.am:2: warning: sort $(tests: non-POSIX variable name
>> Makefile.am:2: (probably a GNU make extension)
>>
>> and also generate the following rule in Makefile.in, which seems
>> apparently wrong:
>>
>> $(sort.log: $(sort
>>         @p='$(sort'; \
>>         b='$(sort'; \
>>         $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
>>         --log-file $$b.log --trs-file $$b.trs \
>>         $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
>>         "$$tst" $(AM_TESTS_FD_REDIRECT)
>>
>> Shall it be fixed in Makefile.am or Automake?
>>
> This is actually a long-standing (and probably not easily lifted) Automake
> limitation, due to the need to perform $(EXEEXT) appending to entries in
> the $(PROGRAMS) primary (and consequently in the $(TESTS) variable too,
> since that can contain programs).
>
Actually, this automake-time processing is required also to be able
to correctly handle extension-less tests; so ...

>  I won't object to a patch that tries to
> mitigate the issue, but given how old this limitation is, I won't attempt
> it myself anytime soon.
> 
> In the meantime, I see two solutions:
> 
>    * Don't use make functions in the TESTS definition; this might not be
>      trivial in general situations, but seems pretty easy in your setup
>      above -- instead of having:
> 
>          noinst_PROGRAMS = foo bar qux zap
>          TESTS = $(filter-out foo bar,$(noinst_PROGRAMS))
> 
>      use:
> 
>          test_progs = foo bar
>          noinst_PROGRAMS = $(test_progs) qux zap
>          TESTS = $(test_progs)
> 
>     And no need to $(sort) the TESTS -- since you are using the parallel
>     driver, they should be able to run in any order anyway.
> 
>   * If you only care for the package to build for UNIX (and not for Cygwin
>     or MinGW), you can do away with the $(EXEEXT) rewrite machinery by
>     using the 'no-exeext' Automake option.
> 
... this proposed work-around won't work :-(

Note that this has come up again recently in automake bug#14561.

There we gave a a possible and quite generic workaround: wrap any
GNU make specific stuff that you want to use in $(TESTS) into a
configure substitution, which will be left alone by automake; e.g.:

  # In configure.ac:
  AC_SUBST([auto_find_tests], ['$(sort $(wildcard *.test))'])

  # In Makefile.am:
  TESTS = @auto_find_tests@

But alas, in order to have this to work correctly, one should only
use tests whose suffix is listed in TESTS_EXTENSIONS (if that
variable is not explicitly defined, it's default value is ".test").
That's because, missing such extension, automake cannot generate
generic suffix rules to create the '.trs' and '.log' files the
testsuite harness relies upon.

So even this workaround wouldn't work in the particular use case
reported by Daiki :-(  Sorry.

Best regards,
  Stefano

Message received at 13771 <at> debbugs.gnu.org:

Received: (at 13771) by debbugs.gnu.org; 5 Mar 2013 15:21:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 05 10:21:38 2013
Received: from localhost ([127.0.0.1]:60801 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UCtgO-00021v-SA
	for submit <at> debbugs.gnu.org; Tue, 05 Mar 2013 10:21:38 -0500
Received: from mail-bk0-f51.google.com ([209.85.214.51]:36233)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <stefano.lattarini@HIDDEN>)
	id 1UCtgF-00021T-2D; Tue, 05 Mar 2013 10:21:34 -0500
Received: by mail-bk0-f51.google.com with SMTP id ik5so75393bkc.38
	for <multiple recipients>; Tue, 05 Mar 2013 07:21:00 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=x-received:message-id:date:from:mime-version:to:cc:subject
	:references:in-reply-to:content-type:content-transfer-encoding;
	bh=Iq5xMXVrKfxu1akXe+0pM50J9DY3/cu32hpNRB20Llc=;
	b=xBfJ1gqn4c73kTLQxRRSmM7EK0WrL59wUacnJXeFKkA7A7TKnWmFyt7FQ1+Qi4g03o
	ljbdkXgfiASzDJv0pAMytg/cE2uY2JnbpjcfXPnMFcxO5XP/zpSfVH8+VFDyKYKYZD6o
	0F0lsBdweyeh/keK8JZh83Fr0XIvGi6QlWq5zZb47BYKvXR30Y25wbrcFtgBY8n0j3L3
	mUTqJkcwNcVbjNuw04YL8C4nFUXgvpUaBqBNQuHablQK0KhiQLTpUAXoofjXrNi/5aD1
	lYnQONVEskEvYq+r3g275/X6SJeZ+ZLeFH4zhBUDylFKqFbdxqpncMOab2OIMEVQM1N6
	s5SQ==
X-Received: by 10.204.150.82 with SMTP id x18mr9498282bkv.109.1362496860311;
	Tue, 05 Mar 2013 07:21:00 -0800 (PST)
Received: from [192.168.178.20]
	(host137-94-dynamic.4-87-r.retail.telecomitalia.it. [87.4.94.137])
	by mx.google.com with ESMTPS id o2sm7239339bkv.3.2013.03.05.07.20.58
	(version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
	Tue, 05 Mar 2013 07:20:59 -0800 (PST)
Message-ID: <51360D58.7010102@HIDDEN>
Date: Tue, 05 Mar 2013 16:20:56 +0100
From: Stefano Lattarini <stefano.lattarini@HIDDEN>
MIME-Version: 1.0
To: Daiki Ueno <ueno@HIDDEN>
Subject: Re: bug#13771: make functions cannot be used in RHS of "TESTS ="
References: <m3r4kbcp3y.fsf-ueno-ueno@HIDDEN>
In-Reply-To: <m3r4kbcp3y.fsf-ueno-ueno@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.1 (/)
X-Debbugs-Envelope-To: 13771
Cc: 13771 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.6 (--)

severity 13771 wishlist
stop

Hi Daiki, sorry for the delay.

On 02/20/2013 10:27 AM, Daiki Ueno wrote:
> Hi,
> 
> I noticed that some modules are using make functions to assign values to
> TESTS.  For example, I found the following in my GNOME jhbuild checkout:
> 
> TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
> TESTS = $(sort $(tests))
> 
> These cause warnings like:
> 
> Makefile.am:2: warning: sort $(tests: non-POSIX variable name
> Makefile.am:2: (probably a GNU make extension)
> 
> and also generate the following rule in Makefile.in, which seems
> apparently wrong:
> 
> $(sort.log: $(sort
>         @p='$(sort'; \
>         b='$(sort'; \
>         $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
>         --log-file $$b.log --trs-file $$b.trs \
>         $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
>         "$$tst" $(AM_TESTS_FD_REDIRECT)
> 
> Shall it be fixed in Makefile.am or Automake?
>
This is actually a long-standing (and probably not easily lifted) Automake
limitation, due to the need to perform $(EXEEXT) appending to entries in
the $(PROGRAMS) primary (and consequently in the $(TESTS) variable too,
since that can contain programs).  I won't object to a patch that tries to
mitigate the issue, but given how old this limitation is, I won't attempt
it myself anytime soon.

In the meantime, I see two solutions:

   * Don't use make functions in the TESTS definition; this might not be
     trivial in general situations, but seems pretty easy in your setup
     above -- instead of having:

         noinst_PROGRAMS = foo bar qux zap
         TESTS = $(filter-out foo bar,$(noinst_PROGRAMS))

     use:

         test_progs = foo bar
         noinst_PROGRAMS = $(test_progs) qux zap
         TESTS = $(test_progs)

    And no need to $(sort) the TESTS -- since you are using the parallel
    driver, they should be able to run in any order anyway.

  * If you only care for the package to build for UNIX (and not for Cygwin
    or MinGW), you can do away with the $(EXEEXT) rewrite machinery by
    using the 'no-exeext' Automake option.

HTH,
  Stefano

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 20 Feb 2013 09:28:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 20 04:28:45 2013
Received: from localhost ([127.0.0.1]:38609 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1U85ym-0000dh-Bc
	for submit <at> debbugs.gnu.org; Wed, 20 Feb 2013 04:28:45 -0500
Received: from eggs.gnu.org ([208.118.235.92]:35437)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <ueno@HIDDEN>) id 1U85yk-0000da-77
	for submit <at> debbugs.gnu.org; Wed, 20 Feb 2013 04:28:42 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ueno@HIDDEN>) id 1U85xZ-0000rp-L4
	for submit <at> debbugs.gnu.org; Wed, 20 Feb 2013 04:27:32 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
	autolearn=unavailable version=3.3.2
Received: from lists.gnu.org ([208.118.235.17]:37978)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ueno@HIDDEN>)
	id 1U85xZ-0000rj-Hs
	for submit <at> debbugs.gnu.org; Wed, 20 Feb 2013 04:27:29 -0500
Received: from eggs.gnu.org ([208.118.235.92]:46788)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ueno@HIDDEN>) id 1U85xW-0007zQ-VO
	for bug-automake@HIDDEN; Wed, 20 Feb 2013 04:27:29 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ueno@HIDDEN>) id 1U85xN-0000pU-NJ
	for bug-automake@HIDDEN; Wed, 20 Feb 2013 04:27:26 -0500
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48280)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ueno@HIDDEN>)
	id 1U85xN-0000pQ-KU
	for bug-automake@HIDDEN; Wed, 20 Feb 2013 04:27:17 -0500
Received: from [2001:e41:db5e:fb14::1] (port=43647 helo=localhost.localdomain)
	by fencepost.gnu.org with esmtpsa
	(TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71)
	(envelope-from <ueno@HIDDEN>) id 1U85xN-0002vM-3B
	for bug-automake@HIDDEN; Wed, 20 Feb 2013 04:27:17 -0500
Message-ID: <m3r4kbcp3y.fsf-ueno-ueno@HIDDEN>
From: Daiki Ueno <ueno@HIDDEN>
To: bug-automake@HIDDEN
Subject: make functions cannot be used in RHS of "TESTS ="
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux)
Date: Wed, 20 Feb 2013 18:27:13 +0900
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
	(bad octet value).
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 208.118.235.17
X-Spam-Score: -4.8 (----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -7.5 (-------)

Hi,

I noticed that some modules are using make functions to assign values to
TESTS.  For example, I found the following in my GNOME jhbuild checkout:

TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
TESTS = $(sort $(tests))

These cause warnings like:

Makefile.am:2: warning: sort $(tests: non-POSIX variable name
Makefile.am:2: (probably a GNU make extension)

and also generate the following rule in Makefile.in, which seems
apparently wrong:

$(sort.log: $(sort
        @p='$(sort'; \
        b='$(sort'; \
        $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
        --log-file $$b.log --trs-file $$b.trs \
        $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
        "$$tst" $(AM_TESTS_FD_REDIRECT)

Shall it be fixed in Makefile.am or Automake?

Regards,
-- 
Daiki Ueno