GNU bug report logs - #12620
Parallel tests vs fast tests (and beyond)

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: automake; Severity: wishlist; Reported by: Reuben Thomas <rrt@HIDDEN>; dated Wed, 10 Oct 2012 19:55:01 UTC; Maintainer for automake is bug-automake@HIDDEN.
Severity set to 'wishlist' from 'normal' Request was from Stefano Lattarini <stefano.lattarini@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 12620 <at> debbugs.gnu.org:


Received: (at 12620) by debbugs.gnu.org; 26 Dec 2013 16:46:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Dec 26 11:46:59 2013
Received: from localhost ([127.0.0.1]:44130 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1VwE5L-00029h-Cs
	for submit <at> debbugs.gnu.org; Thu, 26 Dec 2013 11:46:59 -0500
Received: from mail-ee0-f42.google.com ([74.125.83.42]:65465)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <stefano.lattarini@HIDDEN>)
 id 1VwE5I-00029T-Py; Thu, 26 Dec 2013 11:46:57 -0500
Received: by mail-ee0-f42.google.com with SMTP id e53so3776623eek.29
 for <multiple recipients>; Thu, 26 Dec 2013 08:46:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=message-id:date:from:mime-version:to:cc:subject:content-type
 :content-transfer-encoding;
 bh=Do07e+vDgl+T5P9XNLvqI/8UIsuL9HjDGM1/Zz72PJ0=;
 b=os48ecGC4L31+DeoJlVC5HDVEVQlOflN+/5NHVpoHyNzZA+5YlJSvHxknDRFMta9xx
 SaeYFhqHCvBpRY2YFDYeC8gwB1dYcedjoi/EgKMvJ549J7FF4r74x7HSzLG0PlyEKLT1
 mWzOUgO6JLm++TOsA2qQ049lkrGUo6hNmKudipKVmnnnf+WkqqEUTWQhpMbqhkNI3ZnB
 tlizhG3iQ1xlLcvZ5k1UNx1rZnqlk5ByzlbTbDP2JbLZVR7JKe0hHpDJrTyY91vu9N0x
 a+aKbxT34Z+zclr+Doy7/akG/WbOhshQ991A6qKS9IAlNOebFgIk0QSWf1LxBlGnz9uY
 KIcg==
X-Received: by 10.15.45.135 with SMTP id b7mr8408407eew.88.1388076415906;
 Thu, 26 Dec 2013 08:46:55 -0800 (PST)
Received: from [192.168.0.101]
 (host143-4-dynamic.5-87-r.retail.telecomitalia.it. [87.5.4.143])
 by mx.google.com with ESMTPSA id b41sm74201656eef.16.2013.12.26.08.46.51
 for <multiple recipients>
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Thu, 26 Dec 2013 08:46:55 -0800 (PST)
Message-ID: <52BC5D78.6000006@HIDDEN>
Date: Thu, 26 Dec 2013 17:46:48 +0100
From: Stefano Lattarini <stefano.lattarini@HIDDEN>
MIME-Version: 1.0
To: 12620 <at> debbugs.gnu.org
Subject: Re: Parallel tests vs fast tests (and beyond)
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 12620
Cc: Reuben Thomas <rrt@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

severity 12620 wishlist
stop

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=12620>

Marking this as a "wishlist" rather than a bug, since the current
implementation of parallel tests is working correctly and is mostly
"good enough", and your proposal is thus geared towards improving an
already decent situation rather than fixing errors or misbehaviors
(this doesn't mean I'm denying or belittling the merits of your
proposal, of course).

Just one nit:

Reuben Thomas wrote:
>
> The implication of the 1.12 release notes is that serial-tests
> will be dropped at some point post-1.13.
>
I've been convinced that any idea I had in such a direction was
misguided; the old serial harness is now marked merely as
"discouraged", but there no plans to deprecate pr remove it in
the foreseeable future.

Thanks,
  Stefano




Information forwarded to bug-automake@HIDDEN:
bug#12620; Package automake. Full text available.

Message received at 12620 <at> debbugs.gnu.org:


Received: (at 12620) by debbugs.gnu.org; 11 Oct 2012 21:41:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 11 17:41:11 2012
Received: from localhost ([127.0.0.1]:39997 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1TMQVA-0005b0-F2
	for submit <at> debbugs.gnu.org; Thu, 11 Oct 2012 17:41:08 -0400
Received: from exprod7og116.obsmtp.com ([64.18.2.219]:47340)
	by debbugs.gnu.org with smtp (Exim 4.72)
	(envelope-from <rrt@HIDDEN>) id 1TMQV7-0005aq-GQ
	for 12620 <at> debbugs.gnu.org; Thu, 11 Oct 2012 17:41:08 -0400
Received: from mail-wg0-f72.google.com ([74.125.82.72]) (using TLSv1) by
	exprod7ob116.postini.com ([64.18.6.12]) with SMTP
	ID DSNKUHc8wVuIJ+IKgpjIBNDAHnCRWw1a82hq@HIDDEN;
	Thu, 11 Oct 2012 14:40:18 PDT
Received: by mail-wg0-f72.google.com with SMTP id dr12so1259045wgb.3
	for <12620 <at> debbugs.gnu.org>; Thu, 11 Oct 2012 14:40:16 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=google.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:x-gm-message-state;
	bh=id1NYDndggefKQr5olXXl9k51OzYmpgeSFVxpQ+fEfQ=;
	b=lGCkv/flgMuJgTqryiw79OUJoBeXi/Udv+1+YSQ8m+pB8+OpVjfWItHC4wZDEhbqiv
	2N4JMp+Ntyf3xf5G5565YtOK3GYAYK/pw0datJhpFB/EiOpqm9g1rAKk9xrbfiUJA+1L
	XyIC/rjMzX48hi4blWjZoxtZKtSRA43D2jMwfIreINUlzUZfKkxybnTqQGScPV61+5lR
	KR31eCFPZ4xZ9q7wvLc7vgRkngdoIPkuz15Mw8AVNIjw/qvyd52gtJL54uYdl1UZG6dJ
	tRlGaXTRVaqJ5qtthcDXRVk92tdm1WxUrx9ouUvJEJpPW1X46gUpPW/a/S/R9E0qRsJH
	MgeQ==
Received: by 10.14.4.198 with SMTP id 46mr4118321eej.11.1349991616362;
	Thu, 11 Oct 2012 14:40:16 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.14.4.198 with SMTP id 46mr4118315eej.11.1349991616277; Thu, 11
	Oct 2012 14:40:16 -0700 (PDT)
Received: by 10.14.133.201 with HTTP; Thu, 11 Oct 2012 14:40:16 -0700 (PDT)
In-Reply-To: <CAEnY7O-rikuU3_W_6HTGoy6Q_zedHsFhNXoX=_wFm45eu24AWA@HIDDEN>
References: <CAOnWdoix5PAoUrXPwyq8zeTjpgkpGEnz4VwWVxFUebhKk+pg9g@HIDDEN>
	<CAEnY7O-rikuU3_W_6HTGoy6Q_zedHsFhNXoX=_wFm45eu24AWA@HIDDEN>
Date: Thu, 11 Oct 2012 22:40:16 +0100
Message-ID: <CAOnWdoiX4mGp1Z+vppDJ=DswceGZWutH_HgjK5wM4g86Tt_Ncg@HIDDEN>
Subject: Re: bug#12620: Parallel tests vs fast tests (and beyond)
From: Reuben Thomas <rrt@HIDDEN>
To: Jack Kelly <jack@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1
X-Gm-Message-State: ALoCoQlNipcE+u7URF+XuOR48DPxslEobOgHqc2tnLHS1rIuXcU7sw65RQPiHCfPbukTIHFGWMf4JL+6ZfbcBl1Ni8vwF6mzKD2XVZLpn3dIhWmElP0PaKBZGP+rD54ms6sRXC4GlMgQlyfIhysg5/L5Wh23pkaBEGz2wUBU5v5zcneSyy4PHA0=
X-Spam-Score: -1.5 (-)
X-Debbugs-Envelope-To: 12620
Cc: 12620 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -1.5 (-)

On 11 October 2012 22:12, Jack Kelly <jack@HIDDEN> wrote:
>
> We had a discussion along these lines when refactoring elisp
> compilation: in the past it was all done in one big batch. Now it's
> done with an emacs invocation per .el file. The result of that
> discussion was that while you slow down single core machines, the new
> way of doing things fits make's model much better: instead of fiddling
> around with stamp files, the Makefile just has a suffix rule.

That's very interesting, of course that's the way make works naturally
for C programs, and I never thought of changing it (although I read
about how much faster for compiling several files it is, but it only
ever occurred to me that it would be nice to batch up compilations for
building from source, not for development where typically you may only
need to recompile one file).

-- 
http://rrt.sc3d.org




Information forwarded to bug-automake@HIDDEN:
bug#12620; Package automake. Full text available.

Message received at 12620 <at> debbugs.gnu.org:


Received: (at 12620) by debbugs.gnu.org; 11 Oct 2012 21:13:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 11 17:13:56 2012
Received: from localhost ([127.0.0.1]:39989 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1TMQ4q-0004zV-4n
	for submit <at> debbugs.gnu.org; Thu, 11 Oct 2012 17:13:56 -0400
Received: from mail-wi0-f174.google.com ([209.85.212.174]:47471)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <endgame.dos@HIDDEN>) id 1TMQ4l-0004zF-KO
	for 12620 <at> debbugs.gnu.org; Thu, 11 Oct 2012 17:13:54 -0400
Received: by mail-wi0-f174.google.com with SMTP id hq7so7669418wib.15
	for <12620 <at> debbugs.gnu.org>; Thu, 11 Oct 2012 14:12:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:sender:in-reply-to:references:date
	:x-google-sender-auth:message-id:subject:from:to:cc:content-type;
	bh=+mPkPwqLnnQbWZz+FLFpD2FIj7f4PgplmCp07ZuOH2g=;
	b=vOMpb4Q087rrDDRrsyiTDrlFlfwoHarT8xcJALb0XmYlncEaLWr3ObGXey0sbrSLV7
	H7/5bSNCTQWWnIwuarG/xrVstm3Dk0+UreFby2KzRdhqy3i0lvS4vDWhRe63LvNUlufO
	+0JYd9fYKG3maKMgi6pRLQEHY2bxYZHml+S62/1jyA1hK6/ZbSAVW937r+Iy4Xnt4VY3
	QmoOIN/w1W5DvrIikRM86NfsF8+rFmjasTXPTvobK6dr+8j1E4HEajleVWLa9mtbFmQS
	cVGaVg3JT40DnF0pNbLoaYJYs12j/s385rlgDOUakFUEHmuv35x6PMia6KIIctuHfck7
	6LTw==
MIME-Version: 1.0
Received: by 10.180.79.103 with SMTP id i7mr773621wix.13.1349989979434; Thu,
	11 Oct 2012 14:12:59 -0700 (PDT)
Received: by 10.194.61.116 with HTTP; Thu, 11 Oct 2012 14:12:59 -0700 (PDT)
In-Reply-To: <CAOnWdoix5PAoUrXPwyq8zeTjpgkpGEnz4VwWVxFUebhKk+pg9g@HIDDEN>
References: <CAOnWdoix5PAoUrXPwyq8zeTjpgkpGEnz4VwWVxFUebhKk+pg9g@HIDDEN>
Date: Fri, 12 Oct 2012 08:12:59 +1100
X-Google-Sender-Auth: CYSH8IE-dgQMJSDeRzskGrtyhb0
Message-ID: <CAEnY7O-rikuU3_W_6HTGoy6Q_zedHsFhNXoX=_wFm45eu24AWA@HIDDEN>
Subject: Re: bug#12620: Parallel tests vs fast tests (and beyond)
From: Jack Kelly <jack@HIDDEN>
To: Reuben Thomas <rrt@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1
X-Spam-Score: 0.1 (/)
X-Debbugs-Envelope-To: 12620
Cc: 12620 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: 0.1 (/)

On Thu, Oct 11, 2012 at 6:53 AM, Reuben Thomas <rrt@HIDDEN> wrote:
> This whole problem is an instance of a more general problem, of which
> another example is parallel make: for best performance, it should
> probably batch up calls to gcc, for example, so that multiple source
> files are compiled by each invocation.

We had a discussion along these lines when refactoring elisp
compilation: in the past it was all done in one big batch. Now it's
done with an emacs invocation per .el file. The result of that
discussion was that while you slow down single core machines, the new
way of doing things fits make's model much better: instead of fiddling
around with stamp files, the Makefile just has a suffix rule.

IMHO, your point is valid, but to do it safely would require a tool
smarter than make.

-- Jack




Information forwarded to bug-automake@HIDDEN:
bug#12620; Package automake. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 10 Oct 2012 19:54:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 10 15:54:44 2012
Received: from localhost ([127.0.0.1]:38078 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1TM2Md-0003Lw-Fo
	for submit <at> debbugs.gnu.org; Wed, 10 Oct 2012 15:54:43 -0400
Received: from eggs.gnu.org ([208.118.235.92]:46756)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <rrt@HIDDEN>) id 1TM2Ma-0003Li-Ee
	for submit <at> debbugs.gnu.org; Wed, 10 Oct 2012 15:54:42 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rrt@HIDDEN>) id 1TM2Lp-0000qC-OS
	for submit <at> debbugs.gnu.org; Wed, 10 Oct 2012 15:53:54 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI,
	T_FRT_LOLITA1 autolearn=unavailable version=3.3.2
Received: from lists.gnu.org ([208.118.235.17]:53450)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <rrt@HIDDEN>)
	id 1TM2Lp-0000q7-Ko
	for submit <at> debbugs.gnu.org; Wed, 10 Oct 2012 15:53:53 -0400
Received: from eggs.gnu.org ([208.118.235.92]:33822)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rrt@HIDDEN>) id 1TM2Ln-000333-GT
	for bug-automake@HIDDEN; Wed, 10 Oct 2012 15:53:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rrt@HIDDEN>) id 1TM2Ll-0000pD-Pg
	for bug-automake@HIDDEN; Wed, 10 Oct 2012 15:53:51 -0400
Received: from exprod7og118.obsmtp.com ([64.18.2.8]:57858)
	by eggs.gnu.org with smtp (Exim 4.71) (envelope-from <rrt@HIDDEN>)
	id 1TM2Ll-0000on-Gl
	for bug-automake@HIDDEN; Wed, 10 Oct 2012 15:53:49 -0400
Received: from mail-ie0-f169.google.com ([209.85.223.169]) (using TLSv1) by
	exprod7ob118.postini.com ([64.18.6.12]) with SMTP
	ID DSNKUHXSSmQ+r+9FVttIbH1+kJJ/AsXWKm3o@HIDDEN;
	Wed, 10 Oct 2012 12:53:49 PDT
Received: by mail-ie0-f169.google.com with SMTP id 10so2111228ied.0
	for <bug-automake@HIDDEN>; Wed, 10 Oct 2012 12:53:46 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=google.com; s=20120113;
	h=mime-version:date:message-id:subject:from:to:content-type
	:x-gm-message-state;
	bh=qne/laEsrwO0ypx3E6Xr8G5EeDO72zij6jAo9bhFXl4=;
	b=RX/BS9+asJwv3pFsQvKo/1OqJZOBRCaob7KtG9l9CORNTjQFlXAQetrZnp+ZLS0rMg
	RU7LLNKx9xRsWEHtcZfBB8OqUjwFl5d0RA2GGFj2T+o1i1Pk/791hAEPu3LBEigEoHrA
	y/+GTnqvP3kpjC42TUFRkdpc1sSN2O1/2Lr5aYmne0WQY97WUSwD7XUEaUwD4GrcACbG
	JUcUyAoo8QyRoo4VGKjEDiE+jbAnppencXalkfmjQBtJfMbtYULuvtKMfdeGWbif+Mik
	egY9SLTXjGNP/CKHl5ETtaLci5AHLDboN3RpStuHjwz2nq3qNIr1XALgbEURwV+h6yXw
	iKbw==
MIME-Version: 1.0
Received: by 10.42.156.1 with SMTP id x1mr19280187icw.51.1349898826565; Wed,
	10 Oct 2012 12:53:46 -0700 (PDT)
Received: by 10.231.240.1 with HTTP; Wed, 10 Oct 2012 12:53:46 -0700 (PDT)
Date: Wed, 10 Oct 2012 20:53:46 +0100
Message-ID: <CAOnWdoix5PAoUrXPwyq8zeTjpgkpGEnz4VwWVxFUebhKk+pg9g@HIDDEN>
Subject: Parallel tests vs fast tests (and beyond)
From: Reuben Thomas <rrt@HIDDEN>
To: bug-automake <bug-automake@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1
X-Gm-Message-State: ALoCoQnO5K8RMGdnn2G50YXZBB1wjo5ZTR8G5zP37ZdP+uhrBex1+WbaI67GZNU9f1IuMcAByQhf
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6, seldom 2.4 (older,
	4)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 208.118.235.17
X-Spam-Score: -3.5 (---)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.5 (---)

With the recent work on parallel tests in automake I thought it was
time to give them a spin, so I did, for the "zee" branch of GNU Zile.
This has about 100 tests, the total wall clock time being around 8s on
my 2.5GHz 4-core Sandy Bridge machine, with the following target:

 check-local: $(builddir)/src/zee
	echo $(TESTS) | $(LUA_ENV) $(LUA_TESTS_ENVIRONMENT) xargs $(RUNLUATESTS)

So, I rewrote it as:

LOG_COMPILER = $(LUA_ENV) $(LUA_TESTS_ENVIRONMENT) $(RUNLUATESTS)

and lo! it ran in 2/3 of the time. Not bad for a few minutes' work.
But then I looked closer, and noticed that although the wall-clock
time had gone down, the user+system time had almost doubled. Oops: I
was only winning because I had four cores. Developers using
single-core or dual-core machines might well lose out with parallel
tests.

The test harness is a Lua script, so the parallel tests were starting
an extra Lua interpreter per test, so  this is not a huge surprise,
but it is a pity.

In the end, I found I got a much bigger speed up (down to under 3s), with:

check-local: $(builddir)/src/zee
	NPROC=`nproc`; \
	echo $(LUA_TESTS) | $(LUA_ENV) $(LUA_TESTS_ENVIRONMENT) xargs
--max-procs=$$NPROC --max-args=$$(( `echo $(LUA_TESTS) | wc -w` /
$$NPROC + 1 )) $(RUNLUATESTS) > /dev/null

That would be relatively straightforward to make portable (or rather,
bail out if non-GNU xargs is used, but it's still much clumsier than
parallel-tests, and of course uses the old serial-tests. (I'm working
with automake 1.11.6, in case it matters.) The implication of the 1.12
release notes is that serial-tests will be dropped at some point
post-1.13.

This whole problem is an instance of a more general problem, of which
another example is parallel make: for best performance, it should
probably batch up calls to gcc, for example, so that multiple source
files are compiled by each invocation.

I can imagine a view that we're rapidly increasing the number of
cores, so in fact there's little point working on these intermediate
solutions, but it seems to me that it's not just less-well-off
developers who could benefit, but also compile farms.

Comments? Solutions I've missed?

-- 
http://rrt.sc3d.org




Acknowledgement sent to Reuben Thomas <rrt@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-automake@HIDDEN. Full text available.
Report forwarded to bug-automake@HIDDEN:
bug#12620; Package automake. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 31 Oct 2014 17:00:04 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.