GNU bug report logs - #35139
Rust builds systematically time out

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: important; Reported by: Ludovic Courtès <ludo@HIDDEN>; dated Thu, 4 Apr 2019 09:01:02 UTC; Maintainer for guix is bug-guix@HIDDEN.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 5 Apr 2019 21:20:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Apr 05 17:20:11 2019
Received: from localhost ([127.0.0.1]:46341 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hCWFu-0004CG-M1
	for submit <at> debbugs.gnu.org; Fri, 05 Apr 2019 17:20:11 -0400
Received: from eggs.gnu.org ([209.51.188.92]:49430)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mikadozero@HIDDEN>) id 1hCWFr-0004Bx-Di
 for submit <at> debbugs.gnu.org; Fri, 05 Apr 2019 17:20:09 -0400
Received: from lists.gnu.org ([209.51.188.17]:60953)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <mikadozero@HIDDEN>)
 id 1hCWFl-0003uY-Rd
 for submit <at> debbugs.gnu.org; Fri, 05 Apr 2019 17:20:02 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35520)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <mikadozero@HIDDEN>) id 1hCWFk-0004Gk-BN
 for bug-Guix@HIDDEN; Fri, 05 Apr 2019 17:20:01 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM,
 URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <mikadozero@HIDDEN>) id 1hCWFj-0003oD-34
 for bug-Guix@HIDDEN; Fri, 05 Apr 2019 17:20:00 -0400
Received: from forward100j.mail.yandex.net ([2a02:6b8:0:801:2::100]:50621)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <mikadozero@HIDDEN>)
 id 1hCWFi-0003W6-AD; Fri, 05 Apr 2019 17:19:59 -0400
Received: from mxback17j.mail.yandex.net (mxback17j.mail.yandex.net
 [IPv6:2a02:6b8:0:1619::93])
 by forward100j.mail.yandex.net (Yandex) with ESMTP id CBCF150E11B2;
 Sat,  6 Apr 2019 00:19:37 +0300 (MSK)
Received: from smtp4p.mail.yandex.net (smtp4p.mail.yandex.net
 [2a02:6b8:0:1402::15:6])
 by mxback17j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id 5oGNejGnxA-Jbtqg0Tk;
 Sat, 06 Apr 2019 00:19:37 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.com; s=mail;
 t=1554499177; bh=j+htmYGg4bDgoqttz8nfTg9dlO0PMZxkGfO324g+NX4=;
 h=In-reply-to:Subject:Cc:To:From:Date:References:Message-ID;
 b=Osv2dGdyCra0yTVMAlk2kg9hUnHpp+DwLQDqoTuw2VwtyZIzBAPWeQyjmLUzMBOzQ
 jG0aFHw0Qm5RRdVoNK9Cr0k4SDtUQCk5VJ6mkSvHcilo+PVmEOAjLbztp07C+SCCij
 2AHvmWPPZjVmzQdeZxn3nWjNWiGsFj03vi9uG0c0=
Authentication-Results: mxback17j.mail.yandex.net;
 dkim=pass header.i=@yandex.com
Received: by smtp4p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id
 3ILwuenvaR-JZTuv39h; Sat, 06 Apr 2019 00:19:36 +0300
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client certificate not present)
References: <878swqtabb.fsf@HIDDEN>
User-agent: mu4e 1.0; emacs 26.1
From: mikadoZero <mikadozero@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#35139: Rust builds systematically time out
In-reply-to: <878swqtabb.fsf@HIDDEN>
Date: Fri, 05 Apr 2019 17:18:51 -0400
Message-ID: <cucpnq040ck.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2a02:6b8:0:801:2::100
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
Cc: bug-Guix@HIDDEN, Ivan Petkov <ivanppetkov@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

When I try to install rust I get similar behavior.  It does not finish
building.  The longest I have let it try for was around 12 hours. That
was is a on a machine with 1GB RAM and 10GB SWAP.

Ludovic Court=C3=A8s writes:

> Hello,
>
> On berlin, Rust 1.24.1 builds systematically exceed the timeout:
>
> --8<---------------cut here---------------start------------->8---
> Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-un=
known-linux-gnu)
>    Compiling arena v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc=
-1.24.1-src/src/libarena)
>    Compiling rustc_driver v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-=
0/rustc-1.24.1-src/src/librustc_driver)
>
> [...]
>
>    Compiling rls-data v0.14.0
>    Compiling rustc_data_structures v0.0.0 (file:///tmp/guix-build-rust-1.=
24.1.drv-0/rustc-1.24.1-src/src/librustc_data_structures)
>    Compiling flate2 v1.0.1
>    Compiling syntax_pos v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/=
rustc-1.24.1-src/src/libsyntax_pos)
>    Compiling rustc_errors v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-=
0/rustc-1.24.1-src/src/librustc_errors)
>    Compiling backtrace v0.3.4
> guix offload: error: timeout expired while offloading '/gnu/store/61bd22d=
9mg3xl260jwddisiahh3kmanj-rust-1.24.1.drv'
> --8<---------------cut here---------------end--------------->8---
>
> Strangely, the build lasts ~9000 seconds (2.5 hours) on the front-end
> node of berlin=C2=B9, and the timeout for guix-daemon on berlin is 6h (see
> guix-maintenance.git) while the max-silent-time is 1h.
>
> The build nodes may be slower than the front-end, but still, it seems
> unlikely that it would take more than 6h there.  (That could happen if
> the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly parallel=
=E2=80=9D, but
> we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)
>
> To summarize, there are two problems:
>
>   1. Rust takes too long to build.  What can we do about it?  Enable
>      parallel builds?
>
>   2. Offloaded builds seem to time out prematurely or something.
>
> Thoughts?
>
> Ludo=E2=80=99.
>
> =C2=B9 See <https://ci.guix.info/log/rkrnm3rr7g6fhr17160vn1mz5rdzh9lv-rus=
t-1.24.1>
>   for timings.





Information forwarded to bug-guix@HIDDEN:
bug#35139; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Apr 2019 17:37:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 13:37:20 2019
Received: from localhost ([127.0.0.1]:44650 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hC6Ih-00061t-Q4
	for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 13:37:20 -0400
Received: from eggs.gnu.org ([209.51.188.92]:44705)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ivanppetkov@HIDDEN>) id 1hC6Ig-00061h-L2
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 13:37:19 -0400
Received: from lists.gnu.org ([209.51.188.17]:60365)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <ivanppetkov@HIDDEN>)
 id 1hC6Ib-0004xp-8A
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 13:37:13 -0400
Received: from eggs.gnu.org ([209.51.188.92]:59073)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ivanppetkov@HIDDEN>) id 1hC6IZ-0006M7-Vn
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 13:37:13 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_40,FREEMAIL_FROM,
 HTML_MESSAGE, MIME_QP_LONG_LINE, URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ivanppetkov@HIDDEN>) id 1hC6IY-0004wN-O5
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 13:37:11 -0400
Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]:42234)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <ivanppetkov@HIDDEN>)
 id 1hC6IY-0004tw-AL; Thu, 04 Apr 2019 13:37:10 -0400
Received: by mail-pl1-x631.google.com with SMTP id cv12so1502223plb.9;
 Thu, 04 Apr 2019 10:37:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:subject:from:in-reply-to:date:cc
 :content-transfer-encoding:message-id:references:to;
 bh=je892U3kv3HoQBznfvnzd2DrxOezqrse61iEoHgrJqo=;
 b=VW7zc0uCW2pa2MqIcQGFF4sA3GJ1NFApWMKBSNzs8/mIJSwSIHkRoutrwYTqYi31lZ
 UXKjwSRlT0+ewsyQTchnW37ltNPtwoGBHWS3tF0UIiOlmdTA2MXLuAfOdXA0VrTzPtzW
 5ts0kdPQ1J8n2GGxYiFSq/E5YB+KnQtupawy78H0eR8jr+4DKJKPGqk5z2xL0wEkYOZE
 xGr3ADHoR2mcuPfyt1W/sriuRSioEAZxLfeRdcNRRqnBHCxJPlTs4jSnyfcRdjgrW4FK
 RioTU7Y/CCJdNuQTVEEX2MmAcenrq3cQ+2WH7rTge1qA0QR3rND5alcfE/W0dejY2owS
 4rwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc
 :content-transfer-encoding:message-id:references:to;
 bh=je892U3kv3HoQBznfvnzd2DrxOezqrse61iEoHgrJqo=;
 b=HT4796uytARpDWKp40Jr1Y54lUQOv89SvIYv1eXC95tp9nm5y0d17WAmSQVCIl5A+8
 UsuNC8AUAs8gf1ZWJy5Cazw9qzJjS6OGEhz1dO+MNeqy9Zgrk6WArApRfgOCaugyql1Z
 t/YWB5TIzQMwwkBImXlSqAG7eovtbcJAuPawadwZcGNQWiJ9I34vVQDnX9T2StPNoIMK
 2SMUmELFTyuqkbZQga3n80gf6utJOJOhlaTHLiEqmcvPXKxImdlo0lL5Xckc8nVbp5hb
 aD+z7OtTGpqXwr0pnyQo/Bhj0H6weU4Zv4K5EAcszV10bJ4oDtssfatOYBkq507LQS+0
 fHZA==
X-Gm-Message-State: APjAAAWNWL/WDnUEFzSt9XKPXj8MFWuU69xi7w28JsPUJfTzB3QRod/W
 GntkyLluY6OIdti1QldHCsaU3wzsbrQ=
X-Google-Smtp-Source: APXvYqwOPOHBHUJOiqAkfvDwd2QTzT4eFuT/oh8ijO5SDDCquf9QIzllcpZPxpP5DMYFE9acgCtUqA==
X-Received: by 2002:a17:902:8f92:: with SMTP id
 z18mr7903456plo.123.1554399426240; 
 Thu, 04 Apr 2019 10:37:06 -0700 (PDT)
Received: from [10.207.151.102] (mobile-166-176-184-13.mycingular.net.
 [166.176.184.13])
 by smtp.gmail.com with ESMTPSA id u62sm37721364pfa.124.2019.04.04.10.37.04
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 04 Apr 2019 10:37:05 -0700 (PDT)
Content-Type: multipart/alternative;
 boundary=Apple-Mail-4E652100-E4B6-4614-847F-CF56732B7E73
Mime-Version: 1.0 (1.0)
Subject: Re: bug#35139: Rust builds systematically time out
From: Ivan Petkov <ivanppetkov@HIDDEN>
X-Mailer: iPhone Mail (16D57)
In-Reply-To: <87imvtrc0g.fsf@HIDDEN>
Date: Thu, 4 Apr 2019 10:37:03 -0700
Content-Transfer-Encoding: 7bit
Message-Id: <17B412D1-5D9A-40C8-B37E-D8C08F0E9641@HIDDEN>
References: <878swqtabb.fsf@HIDDEN> <87bm1mglus.fsf@HIDDEN>
 <101FBDE5-97FA-4449-9076-DD24C56B8715@HIDDEN> <87imvtrc0g.fsf@HIDDEN>
To: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 2607:f8b0:4864:20::631
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
Cc: dannym@HIDDEN, Pierre Langlois <pierre.langlois@HIDDEN>,
 bug-guix@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)


--Apple-Mail-4E652100-E4B6-4614-847F-CF56732B7E73
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

Danny=E2=80=99s got a patch for turning on parallel tests in #35126

Not sure why the previous tests were running sequentially, but there is a co=
mment somewhere saying it=E2=80=99s to avoid EAGAIN errors.

--Ivan

> On Apr 4, 2019, at 9:06 AM, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>=20
> Ivan Petkov <ivanppetkov@HIDDEN> skribis:
>=20
>>> On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>>>=20
>>> The build nodes may be slower than the front-end, but still, it seems
>>> unlikely that it would take more than 6h there.  (That could happen if
>>> the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly parallel=E2=
=80=9D, but
>>> we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)
>>>=20
>>> To summarize, there are two problems:
>>>=20
>>> 1. Rust takes too long to build.  What can we do about it?  Enable
>>>    parallel builds?
>>=20
>> Rust tests are designed to run in parallel, as long as you have enough
>> RAM, file descriptors, etc. available on the machine for the amount of
>> concurrency being used. The compiler test suite is largely just compiling=

>> files, so the most important resource is probably available RAM/swap.
>=20
> Perhaps we could start with:
>=20
>  "-j" (number->string (min (parallel-job-count) 2))
>=20
> ?
>=20
>> Maybe if the bootstrapped versions don=E2=80=99t ever change skipping the=
 check
>> phase will be safe, but I think we should try running parallel tests firs=
t
>> and see how far that gets us.
>=20
> Sounds like a good start.
>=20
> So the only reason we=E2=80=99re running tests sequentially is because of m=
emory
> usage concerns?
>=20
> Thanks,
> Ludo=E2=80=99.

--Apple-Mail-4E652100-E4B6-4614-847F-CF56732B7E73
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div style=3D"caret-color: rgb(69, 69, 69);=
 color: rgb(69, 69, 69); text-decoration: -webkit-letterpress;">Danny=E2=80=99=
s got a patch for turning on parallel tests in #35126</div><div style=3D"car=
et-color: rgb(69, 69, 69); color: rgb(69, 69, 69); text-decoration: -webkit-=
letterpress;"><br></div><span style=3D"caret-color: rgb(69, 69, 69); color: r=
gb(69, 69, 69); text-decoration: -webkit-letterpress;">Not sure why the prev=
ious tests were running sequentially, but there is a comment somewhere sayin=
g it=E2=80=99s to avoid EAGAIN errors.<br></span><br><div id=3D"AppleMailSig=
nature" dir=3D"ltr">--Ivan</div><div dir=3D"ltr"><br>On Apr 4, 2019, at 9:06=
 AM, Ludovic Court=C3=A8s &lt;<a href=3D"mailto:ludo@HIDDEN">ludo@HIDDEN</=
a>&gt; wrote:<br><br></div><blockquote type=3D"cite"><div dir=3D"ltr"><span>=
Ivan Petkov &lt;<a href=3D"mailto:ivanppetkov@HIDDEN">ivanppetkov@HIDDEN=
om</a>&gt; skribis:</span><br><span></span><br><blockquote type=3D"cite"><bl=
ockquote type=3D"cite"><span>On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8=
s &lt;<a href=3D"mailto:ludo@HIDDEN">ludo@HIDDEN</a>&gt; wrote:</span><br>=
</blockquote></blockquote><blockquote type=3D"cite"><blockquote type=3D"cite=
"><span></span><br></blockquote></blockquote><blockquote type=3D"cite"><bloc=
kquote type=3D"cite"><span>The build nodes may be slower than the front-end,=
 but still, it seems</span><br></blockquote></blockquote><blockquote type=3D=
"cite"><blockquote type=3D"cite"><span>unlikely that it would take more than=
 6h there. &nbsp;(That could happen if</span><br></blockquote></blockquote><=
blockquote type=3D"cite"><blockquote type=3D"cite"><span>the test suite, whi=
ch lasts 2.1h, were =E2=80=9Cembarrassingly parallel=E2=80=9D, but</span><br=
></blockquote></blockquote><blockquote type=3D"cite"><blockquote type=3D"cit=
e"><span>we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)</span><br>=
</blockquote></blockquote><blockquote type=3D"cite"><blockquote type=3D"cite=
"><span></span><br></blockquote></blockquote><blockquote type=3D"cite"><bloc=
kquote type=3D"cite"><span>To summarize, there are two problems:</span><br><=
/blockquote></blockquote><blockquote type=3D"cite"><blockquote type=3D"cite"=
><span></span><br></blockquote></blockquote><blockquote type=3D"cite"><block=
quote type=3D"cite"><span> 1. Rust takes too long to build. &nbsp;What can w=
e do about it? &nbsp;Enable</span><br></blockquote></blockquote><blockquote t=
ype=3D"cite"><blockquote type=3D"cite"><span> &nbsp;&nbsp;&nbsp;parallel bui=
lds?</span><br></blockquote></blockquote><blockquote type=3D"cite"><span></s=
pan><br></blockquote><blockquote type=3D"cite"><span>Rust tests are designed=
 to run in parallel, as long as you have enough</span><br></blockquote><bloc=
kquote type=3D"cite"><span>RAM, file descriptors, etc. available on the mach=
ine for the amount of</span><br></blockquote><blockquote type=3D"cite"><span=
>concurrency being used. The compiler test suite is largely just compiling</=
span><br></blockquote><blockquote type=3D"cite"><span>files, so the most imp=
ortant resource is probably available RAM/swap.</span><br></blockquote><span=
></span><br><span>Perhaps we could start with:</span><br><span></span><br><s=
pan> &nbsp;"-j" (number-&gt;string (min (parallel-job-count) 2))</span><br><=
span></span><br><span>?</span><br><span></span><br><blockquote type=3D"cite"=
><span>Maybe if the bootstrapped versions don=E2=80=99t ever change skipping=
 the check</span><br></blockquote><blockquote type=3D"cite"><span>phase will=
 be safe, but I think we should try running parallel tests first</span><br><=
/blockquote><blockquote type=3D"cite"><span>and see how far that gets us.</s=
pan><br></blockquote><span></span><br><span>Sounds like a good start.</span>=
<br><span></span><br><span>So the only reason we=E2=80=99re running tests se=
quentially is because of memory</span><br><span>usage concerns?</span><br><s=
pan></span><br><span>Thanks,</span><br><span>Ludo=E2=80=99.</span><br></div>=
</blockquote></body></html>=

--Apple-Mail-4E652100-E4B6-4614-847F-CF56732B7E73--




Information forwarded to bug-guix@HIDDEN:
bug#35139; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Apr 2019 16:06:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 12:06:35 2019
Received: from localhost ([127.0.0.1]:44604 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hC4ss-0003fL-KO
	for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 12:06:35 -0400
Received: from eggs.gnu.org ([209.51.188.92]:53051)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1hC4sr-0003f5-B0
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 12:06:33 -0400
Received: from lists.gnu.org ([209.51.188.17]:42698)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1hC4sm-0001nK-58
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 12:06:28 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39194)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1hC4sl-0005IN-6R
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 12:06:28 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:470:142:3::e]:48009)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1hC4sk-0001lM-Sx; Thu, 04 Apr 2019 12:06:27 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=59714 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1hC4sV-0000Rw-2w; Thu, 04 Apr 2019 12:06:14 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Ivan Petkov <ivanppetkov@HIDDEN>
Subject: Re: bug#35139: Rust builds systematically time out
References: <878swqtabb.fsf@HIDDEN> <87bm1mglus.fsf@HIDDEN>
 <101FBDE5-97FA-4449-9076-DD24C56B8715@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 15 Germinal an 227 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Thu, 04 Apr 2019 18:06:07 +0200
In-Reply-To: <101FBDE5-97FA-4449-9076-DD24C56B8715@HIDDEN> (Ivan Petkov's
 message of "Thu, 4 Apr 2019 08:47:42 -0700")
Message-ID: <87imvtrc0g.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: submit
Cc: Pierre Langlois <pierre.langlois@HIDDEN>, bug-guix@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Ivan Petkov <ivanppetkov@HIDDEN> skribis:

>> On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>>=20
>> The build nodes may be slower than the front-end, but still, it seems
>> unlikely that it would take more than 6h there.  (That could happen if
>> the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly parallel=
=E2=80=9D, but
>> we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)
>>=20
>> To summarize, there are two problems:
>>=20
>>  1. Rust takes too long to build.  What can we do about it?  Enable
>>     parallel builds?
>
> Rust tests are designed to run in parallel, as long as you have enough
> RAM, file descriptors, etc. available on the machine for the amount of
> concurrency being used. The compiler test suite is largely just compiling
> files, so the most important resource is probably available RAM/swap.

Perhaps we could start with:

  "-j" (number->string (min (parallel-job-count) 2))

?

> Maybe if the bootstrapped versions don=E2=80=99t ever change skipping the=
 check
> phase will be safe, but I think we should try running parallel tests first
> and see how far that gets us.

Sounds like a good start.

So the only reason we=E2=80=99re running tests sequentially is because of m=
emory
usage concerns?

Thanks,
Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#35139; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Apr 2019 15:48:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 11:48:17 2019
Received: from localhost ([127.0.0.1]:44591 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hC4bA-0003BY-Ix
	for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 11:48:16 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48168)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ivanppetkov@HIDDEN>) id 1hC4b7-0003BI-PE
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 11:48:14 -0400
Received: from lists.gnu.org ([209.51.188.17]:59700)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <ivanppetkov@HIDDEN>)
 id 1hC4b2-0001Rq-4t
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 11:48:08 -0400
Received: from eggs.gnu.org ([209.51.188.92]:34295)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ivanppetkov@HIDDEN>) id 1hC4az-0004id-Rp
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 11:48:08 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM,
 HTML_MESSAGE,URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ivanppetkov@HIDDEN>) id 1hC4ay-0001OT-N2
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 11:48:05 -0400
Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:39042)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <ivanppetkov@HIDDEN>)
 id 1hC4at-00012z-PQ; Thu, 04 Apr 2019 11:48:00 -0400
Received: by mail-pf1-x432.google.com with SMTP id i17so1572665pfo.6;
 Thu, 04 Apr 2019 08:47:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:subject:from:in-reply-to:date:cc:message-id:references
 :to; bh=q+yRBi+kJPa0MqZjoaZwtE+95gOZNSZBIdBKnaMOiaE=;
 b=DdR2kzYZrv19KU61RHdjVp7XExFpnqlK2BmFdceYIv61lz3FPPVNp6TaHJN2sv4x2+
 sMNZ3p1HYPDZpQDN6xdAe5N2pyfUAnsPxZztY8kQ9kkQChBoGsyGGmt+KFpMPAZnz8ip
 ktX4uQMTt+eiZH/PLDIZ1jLhS2eodduBsLXIy7Ld++LAdinVvtw4HwFmsIcGoCNrJ0JM
 04fBDlhHlusQFMpa7FIycAi7bUXI0J9inE0bu7Ik69h5nJ+8aqFg0S7HBEwcYlQQHRS/
 2AYDxO1kQIgzWAL9pcLRKiabDijd5j/L7YAb5RlCtEh7xFerGL7MfeHZYfa6zwaxIP4j
 bGLA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc
 :message-id:references:to;
 bh=q+yRBi+kJPa0MqZjoaZwtE+95gOZNSZBIdBKnaMOiaE=;
 b=S6dJOXUblxUkA5bKn9CvQv1gfiejZtLRbTZyI/zrX98j/ImvwWT14t8dtAFs0DS5np
 qSUHUY13MF9Il7GjkgvhyPhbiXlmwFZ3uXsYbALsn1AUC8Y2nqFFWMnF8fIV1LtoJs3I
 xdCRmx09NOh7ncmI4B9F9vhzkLhsJO8E2OQrobJoXMQ8CFFaK6r6EOAiqDLymcOLy2uN
 VBV+QaSYPCwga86TSLvRHtxygRkCkUSoaGCNxk/E0woZm5DYiXF/Dv9ddj4LALra22P3
 P1qHCzGpxQb8lvOH4nEa8+V+XuXNEZ9dwrsv5u/xesamhcLx7sgop+/LFcj8tskl8tXH
 Y4lw==
X-Gm-Message-State: APjAAAWpsxHbr+fuCheXpDxotESatUDnAXeAd13fajwMi8k1FUkDEwpz
 kLSUR0mn4KQXC147PFQO2qI=
X-Google-Smtp-Source: APXvYqyu6pByvdYdZb7fSC4p73unnmc9u/usL6PBT8S7gOgvgwpPu3WEfIV4IhJcbG0zzyYDy/i04Q==
X-Received: by 2002:a62:1c07:: with SMTP id c7mr6619940pfc.159.1554392867934; 
 Thu, 04 Apr 2019 08:47:47 -0700 (PDT)
Received: from ?IPv6:2601:602:9a00:1784:9d2f:e342:1236:46d1?
 ([2601:602:9a00:1784:9d2f:e342:1236:46d1])
 by smtp.gmail.com with ESMTPSA id y12sm56689223pgq.64.2019.04.04.08.47.46
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 04 Apr 2019 08:47:47 -0700 (PDT)
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D"
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: Re: bug#35139: Rust builds systematically time out
From: Ivan Petkov <ivanppetkov@HIDDEN>
In-Reply-To: <87bm1mglus.fsf@HIDDEN>
Date: Thu, 4 Apr 2019 08:47:42 -0700
Message-Id: <101FBDE5-97FA-4449-9076-DD24C56B8715@HIDDEN>
References: <878swqtabb.fsf@HIDDEN> <87bm1mglus.fsf@HIDDEN>
To: Pierre Langlois <pierre.langlois@HIDDEN>,
 =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
X-Mailer: Apple Mail (2.3445.9.1)
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 2607:f8b0:4864:20::432
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
Cc: bug-guix@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)


--Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8


> On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>=20
> The build nodes may be slower than the front-end, but still, it seems
> unlikely that it would take more than 6h there.  (That could happen if
> the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly =
parallel=E2=80=9D, but
> we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)
>=20
> To summarize, there are two problems:
>=20
>  1. Rust takes too long to build.  What can we do about it?  Enable
>     parallel builds?

Rust tests are designed to run in parallel, as long as you have enough
RAM, file descriptors, etc. available on the machine for the amount of
concurrency being used. The compiler test suite is largely just =
compiling
files, so the most important resource is probably available RAM/swap.

> On Apr 4, 2019, at 2:28 AM, Pierre Langlois <pierre.langlois@HIDDEN> =
wrote:
>=20
> One thing I suggested in the past was to remove the check phase *only*
> for rust packages used for bootstrapping. This way we still run the
> tests for the final rust but not at every step in the chain.
>=20
> Although, I wonder if we're more likely to miss a bug if we do this, =
I'm
> not sure.

Although that definitely will speed the bootstrap chain, I=E2=80=99m =
concerned that
if a dependency package ever gets updated and breaks things we =
wouldn=E2=80=99t
know without running the test suite.

Maybe if the bootstrapped versions don=E2=80=99t ever change skipping =
the check
phase will be safe, but I think we should try running parallel tests =
first
and see how far that gets us.

=E2=80=94Ivan=

--Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div =
class=3D""><br class=3D""></div><div><blockquote type=3D"cite" =
class=3D""><div class=3D"">On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8=
s &lt;<a href=3D"mailto:ludo@HIDDEN" class=3D"">ludo@HIDDEN</a>&gt; =
wrote:</div><div class=3D""><br class=3D"">The build nodes may be slower =
than the front-end, but still, it seems<br class=3D"">unlikely that it =
would take more than 6h there. &nbsp;(That could happen if<br =
class=3D"">the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly =
parallel=E2=80=9D, but<br class=3D"">we=E2=80=99re running tests with =
=E2=80=98-j1=E2=80=99.)<br class=3D""><br class=3D"">To summarize, there =
are two problems:<br class=3D""><br class=3D"">&nbsp;1. Rust takes too =
long to build. &nbsp;What can we do about it? &nbsp;Enable<br =
class=3D"">&nbsp;&nbsp;&nbsp;&nbsp;parallel builds?<br =
class=3D""></div></blockquote></div><br class=3D""><div class=3D"">Rust =
tests are designed to run in parallel, as long as you have =
enough</div><div class=3D"">RAM, file descriptors, etc. available on the =
machine for the amount of</div><div class=3D"">concurrency being used. =
The compiler test suite is largely just compiling</div><div =
class=3D"">files, so the most important resource is probably available =
RAM/swap.</div><div><br class=3D""><blockquote type=3D"cite" =
class=3D""><div class=3D"">On Apr 4, 2019, at 2:28 AM, Pierre Langlois =
&lt;<a href=3D"mailto:pierre.langlois@HIDDEN" =
class=3D"">pierre.langlois@HIDDEN</a>&gt; wrote:</div><div class=3D""><br=
 class=3D""><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;" class=3D"">One thing I =
suggested in the past was to remove the check phase *only*</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" class=3D"">for rust packages used for =
bootstrapping. This way we still run the</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">tests for the final rust but not at every step in the =
chain.</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;" class=3D"">Although, I =
wonder if we're more likely to miss a bug if we do this, I'm</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" class=3D"">not sure.</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" =
class=3D""></div></blockquote></div><br class=3D""><div =
class=3D"">Although that definitely will speed the bootstrap chain, =
I=E2=80=99m concerned that</div><div class=3D"">if a dependency package =
ever gets updated and breaks things we wouldn=E2=80=99t</div><div =
class=3D"">know without running the test suite.</div><div class=3D""><br =
class=3D""></div><div class=3D"">Maybe if the bootstrapped versions =
don=E2=80=99t ever change skipping the check</div><div class=3D"">phase =
will be safe, but I think we should try running parallel tests =
first</div><div class=3D"">and see how far that gets us.</div><div =
class=3D""><br class=3D""></div><div =
class=3D"">=E2=80=94Ivan</div></body></html>=

--Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D--




Information forwarded to bug-guix@HIDDEN:
bug#35139; Package guix. Full text available.
Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Apr 2019 09:29:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 05:29:13 2019
Received: from localhost ([127.0.0.1]:43700 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hBygL-0000Ew-El
	for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 05:29:13 -0400
Received: from eggs.gnu.org ([209.51.188.92]:54775)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pierre.langlois@HIDDEN>) id 1hBygJ-0000Ek-Qt
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 05:29:12 -0400
Received: from lists.gnu.org ([209.51.188.17]:48783)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <pierre.langlois@HIDDEN>)
 id 1hBygE-0000ti-In
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 05:29:06 -0400
Received: from eggs.gnu.org ([209.51.188.92]:40910)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <pierre.langlois@HIDDEN>) id 1hBygD-0007qA-Bw
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 05:29:06 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.7 required=5.0 tests=BAYES_20,FREEMAIL_FROM,
 RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <pierre.langlois@HIDDEN>) id 1hBygC-0000p1-4K
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 05:29:05 -0400
Received: from mout.gmx.net ([212.227.17.21]:49051)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <pierre.langlois@HIDDEN>)
 id 1hBygB-0000ni-OI
 for bug-guix@HIDDEN; Thu, 04 Apr 2019 05:29:04 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net;
 s=badeba3b8450; t=1554370140;
 bh=2F+xOZlgA7l2FgYpsu2IxvNCAZvRms9/V+ZvZxtB564=;
 h=X-UI-Sender-Class:References:From:To:Cc:Subject:In-reply-to:Date;
 b=BP3WQWJdPwGVWmgt7kFeeb4fH7dtNgpyCLXH5CMECM9vwrtVMemZKZaggThHgLf0Y
 4KTvJ+glcbEVxRnilSepplQ7VQIJ7CHU89tK8FDESF6GitadyKqfmxhs3YZ/G5v7Lh
 34xvPGN1CEwEAfYJujK29DtDFyFzeOoevYdJ5e24=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from labiere ([92.6.236.193]) by mail.gmx.com (mrgmx103
 [212.227.17.174]) with ESMTPSA (Nemesis) id 0McUnM-1hTVWu1Wqm-00Hen6; Thu, 04
 Apr 2019 11:29:00 +0200
References: <878swqtabb.fsf@HIDDEN>
User-agent: mu4e 1.1.0; emacs 26.1
From: Pierre Langlois <pierre.langlois@HIDDEN>
To: bug-guix@HIDDEN
Subject: Re: bug#35139: Rust builds systematically time out
In-reply-to: <878swqtabb.fsf@HIDDEN>
Date: Thu, 04 Apr 2019 10:28:59 +0100
Message-ID: <87bm1mglus.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:MPlsQxbnXnN4pgftVRa8QRk1fTdsNqYrx7+q4BPn6ZfyDBqs05o
 DvJ1ET1cLd+RI1nqwWWz5ke/emgaj06wms6IGZ4HrYLg9qr2MbNRzZ2Un0Sgd04y9IC1/7h
 2bqYZqFq8zie76l9BvNrfvnPnxlwFlTnBlAleZkz9euqTqmbM3QqpOEHNouanK36NBIk5ZD
 /tn9W9fHBvDOJ9oN0S2KQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:zL9DeqYpU20=:AsKPeA11YpHOVpIt6BYAps
 8TDJ+x/zxKGXTR01QpZ/QOxaKGtuczAhLrSC8HOlXS0jRP3aPuS+lDGdoQ/X7OAt33Bjqrmjv
 f0gueuoBItIg4vRfA+ziRTbH1KquL8udCIQQ1lHNDU1SSFhlXjycxNy9Oc3HbwruTCCJDVIvm
 O71t0o9xFBKqYWEmVItR8imfGkdK7i34gOMTcJZF09GVtDOKpSEKnjztwV+hCGQ0pVZd4qwBm
 zGg3m8TOESlp0l5sikLVMC47tAZ7bcoC+SOev19rdZOonk+fos5MBfEczb09vQF7NgAPreChN
 vFeK5UyKoW+s9R6NbYunk45RrGH9NqbbvbO+7u3rvzbvszBIpiR/d6UTSNOBI2HJn9y57QV7S
 hpstLiqpifBLFU7nDqDLverEmSuLpLRufO6apNgMSIdgYuxWUjbXHD9h1iKid5PNbjvLWZjiW
 9phVvjIe96r9IB69vmUUS4UqH/VGUkknezIp+BjSjK1luuxGlN+M0fZP46TbgRLXQCEy2Cp83
 DljPrNwH6DV1w9MR/0QlaFjXzTdL9r+Fg3eRfpq8eBMQRybqXjtyi7C9QYGwshi3VYvitQADm
 1qvFeGql7A69p2LcLjcZ3ucG22V/Ei84AEbX6Un36VSviZ6jwLxGN3v8Ug7EJFFGSpNKPvyNa
 MqP7lGDyCqIOB0p4Iy2ucn8v+kdlV0A9n0MSXOiAUpPSKu7D2sl2Y4Op15ciT/BLBUCNK0UZY
 Ru0ZD6d3fO3VppAiu2iu6UmmkGOMj7C5wEFkPKllimUcveIsV+lNjlwY+Tzrd9gE7+AJLSsup
 JZL0iVl8z1necauQeboEd61kpxk+qOaGf+5ENyJuopsh3Y4av+/COrEprjRKw2ZWUhyUKZfuu
 OI3iYqXwmiC+6WYhXjrAkrFCQUA5UcgToXK934u7cHCHnGQDFpV8hNsaee8/OFl12TM4deElE
 iqg0bz5cE9A==
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 212.227.17.21
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 0.9 (/)
X-Debbugs-Envelope-To: submit
Cc: Ivan Petkov <ivanppetkov@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.1 (/)

Hello!

Ludovic Court=C3=A8s writes:

> Hello,
>
> On berlin, Rust 1.24.1 builds systematically exceed the timeout:
>
> --8<---------------cut here---------------start------------->8---
> Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-un=
known-linux-gnu)
>    Compiling arena v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc=
-1.24.1-src/src/libarena)
>    Compiling rustc_driver v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-=
0/rustc-1.24.1-src/src/librustc_driver)
>
> [...]
>
>    Compiling rls-data v0.14.0
>    Compiling rustc_data_structures v0.0.0 (file:///tmp/guix-build-rust-1.=
24.1.drv-0/rustc-1.24.1-src/src/librustc_data_structures)
>    Compiling flate2 v1.0.1
>    Compiling syntax_pos v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/=
rustc-1.24.1-src/src/libsyntax_pos)
>    Compiling rustc_errors v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-=
0/rustc-1.24.1-src/src/librustc_errors)
>    Compiling backtrace v0.3.4
> guix offload: error: timeout expired while offloading '/gnu/store/61bd22d=
9mg3xl260jwddisiahh3kmanj-rust-1.24.1.drv'
> --8<---------------cut here---------------end--------------->8---
>
> Strangely, the build lasts ~9000 seconds (2.5 hours) on the front-end
> node of berlin=C2=B9, and the timeout for guix-daemon on berlin is 6h (see
> guix-maintenance.git) while the max-silent-time is 1h.
>
> The build nodes may be slower than the front-end, but still, it seems
> unlikely that it would take more than 6h there.  (That could happen if
> the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly parallel=
=E2=80=9D, but
> we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)
>
> To summarize, there are two problems:
>
>   1. Rust takes too long to build.  What can we do about it?  Enable
>      parallel builds?

One thing I suggested in the past was to remove the check phase *only*
for rust packages used for bootstrapping. This way we still run the
tests for the final rust but not at every step in the chain.

Although, I wonder if we're more likely to miss a bug if we do this, I'm
not sure.

For reference: https://lists.gnu.org/archive/html/guix-patches/2018-11/msg0=
0453.html

Thanks,
Pierre




Information forwarded to bug-guix@HIDDEN:
bug#35139; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Apr 2019 09:00:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 05:00:05 2019
Received: from localhost ([127.0.0.1]:43680 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hByE9-0007yu-FN
	for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 05:00:05 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48961)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1hByE7-0007xt-GI
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 05:00:03 -0400
Received: from lists.gnu.org ([209.51.188.17]:55526)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1hByE2-00015D-8g
 for submit <at> debbugs.gnu.org; Thu, 04 Apr 2019 04:59:58 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35104)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1hByE1-0000Uu-1B
 for bug-Guix@HIDDEN; Thu, 04 Apr 2019 04:59:58 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:470:142:3::e]:41689)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1hByE0-00010N-7D; Thu, 04 Apr 2019 04:59:56 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=57276 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1hByDz-0003ZA-61; Thu, 04 Apr 2019 04:59:55 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: bug-Guix@HIDDEN
Subject: Rust builds systematically time out
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 15 Germinal an 227 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Thu, 04 Apr 2019 10:59:52 +0200
Message-ID: <878swqtabb.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: submit
Cc: Danny Milosavljevic <dannym@HIDDEN>,
 Ivan Petkov <ivanppetkov@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello,

On berlin, Rust 1.24.1 builds systematically exceed the timeout:

--8<---------------cut here---------------start------------->8---
Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unkn=
own-linux-gnu)
   Compiling arena v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc-1=
.24.1-src/src/libarena)
   Compiling rustc_driver v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/=
rustc-1.24.1-src/src/librustc_driver)

[...]

   Compiling rls-data v0.14.0
   Compiling rustc_data_structures v0.0.0 (file:///tmp/guix-build-rust-1.24=
.1.drv-0/rustc-1.24.1-src/src/librustc_data_structures)
   Compiling flate2 v1.0.1
   Compiling syntax_pos v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/ru=
stc-1.24.1-src/src/libsyntax_pos)
   Compiling rustc_errors v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/=
rustc-1.24.1-src/src/librustc_errors)
   Compiling backtrace v0.3.4
guix offload: error: timeout expired while offloading '/gnu/store/61bd22d9m=
g3xl260jwddisiahh3kmanj-rust-1.24.1.drv'
--8<---------------cut here---------------end--------------->8---

Strangely, the build lasts ~9000 seconds (2.5 hours) on the front-end
node of berlin=C2=B9, and the timeout for guix-daemon on berlin is 6h (see
guix-maintenance.git) while the max-silent-time is 1h.

The build nodes may be slower than the front-end, but still, it seems
unlikely that it would take more than 6h there.  (That could happen if
the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly parallel=E2=
=80=9D, but
we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.)

To summarize, there are two problems:

  1. Rust takes too long to build.  What can we do about it?  Enable
     parallel builds?

  2. Offloaded builds seem to time out prematurely or something.

Thoughts?

Ludo=E2=80=99.

=C2=B9 See <https://ci.guix.info/log/rkrnm3rr7g6fhr17160vn1mz5rdzh9lv-rust-=
1.24.1>
  for timings.




Acknowledgement sent to Ludovic Courtès <ludo@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#35139; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.