Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 14 Jul 2024 21:51:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 14 17:51:43 2024
Received: from localhost ([127.0.0.1]:58676 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1sT78A-00088J-Mo
for submit <at> debbugs.gnu.org; Sun, 14 Jul 2024 17:51:43 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35964)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1sT788-000884-Ko
for 54447 <at> debbugs.gnu.org; Sun, 14 Jul 2024 17:51:41 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1sT75v-0001pJ-QB; Sun, 14 Jul 2024 17:49:23 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=SmbbPo04OVgKpAlAMeDH24nZIYi43r9DSsIZNYLcpTk=; b=X7C8fff+NhzAK0c7DTTW
fLbs337XKwyffZT8XsgdOsR3nQSmlLI1Y+nnWHFF9eL8V4EiseL6UsaVhHGzTsy+23nWqZj8eKtXT
8FiSi6GkkT4FjdlGk13Fg1aS62MQxwWIIOOG14SKEgMcVb4KAWtVED+wnkdj9BQpgmEiu6TgZWBfK
+nhf0LTlUOLmrQevGsKAZs7W178DUTxjD3Nyiub54w1A+fJRBAuMVL/jBt1xsyhApKNsADuRs9CZs
yUV9tzoqbKMovWnIpwptfgcsslQ+d3vzjDg/dpWef1IrvBjEVBXWEu6zPrvCh+HdPXH7nfbVhl1Pv
zp9544lxlWzZ7Q==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: 54447 <at> debbugs.gnu.org
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87cyr4u825.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Thu, 04 Apr 2024 23:33:38 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87r0m2v5ih.fsf@HIDDEN>
<87a5si7986.fsf@HIDDEN> <87cyr4u825.fsf@HIDDEN>
Date: Sun, 14 Jul 2024 23:49:20 +0200
Message-ID: <87h6cry78f.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
Cc: Mathieu Othacehe <othacehe@HIDDEN>, guix-sysadmin <guix-sysadmin@HIDDEN>,
Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi!
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> News from the everlasting bug!
>
> cannot build missing derivation =E2=80=98/gnu/store/dfgc46q3l8wlnymv49a=
1wjnxypin8p0y-plink-1.07.drv=E2=80=99
[...]
> =E2=80=98guix publish=E2=80=99 replied, but 40s too late (nginx has
> =E2=80=9Cproxy_connect_timeout 10s;=E2=80=9D for .narinfo URLs=C2=B9).
While the exact reason why =E2=80=98guix publish=E2=80=99 exhibits this beh=
avior is
unclear, the good news is that this is =E2=80=9Cfixed=E2=80=9D by having =
=E2=80=98cuirass
remote-worker=E2=80=99 retry when it fails to substitute a .drv (thanks Chr=
is
for the obvious-in-hindsight tip!):
https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3D2365=
ba786c805477fcbae6eaeb358b0dd0501598
https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3D7142=
6663f6ea32152782645e4632168dd2b18602
Furthermore, workers can now reject builds if they fail to substitute
the .drv, in which case =E2=80=98cuirass remote-server=E2=80=99 either resc=
hedules or
cancels the build:
https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3Da909=
fa99340db5e5cd64612ea4e07e929dc643ad
This has been deployed a few days ago on berlin and on its x86_64 build
machines. Working well so far!
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 14 Apr 2024 00:16:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Apr 13 20:16:18 2024 Received: from localhost ([127.0.0.1]:33997 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rvnXb-0006PR-QD for submit <at> debbugs.gnu.org; Sat, 13 Apr 2024 20:16:17 -0400 Received: from mail-4316.protonmail.ch ([185.70.43.16]:25221) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <john.kehayias@HIDDEN>) id 1rvnXU-0006Mn-24 for 54447 <at> debbugs.gnu.org; Sat, 13 Apr 2024 20:16:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1713053751; x=1713312951; bh=abTz5za9O36tpuSSb9MFhXTE+Ft24TMU+q2/Ysj4KJc=; h=Date:To:From:Cc:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=tW7fVttCg2BP6T9zbGTd96P4b9ZaInTZpyQup4JFajUjbBnE/slGtgbcd8XtgNQ9I Exb6TWKh+TRJ3Twf3p3Dma9RMCN1uSXM9mYvdLbwFFZ80fVpBqTyZThR6CTDWtCZNS YSUO8D0uUsHBqdDMat/W3y+fLmH49h2RbNh53hxEQVS7QhqFmSuRBq7J0PmdKMFqsA aemqKaNL9FNYuQz9hknRK/iu4p6CfEA1E+nDduchuuAlo/N4s44S7c3yK1TSqGj+3D H8mx164l0hUnRLyEDeB0/AmIn90sbOxsdyln2X0FHuDYrRI15ggMU8xfAYxAeA0a/N C5HwvL0X95Zeg== Date: Sun, 14 Apr 2024 00:15:45 +0000 To: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> From: John Kehayias <john.kehayias@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error Message-ID: <87mspwbxzn.fsf@HIDDEN> Feedback-ID: 7805494:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 54447 Cc: 54447 <at> debbugs.gnu.org, guix-sysadmin <guix-sysadmin@HIDDEN>, Maxim Cournoyer <maxim.cournoyer@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hi all, On Thu, Apr 04, 2024 at 11:33 PM, Ludovic Court=C3=A8s wrote: > Hello! > > News from the everlasting bug! > > cannot build missing derivation > =E2=80=98/gnu/store/dfgc46q3l8wlnymv49a1wjnxypin8p0y-plink-1.07.drv= =E2=80=99 > > (From <https://ci.guix.gnu.org/build/3861708/>.) > > Why was it missing this time? /var/log/nginx/error.log: > > 2024/04/04 17:15:03 [error] 98751#0: *152293778 upstream timed out (110: = Connection timed out) while reading response header from upstream, client: = 141.80.167.169, server: ci.guix.gnu.org, request: "GET /dfgc46q3l8wlnymv49a= 1wjnxypin8p0y.narinfo HTTP/1.1", upstream: "http://127.0.0.1:3000/dfgc46q3l= 8wlnymv49a1wjnxypin8p0y.narinfo", host: "141.80.167.131" > > > Oops! (There are dozens of upstream timeouts logged on that minute.) > > /var/log/guix-publish.log: > > 2024-04-04 17:14:51 GET /nar/lzip/pz39bkq7pd1hgy5rwiynqa33gyjvpgs5-python= -pygments-2.12.0 > 2024-04-04 17:14:51 GET /z2xxwwxswdd4b8c8iwmxhqnqbp5nwz09.narinfo > 2024-04-04 17:14:51 GET /lgyck285bsxzwrnh3x5ix5dwzd3n3wga.narinfo > 2024-04-04 17:14:51 GET /nar/zstd/jxkglr445f215m2faqz1i2lgmbans4rf-texliv= e-amsmath-66594-doc > 2024-04-04 17:15:33 GET /qg5cxb869i42jn7x2dm6k5l41ikkz21w.narinfo > 2024-04-04 17:15:33 GET /nar/zstd/i2hp3q2pfhsyl0al7z38am7cqpddi4qr-texliv= e-capt-of-66594-doc > 2024-04-04 17:15:33 GET /hh0gdbljj3cjdnjbr88kfm21mhys5sy7.narinfo > 2024-04-04 17:15:33 GET /dfgc46q3l8wlnymv49a1wjnxypin8p0y.narinfo > 2024-04-04 17:15:33 GET /yj63wifalfr6sla42h7mkqg011qrl5d0.narinfo > 2024-04-04 17:15:33 GET /h2s2g2adxbnd67g34mnjnpcr6p3nhr69.narinfo > 2024-04-04 17:15:33 -> GET /h2s2g2adxbnd67g34mnjnpcr6p3nhr69.narinfo: 404 > 2024-04-04 17:15:33 GET /nar/lzip/6zxlrw15b9dsv73s7v5fqabl7iv5v5il-python= -exceptiongroup-1.1.1 > 2024-04-04 17:15:33 GET /nar/zstd/pychjd114abscbqlzcr3s7myf1497vw2-julia-= compilersupportlibraries-jll-0.4.0%2B1 > > =E2=80=98guix publish=E2=80=99 replied, but 40s too late (nginx has > =E2=80=9Cproxy_connect_timeout 10s;=E2=80=9D for .narinfo URLs=C2=B9). > > Notice the 40s pause time between 17:14:51 and 17:15:33. Stop-the-world > GC? Unlikely, because =E2=80=98guix publish=E2=80=99 had been running fo= r ~3h, so even > with a leak=C2=B2, it=E2=80=99s hard to believe GC could take this long. > > Ludo=E2=80=99. > > =C2=B9 > https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/nginx/b= erlin.scm#n103 > =C2=B2 https://issues.guix.gnu.org/69596 I don't have any insight, but if anyone wants to see this in action at a large scale, take look at pretty much any red dot on https://ci.guix.gnu.org/eval/1238471/dashboard?system=3Di686-linux From my quick look all the CL and texlive failures were all missing derivation. I've tried restarting a bunch to get i686 coverage going, so hopefully some will disappear. But I can't/won't manually restart the thousands(?) of failed builds. I didn't see such issues on x86_64, while other architectures take a really long time to build on Berlin so I haven't looked. I don't know if this is helpful, but thought I would chime in if anyone wants potentially a bunch of data. And if there are good ideas to recover (just restart all builds?) that would be great so mesa-updates will be build on i686 since otherwise it looks good. Thanks! John
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 4 Apr 2024 21:33:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 17:33:56 2024
Received: from localhost ([127.0.0.1]:34792 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rsUia-0001NI-B4
for submit <at> debbugs.gnu.org; Thu, 04 Apr 2024 17:33:56 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:44482)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rsUiY-0001N3-Cn
for 54447 <at> debbugs.gnu.org; Thu, 04 Apr 2024 17:33:54 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1rsUiN-0007Rp-Qz; Thu, 04 Apr 2024 17:33:43 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=UFKi6WcF8ZUzwxhuoEZOv4UVsTiA0ldH5gLWme4BU14=; b=XLEiZPhgdz2n5pJ7Nyhe
K+j6mSeQtBG/FIbEVv3DVd+0wMbQueZ3gukrZIWvl+NY0ub1118e040/0caMlU7XovpERxIhX2+Fk
oBGUM94aGtrGPcmNewDD6WBI55sTv8Emm7KSF9shoRb07KFll3N+A6r68wmu1rpJ6AWxbD3aLk4xm
QBidNga2EuwrK2bihB4CSdSU3CkpDjrsl7KLtKY1oFx31VYWBKYMa1mhNc+LseQPlo+6Oj9H9H09k
7rQZCjUWjYSzDM1UoX6S/7Bzu+jNqzJe01ujOqAx5k7AnbHRVNcU32AtYj7zWblq3mtZiZBUzznLY
J8+Mtd3azL5rGQ==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: 54447 <at> debbugs.gnu.org
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87a5si7986.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Mon, 16 Oct 2023 19:44:41 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87r0m2v5ih.fsf@HIDDEN>
<87a5si7986.fsf@HIDDEN>
Date: Thu, 04 Apr 2024 23:33:38 +0200
Message-ID: <87cyr4u825.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
Cc: Mathieu Othacehe <othacehe@HIDDEN>, guix-sysadmin <guix-sysadmin@HIDDEN>,
Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hello!
News from the everlasting bug!
cannot build missing derivation =E2=80=98/gnu/store/dfgc46q3l8wlnymv49a1w=
jnxypin8p0y-plink-1.07.drv=E2=80=99
(From <https://ci.guix.gnu.org/build/3861708/>.)
Why was it missing this time? /var/log/nginx/error.log:
--8<---------------cut here---------------start------------->8---
2024/04/04 17:15:03 [error] 98751#0: *152293778 upstream timed out (110: Co=
nnection timed out) while reading response header from upstream, client: 14=
1.80.167.169, server: ci.guix.gnu.org, request: "GET /dfgc46q3l8wlnymv49a1w=
jnxypin8p0y.narinfo HTTP/1.1", upstream: "http://127.0.0.1:3000/dfgc46q3l8w=
lnymv49a1wjnxypin8p0y.narinfo", host: "141.80.167.131"
--8<---------------cut here---------------end--------------->8---
Oops! (There are dozens of upstream timeouts logged on that minute.)
/var/log/guix-publish.log:
--8<---------------cut here---------------start------------->8---
2024-04-04 17:14:51 GET /nar/lzip/pz39bkq7pd1hgy5rwiynqa33gyjvpgs5-python-p=
ygments-2.12.0
2024-04-04 17:14:51 GET /z2xxwwxswdd4b8c8iwmxhqnqbp5nwz09.narinfo
2024-04-04 17:14:51 GET /lgyck285bsxzwrnh3x5ix5dwzd3n3wga.narinfo
2024-04-04 17:14:51 GET /nar/zstd/jxkglr445f215m2faqz1i2lgmbans4rf-texlive-=
amsmath-66594-doc
2024-04-04 17:15:33 GET /qg5cxb869i42jn7x2dm6k5l41ikkz21w.narinfo
2024-04-04 17:15:33 GET /nar/zstd/i2hp3q2pfhsyl0al7z38am7cqpddi4qr-texlive-=
capt-of-66594-doc
2024-04-04 17:15:33 GET /hh0gdbljj3cjdnjbr88kfm21mhys5sy7.narinfo
2024-04-04 17:15:33 GET /dfgc46q3l8wlnymv49a1wjnxypin8p0y.narinfo
2024-04-04 17:15:33 GET /yj63wifalfr6sla42h7mkqg011qrl5d0.narinfo
2024-04-04 17:15:33 GET /h2s2g2adxbnd67g34mnjnpcr6p3nhr69.narinfo
2024-04-04 17:15:33 -> GET /h2s2g2adxbnd67g34mnjnpcr6p3nhr69.narinfo: 404
2024-04-04 17:15:33 GET /nar/lzip/6zxlrw15b9dsv73s7v5fqabl7iv5v5il-python-e=
xceptiongroup-1.1.1
2024-04-04 17:15:33 GET /nar/zstd/pychjd114abscbqlzcr3s7myf1497vw2-julia-co=
mpilersupportlibraries-jll-0.4.0%2B1
--8<---------------cut here---------------end--------------->8---
=E2=80=98guix publish=E2=80=99 replied, but 40s too late (nginx has
=E2=80=9Cproxy_connect_timeout 10s;=E2=80=9D for .narinfo URLs=C2=B9).
Notice the 40s pause time between 17:14:51 and 17:15:33. Stop-the-world
GC? Unlikely, because =E2=80=98guix publish=E2=80=99 had been running for =
~3h, so even
with a leak=C2=B2, it=E2=80=99s hard to believe GC could take this long.
Ludo=E2=80=99.
=C2=B9 https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/ng=
inx/berlin.scm#n103
=C2=B2 https://issues.guix.gnu.org/69596
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 20 Nov 2023 19:09:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Nov 20 14:09:34 2023
Received: from localhost ([127.0.0.1]:54589 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1r59eE-0003sy-7f
for submit <at> debbugs.gnu.org; Mon, 20 Nov 2023 14:09:34 -0500
Received: from mail-qv1-xf2e.google.com ([2607:f8b0:4864:20::f2e]:56486)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <maxim.cournoyer@HIDDEN>) id 1r59eA-0003sh-To
for 54447 <at> debbugs.gnu.org; Mon, 20 Nov 2023 14:09:29 -0500
Received: by mail-qv1-xf2e.google.com with SMTP id
6a1803df08f44-67089696545so14718746d6.0
for <54447 <at> debbugs.gnu.org>; Mon, 20 Nov 2023 11:09:24 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1700507359; x=1701112159; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=11jZ4im5ieJNvL4k/ZRU3abJ3P2ISyLNHHQ0sO39aNI=;
b=X+5ce9VWU3KWESqFxsmtyYuXYk7WUmnCOb//cI+pO7ZqbBPuhvsvAtLXkIMpEYg6dg
1oUCqBhnIbtVZ2jsXwl2Z+gl9mClueh+jYF7XAJxT1g4Wqjs/a60ql2GkEB4OxCvz6Ul
d70z5oZ48aYfXQpOGJ2/zQO7/wFkePEktjvieG38C+looIeEKlXVk+C2zm3TqkSdc4+v
JuWfJLeQm0p6Gu5fSFltHabKw0sF5iizGH1kuvSQ/pzfMfJZagc3x3qQG9ohrbY3e5QI
HPgaNwDqFs3Ui1E3lWBrFWPMQx9gamkrz17w7TJEbcFBjbKX4cOlSOuo9a/wpn19PIu+
3nBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1700507359; x=1701112159;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from
:to:cc:subject:date:message-id:reply-to;
bh=11jZ4im5ieJNvL4k/ZRU3abJ3P2ISyLNHHQ0sO39aNI=;
b=E0Y2P0M0G20eY0qKPE1D3CbgRY3MRCSke1nvS+Y+H9WobnSxYnHxiaEG21fZsGHJGp
oguXLS1MixebzR+C5nNoR7iB9uSbXhU68NzlMGsqa+d8R104dQnLYAtOrR/pTyutz0gt
MSJQ1csw5c0KqUGJAwc/9sMvp3Fc/jowmWS0QbNWkMzsPmADw6XmtboQVpZ1DWrz0tUR
8q9BEWbDhG8tAu6YTqH1Dwbiccu6sN8/r+IAm9XM27AZePhsMKKhQpZVsaniuw9P1mJG
rxwWYV+29N0mTAW5tmUd8faW8Fbrl7+tWupllzEgxHTnv8Dw5wlTkQ0I5DVvWNq1SPNJ
w8Gg==
X-Gm-Message-State: AOJu0YxI7OnpUBkL196Fb3wN3XJPRofgTn5AIOgQeDAk7ttBEfzODMJe
Xb9zDU3M4aHr9F76+Kk1NrVmW1bnSlw=
X-Google-Smtp-Source: AGHT+IEvzX8ybV1p+A9tibr1w3mO3OHm4j/qTQj5oHAkUgzzv8ac9Nzmw4mDpRLWRSDqzb1S/lrs3A==
X-Received: by 2002:a05:6214:622:b0:66d:a20f:ecfb with SMTP id
a2-20020a056214062200b0066da20fecfbmr10450525qvx.37.1700507358603;
Mon, 20 Nov 2023 11:09:18 -0800 (PST)
Received: from hurd (dsl-154-9.b2b2c.ca. [66.158.154.9])
by smtp.gmail.com with ESMTPSA id
q7-20020ad44027000000b00670a8921170sm3143277qvp.112.2023.11.20.11.09.17
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 20 Nov 2023 11:09:18 -0800 (PST)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87r0lv96mm.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Sun, 15 Oct 2023 18:45:37 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN>
<87y1hsu3lb.fsf@HIDDEN> <87il7detde.fsf@HIDDEN>
<87r0lv96mm.fsf@HIDDEN>
Date: Mon, 20 Nov 2023 14:09:17 -0500
Message-ID: <87zfz89r8i.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 54447
Cc: Mathieu Othacehe <othacehe@HIDDEN>,
=?utf-8?B?5a6L5paH5q2m?= <iyzsong@HIDDEN>, 54447 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi Ludovic,
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:
>
>> Another example: https://ci.guix.gnu.org/build/1982454/details
>>
>> substitute:=20
>> substitute: [Kupdating substitutes from 'http://10.0.0.1'... 0.0%
>> substitute: [Kupdating substitutes from 'http://10.0.0.1'... 100.0%
>> cannot build missing derivation ?/gnu/store/vwhgs9dkj9spryglb180j27dr5vi=
djxv-ecl-23.9.9.drv?
>
> This one is from Sep. 9, which is before I deployed the remote-worker
> fixes, so I=E2=80=99ll dismiss it (happy to look at more recent ones thou=
gh!).
Here's a more recent occurrence:
https://ci.guix.gnu.org/build/2635272/details
I haven't restarted it to leave proof of its existence :-)
--=20
Thanks,
Maxim
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 16 Oct 2023 17:45:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 16 13:45:20 2023
Received: from localhost ([127.0.0.1]:57983 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qsRea-0004gW-6Y
for submit <at> debbugs.gnu.org; Mon, 16 Oct 2023 13:45:20 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:47748)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1qsReV-0004g8-PG
for 54447 <at> debbugs.gnu.org; Mon, 16 Oct 2023 13:45:17 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1qsRe1-0004dd-C1; Mon, 16 Oct 2023 13:44:45 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=YbulGcNbx86hUS48I4/lqCPt4svSxgKo+r8/BYU+Xew=; b=Fa46tExn82IbQX56rL/b
cesX1TgRGEmHoG8uJCFdDU8yOrkoWrjMYJ4aSgGf/7p4I0d6yfaeKWgsTicuuxZ3pfSLBCs7zr+V3
UatUeg9WrnQo6l91x4btIuQ47hRr4zwfEa5D3SgvGg82YqKLsofViLZE+EJmv1CfZlNZFU6SQVGMr
UTlB9AYFqFbGDZxTh1bgeXqm8FFlDAH175oItPikf6eCYxZqlySRroF9DVseG2woKGQQMKPaLQRcI
w3VtBV75Qib47fLigb4ZZySrAFvTY7F0ABW8iRI/EDcQPVE4+pMiBx06L/jDhxXOO+kCrBEy6clUN
wyDutsAkev47Tg==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Mathieu Othacehe <othacehe@HIDDEN>
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87r0m2v5ih.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Tue, 10 Oct 2023 17:52:54 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87r0m2v5ih.fsf@HIDDEN>
Date: Mon, 16 Oct 2023 19:44:41 +0200
Message-ID: <87a5si7986.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
Cc: 54447 <at> debbugs.gnu.org, guix-sysadmin <guix-sysadmin@HIDDEN>,
Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> Turns out that =E2=80=98cleanup-cuirass-roots=E2=80=99 in maintenance.git=
, used as an
> mcron job, explicitly removes GC roots for things like *-os-encrypted
> once they=E2=80=99re more than two days old, as well as GC roots for the
> corresponding .drv.
>
> I think this was increasing the likelihood that a .drv would be GC=E2=80=
=99d by
> the time we run the test: under high load=C2=B9, it=E2=80=99s plausible t=
hat a system
> test wouldn=E2=80=99t be built within two days after it=E2=80=99s been qu=
eued.
>
> I=E2=80=99m proposing the change below to address this; I don=E2=80=99t t=
hink we need
> =E2=80=98--gc-keep-outputs --gc-keep-derivations=E2=80=99 anymore now tha=
t we keep
> things in =E2=80=98guix publish=E2=80=99 cache first and foremost.
I pushed a variant of this patch:
053839d hydra: services: Leave =E2=80=9Cguix-binary.tar.xz=E2=80=9D GC ro=
ots.
e40d961 hydra: services: Preserve Cuirass .drv GC roots.
b8fc66c hydra: cuirass: Fix build product regexps.
I didn=E2=80=99t dare remove =E2=80=9C--gc-keep-derivations=E2=80=9D. I re=
configured berlin
just now from this commit and restarted mcron (I didn=E2=80=99t restart
guix-daemon to avoid downtime; we should do that when the queue is close
to empty).
We=E2=80=99ll have to monitor disk usage to make sure it=E2=80=99s not nega=
tively
affected.
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 16 Oct 2023 17:39:45 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 16 13:39:45 2023 Received: from localhost ([127.0.0.1]:57978 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qsRZB-0004XB-HX for submit <at> debbugs.gnu.org; Mon, 16 Oct 2023 13:39:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53058) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1qsRZ8-0004Wu-Si for 54447 <at> debbugs.gnu.org; Mon, 16 Oct 2023 13:39:43 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qsRYb-0003C1-Jd; Mon, 16 Oct 2023 13:39:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=1a1RwvnYrMBtRR9fAwI0eUcTIaRTg4fi6KRCskZWgKw=; b=MAx+C0EXAhjLSjtAVhOe VLbrenl+/TL/ogl347Ej2Bo0Q+yLaE0NEISyJf/+u/yLLU9MJLNVSjqqxqbe83CXo4AcHo7GNcqHU WDp9RHbm9y568L0kI2HW7n9O8p8BNbVJ5B08haLbn7riJR6uArYYN1vp2zVsxpNiOBZur3mD0rUYQ rDqKDJmKEqpMsPHhj/hFuUhta03BO3Wq4tTHQztal2dHuBOcIovc3QfMrDl98yv2MhiK7QJHiEiQk gdLMxQl0PZsniOp7gBlJpopgTG5Ssxf8/CGG6Qfh5DSEq+hEiV8Y7fdmR4K2dYbEmWnX8gEpdHr96 IqyDiQ9K6nHIZg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error In-Reply-To: <87sf6azolb.fsf@HIDDEN> (Maxim Cournoyer's message of "Mon, 16 Oct 2023 09:25:20 -0400") References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN> <87y1hsu3lb.fsf@HIDDEN> <87il7detde.fsf@HIDDEN> <87r0lv96mm.fsf@HIDDEN> <87sf6azolb.fsf@HIDDEN> Date: Mon, 16 Oct 2023 19:39:01 +0200 Message-ID: <87edhu79hm.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54447 Cc: Mathieu Othacehe <othacehe@HIDDEN>, =?utf-8?B?5a6L5paH5q2m?= <iyzsong@HIDDEN>, 54447 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis: >> Tip of the day: M-: (build-farm-build 1982454) > > I don't have such a function in scope, is this from the guix-emacs > package? It=E2=80=99s from the =E2=80=98emacs-build-farm=E2=80=99 package, which I r= ecommend. :-) Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 16 Oct 2023 13:26:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 16 09:26:02 2023
Received: from localhost ([127.0.0.1]:55478 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qsNbb-0001E7-F1
for submit <at> debbugs.gnu.org; Mon, 16 Oct 2023 09:26:02 -0400
Received: from mail-qv1-xf2d.google.com ([2607:f8b0:4864:20::f2d]:54610)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <maxim.cournoyer@HIDDEN>) id 1qsNbV-0001Dn-72
for 54447 <at> debbugs.gnu.org; Mon, 16 Oct 2023 09:25:57 -0400
Received: by mail-qv1-xf2d.google.com with SMTP id
6a1803df08f44-65af7d102b3so30144476d6.1
for <54447 <at> debbugs.gnu.org>; Mon, 16 Oct 2023 06:25:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1697462722; x=1698067522; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=KnCksx8negqvjA75jRhd/LVb+/3xugd14+FG80MgdMM=;
b=HQhctkwB9aEtc7LENabMLNeFwP99flxQ1h1Dz61XVdYiccfOxB015ZRsVB24aXRR7d
UNYtrNnMK/FzjDdfp538B3B7tiqCwV+GDbF01qnVtoNhuvKaixUO8Tq/dD39ttP9mdDu
EBkhYP8U743pz5UduV6NK5SkUmCr07YrMdE/cYXlfrj4msyK+a/LneAVNusZlODqbdnI
wQKZ6FlCYVOG0xJbox597xGUb3IDi/N8rzn7kvZDJMjNARs1cPXl5w31lYPtUYkCT6wB
3QDIijwghoGmykPuZ80lxeonp9+R1ACA4A89XeitguCtUP+nxTQm8v2rO7hxToLTBHrL
WrVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1697462722; x=1698067522;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from
:to:cc:subject:date:message-id:reply-to;
bh=KnCksx8negqvjA75jRhd/LVb+/3xugd14+FG80MgdMM=;
b=YnTN+dkTLP/VUhVpJfFqpiydYJ/0YOSROjo6m2VB1CQwD1wFwIKDSCg9mIi7zQQCB9
FQIeeysDk3VqjbjKfUlrNrfOIWSOkUJ7j47ivhIJUqjj07MZpDvzUYAB8/1IlkSuVzk8
SIFBoue4FjpQoq1iig2ZTLIjxMUTa865AHSwzzSkGBaMHOCQNm3638Sjh83FEukGxJ/g
hw2YTPZV2Pt9lReQjDdWDpvQfI6vLnBeUT39lrKVieBHR1WmGzAR/kaJSIAODNYd3j7j
gRtYbTrFrHYak74r8ghpsZ/w4ascS8phCzVWSzknnySOkzyZFbiQSXcSk03oFJC6i0UW
mlmA==
X-Gm-Message-State: AOJu0YwuiIHa9cHAWKyfvKKL/MRMCNrDpd4Er6ZBg0AdWkn88YHqEe3S
PHvDY2ttH6z3PFjR9eUtLBB/gcf/vbU=
X-Google-Smtp-Source: AGHT+IEv/qVRkwDrgfq2C+bfOu22c22c37u3wmOCrxXddowx7ekkuztg8Qntq1ZDtPtZ29DvaBg5Lg==
X-Received: by 2002:a0c:df8a:0:b0:66d:1d3f:17cd with SMTP id
w10-20020a0cdf8a000000b0066d1d3f17cdmr9959528qvl.28.1697462722505;
Mon, 16 Oct 2023 06:25:22 -0700 (PDT)
Received: from hurd (dsl-10-149-74.b2b2c.ca. [72.10.149.74])
by smtp.gmail.com with ESMTPSA id
d28-20020a05620a137c00b00774376e6475sm2983939qkl.6.2023.10.16.06.25.21
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 16 Oct 2023 06:25:22 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87r0lv96mm.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Sun, 15 Oct 2023 18:45:37 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN>
<87y1hsu3lb.fsf@HIDDEN> <87il7detde.fsf@HIDDEN>
<87r0lv96mm.fsf@HIDDEN>
Date: Mon, 16 Oct 2023 09:25:20 -0400
Message-ID: <87sf6azolb.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 54447
Cc: Mathieu Othacehe <othacehe@HIDDEN>,
=?utf-8?B?5a6L5paH5q2m?= <iyzsong@HIDDEN>, 54447 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi,
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:
>
>> Another example: https://ci.guix.gnu.org/build/1982454/details
>>
>> substitute:=20
>> substitute: [Kupdating substitutes from 'http://10.0.0.1'... 0.0%
>> substitute: [Kupdating substitutes from 'http://10.0.0.1'... 100.0%
>> cannot build missing derivation ?/gnu/store/vwhgs9dkj9spryglb180j27dr5vi=
djxv-ecl-23.9.9.drv?
>
> This one is from Sep. 9, which is before I deployed the remote-worker
> fixes, so I=E2=80=99ll dismiss it (happy to look at more recent ones thou=
gh!).
>
> Tip of the day: M-: (build-farm-build 1982454)
I don't have such a function in scope, is this from the guix-emacs
package?
--=20
Thanks,
Maxim
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 15 Oct 2023 20:42:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 15 16:42:49 2023
Received: from localhost ([127.0.0.1]:54213 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qs7wn-0006pa-Dw
for submit <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:42:49 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:41686)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1qs7wk-0006pN-MC
for 54447 <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:42:47 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1qs7wH-0004w4-0g; Sun, 15 Oct 2023 16:42:17 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=w1Hoa2C2xh7IXwtbY0HQc1xxD9EjZ8hOE+GJPQYGYxM=; b=MIqwp0bcPKNwNvfZ0bnr
r60lu3eW+NX0s1vgg5KdY4zq9QTT4RIcQVAss8sYT+KmdL4bHPlMyc4ACQQDQ5oDfGTqWCgvP/xxM
df4ZZ6s0hmj9pMEnvHryiXqE1R/5YmBrjzX2MeaAMyKN3VLqnHBt4ih9Zl6hI81Pz3i9kz5H/w/tq
7NaXCPVIKcv8GSmHUC0nv7IOJMbj8PNKeRw3oRnVNvpKT+Iw32j+w90rIszpt0SxYtQCpsr4Lt0LI
EJhP8uLPsMYOqg2OXmWmqLy2FSlraS+AB7mTRROXImmVtSMkmTvdPdUQdETpfLCzDpvWospi3auIq
k8Mgoad/1qfvRw==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Mathieu Othacehe <othacehe@HIDDEN>
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87r0m2v5ih.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Tue, 10 Oct 2023 17:52:54 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87r0m2v5ih.fsf@HIDDEN>
Date: Sun, 15 Oct 2023 22:42:14 +0200
Message-ID: <87h6mr8vo9.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
Cc: 54447 <at> debbugs.gnu.org, guix-sysadmin <guix-sysadmin@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> In addition to the mcron job, Cuirass=E2=80=99s own =E2=80=98register-gc-=
roots=E2=80=99
> procedure periodically deletes GC roots older than =E2=80=98%gc-roots-ttl=
=E2=80=99 (30
> days in practice). That=E2=80=99s okay, except that it would be safer to=
delete
> GC roots for a .drv if and only if it=E2=80=99s been built already.
Fixed in Cuirass commit 55af0f70c0d4938b8eda777382bbc4d8f5698a37.
Ludo'.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 15 Oct 2023 20:35:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 15 16:35:03 2023
Received: from localhost ([127.0.0.1]:54207 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qs7pG-0006Zp-Nf
for submit <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:35:03 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:57722)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1qs7pB-0006ZD-NK
for 54447 <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:35:01 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qs7oh-0003Wr-U2
for 54447 <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:34:27 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=BqLVlI+X/JftPQ67U9pREKBtRUnEY3rlqQ4/mJn9k0E=; b=km1NU8eG2Fe3Dumb3env
ceNxQmTQ9XZgsa3XGRGekgweOeiKVYW1b1lKrimvAfrNud3OJRixg220Wkf+kfDu7kl5MNMXk4OOm
fYB3aJbCQGI6KCcCgpqYMzOXtMqriImnElmMrE0FdZ3WgVcoEwwarpFUpxepwS7rmLbazgEIe9Qpi
ZeN6PnYBO0k29dr4CRwD2iQFwnwxISuBjZAriezUFoW3Gp3oYMcAXMrCoP8MtN4EfpAo8/GLUqu86
7/3mnPN2kSvVzTuzFd9VXkmUWjevL4qmOrPTvdXu6pfRb2/VIMIYS0RfvwTIAEgUP1LdoIaiiGGOm
oubikQ3Qk3hhFA==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: 54447 <at> debbugs.gnu.org
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87pm1f8wm1.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Sun, 15 Oct 2023 22:21:58 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87a63v5xwd.fsf@HIDDEN>
<87pm1f8wm1.fsf@HIDDEN>
Date: Sun, 15 Oct 2023 22:34:25 +0200
Message-ID: <87lec38w1a.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> Looking at the nginx and =E2=80=98guix publish=E2=80=99 logs, I found tha=
t the missing
> substitute is not that of 4r1wij3bzj9zv75ds82a93jl7bcman2x (the .drv
> itself) but rather that of a dependency of that .drv:
>
> [14/Oct/2023:23:22:09 +0200] "GET /wqqzcxrhbnv0nzg64iiiqd5grr4vk2zg.nar=
info HTTP/1.1" 404 58 "-" "GNU Guile"
>
> That item=E2=80=99s size is above the cache bypass threshold of 100=C2=A0=
MiB as
> currently configured on berlin:
>
> $ du -hs /gnu/store/wqqzcxrhbnv0nzg64iiiqd5grr4vk2zg-guix-5a6b1a5
> 124M /gnu/store/wqqzcxrhbnv0nzg64iiiqd5grr4vk2zg-guix-5a6b1a5
>
> The immediate fix/workaround is to raise that threshold.
I raised the threshold to 150=C2=A0MiB in maintenance.git commit
213384e43de63ce3a5a55599e8fb89891ffef7eb.
I reconfigured berlin and restarted =E2=80=98guix publish=E2=80=99 seconds =
ago.
Hopefully next time installation tests won=E2=80=99t have that problem.
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 15 Oct 2023 20:22:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 15 16:22:37 2023
Received: from localhost ([127.0.0.1]:54201 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qs7dF-0006A1-9q
for submit <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:22:37 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:50234)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1qs7dB-00069j-BP
for 54447 <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:22:35 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qs7ch-0001P2-GE
for 54447 <at> debbugs.gnu.org; Sun, 15 Oct 2023 16:22:03 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=YteLRaEBQEn+MZu/DsTP2wCjUkRFz0YxHeQMyOyOBRM=; b=NyBOFbCk3Ju2tDG9Z0tv
fxmtix78KjF8AzmcK5hDDDoV3uh+UmP6db+6g/TJfTdmgC006kr7jcr3b8ON9XKSf2eXZKhhDY2Bv
ib8C+SnmQrwceaWWZlpEYwlk9QGfQNHzkWoAr5H0eEdYG7492Uk3Kb0lyl15aQxpREeIGQMSf1DF1
jMr7J5roLlDkekLjAgXDYYdi8mL6iKs9Xcy2EaC16pLW4Qlj97w5EzfdHubhxWIRkbNylWLE9A75j
zFN4sx2TeV3yPwvYeYGUWBqWDIcBY0xlA4euS5G2CyRh+xxdo28xBE4/np7xVkEe7EHFo2i7UTffv
YTizxsdRo7TlMA==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: 54447 <at> debbugs.gnu.org
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87a63v5xwd.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Sat, 10 Dec 2022 11:57:38 +0100")
References: <877d8r4etz.fsf@HIDDEN> <87a63v5xwd.fsf@HIDDEN>
Date: Sun, 15 Oct 2023 22:21:58 +0200
Message-ID: <87pm1f8wm1.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi!
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> Mathieu Othacehe <othacehe@HIDDEN> skribis:
>
>> A lot of builds, among them ~20 system tests[1], are failing with:
>> "cannot build missing derivation
>> ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-en=
crypted.drv?"
>> errors.
>>
>> Those derivations are present on the CI head node. This means that the
>> errors occur during substitution. This is most likely caused by some
>> issue with the publish server, because:
>>
>> - The publish server serves a 404 error. We should get rid once and for
>> all of this 404 thing, pushing something like:
>> https://issues.guix.gnu.org/50040.
>>
>> or
>>
>> - The publish server is not fast enough and hits an Nginx timeout that
>> closes the communication.
>
> Also being discussed at <https://issues.guix.gnu.org/48468#12>.
I got confirmation that the cache-bypass-threshold hypothesis holds, at
least for system tests.
Namely, looking at <https://ci.guix.gnu.org/build/2258097/details>,
which ends like this:
--8<---------------cut here---------------start------------->8---
@ substituter-succeeded /gnu/store/qh2876i5l1wvxgwhg9fbl9zmb3px3n2m-gc-root=
s.drv
fetching path `/gnu/store/fh9dnmrfsz429pwqmvsjnk0snlm959kc-xdg-mime-databas=
e-builder'...
@ substituter-started /gnu/store/fh9dnmrfsz429pwqmvsjnk0snlm959kc-xdg-mime-=
database-builder substitute
Downloading http://141.80.167.131/nar/lzip/fh9dnmrfsz429pwqmvsjnk0snlm959kc=
-xdg-mime-database-builder...
.=1B[K xdg-mime-database-builder 3.6MiB/s 00:00 | 3KiB t=
ransferred.=1B[K xdg-mime-database-builder 1.9MiB/s 00:0=
0 | 3KiB transferred
@ substituter-succeeded /gnu/store/fh9dnmrfsz429pwqmvsjnk0snlm959kc-xdg-mim=
e-database-builder
cannot build missing derivation =E2=80=98/gnu/store/4r1wij3bzj9zv75ds82a93j=
l7bcman2x-installed-extlinux-os.drv=E2=80=99
--8<---------------cut here---------------end--------------->8---
Looking at the nginx and =E2=80=98guix publish=E2=80=99 logs, I found that =
the missing
substitute is not that of 4r1wij3bzj9zv75ds82a93jl7bcman2x (the .drv
itself) but rather that of a dependency of that .drv:
[14/Oct/2023:23:22:09 +0200] "GET /wqqzcxrhbnv0nzg64iiiqd5grr4vk2zg.narin=
fo HTTP/1.1" 404 58 "-" "GNU Guile"
That item=E2=80=99s size is above the cache bypass threshold of 100=C2=A0Mi=
B as
currently configured on berlin:
--8<---------------cut here---------------start------------->8---
$ du -hs /gnu/store/wqqzcxrhbnv0nzg64iiiqd5grr4vk2zg-guix-5a6b1a5
124M /gnu/store/wqqzcxrhbnv0nzg64iiiqd5grr4vk2zg-guix-5a6b1a5
--8<---------------cut here---------------end--------------->8---
The immediate fix/workaround is to raise that threshold.
A better solution would be for system tests to depend on a fixed-output
derivation for the Guix source instead of the =E2=80=9Csource=E2=80=9D abov=
e (I use
=E2=80=9Csource=E2=80=9D as it is used in the context of <derivation>).
Thanks,
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 15 Oct 2023 16:46:19 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 15 12:46:19 2023 Received: from localhost ([127.0.0.1]:54040 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qs4Fu-0004s8-Nc for submit <at> debbugs.gnu.org; Sun, 15 Oct 2023 12:46:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56456) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1qs4Fs-0004rm-EK for 54447 <at> debbugs.gnu.org; Sun, 15 Oct 2023 12:46:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qs4FL-0008Lc-8A; Sun, 15 Oct 2023 12:45:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=fMqdtvs/conERw8x/oHEr+jyFeFGyP9w5wOXzhr6zCo=; b=FCAspzg7CXR2u1TF77Zl UTLnvwk4N4r7iaXcJRd/T22zMDm8aqyjBP6gOsfutb0A481ufY4wuR8qkOvFxE51DsfER1RI0k6kH z47M2NiHCs5j3t07ETJn2/m6tkfpIpKV2QsOgJU/LYFpI7YUVUn1PvPYJdyj2x0f3xhytnrzw4SPE Dn3nZQ+KWn8/GEC8E5d6+DvV3lLYd5LLCpRBdmwPP35GEbZuLYTpSRiDCn9BD/Up5Se4Ev0n7tutp UmR8XocdZzDqrzo/KFHtckUQj+JOwHzSwg1RN6KP1ITPI8+bLI60+BwsoirDWgUDT5aP/FU2dKuUC HQTQevURTFqfaw==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error In-Reply-To: <87il7detde.fsf@HIDDEN> (Maxim Cournoyer's message of "Tue, 10 Oct 2023 23:21:49 -0400") References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN> <87y1hsu3lb.fsf@HIDDEN> <87il7detde.fsf@HIDDEN> Date: Sun, 15 Oct 2023 18:45:37 +0200 Message-ID: <87r0lv96mm.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54447 Cc: Mathieu Othacehe <othacehe@HIDDEN>, =?utf-8?B?5a6L5paH5q2m?= <iyzsong@HIDDEN>, 54447 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis: > Another example: https://ci.guix.gnu.org/build/1982454/details > > substitute:=20 > substitute: [Kupdating substitutes from 'http://10.0.0.1'... 0.0% > substitute: [Kupdating substitutes from 'http://10.0.0.1'... 100.0% > cannot build missing derivation ?/gnu/store/vwhgs9dkj9spryglb180j27dr5vid= jxv-ecl-23.9.9.drv? This one is from Sep. 9, which is before I deployed the remote-worker fixes, so I=E2=80=99ll dismiss it (happy to look at more recent ones though= !). Tip of the day: M-: (build-farm-build 1982454) Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 11 Oct 2023 03:22:27 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 10 23:22:27 2023 Received: from localhost ([127.0.0.1]:37280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qqPnj-00056w-EE for submit <at> debbugs.gnu.org; Tue, 10 Oct 2023 23:22:27 -0400 Received: from mail-qv1-xf34.google.com ([2607:f8b0:4864:20::f34]:56575) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maxim.cournoyer@HIDDEN>) id 1qqPne-00056Z-EP for 54447 <at> debbugs.gnu.org; Tue, 10 Oct 2023 23:22:22 -0400 Received: by mail-qv1-xf34.google.com with SMTP id 6a1803df08f44-664bd97692dso41432076d6.0 for <54447 <at> debbugs.gnu.org>; Tue, 10 Oct 2023 20:21:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696994511; x=1697599311; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t2nTN4dTrgsdza5edto8LPo7ReV9CHr+HK2gR65eVLY=; b=CxA40oKwshxKs6+Mo/w5JXxyOs/YdzPKEtDXuKwMPCMXw2CIWzPPELxhWaWG4SLFM0 v0qrwZGiQzUCjJD2wSQVLC2DZ0bh+rutIhqZa1X1cqKnK6NAB05C1yxsPYgdeuSx04P7 hNa20+5hEfZQZAY+9oX91/YwgyVOIjANejKuRUfGUUWdeEeaiIUvKabXKcSjk9D3FjKF gjFyyfh6a+E7tS0ZgVxbnvLuFXQqlgsWfSKj1cS9ij2BjD5e5aQzvPc6t24dyqFDJ/Zw J/ky9K3IA2gmnn2PRNXLugWC46hnwSIZjr16LQp//c1aQU2Knd8DV143+WGZb9Bh/oop aqrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696994511; x=1697599311; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=t2nTN4dTrgsdza5edto8LPo7ReV9CHr+HK2gR65eVLY=; b=XW6hUkGNpEbnNUdCoBij/kdUN/LLhOLBuXpN08eQ/VTWKZw9k0K/FJwrLSc3IYVsTD 7H5QdPEZpCl9BrSSOL0SA71OFPzAGBfPRCMzjlugYjw4S5kjpHtwbEDeVdFGBm5lYCs+ xeqolmCfG9EzV/5gRGeN08aqSL7TI6fkQ93exdhTRzsU+axYcXMM/Hj8Gayt/+OVkcdz zrJ7KDwcqT3zQBTkfhEVcXJ0MmpH9l1g2Cwc0eTSwLcFlYjyTo4REOLJtURSkxu4UVzO UhrvnGxw4f25A//8/x9SBAk/YTNR5tB5YEsVJ5l0wxA49gB+yPP0tluFQZcPQuQqnJH7 zymg== X-Gm-Message-State: AOJu0YxSIvFqmVKaK8nIubMpl4Py66kN8f+LHzm/ijyWv9pYijo7f/gZ n4vVVbKkB5qt1d9KwpefxjY= X-Google-Smtp-Source: AGHT+IGF1pa0ePia6d/CKGTRjBrEXJWt9Kuve+H9+CL36q/o19Bh+aJ6zXCCkBE/fLYqQ+F6r/4Hyg== X-Received: by 2002:a0c:b295:0:b0:66c:e3b2:13f4 with SMTP id r21-20020a0cb295000000b0066ce3b213f4mr5901027qve.65.1696994511198; Tue, 10 Oct 2023 20:21:51 -0700 (PDT) Received: from hurd (dsl-10-149-16.b2b2c.ca. [72.10.149.16]) by smtp.gmail.com with ESMTPSA id n4-20020a0ce544000000b0065b0d9b4ee7sm5342589qvm.20.2023.10.10.20.21.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Oct 2023 20:21:50 -0700 (PDT) From: Maxim Cournoyer <maxim.cournoyer@HIDDEN> To: =?utf-8?B?5a6L5paH5q2m?= <iyzsong@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error In-Reply-To: <87y1hsu3lb.fsf@HIDDEN> (=?utf-8?B?IuWui+aWh+atpiIncw==?= message of "Wed, 30 Aug 2023 20:17:20 +0800") References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN> <87y1hsu3lb.fsf@HIDDEN> Date: Tue, 10 Oct 2023 23:21:49 -0400 Message-ID: <87il7detde.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54447 Cc: Mathieu Othacehe <othacehe@HIDDEN>, =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>, 54447 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, =E5=AE=8B=E6=96=87=E6=AD=A6 <iyzsong@HIDDEN> writes: [...] > Hello, this one for ddd: https://ci.guix.gnu.org/build/1372655/log/raw > > cannot build missing derivation ?/gnu/store/anzz2p18b7r9x45y350avnk8br2= yihi2-ddd-3.4.0.drv? > > Restart it on CI still got the same error. Another example: https://ci.guix.gnu.org/build/1982454/details --8<---------------cut here---------------start------------->8--- substitute:=20 substitute: [Kupdating substitutes from 'http://10.0.0.1'... 0.0% substitute: [Kupdating substitutes from 'http://10.0.0.1'... 100.0% cannot build missing derivation ?/gnu/store/vwhgs9dkj9spryglb180j27dr5vidjx= v-ecl-23.9.9.drv? --8<---------------cut here---------------end--------------->8--- --=20 Thanks, Maxim
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 11 Oct 2023 03:08:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 10 23:08:47 2023
Received: from localhost ([127.0.0.1]:37272 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qqPaY-0004cR-PZ
for submit <at> debbugs.gnu.org; Tue, 10 Oct 2023 23:08:47 -0400
Received: from mail-qk1-x732.google.com ([2607:f8b0:4864:20::732]:55709)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <maxim.cournoyer@HIDDEN>) id 1qqPaT-0004c8-Qo
for 54447 <at> debbugs.gnu.org; Tue, 10 Oct 2023 23:08:45 -0400
Received: by mail-qk1-x732.google.com with SMTP id
af79cd13be357-774105e8c37so417035385a.3
for <54447 <at> debbugs.gnu.org>; Tue, 10 Oct 2023 20:08:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1696993694; x=1697598494; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=7embXUi8ltgGfUVF8tM7ff61ZEFkLwGfqPtxhrsSbGk=;
b=cYPq4qySscl2j1vD1Pmq8flmCQROFYNa2RwFcMatEd8SCaw2r9cRUmc8NBesUvWhMa
sEK8KgkvWOIubhtbSdnAtNC8pztJa75OVZYq54gXzbZmwI6nXOnfz4jt5GXw6bdqJS7e
c52IsiuQkeCvTpjzWk1XdJYqnrER/YD7GiaMlV+HwpdLLzdkczWg2XQdylrzRxkgZa5G
necQSnrv6R3erMYvM/P4l+Gs7xY8mHaCsVsN8jikc5ZCXMcoVv0h8hhn4Ppr//h3TlPJ
52hftBjzUJHGZHbSwmpi/y/3LDbenFHXcSV2bDQjoo0ScPbha3+NfFkHjfLMWdOUtYpW
1TOw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1696993694; x=1697598494;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from
:to:cc:subject:date:message-id:reply-to;
bh=7embXUi8ltgGfUVF8tM7ff61ZEFkLwGfqPtxhrsSbGk=;
b=XkwiljdYa/jUzC42i1Bs5/P+F8L9ZfHLNcjshRmrVUnbts89WeZt8FWOfBwJ70E9Fi
rjvlAPOeIeHHM85/cxSzWfoqY/TUBrPbO6TlkmXuQ/xbtXxtCSYRzcMUUbIC05odEAfr
LKpZuS9mmrlyZNptAPWKMlf9v2t+ZneYxjxf5rTgrDXHlvFWhDjOqfuSpXDPs8tQfjPR
+hbtbhv/gJpniTVN8zqYtuOkyVF/eZO3VnM5Y9OB1A0X3hTsFtblzoiyLuNHmWbFMLLQ
FcWwMt21vBl01PwWvnMoztJLysmKTkfcIDl+IhzdNT80HSzjVHIA3jp8RerB+sxB8IFf
v0Wg==
X-Gm-Message-State: AOJu0Yzsad2i1muK90xATOu8JiKyF2S11fQU4fMy8ajLRflVzbWnfmmN
c7+qsYffQHY6bGdN5vgzbgwQ31mQipYEgw==
X-Google-Smtp-Source: AGHT+IHgf9b5lNIIhw/VPYl9f16YyZPdUVsAHZP45AwMp8qoNb+8vJmMdn4KznkkdTQKGUknjrqZzg==
X-Received: by 2002:a05:620a:47b6:b0:765:9391:13e7 with SMTP id
dt54-20020a05620a47b600b00765939113e7mr19980335qkb.49.1696993694557;
Tue, 10 Oct 2023 20:08:14 -0700 (PDT)
Received: from hurd (dsl-10-149-16.b2b2c.ca. [72.10.149.16])
by smtp.gmail.com with ESMTPSA id
c27-20020a05620a165b00b0076cc0a6e127sm4838216qko.116.2023.10.10.20.08.13
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Tue, 10 Oct 2023 20:08:13 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <87r0m2v5ih.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Tue, 10 Oct 2023 17:52:54 +0200")
References: <877d8r4etz.fsf@HIDDEN> <87r0m2v5ih.fsf@HIDDEN>
Date: Tue, 10 Oct 2023 23:08:12 -0400
Message-ID: <87mswpeu03.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 54447
Cc: Mathieu Othacehe <othacehe@HIDDEN>, guix-sysadmin <guix-sysadmin@HIDDEN>,
54447 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi Ludovic,
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Hello!
>
> Mathieu Othacehe <othacehe@HIDDEN> skribis:
>
>> A lot of builds, among them ~20 system tests[1], are failing with:
>> "cannot build missing derivation
>> ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-en=
crypted.drv?"
>> errors.
>
> I have a disappointingly simple hypothesis for this. Remember that
> =E2=80=9Cmissing derivation=E2=80=9D errors happen primarily for system t=
ests.
>
> Turns out that =E2=80=98cleanup-cuirass-roots=E2=80=99 in maintenance.git=
, used as an
> mcron job, explicitly removes GC roots for things like *-os-encrypted
> once they=E2=80=99re more than two days old, as well as GC roots for the
> corresponding .drv.
>
> I think this was increasing the likelihood that a .drv would be GC=E2=80=
=99d by
> the time we run the test: under high load=C2=B9, it=E2=80=99s plausible t=
hat a system
> test wouldn=E2=80=99t be built within two days after it=E2=80=99s been qu=
eued.
>
> I=E2=80=99m proposing the change below to address this; I don=E2=80=99t t=
hink we need
> =E2=80=98--gc-keep-outputs --gc-keep-derivations=E2=80=99 anymore now tha=
t we keep
> things in =E2=80=98guix publish=E2=80=99 cache first and foremost.
>
> Thoughts?
Ah, so that mcron job is kind of a hack to hasten garbage collecting
only *some* items faster than the default policy of 30 days? And we'd
now avoid deleting selected .drv files while still deleting their
outputs, so in the case something that needs it took more than 2 days to
build, it could lead to having to rebuild the garbage collected outputs?
I'm not sure if we need such a fancy hack with the 100 TiB of data we
now have, but your fix seems reasonable (LGTM!)
> In addition to the mcron job, Cuirass=E2=80=99s own =E2=80=98register-gc-=
roots=E2=80=99
> procedure periodically deletes GC roots older than =E2=80=98%gc-roots-ttl=
=E2=80=99 (30
> days in practice). That=E2=80=99s okay, except that it would be safer to=
delete
> GC roots for a .drv if and only if it=E2=80=99s been built already.
Hm. I wonder if this could explain the other cases we've seen. It
could be that building a derivation was interrupted or canceled for some
reason, then 30 days elapsed, then was garbage collected, and after
which it doesn't get recreated and we get the error of the missing .drv?
--=20
Thanks,
Maxim
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 10 Oct 2023 15:53:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 10 11:53:28 2023
Received: from localhost ([127.0.0.1]:36744 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qqF32-0006aU-0S
for submit <at> debbugs.gnu.org; Tue, 10 Oct 2023 11:53:28 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:59882)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1qqF2z-0006aG-8x
for 54447 <at> debbugs.gnu.org; Tue, 10 Oct 2023 11:53:26 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1qqF2X-00006l-C8; Tue, 10 Oct 2023 11:52:58 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=UGjkdV5U8aqc0r2Z2Nt5UyDLOqFbO4V1MEP7zwdyLH8=; b=NFc6HSpX9DLrCFZYeeSu
5yQK0gNsTI/KEHx6yTZnoQ7DRXesYT5iL1A2tEc1rP/rLdS2doEm82L1GzMiyEOTZFDUmrJFIcZMk
l5l+0uVJ/v+IB/TJ357q5eqA/OoiLsKQCZRTi5tYajG0eDfaK1DJWCy0Wc4fKi0eTdaf2AQRQEUxQ
6pxX8Bbf9ISJuXbv/9KUPNF2cqDUkdUPRXXm1rfHKe3YUSu0CMAQh9CyPHgKLkgTGcl+Jf/SS27no
pylNn0qxC9Z6OQCafz0xp9hLzvYfnV2gBBr+YczPYDq62J8mUfcn9LbsyQFyDk1lcnjAbdheVEWgf
F7dFNjl7r7MtXg==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Mathieu Othacehe <othacehe@HIDDEN>
Subject: Re: bug#54447: cuirass: missing derivation error
In-Reply-To: <877d8r4etz.fsf@HIDDEN> (Mathieu Othacehe's message of "Fri, 18
Mar 2022 13:36:56 +0100")
References: <877d8r4etz.fsf@HIDDEN>
Date: Tue, 10 Oct 2023 17:52:54 +0200
Message-ID: <87r0m2v5ih.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 54447
Cc: 54447 <at> debbugs.gnu.org, guix-sysadmin <guix-sysadmin@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hello!
Mathieu Othacehe <othacehe@HIDDEN> skribis:
> A lot of builds, among them ~20 system tests[1], are failing with:
> "cannot build missing derivation
> ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-enc=
rypted.drv?"
> errors.
I have a disappointingly simple hypothesis for this. Remember that
=E2=80=9Cmissing derivation=E2=80=9D errors happen primarily for system tes=
ts.
Turns out that =E2=80=98cleanup-cuirass-roots=E2=80=99 in maintenance.git, =
used as an
mcron job, explicitly removes GC roots for things like *-os-encrypted
once they=E2=80=99re more than two days old, as well as GC roots for the
corresponding .drv.
I think this was increasing the likelihood that a .drv would be GC=E2=80=99=
d by
the time we run the test: under high load=C2=B9, it=E2=80=99s plausible tha=
t a system
test wouldn=E2=80=99t be built within two days after it=E2=80=99s been queu=
ed.
I=E2=80=99m proposing the change below to address this; I don=E2=80=99t thi=
nk we need
=E2=80=98--gc-keep-outputs --gc-keep-derivations=E2=80=99 anymore now that =
we keep
things in =E2=80=98guix publish=E2=80=99 cache first and foremost.
Thoughts?
In addition to the mcron job, Cuirass=E2=80=99s own =E2=80=98register-gc-ro=
ots=E2=80=99
procedure periodically deletes GC roots older than =E2=80=98%gc-roots-ttl=
=E2=80=99 (30
days in practice). That=E2=80=99s okay, except that it would be safer to d=
elete
GC roots for a .drv if and only if it=E2=80=99s been built already.
Thanks,
Ludo=E2=80=99.
=C2=B9 The queue was often processed slowly, with many workers remaining id=
le
due to the bug fixed by
<https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=3D40f=
70d28aed55c404cca6a0760860fb4942e6bee>.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline
diff --git a/hydra/modules/sysadmin/services.scm b/hydra/modules/sysadmin/services.scm
index fecfdde..e6f2b44 100644
--- a/hydra/modules/sysadmin/services.scm
+++ b/hydra/modules/sysadmin/services.scm
@@ -110,9 +110,7 @@
((guix config) => ,(make-config.scm)))
#~(begin
(use-modules (ice-9 ftw)
- (srfi srfi-1)
- (guix store)
- (guix derivations))
+ (srfi srfi-1))
(define %roots-directory
"/var/guix/profiles/per-user/cuirass/cuirass")
@@ -157,28 +155,6 @@
deleted))
deleted))
- (define (root-target root)
- ;; Return the store item ROOT refers to.
- (string-append (%store-prefix) "/" (basename root)))
-
- (define (derivation-referrers store item)
- ;; Return the referrers of the derivers of ITEM.
- (let* ((derivers (valid-derivers store item))
- (referrers (append-map (lambda (drv)
- (referrers store drv))
- derivers)))
- (delete-duplicates referrers)))
-
- (define (delete-gc-root-for-derivation drv)
- ;; Delete the GC root for DRV, if any.
- (catch 'system-error
- (lambda ()
- (let ((item (derivation-path->output-path drv)))
- (delete-file
- (string-append %roots-directory
- "/" (basename drv)))))
- (const #f)))
-
;; Note: 'scandir' would introduce too much overhead due
;; to the large number of entries that it would sort.
(define deleted
@@ -197,17 +173,7 @@
(for-each (lambda (file)
(display file port)
(newline port))
- deleted)))
-
- ;; Since we run 'guix-daemon --gc-keep-outputs
- ;; --gc-keep-derivations', also remove GC roots for the outputs of
- ;; derivations that refer to the derivers of DELETED.
- (for-each delete-gc-root-for-derivation
- (with-store store
- (append-map (lambda (root)
- (derivation-referrers
- store (root-target root)))
- deleted))))))))
+ deleted))))))))
(define (gc-jobs threshold)
"Return the garbage collection mcron jobs. The garbage collection
@@ -251,8 +217,7 @@ collection instead."
(build-accounts (* build-accounts-to-max-jobs-ratio max-jobs))
(extra-options (list "--max-jobs" (number->string max-jobs)
- "--cores" (number->string cores)
- "--gc-keep-outputs" "--gc-keep-derivations"))))
+ "--cores" (number->string cores)))))
;;;
--=-=-=--
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 30 Aug 2023 12:17:17 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Aug 30 08:17:17 2023 Received: from localhost ([127.0.0.1]:52319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qbK8I-0000Lu-7Y for submit <at> debbugs.gnu.org; Wed, 30 Aug 2023 08:17:17 -0400 Received: from mail.envs.net ([5.199.136.28]:46412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <iyzsong@HIDDEN>) id 1qbK8D-0000Li-30 for 54447 <at> debbugs.gnu.org; Wed, 30 Aug 2023 08:17:13 -0400 Received: from localhost (mail.envs.net [127.0.0.1]) by mail.envs.net (Postfix) with ESMTP id 3CEB438A0132; Wed, 30 Aug 2023 12:17:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=envs.net; s=modoboa; t=1693397820; bh=98VAu8Pl5QrVjbaOadfFZ/M4ewDkzCmCoSU9lW0jvHs=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=Bm1/xvDf+wt16chsg3zrADYjG/Etbjtne4TH61DMbAuk74rs0+l07oAL2y9Njs6Dt MsjifV6R7JTFTGpgHkGZIoYEtuLkHmNTxejf9TUQLE2skXg/fvpoZqZU5U+XmVfeK0 E9OgdSGW/Yy1PKPrbeCjTclB1h1J6qmgRLRlWE6YCpYLMEuT78kA1O5/RlvhrUVN4/ JNxlAEPx4tFRtubdsV3Ox9tvOYET7Qmf5ytjU/kDziavZ2sr/9+KzPI3F5cIw62kdl 9+0d7QgMbxUTjHcsRwUKAe5o7BAOBNFhvcRkA/pUd7GL0knPp8ZSPGZF64+o8IJEl/ NxCZcclZ5wzqPtRtV9x/PbaU7DGz/l+znM9wXG10qpYp+ycvngPG2C4gL+mqtwn+/j ZC1KkeX9/qtV0c0G++YgkgnvPFaCIa+6RPi16+lQyibLar9v35JsEsgdUg/DoHe2IB rHP8EmPkLeyHoegLl4ffdcwQEpLmgHBkPKl4/QCjthHMnw22yAaGfheaq4A3aYt11U bJOxWebu/ROCsEsXf427ePqznEIbP/Mq102unEf0v8hS9aBrLu3SHXGT8di5ouyhLZ PhBE/yEpHucRMvoJFPmBk3CLFehX4+P4pspeHvPFkGdIYLAxqsxevlJw+AItWTIufi JB+zlOuhBhIKINOwGqhjemek= X-Virus-Scanned: Debian amavisd-new at mail.envs.net Received: from mail.envs.net ([127.0.0.1]) by localhost (mail.envs.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id xBYOWCD33kKj; Wed, 30 Aug 2023 12:16:55 +0000 (UTC) Received: from localhost (unknown [36.170.58.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.envs.net (Postfix) with ESMTPSA; Wed, 30 Aug 2023 12:16:54 +0000 (UTC) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 4d6a6491; Wed, 30 Aug 2023 12:17:20 +0000 (UTC) From: =?utf-8?B?5a6L5paH5q2m?= <iyzsong@HIDDEN> To: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN> Date: Wed, 30 Aug 2023 20:17:20 +0800 In-Reply-To: <87ttsrbvb2.fsf@HIDDEN> (Maxim Cournoyer's message of "Mon, 21 Aug 2023 23:38:41 -0400") Message-ID: <87y1hsu3lb.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54447 Cc: Mathieu Othacehe <othacehe@HIDDEN>, 54447 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Maxim Cournoyer <maxim.cournoyer@HIDDEN> writes: > I wonder if these could be related to the DDoS protection discovered on > the Berlin network. I'll keep looking for other, potentially different > occurrences. Hello, this one for ddd: https://ci.guix.gnu.org/build/1372655/log/raw cannot build missing derivation ?/gnu/store/anzz2p18b7r9x45y350avnk8br2yihi2-ddd-3.4.0.drv? Restart it on CI still got the same error.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 22 Aug 2023 20:38:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Aug 22 16:38:39 2023 Received: from localhost ([127.0.0.1]:60532 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qYY98-0007oX-VC for submit <at> debbugs.gnu.org; Tue, 22 Aug 2023 16:38:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58456) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1qYY97-0007oJ-0r for 54447 <at> debbugs.gnu.org; Tue, 22 Aug 2023 16:38:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qYY8x-0005yo-F3; Tue, 22 Aug 2023 16:38:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=tD3HwwjFtgIWtqWi0chE3uT0gee9vD6wDipY4IMC9XE=; b=TkNRXQ/RSx68eruuwdtQ jk50RlL/fdm98RT/zwV2RW8U6ocWgyFL45gZSxE78WOJDss2VbwdA75aPfHr3THPKV2A5hNh2qqAs LBZSlthCKV76xiue8AQqlkN/xJ4wCItIUZCcLmwLWwh3QmFzTVoQpCNOA/E1tFspaJYwu4FgUF05h Ah2IoabZCAkFKxGzCIcjIKfcLvHTldl45UuQBDChUOCicl0zIvn38qZ5L40nMSrjArMy4RfuNniE2 WVGwYdwEouOjwR8xy7tl7l63QC2t5l7cqzpacs0E/NhMoTu1t/gsqRlnTKdGjm2/X6VFYaPgU+Act vUd/VOW5DiBS+w==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error References: <877d8r4etz.fsf@HIDDEN> <87ttsrbvb2.fsf@HIDDEN> Date: Tue, 22 Aug 2023 22:38:24 +0200 In-Reply-To: <87ttsrbvb2.fsf@HIDDEN> (Maxim Cournoyer's message of "Mon, 21 Aug 2023 23:38:41 -0400") Message-ID: <871qfu4xtr.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54447 Cc: Mathieu Othacehe <othacehe@HIDDEN>, 54447 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis: > Looking at multiple of recent 'cannot build missing derivation' build > failures on Cuirass, I see for example: > > substitute:=20 > substitute: [Kupdating substitutes from 'http://141.80.167.131'... 0.0% > substitute: [Kcould not fetch http://141.80.167.131/rhgrs3ac6h64siz0krqh2= ia8kkn3h6ym.narinfo 504 > substitute: updating substitutes from 'http://141.80.167.131'... 100.0% > cannot build missing derivation ?/gnu/store/rhgrs3ac6h64siz0krqh2ia8kkn3h= 6ym-python-asdf-standard-1.0.3.drv? > > > So it seems the error originated from guix-publish being too heavily > under load to produce a timely reply, and the nginx proxy issued a 504 > (timeout) error response. > > Looking into /var/log/guix-publish.log for a corresponding entry, I > found: > > 2023-08-21 23:59:35 GET /rhgrs3ac6h64siz0krqh2ia8kkn3h6ym.narinfo > 2023-08-21 23:59:35 In web/server/http.scm: > 2023-08-21 23:59:35 159:7 2 (http-write #<<http-server> socket: #<in= put-output: fi=E2=80=A6> =E2=80=A6) > 2023-08-21 23:59:35 In unknown file: > 2023-08-21 23:59:35 1 (put-bytevector #<input-output: socket 4= 2> #vu8(83 # =E2=80=A6) =E2=80=A6) > 2023-08-21 23:59:35 In ice-9/boot-9.scm: > 2023-08-21 23:59:35 1685:16 0 (raise-exception _ #:continuable? _) > 2023-08-21 23:59:35 In procedure fport_write: Broken pipe > > > So the connection was apparently severed (?), resulting in the "broken > pipe" error. I think it=E2=80=99s just that, when =E2=80=98guix publish=E2=80=99 eventua= lly replied, the client had left, hence EPIPE. The initial problem does look like =E2=80=98guix publish=E2=80=99 being too= slow. Do the corresponding nginx logs confirm the =E2=80=9Cbackend too slow =3D> 504= =E2=80=9D hypothesis? Thanks for investigating! Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 22 Aug 2023 03:38:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 21 23:38:54 2023 Received: from localhost ([127.0.0.1]:58222 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qYIEI-00087O-Fs for submit <at> debbugs.gnu.org; Mon, 21 Aug 2023 23:38:54 -0400 Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]:46537) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maxim.cournoyer@HIDDEN>) id 1qYIEG-00087B-3r for 54447 <at> debbugs.gnu.org; Mon, 21 Aug 2023 23:38:53 -0400 Received: by mail-qk1-x734.google.com with SMTP id af79cd13be357-76d7bcefe8bso264400585a.1 for <54447 <at> debbugs.gnu.org>; Mon, 21 Aug 2023 20:38:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692675523; x=1693280323; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ZRD1hTZ8AGpwGrr9aAi/kGzPIafQBRxxssm+cstrHlU=; b=B0L3LRwdSp8U+nC+twjbno0+vIHr7s+GQQIuHhzA0IK6sArxwJgpgyfjaGFs46hHWj FbcyInPA78BfOSz6hPip5yBxVUwJcFXrHuAfMC8/MHk043Uafdb8/nPmlZUAtW/aqHmn JfAO4Te5DKpyFuWg71O0FafNbIWEuRN086qYMAqhrRrYTWN3+l2Mxdd6SBHlAILSafJH Fu0P4bjLBoDlNiDou1GIARsrMOnO4kuRdNTM8IBDmAU0tXEpH+DYRdWtEC3AeC5TPgia cCJBgPZXV+mH4QYF29rY7UDDd62V1MSXgREfpDPhDvX0stbMphSDCz0CZq1tqec0s9+Y 4wqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692675523; x=1693280323; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZRD1hTZ8AGpwGrr9aAi/kGzPIafQBRxxssm+cstrHlU=; b=UbYAZFPebBOJ5pSsy7TmYijz8B/pPSDYTkmftspyKYlCIaQCzBVa4K7oLaiLdhIqfR tDUmUB+aKKJVnfLZygJ2vyqZXF93Si0KI2Bo0d0tv8+DIJ7dMnrQc37sfUtx3qtps2Zw ws63uQHrRcvkeoCCBCsnvQSLOMAiooV2EmZoas75c572CN+p0s7YP+/BBCAR2dfxyuML Zi4M4Wgtg0lVvMZydPVTxWqPW4R7DqBh5NDrLGcF0RpcSUlThBf/ttBTgK8jiog02IkK NXPf1scpnHCEbgp3d2T1vVolVOJKrvFO5Nxzap0nUBKGoxe40HdhptLrI0Kb1RYQK6jU IJbw== X-Gm-Message-State: AOJu0YyU5jpIa/8kECMhwuFIlN6YsBS34hHeHr0s9T21yjkxa4C+OKpP DDEeuUdttbKAXpXgRjraOSrOMmrWmdk= X-Google-Smtp-Source: AGHT+IHXDDgoxmWk4jpQreMim3eLRTHhXjhiejoEJSK1S1+8KhmKDpVZOMIVx74NjQCqQEAm/stG/Q== X-Received: by 2002:a05:620a:b58:b0:76d:a70f:1bb4 with SMTP id x24-20020a05620a0b5800b0076da70f1bb4mr4424389qkg.66.1692675523571; Mon, 21 Aug 2023 20:38:43 -0700 (PDT) Received: from hurd (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id c1-20020a05620a200100b0076ca401d8c7sm2953675qka.111.2023.08.21.20.38.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 20:38:42 -0700 (PDT) From: Maxim Cournoyer <maxim.cournoyer@HIDDEN> To: Mathieu Othacehe <othacehe@HIDDEN> Subject: Re: bug#54447: cuirass: missing derivation error References: <877d8r4etz.fsf@HIDDEN> Date: Mon, 21 Aug 2023 23:38:41 -0400 In-Reply-To: <877d8r4etz.fsf@HIDDEN> (Mathieu Othacehe's message of "Fri, 18 Mar 2022 13:36:56 +0100") Message-ID: <87ttsrbvb2.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54447 Cc: 54447 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Mathieu Othacehe <othacehe@HIDDEN> writes: > Hello, > > A lot of builds, among them ~20 system tests[1], are failing with: > "cannot build missing derivation > ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-enc= rypted.drv?" > errors. > > Those derivations are present on the CI head node. This means that the > errors occur during substitution. This is most likely caused by some > issue with the publish server, because: > > - The publish server serves a 404 error. We should get rid once and for > all of this 404 thing, pushing something like: > https://issues.guix.gnu.org/50040. > > or > > - The publish server is not fast enough and hits an Nginx timeout that > closes the communication. > > Any other cause I could be missing? Looking at multiple of recent 'cannot build missing derivation' build failures on Cuirass, I see for example: --8<---------------cut here---------------start------------->8--- substitute:=20 substitute: [Kupdating substitutes from 'http://141.80.167.131'... 0.0% substitute: [Kcould not fetch http://141.80.167.131/rhgrs3ac6h64siz0krqh2ia= 8kkn3h6ym.narinfo 504 substitute: updating substitutes from 'http://141.80.167.131'... 100.0% cannot build missing derivation ?/gnu/store/rhgrs3ac6h64siz0krqh2ia8kkn3h6y= m-python-asdf-standard-1.0.3.drv? --8<---------------cut here---------------end--------------->8--- So it seems the error originated from guix-publish being too heavily under load to produce a timely reply, and the nginx proxy issued a 504 (timeout) error response. Looking into /var/log/guix-publish.log for a corresponding entry, I found: --8<---------------cut here---------------start------------->8--- 2023-08-21 23:59:35 GET /rhgrs3ac6h64siz0krqh2ia8kkn3h6ym.narinfo 2023-08-21 23:59:35 In web/server/http.scm: 2023-08-21 23:59:35 159:7 2 (http-write #<<http-server> socket: #<inpu= t-output: fi=E2=80=A6> =E2=80=A6) 2023-08-21 23:59:35 In unknown file: 2023-08-21 23:59:35 1 (put-bytevector #<input-output: socket 42>= #vu8(83 # =E2=80=A6) =E2=80=A6) 2023-08-21 23:59:35 In ice-9/boot-9.scm: 2023-08-21 23:59:35 1685:16 0 (raise-exception _ #:continuable? _) 2023-08-21 23:59:35 In procedure fport_write: Broken pipe --8<---------------cut here---------------end--------------->8--- So the connection was apparently severed (?), resulting in the "broken pipe" error. Here's a different one: --8<---------------cut here---------------start------------->8--- substitute:=20 substitute: [Kupdating substitutes from 'http://141.80.167.131'... 0.0% substitute: [Kcould not fetch http://141.80.167.131/p2lfyvbxicjqsm4qp6368bx= 76gp0g948.narinfo 504 substitute: updating substitutes from 'http://141.80.167.131'... 100.0% cannot build missing derivation ?/gnu/store/p2lfyvbxicjqsm4qp6368bx76gp0g94= 8-python-astropy-healpix-0.7.drv? --8<---------------cut here---------------end--------------->8--- it occurred around the same time, and the failing mode was the same, per guix-publish.log: --8<---------------cut here---------------start------------->8--- 2023-08-21 23:59:35 GET /p2lfyvbxicjqsm4qp6368bx76gp0g948.narinfo 2023-08-21 23:59:35 In web/server/http.scm: 2023-08-21 23:59:35 159:7 2 (http-write #<<http-server> socket: #<inpu= t-output: fi=E2=80=A6> =E2=80=A6) 2023-08-21 23:59:35 In unknown file: 2023-08-21 23:59:35 1 (put-bytevector #<input-output: socket 50>= #vu8(83 # =E2=80=A6) =E2=80=A6) 2023-08-21 23:59:35 In ice-9/boot-9.scm: 2023-08-21 23:59:35 1685:16 0 (raise-exception _ #:continuable? _) 2023-08-21 23:59:35 In procedure fport_write: Broken pipe --8<---------------cut here---------------end--------------->8--- I wonder if these could be related to the DDoS protection discovered on the Berlin network. I'll keep looking for other, potentially different occurrences. --=20 Thanks, Maxim
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Received: (at 54447) by debbugs.gnu.org; 10 Dec 2022 10:57:48 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Dec 10 05:57:48 2022 Received: from localhost ([127.0.0.1]:42093 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1p3xYB-00068x-QI for submit <at> debbugs.gnu.org; Sat, 10 Dec 2022 05:57:48 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55490) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1p3xY9-00068n-NZ for 54447 <at> debbugs.gnu.org; Sat, 10 Dec 2022 05:57:46 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1p3xY4-0002j5-G3 for 54447 <at> debbugs.gnu.org; Sat, 10 Dec 2022 05:57:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=7liOGJjUcZYoZqYbGopnCSFp5uqmtY0YQKmLjvJ2DuM=; b=ZZctsGZMRaPTdWNXNcjQ e+gNhhJ7YwjtBnQMP+1+wuVd+AkvxuLlEkkpYNZIlANlW/I9nx3pgx3KGaCD+iVtSkDl9dzMV6mtK eObL7xUCB+GFKHCNGNwoQp1HIXeoYJhEbbwQeCJILsgfr715DYFVu8A1RLwaUlDt9Vp6oyYH2RJx5 Muy6/N16TmJGueTlajCv4brziOm8ZylLdAm31jZbGUbO1TUm+GA29yolgPcbNAgzUxRM8F0Jej33D 2bzDMCdENNd+LKAzGVl7JJPXhrWeBQa5KfTrypcw5dQ1shDfSl1yQUdWdUt0yAI85EnThTmNSuqSS l9Pk/hsMC9Uvcw==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1p3xY3-0004hg-Ki for 54447 <at> debbugs.gnu.org; Sat, 10 Dec 2022 05:57:39 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: 54447 <at> debbugs.gnu.org Subject: Re: bug#54447: cuirass: missing derivation error References: <877d8r4etz.fsf@HIDDEN> Date: Sat, 10 Dec 2022 11:57:38 +0100 In-Reply-To: <877d8r4etz.fsf@HIDDEN> (Mathieu Othacehe's message of "Fri, 18 Mar 2022 13:36:56 +0100") Message-ID: <87a63v5xwd.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54447 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Mathieu Othacehe <othacehe@HIDDEN> skribis: > A lot of builds, among them ~20 system tests[1], are failing with: > "cannot build missing derivation > ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-enc= rypted.drv?" > errors. > > Those derivations are present on the CI head node. This means that the > errors occur during substitution. This is most likely caused by some > issue with the publish server, because: > > - The publish server serves a 404 error. We should get rid once and for > all of this 404 thing, pushing something like: > https://issues.guix.gnu.org/50040. > > or > > - The publish server is not fast enough and hits an Nginx timeout that > closes the communication. Also being discussed at <https://issues.guix.gnu.org/48468#12>. Ludo=E2=80=99.
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
Received: (at 54447) by debbugs.gnu.org; 10 Aug 2022 15:30:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Aug 10 11:30:44 2022
Received: from localhost ([127.0.0.1]:50498 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1oLnfQ-0003A1-28
for submit <at> debbugs.gnu.org; Wed, 10 Aug 2022 11:30:44 -0400
Received: from andre.telenet-ops.be ([195.130.132.53]:54906)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <maximedevos@HIDDEN>) id 1oLnfN-00039p-83
for 54447 <at> debbugs.gnu.org; Wed, 10 Aug 2022 11:30:42 -0400
Received: from [IPV6:2a02:1811:8c09:9d00:5dba:d409:33f7:a16]
([IPv6:2a02:1811:8c09:9d00:5dba:d409:33f7:a16])
by andre.telenet-ops.be with bizsmtp
id 5rWd2800920ykKC01rWdMF; Wed, 10 Aug 2022 17:30:39 +0200
Message-ID: <66c88669-b72b-d1a0-613c-abb346cf73e7@HIDDEN>
Date: Wed, 10 Aug 2022 17:30:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
Thunderbird/91.12.0
Subject: Re: cuirass: missing derivation error
Content-Language: en-US
From: Maxime Devos <maximedevos@HIDDEN>
To: 54447 <at> debbugs.gnu.org
References: <877d8r4etz.fsf@HIDDEN>
<99749a50-a9db-6437-901a-be8834dd651e@HIDDEN>
In-Reply-To: <99749a50-a9db-6437-901a-be8834dd651e@HIDDEN>
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature";
boundary="------------ZFgW2pf04mfnYTtzAwUQVX2L"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22;
t=1660145439; bh=aLuLI/9ozTk+Vr8AL65NfixU32FBMZ+Y38uEYkljN9E=;
h=Date:Subject:From:To:References:In-Reply-To;
b=h95jfhr83jgPC1UY6dq/wxARq23Hvb+Rs+1YH+d3w/ou1KwtZ4qmRW5gOE2j8/IbX
UMyeYMZMgwMa8gAz1eGIwsAwc7vzp3rtgZHqwbrOUitUfroSJgdTLQkgK7dPsC0gmo
sAsxubJRqJi3GLMHzH5/EW0tEJzGnywhZkiL0DMcO6Eym/aVDAKrLVJb6AW7Fr//pY
Ck1M6QS3iV+JC/mhtVVWYxLUEF8G7CwrROYJiPwHGyq2JgSme/2j1ByAIfjq0DlQPR
1zCYc8sgq7w6D4S84nsCyliPf3Xu+vpgqMxs7juvPdFyYUfrN0Pg3nr8kGS67WNwrM
xee8IdCrfbHsg==
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 54447
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------ZFgW2pf04mfnYTtzAwUQVX2L
Content-Type: multipart/mixed; boundary="------------n92DCcY0e1G0Q9SnCO0TZCE0";
protected-headers="v1"
From: Maxime Devos <maximedevos@HIDDEN>
To: 54447 <at> debbugs.gnu.org
Message-ID: <66c88669-b72b-d1a0-613c-abb346cf73e7@HIDDEN>
Subject: Re: cuirass: missing derivation error
References: <877d8r4etz.fsf@HIDDEN>
<99749a50-a9db-6437-901a-be8834dd651e@HIDDEN>
In-Reply-To: <99749a50-a9db-6437-901a-be8834dd651e@HIDDEN>
--------------n92DCcY0e1G0Q9SnCO0TZCE0
Content-Type: multipart/mixed; boundary="------------zi4w7aQY7P9H2jXzgIUSoW04"
--------------zi4w7aQY7P9H2jXzgIUSoW04
Content-Type: multipart/alternative;
boundary="------------aZPBnMVf70RPXjqAupaidDIg"
--------------aZPBnMVf70RPXjqAupaidDIg
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64
DQpPbiAxMC0wOC0yMDIyIDExOjQzLCBNYXhpbWUgRGV2b3Mgd3JvdGU6DQo+IEhlcmUncyBh
bm90aGVyIGluc3RhbmNlOiBodHRwczovL2NpLmd1aXguZ251Lm9yZy9ldmFsLzUyODcxMA0K
Pg0KTW9yZSBpbmZvcm1hdGlvbjoNCg0KICAqIG5vbi1BU0NJSSBkb2VzIG5vdCBzZWVtIHRv
IGJlIHNldCB1cCAoc2VlOiA/KSAobG9va3MgaXJyZWxldmFudCkNCiAgKiBoZXJlIGFyZSBj
b25uZWN0aW9uIGZhaWx1cmVzDQoNCkxvZzoNCg0KPiBzdWJzdGl0dXRlOg0KPiBzdWJzdGl0
dXRlOiAbW0t1cGRhdGluZyBzdWJzdGl0dXRlcyBmcm9tICdodHRwOi8vMTQxLjgwLjE2Ny4x
MzEnLi4uICAgMC4wJWd1aXggc3Vic3RpdHV0ZTogd2FybmluZzogMTQxLjgwLjE2Ny4xMzE6
IGNvbm5lY3Rpb24gZmFpbGVkOiBDb25uZWN0aW9uIHJlZnVzZWQNCj4gc3Vic3RpdHV0ZToN
Cj4gY2Fubm90IGJ1aWxkIG1pc3NpbmcgZGVyaXZhdGlvbiA/L2dudS9zdG9yZS80Z3FqMmJ5
dmo5enozMHd6dndrYmlqcHlhM3ZuMWJqdy1ydXN0LWRvZ2dlZC0wLjIuMC5kcnY/DQoNCkdy
ZWV0aW5ncywNCk1heGltZS4NCg==
--------------aZPBnMVf70RPXjqAupaidDIg
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DUTF=
-8">
</head>
<body>
<p><br>
</p>
<div class=3D"moz-cite-prefix">On 10-08-2022 11:43, Maxime Devos
wrote:<br>
</div>
<blockquote type=3D"cite"
cite=3D"mid:99749a50-a9db-6437-901a-be8834dd651e@HIDDEN">Here's=
another instance: <a class=3D"moz-txt-link-freetext" href=3D"https:=
//ci.guix.gnu.org/eval/528710">https://ci.guix.gnu.org/eval/528710</a>
<br>
<br>
</blockquote>
<p>More information:</p>
<ul>
<li>non-ASCII does not seem to be set up (see: ?) (looks
irrelevant)<br>
</li>
<li>here are connection failures</li>
</ul>
<p>Log:<br>
</p>
<pre><blockquote type=3D"cite"><pre>substitute:=20
substitute: =1B[Kupdating substitutes from '<a class=3D"moz-txt-link-free=
text" href=3D"http://141.80.167.131">http://141.80.167.131</a>'... 0.0%=
guix substitute: warning: 141.80.167.131: connection failed: Connection r=
efused
substitute:=20
cannot build missing derivation ?/gnu/store/4gqj2byvj9zz30wzvwkbijpya3vn1=
bjw-rust-dogged-0.2.0.drv?</pre></blockquote>
</pre>
Greetings,<br>
Maxime.<br>
</body>
</html>
--------------aZPBnMVf70RPXjqAupaidDIg--
--------------zi4w7aQY7P9H2jXzgIUSoW04
Content-Type: application/pgp-keys; name="OpenPGP_0x49E3EE22191725EE.asc"
Content-Disposition: attachment; filename="OpenPGP_0x49E3EE22191725EE.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable
-----BEGIN PGP PUBLIC KEY BLOCK-----
xjMEX4ch6BYJKwYBBAHaRw8BAQdANPb/d6MrGnGi5HyvODCkBUJPRjiFQcRU5V+m
xvMaAa/NL01heGltZSBEZXZvcyA8bWF4aW1lLmRldm9zQHN0dWRlbnQua3VsZXV2
ZW4uYmU+wpAEExYIADgWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCX4ch6AIbAwUL
CQgHAwUVCgkICwUWAgMBAAIeAQIXgAAKCRBJ4+4iGRcl7japAQC3opZ2KGWzWmRc
/gIWSu0AAcfMwyinFEEPa/QhUt2CogD/e2RdF4CYAgaRHJJmZ9WU7piKbLZ7llB4
LzgezVDHggzNJU1heGltZSBEZXZvcyA8bWF4aW1lZGV2b3NAdGVsZW5ldC5iZT7C
kAQTFggAOBYhBMHzPuIMUo/bfdcBH0nj7iIZFyXuBQJf56ycAhsDBQsJCAcDBRUK
CQgLBRYCAwEAAh4BAheAAAoJEEnj7iIZFyXujpQBAKV1SwDDl4f24rXciDlB9L8W
ycZt30CgbewMSRQk4mvbAP9dFMbVVixYBd6C8cfhR+NsOBGiOJnQABlUmgNuqGFJ
Dc44BF+HIegSCisGAQQBl1UBBQEBB0BOlzIWiJzgobMF6/cqwLaLk7jIcFSZ++c0
k9cCNT6YXwMBCAfCeAQYFggAIBYhBMHzPuIMUo/bfdcBH0nj7iIZFyXuBQJfhyHo
AhsMAAoJEEnj7iIZFyXuMr0BAJc8cl5PGvVmVuSQVKjleNl4DK1/XAaPAYPe34AE
fZJPAP9IqLCQhH/FeJanHqBP8gNdGNI2qn8RnnLVfRJgUjZ1BA=3D=3D
=3DOVqp
-----END PGP PUBLIC KEY BLOCK-----
--------------zi4w7aQY7P9H2jXzgIUSoW04--
--------------n92DCcY0e1G0Q9SnCO0TZCE0--
--------------ZFgW2pf04mfnYTtzAwUQVX2L
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"
-----BEGIN PGP SIGNATURE-----
wnsEABYIACMWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYvPPHQUDAAAAAAAKCRBJ4+4iGRcl7i1F
AQCm8dDfJhYRlPAW06na8YaAFDEGiMTUQLPPbtofLO8iFgD/UfGv/mXQWJYlMvMCWWr5KCnvN7bi
TZFPH3n8WCndEA0=
=/ws3
-----END PGP SIGNATURE-----
--------------ZFgW2pf04mfnYTtzAwUQVX2L--
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at 54447) by debbugs.gnu.org; 10 Aug 2022 09:43:56 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Aug 10 05:43:56 2022 Received: from localhost ([127.0.0.1]:46118 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1oLiFZ-00015O-5N for submit <at> debbugs.gnu.org; Wed, 10 Aug 2022 05:43:56 -0400 Received: from baptiste.telenet-ops.be ([195.130.132.51]:38394) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1oLiFV-00015C-73 for 54447 <at> debbugs.gnu.org; Wed, 10 Aug 2022 05:43:40 -0400 Received: from [IPV6:2a02:1811:8c09:9d00:5dba:d409:33f7:a16] ([IPv6:2a02:1811:8c09:9d00:5dba:d409:33f7:a16]) by baptiste.telenet-ops.be with bizsmtp id 5ljZ2800A20ykKC01ljawB; Wed, 10 Aug 2022 11:43:35 +0200 Message-ID: <99749a50-a9db-6437-901a-be8834dd651e@HIDDEN> Date: Wed, 10 Aug 2022 11:43:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.12.0 Content-Language: en-US To: 54447 <at> debbugs.gnu.org From: Maxime Devos <maximedevos@HIDDEN> Subject: cuirass: missing derivation error Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------quqOvwD0IkUvLRQGX3SajPS9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1660124615; bh=/rIC9G1gxDUyGnNVciUvuSFGg0jFTyevk4PAWjOa7Z0=; h=Date:To:From:Subject; b=DzfqVXfW+WRVXPH2o/Lr8IbYsRoX0Ctou9oC/68LfnU1ldAo8jBVqwVjc9ccP6FHI nljKFzUUIH29qrVqgKwrN9rQ+vx/Fr9nLA6lzdL9PSVLbPbsTiSxb+yANMdTIWxA4Y oDYBCpX1vZErZtdj+2CmWaBTf5LN4l0b2oDHv+NwaaHdDEl+YKCpZzs+I1Wb8IYwuU Yr3ZkRNi50TXXE7X9mpMyclPKFhPOzZvVYzjE9GXHAARo1vzo9z8k/SJ6UdxZ7qTff aN3i4DMqQ8ghbsWGNFjADLnWu2PeVUUpgkZ2pMK+n/+fYw0636gU3bVaq37YXFQAc7 Z3C9a/sifkQJw== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 54447 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------quqOvwD0IkUvLRQGX3SajPS9 Content-Type: multipart/mixed; boundary="------------nMZxPmIhuDH03MWoMaszAnbX"; protected-headers="v1" From: Maxime Devos <maximedevos@HIDDEN> To: 54447 <at> debbugs.gnu.org Message-ID: <99749a50-a9db-6437-901a-be8834dd651e@HIDDEN> Subject: cuirass: missing derivation error --------------nMZxPmIhuDH03MWoMaszAnbX Content-Type: multipart/mixed; boundary="------------r6o2sy0ul2lrmUQhnyALR8kv" --------------r6o2sy0ul2lrmUQhnyALR8kv Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 SGVyZSdzIGFub3RoZXIgaW5zdGFuY2U6IGh0dHBzOi8vY2kuZ3VpeC5nbnUub3JnL2V2YWwv NTI4NzEwDQoNCg== --------------r6o2sy0ul2lrmUQhnyALR8kv Content-Type: application/pgp-keys; name="OpenPGP_0x49E3EE22191725EE.asc" Content-Disposition: attachment; filename="OpenPGP_0x49E3EE22191725EE.asc" Content-Description: OpenPGP public key Content-Transfer-Encoding: quoted-printable -----BEGIN PGP PUBLIC KEY BLOCK----- xjMEX4ch6BYJKwYBBAHaRw8BAQdANPb/d6MrGnGi5HyvODCkBUJPRjiFQcRU5V+m xvMaAa/NL01heGltZSBEZXZvcyA8bWF4aW1lLmRldm9zQHN0dWRlbnQua3VsZXV2 ZW4uYmU+wpAEExYIADgWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCX4ch6AIbAwUL CQgHAwUVCgkICwUWAgMBAAIeAQIXgAAKCRBJ4+4iGRcl7japAQC3opZ2KGWzWmRc /gIWSu0AAcfMwyinFEEPa/QhUt2CogD/e2RdF4CYAgaRHJJmZ9WU7piKbLZ7llB4 LzgezVDHggzNJU1heGltZSBEZXZvcyA8bWF4aW1lZGV2b3NAdGVsZW5ldC5iZT7C kAQTFggAOBYhBMHzPuIMUo/bfdcBH0nj7iIZFyXuBQJf56ycAhsDBQsJCAcDBRUK CQgLBRYCAwEAAh4BAheAAAoJEEnj7iIZFyXujpQBAKV1SwDDl4f24rXciDlB9L8W ycZt30CgbewMSRQk4mvbAP9dFMbVVixYBd6C8cfhR+NsOBGiOJnQABlUmgNuqGFJ Dc44BF+HIegSCisGAQQBl1UBBQEBB0BOlzIWiJzgobMF6/cqwLaLk7jIcFSZ++c0 k9cCNT6YXwMBCAfCeAQYFggAIBYhBMHzPuIMUo/bfdcBH0nj7iIZFyXuBQJfhyHo AhsMAAoJEEnj7iIZFyXuMr0BAJc8cl5PGvVmVuSQVKjleNl4DK1/XAaPAYPe34AE fZJPAP9IqLCQhH/FeJanHqBP8gNdGNI2qn8RnnLVfRJgUjZ1BA=3D=3D =3DOVqp -----END PGP PUBLIC KEY BLOCK----- --------------r6o2sy0ul2lrmUQhnyALR8kv-- --------------nMZxPmIhuDH03MWoMaszAnbX-- --------------quqOvwD0IkUvLRQGX3SajPS9 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYvN9xQUDAAAAAAAKCRBJ4+4iGRcl7swo AQCTbOMaOo8Zrw/Yx9QYY0go8RovpaMWr7HpAITIgcZwTgD9FeXXbAIq4VhWPmA4zUfmSlDtI5El Bka/HThMKVjxJAE= =2lfZ -----END PGP SIGNATURE----- --------------quqOvwD0IkUvLRQGX3SajPS9--
bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.Received: (at submit) by debbugs.gnu.org; 18 Mar 2022 12:37:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 18 08:37:01 2022 Received: from localhost ([127.0.0.1]:56496 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nVBqm-0001A3-Ow for submit <at> debbugs.gnu.org; Fri, 18 Mar 2022 08:37:00 -0400 Received: from lists.gnu.org ([209.51.188.17]:55526) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <othacehe@HIDDEN>) id 1nVBql-00019u-Kn for submit <at> debbugs.gnu.org; Fri, 18 Mar 2022 08:36:59 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37000) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1nVBql-0008CX-E9 for bug-guix@HIDDEN; Fri, 18 Mar 2022 08:36:59 -0400 Received: from [2001:470:142:3::e] (port=45650 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1nVBql-000258-4f for bug-guix@HIDDEN; Fri, 18 Mar 2022 08:36:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=7ldhER1phncD8wtCLZF1/kF77C4t4ZAnLFkZYrGKabQ=; b=Hnz09RZyhoSjpq oNQeZm7r4TJZnrqQex2absCpsLm5CFZHnIiYhajbdZOgxIcuG8EH5E+IuMSVJLAkGvi9EGrLgzSEa EdsyTOCeMCreQTLd1hzV+szFKl4wIZywPA0SLjEJQyyeykWYCPqy03UwuDTPG40+6k+szmtqXzWm1 b8p5eiZ3nuzLMG37dh430om7BGZNqI0Tdi4r381QweEOwA3NHKJDAsLZFj1EymicYYU+vXazItp9K rAUgg3oMbCZk9/2DBHzpCxVv7Bt60E9WFGr4YAtsFDJxrgx574d/AfrxpAuneKZZ403HjcWPz6Fao 9XOUcLqxxcCNGCOnGeJA==; Received: from 71.125.192.77.rev.sfr.net ([77.192.125.71]:59572 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1nVBqk-0001xb-OE for bug-guix@HIDDEN; Fri, 18 Mar 2022 08:36:58 -0400 From: Mathieu Othacehe <othacehe@HIDDEN> To: bug-guix@HIDDEN Subject: cuirass: missing derivation error Date: Fri, 18 Mar 2022 13:36:56 +0100 Message-ID: <877d8r4etz.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hello, A lot of builds, among them ~20 system tests[1], are failing with: "cannot build missing derivation ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-encrypted.drv?" errors. Those derivations are present on the CI head node. This means that the errors occur during substitution. This is most likely caused by some issue with the publish server, because: - The publish server serves a 404 error. We should get rid once and for all of this 404 thing, pushing something like: https://issues.guix.gnu.org/50040. or - The publish server is not fast enough and hits an Nginx timeout that closes the communication. Any other cause I could be missing? Thanks, Mathieu [1]: https://ci.guix.gnu.org/eval/159975?status=failed
Mathieu Othacehe <othacehe@HIDDEN>:bug-guix@HIDDEN.
Full text available.bug-guix@HIDDEN:bug#54447; Package guix.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.