GNU bug report logs - #38232
Cuirass: Some builds fail although their log file ends with 'build-succeeded'

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 38232 in the body.
You can then email your comments to 38232 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: bug-guix <at> gnu.org
Subject: Cuirass: Some builds fail although their log file ends with
 'build-succeeded'
Date: Sat, 16 Nov 2019 19:26:06 +0100

See https://ci.guix.info/build/1934383/details.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: bug-guix <at> gnu.org
Cc: 38232 <at> debbugs.gnu.org
Subject: Re: bug#38232: Cuirass: Some builds fail although their log file ends
 with 'build-succeeded'
Date: Sun, 17 Nov 2019 09:27:45 +0100

Hello,

It seems to be also an issue for evaluations that look successful but
are reported as failed, see here:

https://ci.guix.gnu.org/eval/8641/log/raw -> mark as failed
https://ci.guix.gnu.org/eval/8725/log/raw -> mark as failed

Mathieu

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: bug-guix <at> gnu.org
Cc: 38232 <at> debbugs.gnu.org
Subject: Re: bug#38232: Cuirass: Some builds fail although their log file ends
 with 'build-succeeded'
Date: Sun, 17 Nov 2019 11:56:41 +0100

> It seems to be also an issue for evaluations that look successful but
> are reported as failed, see here:

Looking at those evaluation logs, I think there is yet another issue. On
core-updates:

* Evaluation 8747 fails because it cannot build curl
(https://ci.guix.gnu.org/eval/8748/log/raw).

* Next evaluation 8762, considers curl build as previously failed:
--8<---------------cut here---------------start------------->8---
Computing Guix derivation for 'x86_64-linux'...  builder for `/gnu/store/1510sg9h2ivq4841gqm9dhmq5l50a52b-curl-7.66.0' failed previously (cached)
@ build-failed /gnu/store/pqfnxfk5xx0ngc32wlq87vhchh80mnkw-curl-7.66.0.drv - cached--8<---------------cut here---------------end--------------->8---
and fails immediately.

The curl build problem (at 8747 evaluation) seems related to some failed
test case (1242) that I cannot reproduce locally.

Will all future evaluations re-use this cache and also fail immediately?
If yes, how can we unblock the situation?

Mathieu

Message #20 received at 38232 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Clément Lassieur <clement <at> lassieur.org>
Cc: 38232 <at> debbugs.gnu.org
Subject: Re: bug#38232: Cuirass: Some builds fail although their log file ends
 with 'build-succeeded'
Date: Sun, 17 Nov 2019 22:24:08 +0100

Hi Clément,

Clément Lassieur <clement <at> lassieur.org> skribis:

> See https://ci.guix.info/build/1934383/details.

Perhaps the build failed (transient failure or something) but was
eventually restarted either as a dependency of some other build or
manually on berlin.  In that case Cuirass may be unaware that the build
eventually succeeded, hence the discrepancy.

Thanks,
Ludo’.

Message #23 received at 38232 <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 38232 <at> debbugs.gnu.org
Subject: Re: bug#38232: Cuirass: Some builds fail although their log file ends
 with 'build-succeeded'
Date: Mon, 18 Nov 2019 18:03:46 +0100

Hi Ludovic,

> Perhaps the build failed (transient failure or something) but was
> eventually restarted either as a dependency of some other build or
> manually on berlin.  In that case Cuirass may be unaware that the build
> eventually succeeded, hence the discrepancy.

Thank you for the explanation.  Would it make sense to copy the log file
to some other place when the build is done?  The web interface would
fetch it there, and we would be sure it matches the associated Cuirass
build.

Clément

Message #26 received at 38232 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Clément Lassieur <clement <at> lassieur.org>
Cc: 38232 <at> debbugs.gnu.org
Subject: Re: bug#38232: Cuirass: Some builds fail although their log file ends
 with 'build-succeeded'
Date: Mon, 18 Nov 2019 21:22:20 +0100

Hello,

Clément Lassieur <clement <at> lassieur.org> skribis:

>> Perhaps the build failed (transient failure or something) but was
>> eventually restarted either as a dependency of some other build or
>> manually on berlin.  In that case Cuirass may be unaware that the build
>> eventually succeeded, hence the discrepancy.
>
> Thank you for the explanation.  Would it make sense to copy the log file
> to some other place when the build is done?  The web interface would
> fetch it there, and we would be sure it matches the associated Cuirass
> build.

Build logs are kept by guix-daemon under /var/log/guix/drvs.  They are
indexed by derivation, meaning that there can be only one log per
derivation.

This is a limitation in cases where builds are retried.  Initially
Mathieu Lirzin thought about keeping logs elsewhere so we could
distinguish between several attempts to build a derivation.  However,
that’s kind of redundant with what the daemon does, and not frequently
useful.

Now, it is indeed useful in some cases, so we could do something like
you describe.

Thanks,
Ludo’.

Message #31 received at 38232-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: 38232-done <at> debbugs.gnu.org
Subject: Re: bug#38232: Cuirass: Some builds fail although their log file
 ends with 'build-succeeded'
Date: Thu, 25 Mar 2021 14:06:42 +0100

Hello,

This is no longer an issue on Cuirass master.

Thanks,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.