GNU bug report logs -
#67250
builtin:git-download capability detection not working for the bordeaux build farm
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 67250 in the body.
You can then email your comments to 67250 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Fri, 17 Nov 2023 21:45:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Christopher Baines <mail <at> cbaines.net>
:
New bug report received and forwarded. Copy sent to
bug-guix <at> gnu.org
.
(Fri, 17 Nov 2023 21:45:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
The bordeaux build farm depends on computing the derivations on one
machine, then potentially building them on a different machine.
Some of the build machines don't have a new enough guix-daemon that
understands builtin:git-download, so derivations that use this are
sometimes failing (e.g. [1])
1: https://bordeaux.guix.gnu.org/build/10cc5622-6b1f-4f28-ad9a-41cf796d7a15
One potential approach to address this is somehow have the data service
indicate that it wants compatible derivations when computing them,
rather than to have guix do feature detection against the guix-daemon
that is being used at the point the derivations are being computed.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Mon, 20 Nov 2023 10:23:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 67250 <at> debbugs.gnu.org (full text, mbox):
Hi Chris,
On Fri, 17 Nov 2023 at 21:39, Christopher Baines <mail <at> cbaines.net> wrote:
> The bordeaux build farm depends on computing the derivations on one
> machine, then potentially building them on a different machine.
>
> Some of the build machines don't have a new enough guix-daemon that
> understands builtin:git-download, so derivations that use this are
> sometimes failing (e.g. [1])
Do you mean that the drv file is generated using a new daemon then this
drv file is “offloaded” to another machine using an older daemon and
thus fails to build it?
> One potential approach to address this is somehow have the data service
> indicate that it wants compatible derivations when computing them,
> rather than to have guix do feature detection against the guix-daemon
> that is being used at the point the derivations are being computed.
Other solutions are: downgrade the daemon of one of the machine or
update the daemon of other machines, no?
Why would the latter not be possible?
Cheers
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Mon, 20 Nov 2023 10:32:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 67250 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Simon Tournier <zimon.toutoune <at> gmail.com> writes:
> Hi Chris,
>
> On Fri, 17 Nov 2023 at 21:39, Christopher Baines <mail <at> cbaines.net> wrote:
>
>> The bordeaux build farm depends on computing the derivations on one
>> machine, then potentially building them on a different machine.
>>
>> Some of the build machines don't have a new enough guix-daemon that
>> understands builtin:git-download, so derivations that use this are
>> sometimes failing (e.g. [1])
>
> Do you mean that the drv file is generated using a new daemon then this
> drv file is “offloaded” to another machine using an older daemon and
> thus fails to build it?
Roughly, yep.
>> One potential approach to address this is somehow have the data service
>> indicate that it wants compatible derivations when computing them,
>> rather than to have guix do feature detection against the guix-daemon
>> that is being used at the point the derivations are being computed.
>
> Other solutions are: downgrade the daemon of one of the machine or
> update the daemon of other machines, no?
>
> Why would the latter not be possible?
It's possible, just difficult. Because with the current guix-daemon, the
build coordinator can only build things that can be GC'd, some machines
deliberately run older daemons to allow building newer things without
hitting this issue.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Wed, 22 Nov 2023 10:20:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 67250 <at> debbugs.gnu.org (full text, mbox):
Hi,
Christopher Baines <mail <at> cbaines.net> skribis:
> The bordeaux build farm depends on computing the derivations on one
> machine, then potentially building them on a different machine.
>
> Some of the build machines don't have a new enough guix-daemon that
> understands builtin:git-download, so derivations that use this are
> sometimes failing (e.g. [1])
>
> 1: https://bordeaux.guix.gnu.org/build/10cc5622-6b1f-4f28-ad9a-41cf796d7a15
>
> One potential approach to address this is somehow have the data service
> indicate that it wants compatible derivations when computing them,
> rather than to have guix do feature detection against the guix-daemon
> that is being used at the point the derivations are being computed.
Would it work for you if we added a keyword argument to
‘open-connection’ in (guix store) that would let you select which
builtins to use?
As in:
(open-connection
#:assume-available-builtin-builders '("download"))
Ludo’.
Severity set to 'important' from 'normal'
Request was from
Ludovic Courtès <ludo <at> gnu.org>
to
control <at> debbugs.gnu.org
.
(Wed, 22 Nov 2023 10:21:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Tue, 28 Nov 2023 15:44:04 GMT)
Full text and
rfc822 format available.
Message #19 received at 67250 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Wed, 22 Nov 2023 at 11:19, Ludovic Courtès <ludo <at> gnu.org> wrote:
> As in:
>
> (open-connection
> #:assume-available-builtin-builders '("download"))
Instead, why not check in ’git-fetch’? Currently, the test is done
against the local daemon, right?
--8<---------------cut here---------------start------------->8---
(define* (git-fetch ref hash-algo hash
#:optional name
#:key (system (%current-system))
guile git)
"Return a fixed-output derivation that fetches REF, a <git-reference>
object. The output is expected to have recursive hash HASH of type
HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f."
(mlet %store-monad ((builtins (built-in-builders*)))
(if (member "git-download" builtins)
(git-fetch/built-in ref hash-algo hash name
#:system system)
(git-fetch/in-band ref hash-algo hash name
#:system system
#:guile guile
#:git git))))
--8<---------------cut here---------------end--------------->8---
For example, why not a test as:
(if (and SOMETHING
(member "git-download" builtins))
And this SOMETHING could be a global variable, as
%assume-builtin-builder, set to true by default and turn to false with
an environment variable, as GUIX_ASSUME_BUILTIN.
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Tue, 28 Nov 2023 16:57:01 GMT)
Full text and
rfc822 format available.
Message #22 received at 67250 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes:
> Hi,
>
> Christopher Baines <mail <at> cbaines.net> skribis:
>
>> The bordeaux build farm depends on computing the derivations on one
>> machine, then potentially building them on a different machine.
>>
>> Some of the build machines don't have a new enough guix-daemon that
>> understands builtin:git-download, so derivations that use this are
>> sometimes failing (e.g. [1])
>>
>> 1: https://bordeaux.guix.gnu.org/build/10cc5622-6b1f-4f28-ad9a-41cf796d7a15
>>
>> One potential approach to address this is somehow have the data service
>> indicate that it wants compatible derivations when computing them,
>> rather than to have guix do feature detection against the guix-daemon
>> that is being used at the point the derivations are being computed.
>
> Would it work for you if we added a keyword argument to
> ‘open-connection’ in (guix store) that would let you select which
> builtins to use?
>
> As in:
>
> (open-connection
> #:assume-available-builtin-builders '("download"))
Yep, that would at least allow freezing the available builtins going
forward and separating it from the running daemon.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Sat, 18 May 2024 13:24:02 GMT)
Full text and
rfc822 format available.
Message #25 received at 67250 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Christopher Baines <mail <at> cbaines.net> writes:
> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Hi,
>>
>> Christopher Baines <mail <at> cbaines.net> skribis:
>>
>>> The bordeaux build farm depends on computing the derivations on one
>>> machine, then potentially building them on a different machine.
>>>
>>> Some of the build machines don't have a new enough guix-daemon that
>>> understands builtin:git-download, so derivations that use this are
>>> sometimes failing (e.g. [1])
>>>
>>> 1: https://bordeaux.guix.gnu.org/build/10cc5622-6b1f-4f28-ad9a-41cf796d7a15
>>>
>>> One potential approach to address this is somehow have the data service
>>> indicate that it wants compatible derivations when computing them,
>>> rather than to have guix do feature detection against the guix-daemon
>>> that is being used at the point the derivations are being computed.
>>
>> Would it work for you if we added a keyword argument to
>> ‘open-connection’ in (guix store) that would let you select which
>> builtins to use?
>>
>> As in:
>>
>> (open-connection
>> #:assume-available-builtin-builders '("download"))
>
> Yep, that would at least allow freezing the available builtins going
> forward and separating it from the running daemon.
I've now sent some patches which add this option to #71038.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#67250
; Package
guix
.
(Sat, 18 May 2024 13:33:01 GMT)
Full text and
rfc822 format available.
Message #28 received at 67250 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Simon Tournier <zimon.toutoune <at> gmail.com> writes:
> On Wed, 22 Nov 2023 at 11:19, Ludovic Courtès <ludo <at> gnu.org> wrote:
>
>> As in:
>>
>> (open-connection
>> #:assume-available-builtin-builders '("download"))
>
> Instead, why not check in ’git-fetch’? Currently, the test is done
> against the local daemon, right?
>
> --8<---------------cut here---------------start------------->8---
> (define* (git-fetch ref hash-algo hash
> #:optional name
> #:key (system (%current-system))
> guile git)
> "Return a fixed-output derivation that fetches REF, a <git-reference>
> object. The output is expected to have recursive hash HASH of type
> HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f."
> (mlet %store-monad ((builtins (built-in-builders*)))
> (if (member "git-download" builtins)
> (git-fetch/built-in ref hash-algo hash name
> #:system system)
> (git-fetch/in-band ref hash-algo hash name
> #:system system
> #:guile guile
> #:git git))))
> --8<---------------cut here---------------end--------------->8---
>
> For example, why not a test as:
>
> (if (and SOMETHING
> (member "git-download" builtins))
>
> And this SOMETHING could be a global variable, as
> %assume-builtin-builder, set to true by default and turn to false with
> an environment variable, as GUIX_ASSUME_BUILTIN.
I can think of a couple of reasons not to do this via an environment
variable.
Having the logic in git-fetch via an environment variable would mean
that if other builtins are added, the code generating derivations using
them would also have to check the environment variable. Specifying the
builtin's that are available on a store connection would work
automatically in the case of new builtins being added..
There's also trade-offs in control when using environment variables, or
deciding when to take the value from an environment variable and switch
to passing the value around through arguments or other means. Consider
the case where you have a single process which has multiple store
connections open and it wants to specify different available builtin's
on particular store connections. This would become quite difficult when
using an environment variable, but work easily when specifying the
builtins available on the store connection.
[signature.asc (application/pgp-signature, inline)]
Reply sent
to
Ludovic Courtès <ludo <at> gnu.org>
:
You have taken responsibility.
(Fri, 06 Sep 2024 20:54:01 GMT)
Full text and
rfc822 format available.
Notification sent
to
Christopher Baines <mail <at> cbaines.net>
:
bug acknowledged by developer.
(Fri, 06 Sep 2024 20:54:02 GMT)
Full text and
rfc822 format available.
Message #33 received at 67250-done <at> debbugs.gnu.org (full text, mbox):
Christopher Baines <mail <at> cbaines.net> skribis:
> I've now sent some patches which add this option to #71038.
These were committed as f002371767df04238e3fc6dfacb6930f5529144b.
Closing,
Ludo’.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Sat, 05 Oct 2024 11:24:14 GMT)
Full text and
rfc822 format available.
This bug report was last modified 218 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.