GNU bug report logs - #58903
[PATCH] gnu: nix: Update to 2.11.1.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 58903 in the body.
You can then email your comments to 58903 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: phodina <phodina <at> protonmail.com>
To: Guix Patches <guix-patches <at> gnu.org>
Subject: [PATCH] gnu: nix: Update to 2.11.1.
Date: Sun, 30 Oct 2022 20:45:24 +0000

[Message part 1 (text/plain, inline)]

Hi,

here's update of Nix daemon. Unfortunately, there is one test that doesn't return the location '/nix/store' so it's substituted.

----
Petr

[Message part 2 (text/html, inline)]

[0001-gnu-nix-Update-to-2.11.1.patch (text/x-patch, attachment)]

Message #8 received at 58903 <at> debbugs.gnu.org (full text, mbox):

From: phodina <phodina <at> protonmail.com>
To: "58903 <at> debbugs.gnu.org" <58903 <at> debbugs.gnu.org>
Cc: "go.wigust <at> gmail.com" <go.wigust <at> gmail.com>,
 Tobias Geerinckx-Rice <me <at> tobias.gr>, "i <at> pengmeiyu.com" <i <at> pengmeiyu.com>
Subject: Specify the build dir for Nix
Date: Mon, 31 Oct 2022 15:37:09 +0000

[Message part 1 (text/plain, inline)]

Hello,

I'd like to upstream this patch to Nix service and hear your thoughts on the matter.

The issue is that there are some packages that fail to build even on realtively powerful machine due to resource exhaustion - not enough RAM.

The solution for that is to extract and build the derivation on block device instead of tmpfs.

The solution for this is to specify the environment variable TMPDIR​ and point it e.g. to `/var/tmp​`.

Unfortunately I'm not aware of any other way how to specify I'd like to build just this package in other directory (other than `/tmp`).

Also is there some other way how Guix handles this issue? I might be also used in Nix.

FIY the reason I have to use Nix is to build the Android image [1]

1 https://github.com/danielfullmer/robotnix

----
Petr

[Message part 2 (text/html, inline)]

[0001-services-nix-Add-more-configuration-fields.patch (text/x-patch, attachment)]

Message #11 received at 58903 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: phodina <phodina <at> protonmail.com>
Cc: "58903 <at> debbugs.gnu.org" <58903 <at> debbugs.gnu.org>,
 "go.wigust <at> gmail.com" <go.wigust <at> gmail.com>,
 "i <at> pengmeiyu.com" <i <at> pengmeiyu.com>
Subject: Re: Specify the build dir for Nix
Date: Mon, 31 Oct 2022 16:45:30 +0100

[Message part 1 (text/plain, inline)]

Heyo,

phodina 写道：
> The issue is that there are some packages that fail to build 
> even on realtively powerful machine due to resource exhaustion - 
> not enough
> RAM.

Yes, this is an issue in Guix as well, e.g., when building disc 
images.

> Unfortunately I'm not aware of any other way how to specify I'd 
> like to build just this package in other directory (other than 
> `/tmp`).
>
> Also is there some other way how Guix handles this issue? I 
> might be also used in Nix.

No, Guix uses the exact same mechanism (and possibly code), 
inherited directly from Nix.

>  (build-dir           nix-configuration-build-dir ;string
>                       (default "/tmp"))

‘directory’.  Plz, no unnec. abbrev.

The concept LGTM.  Calling it a ‘build directory’ might be 
editorialising too much.  Does Nix use it for anything else?

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 58903 <at> debbugs.gnu.org (full text, mbox):

From: phodina <phodina <at> protonmail.com>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: "58903 <at> debbugs.gnu.org" <58903 <at> debbugs.gnu.org>,
 "go.wigust <at> gmail.com" <go.wigust <at> gmail.com>,
 Ludovic Courtès <ludo <at> gnu.org>,
 "i <at> pengmeiyu.com" <i <at> pengmeiyu.com>
Subject: Re: Specify the build dir for Nix
Date: Mon, 31 Oct 2022 19:18:29 +0000

[Message part 1 (text/plain, inline)]

Hi Tobias,

thanks for feedback!

> 
> > (build-dir nix-configuration-build-dir ;string
> > (default "/tmp"))
> 
> 
> ‘directory’. Plz, no unnec. abbrev.
> 
> The concept LGTM. Calling it a ‘build directory’ might be editorialising too much. Does Nix use it for anything else?

I've updated the patch so there is no abbreviation. I asked on Nix matrix channel and was told just for derivations. But I'm not expert so take it with grain of salt.

Ludo',
you have a lot insight into the build infrastructure and you also took the concepts from Nix.

What do you think about this approach? It should not be premanently set to block device as you loose throughput and increase latency, but it's neccessary for some derivations as not everybody can have powerful machine at their disposal.

Is there some way to share the approach with guix daemon? If I remeber correctly the derivation is offloaded to nix backend - the contents of `nix` directory (haven't yet had the chance to go through the C code).

I guess the machines behind 'https://ci.guix.gnu.org' have enough resources. Or is there some script that handles large derivations? 

Also this might be interesting problem for embedded devices though offloading is definitely the best approach.

----
Petr

[v2-0001-services-nix-Add-more-configuration-fields.patch (text/x-patch, attachment)]

Message #17 received at 58903 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: phodina <phodina <at> protonmail.com>
Cc: "58903 <at> debbugs.gnu.org" <58903 <at> debbugs.gnu.org>,
 "go.wigust <at> gmail.com" <go.wigust <at> gmail.com>,
 Tobias Geerinckx-Rice <me <at> tobias.gr>, "i <at> pengmeiyu.com" <i <at> pengmeiyu.com>
Subject: Re: bug#58903: [PATCH] gnu: nix: Update to 2.11.1.
Date: Wed, 09 Nov 2022 23:45:49 +0100

Hi,

phodina <phodina <at> protonmail.com> skribis:

> I've updated the patch so there is no abbreviation. I asked on Nix matrix channel and was told just for derivations. But I'm not expert so take it with grain of salt.
>
> Ludo',
> you have a lot insight into the build infrastructure and you also took the concepts from Nix.

The manual documents it in the context of guix-daemon (info "(guix)
Build Environment Setup"):

     You can influence the directory where the daemon stores build trees
  via the ‘TMPDIR’ environment variable.  However, the build tree within
  the chroot is always called ‘/tmp/guix-build-NAME.drv-0’, where NAME is
  the derivation name—e.g., ‘coreutils-8.24’.  This way, the value of
  ‘TMPDIR’ does not leak inside build environments, which avoids
  discrepancies in cases where build processes capture the name of their
  build tree.

> I guess the machines behind 'https://ci.guix.gnu.org' have enough resources. Or is there some script that handles large derivations? 

On Guix System, /tmp is not on tmpfs by default (maybe something to
change?); that’s also the case on build machines.

> From d71d48d0fd5d4803abbb50900e0d9aa3c374e7e1 Mon Sep 17 00:00:00 2001
> From: Petr Hodina <phodina <at> protonmail.com>
> Date: Mon, 31 Oct 2022 16:12:38 +0100
> Subject: [PATCH v2] services: nix: Add more configuration fields.
>
> * gnu/services/nix.scm (<nix-configuration>)[build-directory]: New field.
> (nix-service-etc, nix-shepherd-service): Take them into account.
> * doc/guix.texi (Nix): Update it.

[...]

> +@item @code{build-directory} (default: @code{"/tmp"})
> +Specifies build directory.

Rather something like: “The directory where build directory are stored
during builds.”

> This is useful to change if there is not enough RAM
> +on the machine (e.g. embedded targets) or if building big packages (e.g.
> +chromium).

“This is useful to change if, for example, the default location does not
have enough space to hold build trees for big packages.”

(There’s no ‘chromium’ package in Guix.)

Then, instead of a long explanation, I’d just write something like:

  This is similar to setting the @env{TMPDIR} environment variable for
  @command{guix-daemon}.  @ref{Build Environment Setup, @env{TMPDIR}},
  for more info.

Could you send an updated patch?

Thanks,
Ludo’.

Message #22 received at 58903 <at> debbugs.gnu.org (full text, mbox):

From: phodina <phodina <at> protonmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: "58903 <at> debbugs.gnu.org" <58903 <at> debbugs.gnu.org>,
 "go.wigust <at> gmail.com" <go.wigust <at> gmail.com>,
 Tobias Geerinckx-Rice <me <at> tobias.gr>, "i <at> pengmeiyu.com" <i <at> pengmeiyu.com>
Subject: Re: bug#58903: [PATCH] gnu: nix: Update to 2.11.1.
Date: Tue, 13 Dec 2022 12:27:28 +0000

[Message part 1 (text/plain, inline)]

Hello Ludo’,

here's the updated patch with the text as you suggested. I removed the previous long paragraph which was to detailed and would only lengthen the Guix documentation :)

Also good idea to reference the guix daemon. Thanks.

----
Petr

[v3-0001-services-nix-Add-more-configuration-fields.patch (text/x-patch, attachment)]

Message #27 received at 58903-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: phodina <phodina <at> protonmail.com>
Cc: "58903 <at> debbugs.gnu.org" <58903-done <at> debbugs.gnu.org>,
 "go.wigust <at> gmail.com" <go.wigust <at> gmail.com>,
 Tobias Geerinckx-Rice <me <at> tobias.gr>, "i <at> pengmeiyu.com" <i <at> pengmeiyu.com>
Subject: Re: bug#58903: [PATCH] gnu: nix: Update to 2.11.1.
Date: Tue, 13 Dec 2022 17:52:49 +0100

Hi,

phodina <phodina <at> protonmail.com> skribis:

> From 5b38fc4945224df40ceca3736ee40becbda51853 Mon Sep 17 00:00:00 2001
> From: Petr Hodina <phodina <at> protonmail.com>
> Date: Mon, 31 Oct 2022 16:12:38 +0100
> Subject: [PATCH v3] services: nix: Add more configuration fields.
>
> * gnu/services/nix.scm (<nix-configuration>)[build-directory]: New field.
> (nix-service-etc, nix-shepherd-service): Take them into account.
> * doc/guix.texi (Nix): Update it.

Applied, thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.