GNU bug report logs - #67238
[PATCH] derivations: Avoid readlink syscalls in read-derivation-from-file.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 67238 in the body.
You can then email your comments to 67238 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: [PATCH] derivations: Avoid readlink syscalls in
 read-derivation-from-file.
Date: Fri, 17 Nov 2023 11:35:34 +0000

strace -c reports over 10,000 readlink syscalls when reading the derivation
for the hello package. By just setting the %file-port-name-canonicalization
fluid, this drops to less than 10.

I'm not sure if this actually improves performance, but doing less is surely
better.

* guix/derivations.scm (read-derivation-from-file): Set
%file-port-name-canonicalization to 'none when calling call-with-input-file.

Change-Id: I1ff16a059160576a576f2e9ed881379596e66af3
---
 guix/derivations.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/guix/derivations.scm b/guix/derivations.scm
index 9fec7f4f0b..e6ecb570c4 100644
--- a/guix/derivations.scm
+++ b/guix/derivations.scm
@@ -556,7 +556,12 @@ (define (read-derivation-from-file file)
   ;; and because the same argument is read more than 15 times on average
   ;; during something like (package-derivation s gdb).
   (or (and file (hash-ref %derivation-cache file))
-      (let ((drv (call-with-input-file file read-derivation)))
+      (let ((drv
+             ;; Avoid calling scm_i_relativize_path in
+             ;; fport_canonicalize_filename since this leads to lots of
+             ;; readlink calls
+             (with-fluids ((%file-port-name-canonicalization 'none))
+               (call-with-input-file file read-derivation))))
         (hash-set! %derivation-cache file drv)
         drv)))
 

base-commit: e35b7c5386c1bfacf47ed31bac9b503373dd26fc
-- 
2.41.0

Message #8 received at 67238 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>,
 67238 <at> debbugs.gnu.org, Ricardo Wurmus <rekado <at> elephly.net>,
 Christopher Baines <guix <at> cbaines.net>
Subject: Re: [bug#67238] [PATCH] derivations: Avoid readlink syscalls in
 read-derivation-from-file.
Date: Fri, 24 Nov 2023 12:28:33 +0100

Hi,

Christopher Baines <mail <at> cbaines.net> skribis:

> strace -c reports over 10,000 readlink syscalls when reading the derivation
> for the hello package. By just setting the %file-port-name-canonicalization
> fluid, this drops to less than 10.
>
> I'm not sure if this actually improves performance, but doing less is surely
> better.
>
> * guix/derivations.scm (read-derivation-from-file): Set
> %file-port-name-canonicalization to 'none when calling call-with-input-file.
>
> Change-Id: I1ff16a059160576a576f2e9ed881379596e66af3

[...]

> +      (let ((drv
> +             ;; Avoid calling scm_i_relativize_path in
> +             ;; fport_canonicalize_filename since this leads to lots of
> +             ;; readlink calls
> +             (with-fluids ((%file-port-name-canonicalization 'none))
> +               (call-with-input-file file read-derivation))))

This is already done in ‘run-guix’ in (guix ui), for all the ‘guix’
commands (so this patch would be a slight performance regression for
Guix itself).

I’d suggest setting this fluid globally in applications that use Guix
(the Build Coordinator, etc.), as is done in Guix itself.

WDYT?

Ludo’.

Message #13 received at 67238-close <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 67238-close <at> debbugs.gnu.org
Subject: Re: [bug#67238] [PATCH] derivations: Avoid readlink syscalls in
 read-derivation-from-file.
Date: Sat, 25 Nov 2023 20:58:03 +0000

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Christopher Baines <mail <at> cbaines.net> skribis:
>
>> strace -c reports over 10,000 readlink syscalls when reading the derivation
>> for the hello package. By just setting the %file-port-name-canonicalization
>> fluid, this drops to less than 10.
>>
>> I'm not sure if this actually improves performance, but doing less is surely
>> better.
>>
>> * guix/derivations.scm (read-derivation-from-file): Set
>> %file-port-name-canonicalization to 'none when calling call-with-input-file.
>>
>> Change-Id: I1ff16a059160576a576f2e9ed881379596e66af3
>
> [...]
>
>> +      (let ((drv
>> +             ;; Avoid calling scm_i_relativize_path in
>> +             ;; fport_canonicalize_filename since this leads to lots of
>> +             ;; readlink calls
>> +             (with-fluids ((%file-port-name-canonicalization 'none))
>> +               (call-with-input-file file read-derivation))))
>
> This is already done in ‘run-guix’ in (guix ui), for all the ‘guix’
> commands (so this patch would be a slight performance regression for
> Guix itself).
>
> I’d suggest setting this fluid globally in applications that use Guix
> (the Build Coordinator, etc.), as is done in Guix itself.
>
> WDYT?

Ah, I didn't realise it was already set for Guix scripts. But yeah,
setting it in other places that read derivations makes sense.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.