GNU bug report logs - #69596
‘guix publish’ memory leak

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 69596 in the body.
You can then email your comments to 69596 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: bug-guix <at> gnu.org
Subject: ‘guix publish’ memory leak
Date: Wed, 06 Mar 2024 22:39:09 +0100

It seems that ‘guix publish’ has been leaking memory noticeably I’d say
since the beginning of the year on berlin.  After roughly 10 days, it
has several GiB resident and needs to be restarted or it becomes too
unresponsive.

I wonder what could be causing this because we used to let it run for
months without problems I believe.

This is what’s currently running on berlin:

  /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libexec/guix/guile \
  /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/guix \
  publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=nar \
  --listen=localhost --workers=8 --ttl=15552000s \
  --cache=/var/cache/guix/publish --cache-bypass-threshold=157286400

… coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.

I don’t see anything that changed recently, be it in maintenance.git
(for the parameters), ‘guix publish’, guile-{lzlib,zstd}, or Guile.
Maybe we had just been lucky?  Anyone else seeing this?

Ludo’.

Message #10 received at 69596 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 69596 <at> debbugs.gnu.org
Subject: Re: bug#69596: ‘guix publish’ memory leak
Date: Mon, 29 Apr 2024 00:03:48 +0200

Hi!

Ludovic Courtès <ludo <at> gnu.org> skribis:

> It seems that ‘guix publish’ has been leaking memory noticeably I’d say
> since the beginning of the year on berlin.  After roughly 10 days, it
> has several GiB resident and needs to be restarted or it becomes too
> unresponsive.
>
> I wonder what could be causing this because we used to let it run for
> months without problems I believe.
>
> This is what’s currently running on berlin:
>
>   /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libexec/guix/guile \
>   /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/guix \
>   publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=nar \
>   --listen=localhost --workers=8 --ttl=15552000s \
>   --cache=/var/cache/guix/publish --cache-bypass-threshold=157286400
>
> … coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.

It turned out to be a guile-lzlib leak that had always been present:

  https://notabug.org/guile-lzlib/guile-lzlib/commit/74bd35b690801a10ed775d486fffc7372b1b341c

The reason we were seeing it more on berlin is probably because we
increased the cache-bypass-threshold, which goes through the
‘make-lzip-output-port’ code path (as opposed to
‘call-with-lzip-output-port’).

The bug could be reproduced with:

  guix publish -p 8124 … &
  while true ; do wget -q -O/dev/full http://localhost:8124/nar/lzip/…-coreutils-9.1 ; done

(Replace the ellipses with the actual store file name of coreutils.)

Fixed by commit 7cef6b7ba555a9dfaf6d09cb7e112b0df77d5141, which updates
guile-lzlib.

Ludo’.

Message #15 received at 69596 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Roberts <jonrob.one <at> googlemail.com>
To: 69596 <at> debbugs.gnu.org
Subject: Possibly breaks Guix pull on foreign distros
Date: Tue, 30 Apr 2024 09:38:33 +0100

Ran in to the error that a lot of people are reporting - too many heap sections on Guix pull. 

Worked through commits to see where Guix pull stopped working for me, and the answer seems to be that it broke with 7cef6b7ba - gnu: guile-lzlib: Update to 0.3.0.

Could this be confirmed and reverted if it is breaking pull? 

Sent from my iPhone

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.