GNU logs - #69596, boring messages

Message sent to bug-guix@HIDDEN:

Subject: bug#69596: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?publish=E2=80=99?= memory leak
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Wed, 06 Mar 2024 21:40:01 +0000
Resent-Message-ID: <handler.69596.B.170976119121679 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 69596 <at> debbugs.gnu.org
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Date: Wed, 06 Mar 2024 22:39:09 +0100
Message-ID: <87zfvb6nwi.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory notice=
ably I=E2=80=99d say
since the beginning of the year on berlin.  After roughly 10 days, it
has several GiB resident and needs to be restarted or it becomes too
unresponsive.

I wonder what could be causing this because we used to let it run for
months without problems I believe.

This is what=E2=80=99s currently running on berlin:

  /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libexec=
/guix/guile \
  /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/gui=
x \
  publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \
  --listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \
  --cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400

=E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.

I don=E2=80=99t see anything that changed recently, be it in maintenance.git
(for the parameters), =E2=80=98guix publish=E2=80=99, guile-{lzlib,zstd}, o=
r Guile.
Maybe we had just been lucky?  Anyone else seeing this?

Ludo=E2=80=99.

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: bug#69596: Acknowledgement (=?UTF-8?Q?=E2=80=98guix_?=
 =?UTF-8?Q?publish=E2=80=99?= memory leak)
Message-ID: <handler.69596.B.170976119121679.ack <at> debbugs.gnu.org>
References: <87zfvb6nwi.fsf@HIDDEN>
X-Gnu-PR-Message: ack 69596
X-Gnu-PR-Package: guix
Reply-To: 69596 <at> debbugs.gnu.org
Date: Wed, 06 Mar 2024 21:40:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-guix@HIDDEN

If you wish to submit further information on this problem, please
send it to 69596 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
69596: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D69596
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 4 Apr 2024 21:20:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Apr 04 17:20:27 2024
Received: from localhost ([127.0.0.1]:34779 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rsUVX-0005ce-Ek
	for submit <at> debbugs.gnu.org; Thu, 04 Apr 2024 17:20:27 -0400
Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:36082)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1rsUVT-0005bK-4k
 for control <at> debbugs.gnu.org; Thu, 04 Apr 2024 17:20:25 -0400
Authentication-Results: mail3-relais-sop.national.inria.fr;
 dkim=none (message not signed) header.i=none;
 spf=SoftFail smtp.mailfrom=ludo@HIDDEN;
 dmarc=fail (p=none dis=none) d=gnu.org
X-IronPort-AV: E=Sophos;i="6.07,180,1708383600"; d="scan'208";a="83931838"
Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201])
 by mail3-relais-sop.national.inria.fr with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2024 23:20:11 +0200
Date: Thu, 04 Apr 2024 23:20:10 +0200
Message-Id: <87il0wu8ol.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #69596
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

severity 69596 important
quit

Message sent to bug-guix@HIDDEN:

Subject: bug#69596: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?publish=E2=80=99?= memory leak
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Sun, 28 Apr 2024 22:05:01 +0000
Resent-Message-ID: <handler.69596.B69596.171434186810383 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 69596 <at> debbugs.gnu.org
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <87zfvb6nwi.fsf@HIDDEN> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Wed, 06 Mar 2024 22:39:09 +0100")
References: <87zfvb6nwi.fsf@HIDDEN>
Date: Mon, 29 Apr 2024 00:03:48 +0200
Message-ID: <87le4xgn5n.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hi!

Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:

> It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory noti=
ceably I=E2=80=99d say
> since the beginning of the year on berlin.  After roughly 10 days, it
> has several GiB resident and needs to be restarted or it becomes too
> unresponsive.
>
> I wonder what could be causing this because we used to let it run for
> months without problems I believe.
>
> This is what=E2=80=99s currently running on berlin:
>
>   /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libex=
ec/guix/guile \
>   /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/g=
uix \
>   publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \
>   --listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \
>   --cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400
>
> =E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.

It turned out to be a guile-lzlib leak that had always been present:

  https://notabug.org/guile-lzlib/guile-lzlib/commit/74bd35b690801a10ed775d=
486fffc7372b1b341c

The reason we were seeing it more on berlin is probably because we
increased the cache-bypass-threshold, which goes through the
=E2=80=98make-lzip-output-port=E2=80=99 code path (as opposed to
=E2=80=98call-with-lzip-output-port=E2=80=99).

The bug could be reproduced with:

  guix publish -p 8124 =E2=80=A6 &
  while true ; do wget -q -O/dev/full http://localhost:8124/nar/lzip/=E2=80=
=A6-coreutils-9.1 ; done

(Replace the ellipses with the actual store file name of coreutils.)

Fixed by commit 7cef6b7ba555a9dfaf6d09cb7e112b0df77d5141, which updates
guile-lzlib.

Ludo=E2=80=99.

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 28 Apr 2024 22:04:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 28 18:04:38 2024
Received: from localhost ([127.0.0.1]:53136 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1s1CdS-0002hh-0k
	for submit <at> debbugs.gnu.org; Sun, 28 Apr 2024 18:04:38 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:42872)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1s1CdP-0002hW-W7
 for control <at> debbugs.gnu.org; Sun, 28 Apr 2024 18:04:36 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1s1Cd1-0003Oy-3R
 for control <at> debbugs.gnu.org; Sun, 28 Apr 2024 18:04:11 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=WX3eg4nXR5stbYbQquqk6b7auanIWHWr1ZNh8dKbG3c=; b=seCDcrZ1gwJXdh
 AcJzY7b7C+qreja8RuDf+5wtO2YB2A0d4wiFyiwqXIsdYMsywbYGveFY9rToHfSSJW3gMzeNQOvZY
 OPgEOJZWVTnuec0ThVoX3eR/UWqTOviTNV5Zm6dOSb/cCWJmaJJKlDd+3KrbIr3VnWafRLth1wyyA
 3CKkYm38hJMDE6z8EH4ZvMoLMUhq/Q94Ygd9IjBq8yX4WeQT53Btdj9JpgM0dPL4MqAS5mlBMuUN3
 Hkpo8aVwjQQmFnbESthhYdkrCdHPq4/LP0exqV7UeMBZawqMRIFZ8LzAcCUtUZomeUmpz+28YxH8w
 Yapaj75yKKQAjjMzx3Gw==;
Date: Mon, 29 Apr 2024 00:04:00 +0200
Message-Id: <87jzkhgn5b.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #69596
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

close 69596 
quit

Last modified: Sun, 28 Apr 2024 22:15:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.