Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 69596) by debbugs.gnu.org; 28 Apr 2024 22:04:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 28 18:04:28 2024
Received: from localhost ([127.0.0.1]:53129 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1s1CdH-0002hO-L2
for submit <at> debbugs.gnu.org; Sun, 28 Apr 2024 18:04:27 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:42008)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1s1CdF-0002hG-Ng
for 69596 <at> debbugs.gnu.org; Sun, 28 Apr 2024 18:04:26 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1s1Ccq-0003O6-Qj
for 69596 <at> debbugs.gnu.org; Sun, 28 Apr 2024 18:04:00 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=AB0aSjCjRLKl/piNrs9wyI4nJqdpC3fBD02dxcPerBY=; b=AQHkT6xLgNar6O+8HlxH
655exyDUmvhqyDuXL+VgQgF1lpKAIsy18GSl0x25iUysOjW4rkuGZQaH/eRImsBk0WMze/M+osNOd
vNYXPEyPFAlEysPqvDlC4NfcpEocq8m6bQ/GmZ+UJifPY0Nl/oNafis9cejIghTvgumv7lxS5HypT
q9DtEGHTYOFxgFIkcemso34ThWC6Hv4LHSNKHXniEZG5yW2s3dseidpvDCE1UPhdspVxs1gN5Ajf5
2vx8hj9Yf2ZD0hwVWZ9nTajkZGNP9qmwn9J1QYg+NfcBURthLj1+/2W1XlbuptshK0/WQMKcngXaZ
/g6TapgFPbwhoA==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: 69596 <at> debbugs.gnu.org
Subject: Re: bug#69596: =?utf-8?Q?=E2=80=98guix_publish=E2=80=99?= memory leak
In-Reply-To: <87zfvb6nwi.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?=
=?utf-8?Q?'s?= message of "Wed, 06 Mar 2024 22:39:09 +0100")
References: <87zfvb6nwi.fsf@HIDDEN>
Date: Mon, 29 Apr 2024 00:03:48 +0200
Message-ID: <87le4xgn5n.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 69596
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi!
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory noti=
ceably I=E2=80=99d say
> since the beginning of the year on berlin. After roughly 10 days, it
> has several GiB resident and needs to be restarted or it becomes too
> unresponsive.
>
> I wonder what could be causing this because we used to let it run for
> months without problems I believe.
>
> This is what=E2=80=99s currently running on berlin:
>
> /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libex=
ec/guix/guile \
> /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/g=
uix \
> publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \
> --listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \
> --cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400
>
> =E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.
It turned out to be a guile-lzlib leak that had always been present:
https://notabug.org/guile-lzlib/guile-lzlib/commit/74bd35b690801a10ed775d=
486fffc7372b1b341c
The reason we were seeing it more on berlin is probably because we
increased the cache-bypass-threshold, which goes through the
=E2=80=98make-lzip-output-port=E2=80=99 code path (as opposed to
=E2=80=98call-with-lzip-output-port=E2=80=99).
The bug could be reproduced with:
guix publish -p 8124 =E2=80=A6 &
while true ; do wget -q -O/dev/full http://localhost:8124/nar/lzip/=E2=80=
=A6-coreutils-9.1 ; done
(Replace the ellipses with the actual store file name of coreutils.)
Fixed by commit 7cef6b7ba555a9dfaf6d09cb7e112b0df77d5141, which updates
guile-lzlib.
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#69596; Package guix.
Full text available.Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at submit) by debbugs.gnu.org; 6 Mar 2024 21:39:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 06 16:39:51 2024
Received: from localhost ([127.0.0.1]:51797 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rhyzP-0005dZ-2V
for submit <at> debbugs.gnu.org; Wed, 06 Mar 2024 16:39:51 -0500
Received: from lists.gnu.org ([209.51.188.17]:40462)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rhyzN-0005dS-9p
for submit <at> debbugs.gnu.org; Wed, 06 Mar 2024 16:39:49 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rhyyq-00075o-U6
for bug-guix@HIDDEN; Wed, 06 Mar 2024 16:39:17 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rhyyq-0001LX-I0
for bug-guix@HIDDEN; Wed, 06 Mar 2024 16:39:16 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to:
references; bh=cR2wWvr0foz1YvKiA76O2Vr+w000yQC5TGnQ9hAQ0Dw=; b=MlswYMaH/4xxjH
tYy8LMeHGdYk+akwD1qkm5Enn8d1msQJxVukJnFvYXHJXPLH6CJBBBmZObbpKw9b6Zmd8LQ7vk5Z/
TuZgIsVUGHcSJtePmZMMrc4M5dHcCa6ze2x6P9vpZR0h623BbZAmXtbhwYLBwC+fKMZbzPo36HcSz
m138EYakx2uW1WRisGzUCceUVuvNNJLY2oCwFWs/nk9SG2D4JSCgFBibma20vgDkGc+d95HVTAKI3
25e0PQBj+ijN4zvyJzbwwCNL5aQalWbabGR7y3+lwsgzdOkIowyCnY+BDOIkVxSDeg1GaEwftg1FF
4x5AHiMjvvH0T38bEoaQ==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: bug-guix@HIDDEN
Subject: =?utf-8?Q?=E2=80=98guix_publish=E2=80=99?= memory leak
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Septidi 17 =?utf-8?Q?Vent=C3=B4se?= an 232 de la
=?utf-8?Q?R=C3=A9volution=2C?= jour du Doronic
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Wed, 06 Mar 2024 22:39:09 +0100
Message-ID: <87zfvb6nwi.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory notice=
ably I=E2=80=99d say
since the beginning of the year on berlin. After roughly 10 days, it
has several GiB resident and needs to be restarted or it becomes too
unresponsive.
I wonder what could be causing this because we used to let it run for
months without problems I believe.
This is what=E2=80=99s currently running on berlin:
/gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libexec=
/guix/guile \
/gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/gui=
x \
publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \
--listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \
--cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400
=E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8.
I don=E2=80=99t see anything that changed recently, be it in maintenance.git
(for the parameters), =E2=80=98guix publish=E2=80=99, guile-{lzlib,zstd}, o=
r Guile.
Maybe we had just been lucky? Anyone else seeing this?
Ludo=E2=80=99.
Ludovic Courtès <ludo@HIDDEN>:bug-guix@HIDDEN.
Full text available.bug-guix@HIDDEN:bug#69596; Package guix.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.