GNU bug report logs - #45200
Memory leaks: (garbage-collect) fails to reclaim memory

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Konstantin Kharlamov <hi-angel@HIDDEN>; dated Sat, 12 Dec 2020 18:44:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:41:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 16:41:19 2021
Received: from localhost ([127.0.0.1]:37434 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3n8J-00064k-GZ
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 16:41:19 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:20566)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1l3n8I-00064W-GG
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 16:41:18 -0500
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 2E1F7101136;
 Sun, 24 Jan 2021 16:41:13 -0500 (EST)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 79C0D10022E;
 Sun, 24 Jan 2021 16:41:05 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1611524465;
 bh=Jop/6kD6RyxWduTAk6dUXwT3KlnMtUp0Zl4rU36QqTA=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=Txz/b+WXnDs5bcR139OpTPLsFjvGHv1uHfDqrSnH3QAvSU5rqVdzvBm8A650e1H+l
 rRsjdy3IwSGuwmCMamPEoGFRQN/7f1K1kQaJCtC6JGpAJFMFIL26N2tQpqkU6WP1XH
 DWoRHfLueeSOBfg589ZhnTgQxlDw/YeDEVHDsb2344tKYqcTqvMrjoZZveKXftpLJ+
 c3WMS744wkGZrw8pM4OZb1oMP5eXn+uyiFZ6qlA42BrslnBd8Yc+etEiurAIXZgPsx
 WgRhwISc0nZgRbBhb+4U9opiJsqB9lME75KZNyxzXiFfwv7g8XXLmzsy1zHblNlkzg
 /kG5s0T2DPgwQ==
Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 4367E120155;
 Sun, 24 Jan 2021 16:41:05 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
Message-ID: <jwvk0s2m32s.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
 <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
 <83tur62iyl.fsf@HIDDEN>
 <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@HIDDEN>
 <jwv1reanije.fsf-monnier+emacs@HIDDEN>
 <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@HIDDEN>
Date: Sun, 24 Jan 2021 16:41:04 -0500
In-Reply-To: <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@HIDDEN>
 (Konstantin Kharlamov's message of "Mon, 25 Jan 2021 00:26:45 +0300")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL -0.006 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: Eli Zaretskii <eliz@HIDDEN>, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Why unrealistic? To me the situation is pretty clear: I needed 200M, but =
I don't
> need them anymore, so I want them released. What's unrealistic about that?

What can I say: it's not realistic and I already gave you two technical
reasons why that is.

> If that was your point about that "maybe the 200M of memory will be
> needed again", then as I said, yeah, maybe, maybe not =E2=80=94 unless Gl=
ibc
> has a prophet module built-in, it can't know, so it should not behave
> as if it does.

Maybe so.  But to the extent that you consciously decided it's OK for
Emacs to use 200MB at time-step A, then you basically lost the "moral
authority" to say that it's not OK for Emacs to still uses up 200MB at
time-step B.

If you can show a similar excessive use of memory without doing
something like (setq gc-cons-threshold most-positive-fixnum), the
question becomes quite different.


        Stefan





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:26:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 16:26:53 2021
Received: from localhost ([127.0.0.1]:37418 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3muL-0005ex-KC
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 16:26:53 -0500
Received: from forward100j.mail.yandex.net ([5.45.198.240]:57839)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1l3muK-0005eg-5W
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 16:26:52 -0500
Received: from myt5-36442628f0be.qloud-c.yandex.net
 (myt5-36442628f0be.qloud-c.yandex.net
 [IPv6:2a02:6b8:c12:1c0c:0:640:3644:2628])
 by forward100j.mail.yandex.net (Yandex) with ESMTP id 2153150E0597;
 Mon, 25 Jan 2021 00:26:46 +0300 (MSK)
Received: from myt6-efff10c3476a.qloud-c.yandex.net
 (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3])
 by myt5-36442628f0be.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 YJIA7gGm9j-QkG4XOMu; Mon, 25 Jan 2021 00:26:46 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1611523606; bh=Qycj0hjBfBNbcdVa43YeYyb+TjO/g5CXDmU8CLFTuWs=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=EUijUzxWeYCrB/0/hqnNQUC9k3k1hNntMHHywmE4rU2mkfdiy78wTimm6Rr8jJi8/
 EMxxnfqNs0ivL4zzRRaFmBr5/oln58zpyT5UwYVLBnyMpn+chCfGR0p1MMTZ/7bFrT
 GQcCGOglpZql64k/y7sgidkNxVSpZti/Wo14PAqY=
Authentication-Results: myt5-36442628f0be.qloud-c.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id JWczKzgOJq-QjHuoa55; Mon, 25 Jan 2021 00:26:45 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <823c72fe5b0d4bfce3c758af4c3333c199d1e3df.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Date: Mon, 25 Jan 2021 00:26:45 +0300
In-Reply-To: <jwv1reanije.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
 <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
 <83tur62iyl.fsf@HIDDEN>
 <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@HIDDEN>
 <jwv1reanije.fsf-monnier+emacs@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.3 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 45200
Cc: Eli Zaretskii <eliz@HIDDEN>, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Sun, 2021-01-24 at 16:20 -0500, Stefan Monnier wrote:
> > Sure, they have my report on the problem. In my text I was just trying to
> > convince Stefan that this is not correct behaviour, whereas he was
> > implying otherwise.
> 
> I'm not saying it's necessarily correct behavior.
> What I'm saying is that the expectation that the temporary use of 200MB
> should not affect the memory use later is unrealistic.
> 
> And also that if you (setq gc-cons-threshold most-positive-fixnum)
> then you're simply asking for trouble unless you *really* know what
> you're doing (in which case you'd understand that the above expectation
> is unrealistic).

Why unrealistic? To me the situation is pretty clear: I needed 200M, but I don't
need them anymore, so I want them released. What's unrealistic about that?

If that was your point about that "maybe the 200M of memory will be needed
again", then as I said, yeah, maybe, maybe not — unless Glibc has a prophet
module built-in, it can't know, so it should not behave as if it does.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 21:20:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 16:20:32 2021
Received: from localhost ([127.0.0.1]:37410 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3moC-0005Uz-Iv
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 16:20:32 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:49489)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1l3moA-0005Uk-S6
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 16:20:31 -0500
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 3E15A101136;
 Sun, 24 Jan 2021 16:20:25 -0500 (EST)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id CF83A100707;
 Sun, 24 Jan 2021 16:20:23 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1611523223;
 bh=aFRM3icbGp6HZvFLVY4AGVL1sfwS1MI5QvyfPniiiEg=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=VBedqCDazqQx0squT5QF/ARGO0kQUYDs4nEg2EGobk9iDQozl8Cn2yBkqwzwHcAeO
 OfE8zE41+QM1sE6jD+SXWwYn7QzmmrqaMG11GTyD/Z0tTQtoxCGC1/FgH9Dk0Saa6p
 ypfFrPpDLHes1aXTzL7DWEmZBp526MraiJmSAgUn3B0Ww20q/V4IRpXOjEFbgdGq46
 5cB/LqmVf/mqAdtpJfZGWdRa496wT66YDnpNbDsZpJBXELh8QlLiyGAgk6mEVkTGYU
 Ftj0vN7rQqrvLFb3OSvHFSbIpfLxxGikHzl4gA2edd+fw9W02lNfiI+zE1A1bZb7gF
 GJ4SqpBTg/WqA==
Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 851AE12014C;
 Sun, 24 Jan 2021 16:20:23 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
Message-ID: <jwv1reanije.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
 <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
 <83tur62iyl.fsf@HIDDEN>
 <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@HIDDEN>
Date: Sun, 24 Jan 2021 16:20:22 -0500
In-Reply-To: <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@HIDDEN>
 (Konstantin Kharlamov's message of "Sun, 24 Jan 2021 23:21:09 +0300")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL -0.006 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: Eli Zaretskii <eliz@HIDDEN>, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Sure, they have my report on the problem. In my text I was just trying to
> convince Stefan that this is not correct behaviour, whereas he was
> implying otherwise.

I'm not saying it's necessarily correct behavior.
What I'm saying is that the expectation that the temporary use of 200MB
should not affect the memory use later is unrealistic.

And also that if you (setq gc-cons-threshold most-positive-fixnum)
then you're simply asking for trouble unless you *really* know what
you're doing (in which case you'd understand that the above expectation
is unrealistic).

> Glibc could for example use statistics of previous allocation patterns
> to see if it's needed to release memory and how much, but this is not
> what happens. And I am not the only dissatisfied user. Ruby for
> example too=C2=B9. Also, many people experience strange memory usage grow
> with Qutebrowser that nobody knows where it's coming from; and after
> today's research I suspect Glibc is the culprit (I don't have yet
> enough evidence because my Qutebrowser instance doesn't have much
> uptime ATM, but attaching to it with gdb and calling `malloc_trim`
> inside it already freed =E2=89=8840M of memory to me today). Glib also se=
ems
> affected=C2=B2.

Memory management is hard, and lots and lots of things can go wrong.
Maybe you're right and they're all due to some underlying problem
in glibc.  I suspect that instead the only thing in common is that
they're all problems with the memory management.


        Stefan





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 20:21:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 15:21:18 2021
Received: from localhost ([127.0.0.1]:37391 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3lss-000414-C1
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 15:21:18 -0500
Received: from forward103j.mail.yandex.net ([5.45.198.246]:54285)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1l3lsq-00040p-Hb
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 15:21:17 -0500
Received: from forward101q.mail.yandex.net (forward101q.mail.yandex.net
 [IPv6:2a02:6b8:c0e:4b:0:640:4012:bb98])
 by forward103j.mail.yandex.net (Yandex) with ESMTP id AAC0F6740FB7;
 Sun, 24 Jan 2021 23:21:10 +0300 (MSK)
Received: from vla1-5c8ce23a551e.qloud-c.yandex.net
 (vla1-5c8ce23a551e.qloud-c.yandex.net
 [IPv6:2a02:6b8:c0d:400d:0:640:5c8c:e23a])
 by forward101q.mail.yandex.net (Yandex) with ESMTP id A585ACF40002;
 Sun, 24 Jan 2021 23:21:10 +0300 (MSK)
Received: from vla4-d1b041059520.qloud-c.yandex.net
 (vla4-d1b041059520.qloud-c.yandex.net [2a02:6b8:c17:914:0:640:d1b0:4105])
 by vla1-5c8ce23a551e.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 BqfOFRLV27-LAGGdOui; Sun, 24 Jan 2021 23:21:10 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1611519670; bh=fejtMT4drRKGDP0WOrNBh0AjQrQReMLp0PtRt/p6Dl8=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=BhizMNPx20KZHlWTNRX9ARuFWmcwnUSr9HJ3wszrFnvzZSSwqMGmrROAUWMWdoK9N
 0UEjaFK21gR9bbMOoK4gyrTLu2dN+i0h+74fNJVNUfI7eFzyJWzOs/7T8N20GsbZTj
 8hG5QyqMzWo4z5YQj+tPx0/2BBfq8U7NUbMdOQmI=
Authentication-Results: vla1-5c8ce23a551e.qloud-c.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by vla4-d1b041059520.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id 9wHMxPrxjX-LAmqTha2; Sun, 24 Jan 2021 23:21:10 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <6b661d5f325efb3f49e83e2de093b1e271751bc0.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Date: Sun, 24 Jan 2021 23:21:09 +0300
In-Reply-To: <83tur62iyl.fsf@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
 <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
 <83tur62iyl.fsf@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.3 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 45200
Cc: monnier@HIDDEN, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Sun, 2021-01-24 at 22:11 +0200, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <hi-angel@HIDDEN>
> > Date: Sun, 24 Jan 2021 23:00:50 +0300
> > Cc: 45200 <at> debbugs.gnu.org
> > 
> > With so many unhappy users, do you still think nothing wrong with the glibc
> > allocator?
> 
> Please be aware that this is not the right place to send bug reports
> about glibc's memory allocation algorithms and implementation.  They
> have their own bug tracker.

Sure, they have my report on the problem. In my text I was just trying to convince Stefan that this is not correct behaviour, whereas he was implying otherwise.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 20:11:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 15:11:51 2021
Received: from localhost ([127.0.0.1]:37377 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3lji-0003jc-Vl
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 15:11:51 -0500
Received: from eggs.gnu.org ([209.51.188.92]:45674)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1l3ljh-0003jN-6e
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 15:11:49 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:36236)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1l3ljb-00089A-T7; Sun, 24 Jan 2021 15:11:43 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3853
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1l3ljY-0004te-Fh; Sun, 24 Jan 2021 15:11:41 -0500
Date: Sun, 24 Jan 2021 22:11:46 +0200
Message-Id: <83tur62iyl.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
In-Reply-To: <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
 (message from Konstantin Kharlamov on Sun, 24 Jan 2021 23:00:50 +0300)
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
 <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: monnier@HIDDEN, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Konstantin Kharlamov <hi-angel@HIDDEN>
> Date: Sun, 24 Jan 2021 23:00:50 +0300
> Cc: 45200 <at> debbugs.gnu.org
> 
> With so many unhappy users, do you still think nothing wrong with the glibc
> allocator?

Please be aware that this is not the right place to send bug reports
about glibc's memory allocation algorithms and implementation.  They
have their own bug tracker.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 20:01:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 15:01:01 2021
Received: from localhost ([127.0.0.1]:37369 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3lZF-0003Te-Ie
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 15:01:01 -0500
Received: from forward101j.mail.yandex.net ([5.45.198.241]:49730)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1l3lZD-0003TG-Gb
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 15:01:01 -0500
Received: from myt2-4597fff41cb2.qloud-c.yandex.net
 (myt2-4597fff41cb2.qloud-c.yandex.net
 [IPv6:2a02:6b8:c00:3b2c:0:640:4597:fff4])
 by forward101j.mail.yandex.net (Yandex) with ESMTP id 587741BE022B;
 Sun, 24 Jan 2021 23:00:52 +0300 (MSK)
Received: from myt6-efff10c3476a.qloud-c.yandex.net
 (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3])
 by myt2-4597fff41cb2.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 YPeZnxVA9D-0qH8wQt8; Sun, 24 Jan 2021 23:00:52 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1611518452; bh=DGHB7wTmROhuQJyXdW2kGLCrWOr9FKnXoWpERvnKdyo=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=p8evXnbDVwjP27fx+KfRrHA0PUuZqm2caDNYaOyqdXfJGqJ5kqKxm5CaJYuVlCQbh
 10p2shh2hBVMMcm6Xiz1A3vwMdvSG7rBog0XDJPurLdvsOm6wxdPTrHJ5XF9mJae60
 1H/HtZ4FClOfNWnRRaOr0w0nv0M+UieaX0b1p5e0=
Authentication-Results: myt2-4597fff41cb2.qloud-c.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id wghFCbz68h-0pHiXOJg; Sun, 24 Jan 2021 23:00:51 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <5c5696670dea31a4647c901be1f87ddb1474f820.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Date: Sun, 24 Jan 2021 23:00:50 +0300
In-Reply-To: <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.3 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Sun, 2021-01-24 at 14:12 -0500, Stefan Monnier wrote:
> > I'm fine with Emacs possibly keeping a dozen of megabytes for personal
> > use. The problem though is that the issue easily manifests itself in
> > hundreds of megabytes, as can be seen with the testcase marked as "Message
> > #11" in web-ui.
> 
> I don't think the exact number matters: you were willing to make use of
> *any* amount of extra memory by delaying all GC until idle time.
> The fact that the GC you finally do after that long wait doesn't recover
> that memory is not a failure in my book: there's no reason Emacs or
> glibc should presume that you won't be reusing just as much heap space
> before the next GC.
> 
> IOW in my book, you got what you asked for ;-)

This doesn't look right. I mean, sure, I might need the heap again. Or I might
not. I don't know. Neither you can. Nobody knows. If glibc can predict the
future then fine. But my experience as part of this report suggests glibc can't.

Glibc could for example use statistics of previous allocation patterns to see if
it's needed to release memory and how much, but this is not what happens. And I
am not the only dissatisfied user. Ruby for example too¹. Also, many people
experience strange memory usage grow with Qutebrowser that nobody knows where
it's coming from; and after today's research I suspect Glibc is the culprit (I
don't have yet enough evidence because my Qutebrowser instance doesn't have much
uptime ATM, but attaching to it with gdb and calling `malloc_trim` inside it
already freed ≈40M of memory to me today). Glib also seems affected².

With so many unhappy users, do you still think nothing wrong with the glibc
allocator? It seems to me, we're doing perfectly valid usecase: we used memory,
we don't need it anymore, we free it. If somebody ever notes that malloc/free
cycle takes too much time for them, then they know they need to optimize the
specific codepath where this happens. It's a nice bonus if Glibc could do
malloc/free faster, but not so much if this causes other problems, especially
given it's a core system library.

> BTW, I wish `malloc_trim` could be passed an argument that says
> something like "release about half of the free memory", which would
> provide a better balance between "hoard free memory indefinitely" and
> "constantly free&reallocate memory from the OS".
> 
> Also, the manpage of mallopt suggests that glibc should automatically
> do the trim on its own every once in a while anyway, so maybe the
> problem won't be solved by `malloc_trim`.

Yeah, I've seen that too. Idk why it doesn't work, but that `malloc_trim()` does
I know because I tested it with the testcase where previously Emacs was never
returning ≈200M of memory.

1:
https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#investigating-fragmentation-at-the-memory-allocator-level
2: https://gitlab.gnome.org/GNOME/glib/-/issues/2057





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:55:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 14:55:04 2021
Received: from localhost ([127.0.0.1]:37353 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3lTU-0003J1-3U
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:55:04 -0500
Received: from eggs.gnu.org ([209.51.188.92]:42284)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1l3lTS-0003IT-7A
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:55:02 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:35861)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1l3lTM-00022h-N7; Sun, 24 Jan 2021 14:54:56 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2791
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1l3lTL-0008Ob-MY; Sun, 24 Jan 2021 14:54:56 -0500
Date: Sun, 24 Jan 2021 21:55:00 +0200
Message-Id: <83y2gi2jqj.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
In-Reply-To: <0457437ff45b535a61f7c35011de1d133cdd3496.camel@HIDDEN>
 (message from Konstantin Kharlamov on Sun, 24 Jan 2021 22:06:21 +0300)
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 <0457437ff45b535a61f7c35011de1d133cdd3496.camel@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: monnier@HIDDEN, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Konstantin Kharlamov <hi-angel@HIDDEN>
> Date: Sun, 24 Jan 2021 22:06:21 +0300
> Cc: 45200 <at> debbugs.gnu.org
> 
> Case in point btw: after the previous weekend I found my Emacs instance that I did not touch during the weekend to bloat to as much as 6xxMB. I think that happened because the on-idle hook that I hooked garbage collector onto was not run for 2 days, whilst Emacs might have been doing something like communication to LSP server from time to time, so this is why it ended up taking so much memory. I think it's needless to say that I had to restart Emacs for the half-gigabyte of memory to get returned to my system.

That bug (bug#43389) was recently tracked down and fixed.  I don't
think it has anything to do with malloc_trim and friends.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:12:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 14:12:38 2021
Received: from localhost ([127.0.0.1]:37308 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3koP-0002IV-Q0
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:12:37 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:61613)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1l3koO-0002IF-Fe
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:12:36 -0500
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 86AD044061A;
 Sun, 24 Jan 2021 14:12:30 -0500 (EST)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id DBB8A44059C;
 Sun, 24 Jan 2021 14:12:21 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1611515541;
 bh=hyRyxFPMOW0abSrju6coe8g1J4SJN+WSdbhe1fOKa+E=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=iZMVQbuzPvcpqu1NbhlQzQDjAsPEsepHnTkzerXDDMun5vuQbQkFItoKRVA+5jY9k
 iKTQ8tuV/kk38NAbt476fyXq093JiCYEAiWBSpEQv8I87tADHv03dKmhvsGKHd8hSd
 X/y0doBWbxXxfsT2SqQWpXzSDPrJCNSBZQe/PSRCL7MW87XKiMQu57NB5YkYPcECeH
 gFAkfrbmBqPsCSgZGvnGaW23EAGfrFKY8PKP3nTHJPZzaEwKByRFL4Us1FQoY/a13x
 xAonBlacVRueoRAscDppisS7yqngXvg4RDvQB9em3zv2+OWeGHxgpG7ghfUc3JdlJ7
 iNKrzAKyMavHw==
Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id AC7CA1202B6;
 Sun, 24 Jan 2021 14:12:21 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
Message-ID: <jwvbldenom2.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
Date: Sun, 24 Jan 2021 14:12:20 -0500
In-Reply-To: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
 (Konstantin Kharlamov's message of "Sun, 24 Jan 2021 22:00:31 +0300")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL 0.085 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> I'm fine with Emacs possibly keeping a dozen of megabytes for personal
> use. The problem though is that the issue easily manifests itself in
> hundreds of megabytes, as can be seen with the testcase marked as "Message
> #11" in web-ui.

I don't think the exact number matters: you were willing to make use of
*any* amount of extra memory by delaying all GC until idle time.
The fact that the GC you finally do after that long wait doesn't recover
that memory is not a failure in my book: there's no reason Emacs or
glibc should presume that you won't be reusing just as much heap space
before the next GC.

IOW in my book, you got what you asked for ;-)

BTW, I wish `malloc_trim` could be passed an argument that says
something like "release about half of the free memory", which would
provide a better balance between "hoard free memory indefinitely" and
"constantly free&reallocate memory from the OS".

Also, the manpage of mallopt suggests that glibc should automatically
do the trim on its own every once in a while anyway, so maybe the
problem won't be solved by `malloc_trim`.


        Stefan





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:06:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 14:06:32 2021
Received: from localhost ([127.0.0.1]:37287 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3kiW-00028G-G9
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:06:32 -0500
Received: from forward103o.mail.yandex.net ([37.140.190.177]:53439)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1l3kiT-00027y-G3
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:06:30 -0500
Received: from sas1-37a648709e20.qloud-c.yandex.net
 (sas1-37a648709e20.qloud-c.yandex.net
 [IPv6:2a02:6b8:c08:fe03:0:640:37a6:4870])
 by forward103o.mail.yandex.net (Yandex) with ESMTP id B88AC5F80D98;
 Sun, 24 Jan 2021 22:06:22 +0300 (MSK)
Received: from sas1-e20a8b944cac.qloud-c.yandex.net
 (sas1-e20a8b944cac.qloud-c.yandex.net [2a02:6b8:c14:6696:0:640:e20a:8b94])
 by sas1-37a648709e20.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 1Z1ZMHGEDe-6MGO7SHw; Sun, 24 Jan 2021 22:06:22 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1611515182; bh=uMD8zf8vnMHzN04u2mb17Tq3FpEpsyUiU1ecPRJhjsE=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=kB6Iv0QnGDJwAJKIhWCIv0Gxf6ICzSTnkR0lS14EB4VfBymMg8LXCBIQIqgnB244g
 crDMNckmKXOSErArpdJhPOoHPuwvhbuTucujFVBX/KAnbsg86Exj/T6GiQ6hEnF2UX
 z2GU3x6ODA+mQVRMB0E4grJrUh+qumPPUklhHqFg=
Authentication-Results: sas1-37a648709e20.qloud-c.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by sas1-e20a8b944cac.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id nl5csKbbqS-6MIaVelL; Sun, 24 Jan 2021 22:06:22 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <0457437ff45b535a61f7c35011de1d133cdd3496.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Date: Sun, 24 Jan 2021 22:06:21 +0300
In-Reply-To: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
 <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.3 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Sun, 2021-01-24 at 22:00 +0300, Konstantin Kharlamov wrote:
> On Sun, 2021-01-24 at 13:51 -0500, Stefan Monnier wrote:
> > > # Steps to reproduce:
> > > 
> > > 1. Run `mkdir /tmp/.emacs.d`
> > > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS
> > > 3. Create a file /tmp/.emacs.d/early-init.el with content:
> > > 
> > >     ;; only run garbage collection on idle
> > >     (setq gc-cons-threshold most-positive-fixnum)
> > >     (run-with-idle-timer 2 t (lambda () (garbage-collect)))
> > > 
> > > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then
> > > measure
> > > its PSS
> > > 
> > > ## Expected
> > > 
> > > Size has no statistically-significant difference, because in both
> > > cases we garbage-collected memory.
> > 
> > I disagree with this expectation: it is perfectly normal for the amount
> > of memory allocated to the Emacs process to be left higher if you delay
> > the GC.  There are various reasons for that:
> > - fragmentation, of course.  Not much we can do about it short of using
> >   a moving collector.
> > - the desire to keep memory around rather than return it to the OS,
> >   under the assumption that we'll need it again soon.
> > 
> > And it's not considered as a memory leak as long as that memory has
> > indeed been needed in the past and that future allocations can still
> > make use of it.
> 
> I'm fine with Emacs possibly keeping a dozen of megabytes for personal use.
> The problem though is that the issue easily manifests itself in hundreds of
> megabytes, as can be seen with the testcase marked as "Message #11" in web-ui.

Case in point btw: after the previous weekend I found my Emacs instance that I did not touch during the weekend to bloat to as much as 6xxMB. I think that happened because the on-idle hook that I hooked garbage collector onto was not run for 2 days, whilst Emacs might have been doing something like communication to LSP server from time to time, so this is why it ended up taking so much memory. I think it's needless to say that I had to restart Emacs for the half-gigabyte of memory to get returned to my system.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 19:00:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 14:00:42 2021
Received: from localhost ([127.0.0.1]:37275 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3kcs-0001zQ-9E
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:00:42 -0500
Received: from forward106j.mail.yandex.net ([5.45.198.249]:35443)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1l3kcq-0001zA-42
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 14:00:41 -0500
Received: from iva3-3aa7bbf53bdf.qloud-c.yandex.net
 (iva3-3aa7bbf53bdf.qloud-c.yandex.net
 [IPv6:2a02:6b8:c0c:4983:0:640:3aa7:bbf5])
 by forward106j.mail.yandex.net (Yandex) with ESMTP id E1EA711A0D7F;
 Sun, 24 Jan 2021 22:00:32 +0300 (MSK)
Received: from iva8-174eb672ffa9.qloud-c.yandex.net
 (iva8-174eb672ffa9.qloud-c.yandex.net [2a02:6b8:c0c:b995:0:640:174e:b672])
 by iva3-3aa7bbf53bdf.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 4G3T6JmRog-0WG0OMjd; Sun, 24 Jan 2021 22:00:32 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1611514832; bh=bHettI+ZzAnUuZGNz0T0xIOWtjvdJd8nsWDkhcqJBME=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=DVYwPw1989BZeEJfI33lXl3+PR2T18S59740qPN4+9m4Yd/5RinU/jKibkPtteTaD
 iy2fhR15/HGSHMGuBVL/MRy0ODPsVhJzFbakWZDX+Pe57Wqg4O4Ey3sIMeixXwpJaF
 le8/KoGt8aezRBXkvJLoBQXeH0vurZRz/f2HsKaI=
Authentication-Results: iva3-3aa7bbf53bdf.qloud-c.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by iva8-174eb672ffa9.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id r3NgoBo7Lp-0WImEXdo; Sun, 24 Jan 2021 22:00:32 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <2a4f3f31f30db28387055821a8edf44fd1d66ea5.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Date: Sun, 24 Jan 2021 22:00:31 +0300
In-Reply-To: <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.3 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Sun, 2021-01-24 at 13:51 -0500, Stefan Monnier wrote:
> > # Steps to reproduce:
> > 
> > 1. Run `mkdir /tmp/.emacs.d`
> > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS
> > 3. Create a file /tmp/.emacs.d/early-init.el with content:
> > 
> >     ;; only run garbage collection on idle
> >     (setq gc-cons-threshold most-positive-fixnum)
> >     (run-with-idle-timer 2 t (lambda () (garbage-collect)))
> > 
> > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure
> > its PSS
> > 
> > ## Expected
> > 
> > Size has no statistically-significant difference, because in both
> > cases we garbage-collected memory.
> 
> I disagree with this expectation: it is perfectly normal for the amount
> of memory allocated to the Emacs process to be left higher if you delay
> the GC.  There are various reasons for that:
> - fragmentation, of course.  Not much we can do about it short of using
>   a moving collector.
> - the desire to keep memory around rather than return it to the OS,
>   under the assumption that we'll need it again soon.
> 
> And it's not considered as a memory leak as long as that memory has
> indeed been needed in the past and that future allocations can still
> make use of it.

I'm fine with Emacs possibly keeping a dozen of megabytes for personal use. The problem though is that the issue easily manifests itself in hundreds of megabytes, as can be seen with the testcase marked as "Message #11" in web-ui.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 18:51:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 13:51:58 2021
Received: from localhost ([127.0.0.1]:37260 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3kUP-0001kF-SL
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 13:51:58 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:34091)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1l3kUO-0001k0-5o
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 13:51:56 -0500
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
 by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7DB0D80B69;
 Sun, 24 Jan 2021 13:51:50 -0500 (EST)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id EEDD280057;
 Sun, 24 Jan 2021 13:51:48 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1611514309;
 bh=tiaED2ZbqzCrNjpdzBUTKfq/fzgmO6rpkQQf+lPUOSc=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=T65lOMKDRlIcdDVJPKfMx/IAgzUMYq9/iHIrmzfg6ye3UvaPrBBqsXn1O9L5i+pK4
 1SxeoXJEcbv7EaomNTtToRcoR6jirgL5sAfu3RAubfB4qyPqsKce+cR3am1XvQh0IC
 zi5L8pW5CQpgHuKubTuNGuc9Wy1gPJEm1zEivJ5GGL1RQCB4HlOTJgmV/uYbQtAR3y
 +KCUde6jjtjGc94RchG8mipe2YiF9Km9QSDO/NF+oAmyG1orQs844xFoifu6B2FjTz
 WvGVnpvf4RNiZbPAN5pXgp7rgJeg4KMyMIhuiBcYt41T4j48txufh48x39p/W9sXjd
 EpMnJ7OeK4V9Q==
Received: from alfajor (69-196-141-46.dsl.teksavvy.com [69.196.141.46])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BA4801201CC;
 Sun, 24 Jan 2021 13:51:48 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
Message-ID: <jwvlfcinpmk.fsf-monnier+emacs@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
Date: Sun, 24 Jan 2021 13:51:41 -0500
In-Reply-To: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 (Konstantin Kharlamov's message of "Sat, 12 Dec 2020 21:43:10 +0300")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL -0.010 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> # Steps to reproduce:
>
> 1. Run `mkdir /tmp/.emacs.d`
> 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS
> 3. Create a file /tmp/.emacs.d/early-init.el with content:
>
>     ;; only run garbage collection on idle
>     (setq gc-cons-threshold most-positive-fixnum)
>     (run-with-idle-timer 2 t (lambda () (garbage-collect)))
>
> 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure its PSS
>
> ## Expected
>
> Size has no statistically-significant difference, because in both
> cases we garbage-collected memory.

I disagree with this expectation: it is perfectly normal for the amount
of memory allocated to the Emacs process to be left higher if you delay
the GC.  There are various reasons for that:
- fragmentation, of course.  Not much we can do about it short of using
  a moving collector.
- the desire to keep memory around rather than return it to the OS,
  under the assumption that we'll need it again soon.

And it's not considered as a memory leak as long as that memory has
indeed been needed in the past and that future allocations can still
make use of it.

Eli wrote:
> Thanks, but is it really a good idea to call malloc_trim each time we
> free some chunk of memory?

I think if we want to call `malloc_trim`, the obvious place would be to
do it at the end of `garbage_collect`.


        Stefan





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 15:40:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 10:40:37 2021
Received: from localhost ([127.0.0.1]:36992 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3hVF-00017e-KV
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 10:40:37 -0500
Received: from eggs.gnu.org ([209.51.188.92]:33922)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1l3hVB-00017O-GM
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 10:40:37 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:60069)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1l3hV5-0005z8-4l; Sun, 24 Jan 2021 10:40:27 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2771
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1l3hV2-000592-NG; Sun, 24 Jan 2021 10:40:26 -0500
Date: Sun, 24 Jan 2021 17:40:29 +0200
Message-Id: <834kj64a36.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Konstantin Kharlamov <Hi-Angel@HIDDEN>
In-Reply-To: <20210124152402.40270-1-Hi-Angel@HIDDEN> (message from
 Konstantin Kharlamov on Sun, 24 Jan 2021 18:24:02 +0300)
Subject: Re: bug#45200: [PATCH] Force Glibc to free the memory freed
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <20210124152402.40270-1-Hi-Angel@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: carlos@HIDDEN, fweimer@HIDDEN, dj@HIDDEN,
 monnier@HIDDEN, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Konstantin Kharlamov <Hi-Angel@HIDDEN>
> Date: Sun, 24 Jan 2021 18:24:02 +0300
> 
> configure.ac: check whether malloc_trim is suported
> src/alloc.c (lisp_free): call malloc_trim() if possible
> (bug#45200)
> ---
>  configure.ac | 3 +++
>  src/alloc.c  | 3 +++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/configure.ac b/configure.ac
> index bcc0be7de0..3e0459a0e2 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -4544,6 +4544,9 @@ AC_DEFUN
>  dnl the current CFLAGS etc.
>  AC_CHECK_FUNCS(snprintf)
>  
> +
> +AC_CHECK_FUNCS(malloc_trim)
> +
>  dnl Check for glib.  This differs from other library checks in that
>  dnl Emacs need not link to glib unless some other library is already
>  dnl linking to glib.  Although glib provides no facilities that Emacs
> diff --git a/src/alloc.c b/src/alloc.c
> index c0a55e61b9..97e3ceb52c 100644
> --- a/src/alloc.c
> +++ b/src/alloc.c
> @@ -1047,6 +1047,9 @@ lisp_free (void *block)
>  
>    MALLOC_BLOCK_INPUT;
>    free (block);
> +#ifdef HAVE_MALLOC_TRIM
> +  malloc_trim(0); /* work around for high memory consumption, see bug 45200 */
> +#endif /* HAVE_MALLOC_TRIM */
>  #ifndef GC_MALLOC_CHECK
>    mem_delete (mem_find (block));
>  #endif

Thanks, but is it really a good idea to call malloc_trim each time we
free some chunk of memory?

Carlos, Florian, DJ: any thoughts or comments on this proposal?  (Let
me know if you need some background about the code involved in this.)




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 24 Jan 2021 15:24:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 24 10:24:22 2021
Received: from localhost ([127.0.0.1]:36987 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l3hFV-0000h0-Vp
	for submit <at> debbugs.gnu.org; Sun, 24 Jan 2021 10:24:22 -0500
Received: from forward100p.mail.yandex.net ([77.88.28.100]:59195)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <Hi-Angel@HIDDEN>) id 1l3hFQ-0000gi-DY
 for 45200 <at> debbugs.gnu.org; Sun, 24 Jan 2021 10:24:20 -0500
Received: from myt3-a3d1bb5f3f13.qloud-c.yandex.net
 (myt3-a3d1bb5f3f13.qloud-c.yandex.net [IPv6:2a02:6b8:c12:28:0:640:a3d1:bb5f])
 by forward100p.mail.yandex.net (Yandex) with ESMTP id 2C5195980937
 for <45200 <at> debbugs.gnu.org>; Sun, 24 Jan 2021 18:24:09 +0300 (MSK)
Received: from myt6-efff10c3476a.qloud-c.yandex.net
 (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3])
 by myt3-a3d1bb5f3f13.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 7DLljfbBP1-O8G81gaj; Sun, 24 Jan 2021 18:24:09 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1611501849; bh=WpgoFbZUg3vTl+wC5eMn1D+d9Rt7AY/rmlsl2+v1W50=;
 h=Date:Subject:To:From:Message-Id;
 b=pDXxk1V+TA1QQrRZxxWK3QaKeSJaHOCRocszIgAPpoF9dhNbRw29NrUTM7eKXR232
 +kfVShAV2sMAqEscN9dE9I1IVtn+FH7IU8ni9K26UXf3bNX+z6TUZIyjyWMSV2bBfY
 O6LamhIcgvL84toSmobC6NzsJwUQeny1BhCxSxZg=
Authentication-Results: myt3-a3d1bb5f3f13.qloud-c.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id MkwfNY9RdE-O8H4ZoA6; Sun, 24 Jan 2021 18:24:08 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
From: Konstantin Kharlamov <Hi-Angel@HIDDEN>
To: 45200 <at> debbugs.gnu.org
Subject: [PATCH] Force Glibc to free the memory freed
Date: Sun, 24 Jan 2021 18:24:02 +0300
Message-Id: <20210124152402.40270-1-Hi-Angel@HIDDEN>
X-Mailer: git-send-email 2.30.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 45200
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

configure.ac: check whether malloc_trim is suported
src/alloc.c (lisp_free): call malloc_trim() if possible
(bug#45200)
---
 configure.ac | 3 +++
 src/alloc.c  | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/configure.ac b/configure.ac
index bcc0be7de0..3e0459a0e2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -4544,6 +4544,9 @@ AC_DEFUN
 dnl the current CFLAGS etc.
 AC_CHECK_FUNCS(snprintf)
 
+
+AC_CHECK_FUNCS(malloc_trim)
+
 dnl Check for glib.  This differs from other library checks in that
 dnl Emacs need not link to glib unless some other library is already
 dnl linking to glib.  Although glib provides no facilities that Emacs
diff --git a/src/alloc.c b/src/alloc.c
index c0a55e61b9..97e3ceb52c 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -1047,6 +1047,9 @@ lisp_free (void *block)
 
   MALLOC_BLOCK_INPUT;
   free (block);
+#ifdef HAVE_MALLOC_TRIM
+  malloc_trim(0); /* work around for high memory consumption, see bug 45200 */
+#endif /* HAVE_MALLOC_TRIM */
 #ifndef GC_MALLOC_CHECK
   mem_delete (mem_find (block));
 #endif
-- 
2.30.0





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 13 Dec 2020 12:07:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Dec 13 07:07:56 2020
Received: from localhost ([127.0.0.1]:47585 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1koQAO-0008SB-Ip
	for submit <at> debbugs.gnu.org; Sun, 13 Dec 2020 07:07:56 -0500
Received: from forward104p.mail.yandex.net ([77.88.28.107]:55659)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1koQAL-0008Rc-S3
 for 45200 <at> debbugs.gnu.org; Sun, 13 Dec 2020 07:07:55 -0500
Received: from mxback14o.mail.yandex.net (mxback14o.mail.yandex.net
 [IPv6:2a02:6b8:0:1a2d::65])
 by forward104p.mail.yandex.net (Yandex) with ESMTP id 7E8244B00D54;
 Sun, 13 Dec 2020 15:07:47 +0300 (MSK)
Received: from myt5-95c1fb78270f.qloud-c.yandex.net
 (myt5-95c1fb78270f.qloud-c.yandex.net [2a02:6b8:c12:1725:0:640:95c1:fb78])
 by mxback14o.mail.yandex.net (mxback/Yandex) with ESMTP id E1pNK9OF5a-7lVq638b;
 Sun, 13 Dec 2020 15:07:47 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1607861267; bh=izcAo6NbdFJ4KXQv4oPCQvEna4EUceJEw9aT8DRatFA=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=aXjqOMH/g5MdYjPTxajlU5Cd8nI6w58lXd83bMJE33i93QDtxD4hRoJWnM6mpGmhl
 18h+eqUN/WazlPlRWl2NgJWAEfcn5OryiRdkwAbRmcCVFkW8CJ2nhuwAoLCnOGKanH
 cwNEYldz8RXR8HNzLZG2XL7kIBtnGXm0WW1BElig=
Authentication-Results: mxback14o.mail.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by myt5-95c1fb78270f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id Rpbu72hy8I-7kn0dX9E; Sun, 13 Dec 2020 15:07:46 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <72e40cc93232dd0054977bb7037a5ac38e2165ba.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Date: Sun, 13 Dec 2020 15:07:46 +0300
In-Reply-To: <E1koKKT-0007bp-7N@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <83k0tmeq6f.fsf@HIDDEN>
 <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
 <E1koKKT-0007bp-7N@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.2 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Sun, 2020-12-13 at 00:53 -0500, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <hi-angel@HIDDEN>
> > Cc: 45200 <at> debbugs.gnu.org
> > Date: Sun, 13 Dec 2020 01:44:13 +0300
> > 
> > Alright, fair enough. I crafted up another testcase, it may be better. The
> > following code first temporarily disables GC, then it prints "hello" 1000000
> > times, and finally it calls GC manually.
> > 
> > I call `emacs -Q`, then measure PSS, then evaluate the code below, then
> > again measure PSS.
> > 
> >     (let ((i 1000000))
> >       (setq gc-cons-threshold most-positive-fixnum)
> >       (while (> i 0)
> >         (print "hello")
> >         (setq i (- i 1)))
> >       (garbage-collect))
> > 
> > The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and PSS
> > after is 266.3M. That is ≈200M of memory just vanished.
> 
> That memory hasn't vanished, it is in your libc's malloc arena,
> available for future allocations.  When and if it will be given back
> to the OS is up to the specifics of the malloc implementation.  E.g.,
> when I do the above on MS-Windows, where malloc is more eager to return
> memory to the OS, I end up with just 40 MB footprint, and if I then
> invoke GC manually, the memory goes down almost to the original value:
> 14 MB vs 12 MB after startup.

Your explanation seems likely, because such memory is reused by glibc on later
mallocs, and if I repeat the testcase, I can see that memory does not increase
further, meaning it got reused.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 13 Dec 2020 06:08:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Dec 13 01:08:47 2020
Received: from localhost ([127.0.0.1]:47313 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1koKYp-0004zn-82
	for submit <at> debbugs.gnu.org; Sun, 13 Dec 2020 01:08:47 -0500
Received: from eggs.gnu.org ([209.51.188.92]:56344)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1koKYn-0004zW-Bd
 for 45200 <at> debbugs.gnu.org; Sun, 13 Dec 2020 01:08:45 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:59664)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1koKYh-0000h9-R1; Sun, 13 Dec 2020 01:08:39 -0500
Received: from eliz by fencepost.gnu.org with local (Exim 4.82)
 (envelope-from <eliz@HIDDEN>)
 id 1koKYh-0004uX-IT; Sun, 13 Dec 2020 01:08:39 -0500
From: Eli Zaretskii <eliz@HIDDEN>
To: Lars Ingebrigtsen <larsi@HIDDEN>
In-Reply-To: <87czze7hrf.fsf@HIDDEN> (message from Lars Ingebrigtsen on Sat, 
 12 Dec 2020 23:59:32 +0100)
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <83k0tmeq6f.fsf@HIDDEN>
 <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
 <87czze7hrf.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Message-Id: <E1koKYh-0004uX-IT@HIDDEN>
Date: Sun, 13 Dec 2020 01:08:39 -0500
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org, hi-angel@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Lars Ingebrigtsen <larsi@HIDDEN>
> Cc: Eli Zaretskii <eliz@HIDDEN>,  45200 <at> debbugs.gnu.org
> Date: Sat, 12 Dec 2020 23:59:32 +0100
> 
> Konstantin Kharlamov <hi-angel@HIDDEN> writes:
> 
> > The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and
> > PSS after is 266.3M. That is ≈200M of memory just vanished.
> 
> I get similar results.  `M-x memory-report' shows that *Messages* and
> *Echo Area* each group to 9MB, and almost all of that is in `gap-size'.
> 
> But that's still a long way from 200MB.  Perhaps Emacs is bouncing the
> buffer area around a lot and isn't returning the data to the OS?  (Does
> Emacs ever return memory used by buffers to the OS?)

Yes, Emacs returns buffer memory to the OS when a buffer is killed.
Buffer text memory is usually obtained via mmap, and is returned when
no longer needed.  Buffer text memory is also returned to the OS when
buffers have their gap resized.

I think it isn't buffer memory that takes up most of the space here,
it's memory allocated for all the strings consed by this snippet.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 13 Dec 2020 05:54:05 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Dec 13 00:54:05 2020
Received: from localhost ([127.0.0.1]:47299 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1koKKb-0004PU-Bi
	for submit <at> debbugs.gnu.org; Sun, 13 Dec 2020 00:54:05 -0500
Received: from eggs.gnu.org ([209.51.188.92]:54838)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1koKKY-0004Ov-Se
 for 45200 <at> debbugs.gnu.org; Sun, 13 Dec 2020 00:54:03 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:59427)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1koKKT-0002Hx-FX; Sun, 13 Dec 2020 00:53:57 -0500
Received: from eliz by fencepost.gnu.org with local (Exim 4.82)
 (envelope-from <eliz@HIDDEN>)
 id 1koKKT-0007bp-7N; Sun, 13 Dec 2020 00:53:57 -0500
From: Eli Zaretskii <eliz@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
In-Reply-To: <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
 (message from Konstantin Kharlamov on Sun, 13 Dec 2020 01:44:13 +0300)
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <83k0tmeq6f.fsf@HIDDEN>
 <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Message-Id: <E1koKKT-0007bp-7N@HIDDEN>
Date: Sun, 13 Dec 2020 00:53:57 -0500
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Konstantin Kharlamov <hi-angel@HIDDEN>
> Cc: 45200 <at> debbugs.gnu.org
> Date: Sun, 13 Dec 2020 01:44:13 +0300
> 
> Alright, fair enough. I crafted up another testcase, it may be better. The following code first temporarily disables GC, then it prints "hello" 1000000 times, and finally it calls GC manually.
> 
> I call `emacs -Q`, then measure PSS, then evaluate the code below, then again measure PSS.
> 
>     (let ((i 1000000))
>       (setq gc-cons-threshold most-positive-fixnum)
>       (while (> i 0)
>         (print "hello")
>         (setq i (- i 1)))
>       (garbage-collect))
> 
> The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and PSS after is 266.3M. That is ≈200M of memory just vanished.

That memory hasn't vanished, it is in your libc's malloc arena,
available for future allocations.  When and if it will be given back
to the OS is up to the specifics of the malloc implementation.  E.g.,
when I do the above on MS-Windows, where malloc is more eager to return
memory to the OS, I end up with just 40 MB footprint, and if I then
invoke GC manually, the memory goes down almost to the original value:
14 MB vs 12 MB after startup.

There are many places on the Internet which explain why the memory
footprint of a program doesn't go back to the original value even
though the program frees all the heap memory it allocated.  I suggest
to read some of those explanations.

> Regarding, whether it is stack size:
> 
>      λ grep VmStk /proc/283047/status
>     VmStk:       132 kB
> 
> Apparently, it is not stack size.

This is a misunderstanding.  The space allocated for the stack doesn't
need to grow.  Values are pushed and popped there depending on the
callstack depth, and Emacs regards anything on the stack that looks
like a Lisp value or a pointer to a Lisp value as an indication that
this Lisp value is in use, and shouldn't be GC'd.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 12 Dec 2020 22:59:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Dec 12 17:59:43 2020
Received: from localhost ([127.0.0.1]:47101 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1koDrb-0001Zr-Nv
	for submit <at> debbugs.gnu.org; Sat, 12 Dec 2020 17:59:43 -0500
Received: from quimby.gnus.org ([95.216.78.240]:52088)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1koDra-0001Zc-95
 for 45200 <at> debbugs.gnu.org; Sat, 12 Dec 2020 17:59:42 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org;
 s=20200322; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID
 :In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=TDcnhUf7LkLJWm2aKW38GCv2UT1WRgLFFfgVVA9rfOk=; b=d+6r8CB6ij0YlLq1Y6ktF0EQ3p
 ehqpRsdvx7yHc26xczuhlikWBxBlwcasrfwIDwLkwyNLXdJD9wtd4kfPw3isXwGE+SMOsHJUHedoh
 ZPSx4HW1qxS4FgiwSMTJ7Seat96xEER8KxdWFE+zYp9j2A6MEd4aUIbxOYCCp6tqe75s=;
Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=xo)
 by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1koDrR-0003ZT-Gn; Sat, 12 Dec 2020 23:59:35 +0100
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <83k0tmeq6f.fsf@HIDDEN>
 <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAABGdBTUEAALGPC/xhBQAAACBj
 SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAADFBMVEWXaIHevcLBv03/
 //8lKWoeAAAAAWJLR0QDEQxM8gAAAAd0SU1FB+QMDBYpNyp3RKsAAAGOSURBVCjPTdDNatwwEAfw
 v428mD2ZIoekp01oSqqncEoDOSpgBbwn92DD+ilcQ8Pik09huyfXZI09T9kZd/shjNDPMxqNhMDi
 3wiW2RCPHIpoPlbPpzHLLFS2LV6z41js5k5QvrrjWGYCJ+jG8scZz8XhY7HsKYtyVxxuaVt95Woz
 URv+PkXx7FJcJ/FCD4H1WsV/zh141fdU/cXdKZU051ziOTMlZ1jnaLYLnpiOSNIQPxUucYbSIBGc
 OM9Q7Tgx3t0u+OkcoB8EFZ0E8f8oXuLUui3Ngp3SNno445PWXOY9keBKR4x3ZAShjvhon4zcLOwE
 IPoGKGU7rBltz42Gtsaav6GfoC454neJGfoKKso1g19+aqD8/AC/NgNNn6Gu8wkRBBy5T4jrmGmp
 fd+YidET8QFBvV1ghhd5OJ8Ed/24gCP0Fm5a4MMeggkbrqb33OaGavgDd8DrVS+38LGqvQQXNa9v
 WqjmApGWAAasGnA3OW/b9wgEaz4pbG8gz70A/RrBF0mTMfkIHv+Aml/OxZANo//3HgAAACV0RVh0
 ZGF0ZTpjcmVhdGUAMjAyMC0xMi0xMlQyMjo0MTo1NCswMDowMH/P9kgAAAAldEVYdGRhdGU6bW9k
 aWZ5ADIwMjAtMTItMTJUMjI6NDE6NTQrMDA6MDAOkk70AAAAAElFTkSuQmCC
X-Now-Playing: JD Twitch, Optimo's _Mutant Disco Vol 4_: "Contort Yourself
 Optimo (Re Edit Mix)"
Date: Sat, 12 Dec 2020 23:59:32 +0100
In-Reply-To: <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
 (Konstantin Kharlamov's message of "Sun, 13 Dec 2020 01:44:13 +0300")
Message-ID: <87czze7hrf.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 
 Content preview:  Konstantin Kharlamov <hi-angel@HIDDEN> writes: > The loop
    takes 20-30 seconds for me, I think. PSS before is ≈41M, and > PSS after
    is 266.3M. That is ≈200M of memory just vanished. I get similar results.
    `M-x memory-report' shows that *Messages* and *Echo Area* each group to 9MB,
    and almost all of that is in `gap-size'. 
 
 Content analysis details:   (-2.9 points, 5.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
                             [score: 0.0000]
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 45200
Cc: Eli Zaretskii <eliz@HIDDEN>, 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Konstantin Kharlamov <hi-angel@HIDDEN> writes:

> The loop takes 20-30 seconds for me, I think. PSS before is =E2=89=8841M,=
 and
> PSS after is 266.3M. That is =E2=89=88200M of memory just vanished.

I get similar results.  `M-x memory-report' shows that *Messages* and
*Echo Area* each group to 9MB, and almost all of that is in `gap-size'.

But that's still a long way from 200MB.  Perhaps Emacs is bouncing the
buffer area around a lot and isn't returning the data to the OS?  (Does
Emacs ever return memory used by buffers to the OS?)

--=20
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 12 Dec 2020 22:44:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Dec 12 17:44:23 2020
Received: from localhost ([127.0.0.1]:47081 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1koDcl-0001Ao-Jb
	for submit <at> debbugs.gnu.org; Sat, 12 Dec 2020 17:44:23 -0500
Received: from forward100j.mail.yandex.net ([5.45.198.240]:48976)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1koDcj-0001AX-V7
 for 45200 <at> debbugs.gnu.org; Sat, 12 Dec 2020 17:44:22 -0500
Received: from mxback30g.mail.yandex.net (mxback30g.mail.yandex.net
 [IPv6:2a02:6b8:c03:7b3:0:640:11da:816c])
 by forward100j.mail.yandex.net (Yandex) with ESMTP id 7115050E05EA;
 Sun, 13 Dec 2020 01:44:15 +0300 (MSK)
Received: from sas1-f4dc5f2fc86f.qloud-c.yandex.net
 (sas1-f4dc5f2fc86f.qloud-c.yandex.net [2a02:6b8:c08:cb28:0:640:f4dc:5f2f])
 by mxback30g.mail.yandex.net (mxback/Yandex) with ESMTP id lJ20sGCHhX-iF68mw13;
 Sun, 13 Dec 2020 01:44:15 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1607813055; bh=CsczHDEKLHpHQTtgc48u6y+nG1g/IH3PtkJUT8dU74k=;
 h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date;
 b=j8O+CCChqjnDxzp8DcAh1XHwHEH9wg9KYOmmv/raicWSXyf9urLD9INgcvdbPCmT7
 j9ZmM1GxmR2gEY88hrlEXeVLk3ykiORHmedr6w8UuI/gdDgZ6OSX/wPUzH1ul17aBJ
 T36N+O5qrC5pS/FtTXcOLZCZbpBF8FADI18LC+S8=
Authentication-Results: mxback30g.mail.yandex.net;
 dkim=pass header.i=@yandex.ru
Received: by sas1-f4dc5f2fc86f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id xeQDxoCdlM-iEI4txB7; Sun, 13 Dec 2020 01:44:14 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@HIDDEN>
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Date: Sun, 13 Dec 2020 01:44:13 +0300
In-Reply-To: <83k0tmeq6f.fsf@HIDDEN>
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 <83k0tmeq6f.fsf@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.2 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Sat, 2020-12-12 at 22:15 +0200, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <hi-angel@HIDDEN>
> > Date: Sat, 12 Dec 2020 21:43:10 +0300
> >
> > # Steps to reproduce:
> >
> > 1. Run `mkdir /tmp/.emacs.d`
> > 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS
> > 3. Create a file /tmp/.emacs.d/early-init.el with content:
> >
> >     ;; only run garbage collection on idle
> >     (setq gc-cons-threshold most-positive-fixnum)
> >     (run-with-idle-timer 2 t (lambda () (garbage-collect)))
> >
> > 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure
> > its PSS
> >
> > ## Expected
> >
> > Size has no statistically-significant difference, because in both cases we
> > garbage-collected memory.
> >
> > ## Actual
> >
> > Size without calling explicit garbage-collect, from 3 runs, varied around
> > 41.2..41.7 MB.
> >
> > Size afterwards, also 3 runs, varied around 45.4..45.5 MB.
>
> Could be simply the effect of different stack size, since Emacs's GC
> is conservative, and when there's doubt whether something is a live
> object, it won't GC it.
>
> I think more specific and detailed evidence is needed to prove your
> case: which objects were not GC'ed and why.

Alright, fair enough. I crafted up another testcase, it may be better. The following code first temporarily disables GC, then it prints "hello" 1000000 times, and finally it calls GC manually.

I call `emacs -Q`, then measure PSS, then evaluate the code below, then again measure PSS.

    (let ((i 1000000))
      (setq gc-cons-threshold most-positive-fixnum)
      (while (> i 0)
        (print "hello")
        (setq i (- i 1)))
      (garbage-collect))

The loop takes 20-30 seconds for me, I think. PSS before is ≈41M, and PSS after is 266.3M. That is ≈200M of memory just vanished.

Regarding, whether it is stack size:

     λ grep VmStk /proc/283047/status
    VmStk:       132 kB

Apparently, it is not stack size.






Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at 45200 <at> debbugs.gnu.org:


Received: (at 45200) by debbugs.gnu.org; 12 Dec 2020 20:16:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Dec 12 15:16:15 2020
Received: from localhost ([127.0.0.1]:46762 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1koBJP-0001K0-39
	for submit <at> debbugs.gnu.org; Sat, 12 Dec 2020 15:16:15 -0500
Received: from eggs.gnu.org ([209.51.188.92]:53982)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1koBJO-0001Jc-42
 for 45200 <at> debbugs.gnu.org; Sat, 12 Dec 2020 15:16:14 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:51314)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1koBJH-0002dH-DS; Sat, 12 Dec 2020 15:16:08 -0500
Received: from [176.228.60.248] (port=1537 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1koBJG-000875-PR; Sat, 12 Dec 2020 15:16:07 -0500
Date: Sat, 12 Dec 2020 22:15:52 +0200
Message-Id: <83k0tmeq6f.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Konstantin Kharlamov <hi-angel@HIDDEN>
In-Reply-To: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
 (message from Konstantin Kharlamov on Sat, 12 Dec 2020 21:43:10 +0300)
Subject: Re: bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory
References: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 45200
Cc: 45200 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Konstantin Kharlamov <hi-angel@HIDDEN>
> Date: Sat, 12 Dec 2020 21:43:10 +0300
> 
> # Steps to reproduce:
> 
> 1. Run `mkdir /tmp/.emacs.d`
> 2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS
> 3. Create a file /tmp/.emacs.d/early-init.el with content:
> 
>     ;; only run garbage collection on idle
>     (setq gc-cons-threshold most-positive-fixnum)
>     (run-with-idle-timer 2 t (lambda () (garbage-collect)))
> 
> 4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure its PSS
> 
> ## Expected
> 
> Size has no statistically-significant difference, because in both cases we garbage-collected memory.
> 
> ## Actual
> 
> Size without calling explicit garbage-collect, from 3 runs, varied around 41.2..41.7 MB.
> 
> Size afterwards, also 3 runs, varied around 45.4..45.5 MB.

Could be simply the effect of different stack size, since Emacs's GC
is conservative, and when there's doubt whether something is a live
object, it won't GC it.

I think more specific and detailed evidence is needed to prove your
case: which objects were not GC'ed and why.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 12 Dec 2020 18:43:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Dec 12 13:43:23 2020
Received: from localhost ([127.0.0.1]:46507 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ko9rX-0000u4-73
	for submit <at> debbugs.gnu.org; Sat, 12 Dec 2020 13:43:23 -0500
Received: from lists.gnu.org ([209.51.188.17]:38118)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hi-angel@HIDDEN>) id 1ko9rV-0000tw-UO
 for submit <at> debbugs.gnu.org; Sat, 12 Dec 2020 13:43:22 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:41584)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <hi-angel@HIDDEN>)
 id 1ko9rV-0003wd-I9
 for bug-gnu-emacs@HIDDEN; Sat, 12 Dec 2020 13:43:21 -0500
Received: from forward106p.mail.yandex.net ([77.88.28.109]:57409)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <hi-angel@HIDDEN>)
 id 1ko9rS-0002Tr-D0
 for bug-gnu-emacs@HIDDEN; Sat, 12 Dec 2020 13:43:20 -0500
Received: from mxback9g.mail.yandex.net (mxback9g.mail.yandex.net
 [IPv6:2a02:6b8:c03:7aa:0:640:d10:d80f])
 by forward106p.mail.yandex.net (Yandex) with ESMTP id 73FF11C816EE
 for <bug-gnu-emacs@HIDDEN>; Sat, 12 Dec 2020 21:43:11 +0300 (MSK)
Received: from iva8-174eb672ffa9.qloud-c.yandex.net
 (iva8-174eb672ffa9.qloud-c.yandex.net [2a02:6b8:c0c:b995:0:640:174e:b672])
 by mxback9g.mail.yandex.net (mxback/Yandex) with ESMTP id ZktHbZDohB-hBlKKG4K; 
 Sat, 12 Dec 2020 21:43:11 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail;
 t=1607798591; bh=oxv1AY/uOWizwjT7CWACUwRcNqxVlyMGswP6MDIAXkE=;
 h=To:From:Subject:Message-ID:Date;
 b=fvJ37UX4cPhGpjoZHd1jmR4+Q/RJTiOLQLEnkgzTqS2kWOgqFYZuXlEPQb/74VlAw
 JBWnzEARCupotd5Pjt0VPnpVIAPZKSetO1H1ChSmbkkb4uJVgFdW+8G2cKYGoDsgGb
 R2E1n3mZhyriGoDP/rYPHKORu07AjMCIZ+aozkeY=
Authentication-Results: mxback9g.mail.yandex.net; dkim=pass header.i=@yandex.ru
Received: by iva8-174eb672ffa9.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id XHIvb3LCoV-hAI0l86g; Sat, 12 Dec 2020 21:43:10 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
Message-ID: <dd4a240ab3ff14d8f04d02b2504611a4d9f3f816.camel@HIDDEN>
Subject: Memory leaks: (garbage-collect) fails to reclaim memory
From: Konstantin Kharlamov <hi-angel@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Date: Sat, 12 Dec 2020 21:43:10 +0300
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.2 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=77.88.28.109; envelope-from=hi-angel@HIDDEN;
 helo=forward106p.mail.yandex.net
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.7 (/)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

The problem is basically: (garbage-collect) refuses to collect some of the memory. It is more visible if you delay garbage collection, which is what I do for for performance-related reasons. My config has the following snippet:

    ;; only run garbage collection on idle
    (setq gc-cons-threshold most-positive-fixnum)
    (run-with-idle-timer 2 t (lambda () (garbage-collect)))

How much memory gets lost depends on configuration. For me right after start the difference is 40 MB: ≈60 MB is Emacs PSS size without above code, and ≈100 MB it is when garbage-collection is delayed, *after* I run explicitly (garbage-collect). It is less visible without any other configs, nonetheless it's visible.

# Steps to reproduce:

1. Run `mkdir /tmp/.emacs.d`
2. Run emacs as `HOME=/tmp/ emacs`, and measure its PSS
3. Create a file /tmp/.emacs.d/early-init.el with content:

    ;; only run garbage collection on idle
    (setq gc-cons-threshold most-positive-fixnum)
    (run-with-idle-timer 2 t (lambda () (garbage-collect)))

4. Run emacs as `HOME=/tmp/ emacs`, evaluate (garbage-collect), then measure its PSS

## Expected

Size has no statistically-significant difference, because in both cases we garbage-collected memory.

## Actual

Size without calling explicit garbage-collect, from 3 runs, varied around 41.2..41.7 MB.

Size afterwards, also 3 runs, varied around 45.4..45.5 MB.

This is 4 MB lost. While not much, but as I mentioned it grows as much as to 40MB. The Emacs whose emacslient I'm using to write this email has size 218 MB, and now I wouldn't be surprised if much of that is actually a leaked memory.

# Version

Emacs-git 28.0.50, build from commit abd15e088e99





Acknowledgement sent to Konstantin Kharlamov <hi-angel@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#45200; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 24 Jan 2021 21:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.