GNU bug report logs - #46900
28.0.50; GC uses excessive amounts of stack

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46900 in the body.
You can then email your comments to 46900 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; GC uses excessive amounts of stack
Date: Wed, 3 Mar 2021 15:44:33 +0000

This is a long-standing issue.

Currently, GC is written to recurse when marking objects. That means
for every level of recursion that we see, a new function stack frame
is being created on the C stack; on this x86_64-pc-linux-gnu system
running Debian GNU/Linux, in an optimized build, that means 80 bytes
of stack space are reserved for each level of recursion.

That number could be much closer to 8, which is the number of bytes of
relevant data in those 80 bytes of stack space.

My proposal is to continue allocating the mark queue on the stack (not
the heap), but to do so in larger chunks, allocating enough space for
N objects and recursing, in the C sense, only when all N object slots
have been used.

In addition to saving stack space, this should also help, at least,
older systems with limited return address prediction stacks.

IMHO, it would also help, not hinder, debugging of GC internals, as
more context would be available in the function frame. Indeed, the
patch I will send once this has a bug number keeps all of the large
"soft" stack frames in a linked list so we can easily walk them.

In further patches, I'd like to change things so we no longer recurse
through symbols so enthusiastically, building up tens of thousands of
objects on the mark queue.

In the interest of full disclosure, I have several ulterior motives
for this: the WebAssembly implementation I'm using doesn't allow deep
nesting of calls; I'd like to look into parallelizing GC for which
this might be useful; and I suspect native-comp will benefit more from
further work in this area, too.

Message #8 received at 46900 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pip Cet <pipcet <at> gmail.com>
Cc: 46900 <at> debbugs.gnu.org
Subject: Re: bug#46900: 28.0.50; GC uses excessive amounts of stack
Date: Wed, 03 Mar 2021 18:28:23 +0200

> From: Pip Cet <pipcet <at> gmail.com>
> Date: Wed, 3 Mar 2021 15:44:33 +0000
> 
> Currently, GC is written to recurse when marking objects. That means
> for every level of recursion that we see, a new function stack frame
> is being created on the C stack; on this x86_64-pc-linux-gnu system
> running Debian GNU/Linux, in an optimized build, that means 80 bytes
> of stack space are reserved for each level of recursion.
> 
> That number could be much closer to 8, which is the number of bytes of
> relevant data in those 80 bytes of stack space.
> 
> My proposal is to continue allocating the mark queue on the stack (not
> the heap), but to do so in larger chunks, allocating enough space for
> N objects and recursing, in the C sense, only when all N object slots
> have been used.
> 
> In addition to saving stack space, this should also help, at least,
> older systems with limited return address prediction stacks.

FWIW, I'm only interested in changes to GC if they bring significant
gains, such as replacing the mark & sweep algorithm with something
more modern and powerful.  Otherwise, experience shows that messing
with GC for smaller gains is not worth it: it destabilizes Emacs for
no good reason, since users rarely if ever see any significant
improvements.

> In further patches, I'd like to change things so we no longer recurse
> through symbols so enthusiastically, building up tens of thousands of
> objects on the mark queue.

I'd like to hear and understand the long-term plan before I agree to
any such changes.  And in general, I'd like to see the plan
implemented in large steps, not small changes.  We have bad experience
with the latter: over the years there were a couple of attempts to do
it that led nowhere, and we are left with their leftovers that don't
make much sense on their own, and just make the sources harder to read
and understand, let alone change.

So please: no small changes in GC, thank you.

> In the interest of full disclosure, I have several ulterior motives
> for this: the WebAssembly implementation I'm using doesn't allow deep
> nesting of calls; I'd like to look into parallelizing GC for which
> this might be useful; and I suspect native-comp will benefit more from
> further work in this area, too.

If the goal is to make GC parallel, let's first discuss the idea for
how to do it, and (assuming we agree to go that way) let's start with
it, not present it as some long-term goal.

Thanks.

Message #11 received at 46900 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Pip Cet <pipcet <at> gmail.com>
Cc: 46900 <at> debbugs.gnu.org
Subject: Re: bug#46900: 28.0.50; GC uses excessive amounts of stack
Date: Thu, 04 Mar 2021 00:32:05 -0500

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Could you please tell me what you're doing in Emacs with webassembly?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Message #14 received at 46900 <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: rms <at> gnu.org
Cc: 46900 <at> debbugs.gnu.org
Subject: Re: bug#46900: 28.0.50; GC uses excessive amounts of stack
Date: Thu, 4 Mar 2021 07:53:08 +0000

On Thu, Mar 4, 2021 at 5:32 AM Richard Stallman <rms <at> gnu.org> wrote:
> Could you please tell me what you're doing in Emacs with webassembly?

Running it locally, in batch mode, with the SpiderMonkey shell (not a
web browser). There's no "Web" involved, at present. I'm vaguely
considering running Emacs on the non-free hand-me-down Android phone I
have, which I appear to be unable to install real software on, but my
interest in environments which aren't clearly free is somewhat
limited.

It did help shake out a couple of bugs, though. Some of them, unlike
this one, may even be fixed.

Pip

Message #19 received at 46900-done <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: 31362-done <at> debbugs.gnu.org, 46900-done <at> debbugs.gnu.org
Cc: Noam Postavsky <npostavs <at> gmail.com>, Pip Cet <pipcet <at> gmail.com>,
 Tino Calancha <tino.calancha <at> gmail.com>
Subject: GC stack overflow fixed
Date: Mon, 4 Apr 2022 19:24:06 +0200

Now that bug#54698 is resolved, we can close bug#31362 and bug#46900 which seem to have been about that, essentially.

If that didn't do it for you, please let us know.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.