GNU bug report logs - #46916
28.0.50; pure_alloc(10424, ...) fails badly when pure space is exhausted

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46916 in the body.
You can then email your comments to 46916 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50;
 pure_alloc(10424, ...) fails badly when pure space is exhausted
Date: Thu, 4 Mar 2021 12:24:21 +0000

Pure space should be removed. But until it is, it shouldn't exhaust
all virtual memory, crashing the machines of the unwary ones who tried
to build emacs without a ulimit...

The problem I ran into was that the native-comp branch calls Fpurecopy
on comp_u->data_vec. In my case, pure space was already exhausted and
data_vec had 1302 elements.

That caused purecopy() to call pure_alloc with a size argument of
10424. pure_alloc allocated another 10000 bytes of fake pure space,
found out those weren't enough, so it allocated another 10000 bytes of
fake pure space, which weren't enough, etc., until the process started
swapping.

I'm not sure who's the culprit here: the native-comp branch tries to
purecopy a large vector, which is a problem; purecopy() saw that
request and passed it on to pure_alloc(), rather than returning the
impure vector because it was too large. And pure_alloc() certainly
should fail more gracefully than it did (there's an eassert() to
prevent this situation, but that should be unconditionally compiled
instead).

I was going to write a message to emacs-devel (I'd started before
hitting this bug) proposing to remove pure-space now, but I think it's
fair to have a separate bug report if we decide to keep pure space
after all.

Message #8 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: 46916 <at> debbugs.gnu.org
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Thu, 4 Mar 2021 12:44:49 +0000

[Message part 1 (text/plain, inline)]

On Thu, Mar 4, 2021 at 12:26 PM Pip Cet <pipcet <at> gmail.com> wrote:
> Pure space should be removed. But until it is, it shouldn't exhaust
> all virtual memory, crashing the machines of the unwary ones who tried
> to build emacs without a ulimit...

This minimal patch "fixes" the issue here, by throwing a fatal error
in this unusual and avoidable situation. No new test as not testable.


Pip

[0001-Don-t-exhaust-memory-on-large-pure-space-allocations.patch (application/x-patch, attachment)]

Message #11 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pip Cet <pipcet <at> gmail.com>
Cc: 46916 <at> debbugs.gnu.org
Subject: Re: bug#46916: 28.0.50;
 pure_alloc(10424, ...) fails badly when pure space is exhausted
Date: Thu, 04 Mar 2021 16:15:24 +0200

> From: Pip Cet <pipcet <at> gmail.com>
> Date: Thu, 4 Mar 2021 12:44:49 +0000
> 
> On Thu, Mar 4, 2021 at 12:26 PM Pip Cet <pipcet <at> gmail.com> wrote:
> > Pure space should be removed. But until it is, it shouldn't exhaust
> > all virtual memory, crashing the machines of the unwary ones who tried
> > to build emacs without a ulimit...
> 
> This minimal patch "fixes" the issue here, by throwing a fatal error
> in this unusual and avoidable situation. No new test as not testable.

Just enlarge the pure space amount, and be done with it.  That's what
we always do when this happens.

Message #14 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 46916 <at> debbugs.gnu.org, Pip Cet <pipcet <at> gmail.com>
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Sat, 30 Oct 2021 19:56:42 -0700

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Pip Cet <pipcet <at> gmail.com>
>> Date: Thu, 4 Mar 2021 12:44:49 +0000
>>
>> On Thu, Mar 4, 2021 at 12:26 PM Pip Cet <pipcet <at> gmail.com> wrote:
>> > Pure space should be removed. But until it is, it shouldn't exhaust
>> > all virtual memory, crashing the machines of the unwary ones who tried
>> > to build emacs without a ulimit...
>>
>> This minimal patch "fixes" the issue here, by throwing a fatal error
>> in this unusual and avoidable situation. No new test as not testable.
>
> Just enlarge the pure space amount, and be done with it.  That's what
> we always do when this happens.

There's a patch here that was never installed.  Eli didn't sound to
enthusiastic.  Is there anything more left to do here, or should this
just be closed?  Or should purespace be enlarged?

Message #19 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 46916 <at> debbugs.gnu.org, pipcet <at> gmail.com
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Sun, 31 Oct 2021 09:13:57 +0200

> From: Stefan Kangas <stefan <at> marxist.se>
> Date: Sat, 30 Oct 2021 19:56:42 -0700
> Cc: Pip Cet <pipcet <at> gmail.com>, 46916 <at> debbugs.gnu.org
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Pip Cet <pipcet <at> gmail.com>
> >> Date: Thu, 4 Mar 2021 12:44:49 +0000
> >>
> >> On Thu, Mar 4, 2021 at 12:26 PM Pip Cet <pipcet <at> gmail.com> wrote:
> >> > Pure space should be removed. But until it is, it shouldn't exhaust
> >> > all virtual memory, crashing the machines of the unwary ones who tried
> >> > to build emacs without a ulimit...
> >>
> >> This minimal patch "fixes" the issue here, by throwing a fatal error
> >> in this unusual and avoidable situation. No new test as not testable.
> >
> > Just enlarge the pure space amount, and be done with it.  That's what
> > we always do when this happens.
> 
> There's a patch here that was never installed.  Eli didn't sound to
> enthusiastic.  Is there anything more left to do here, or should this
> just be closed?  Or should purespace be enlarged?

We could enlarge the pure space size, yes.  but since no one has yet
reported any problems with that, I tend to do nothing for now.

Message #22 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 46916 <at> debbugs.gnu.org
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Sun, 31 Oct 2021 09:41:58 +0000

On Sun, Oct 31, 2021 at 2:56 AM Stefan Kangas <stefan <at> marxist.se> wrote:
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> From: Pip Cet <pipcet <at> gmail.com>
> >> Date: Thu, 4 Mar 2021 12:44:49 +0000
> >>
> >> On Thu, Mar 4, 2021 at 12:26 PM Pip Cet <pipcet <at> gmail.com> wrote:
> >> > Pure space should be removed. But until it is, it shouldn't exhaust
> >> > all virtual memory, crashing the machines of the unwary ones who tried
> >> > to build emacs without a ulimit...
> >>
> >> This minimal patch "fixes" the issue here, by throwing a fatal error
> >> in this unusual and avoidable situation. No new test as not testable.
> >
> > Just enlarge the pure space amount, and be done with it.  That's what
> > we always do when this happens.
>
> There's a patch here that was never installed.  Eli didn't sound to
> enthusiastic.  Is there anything more left to do here, or should this
> just be closed?  Or should purespace be enlarged?

I think Eli's suggestion was sarcastic, but I'm not sure.

I have no objections to this report being closed, in case that's
relevant: it's a clear bug but Eli appears to prefer not to fix it,
and I can live with that. In fact, isn't that what WONTFIX is for?

Thanks for taking the time to look into these issues!

Pip

Message #25 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pip Cet <pipcet <at> gmail.com>
Cc: stefan <at> marxist.se, 46916 <at> debbugs.gnu.org
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Sun, 31 Oct 2021 13:29:50 +0200

> From: Pip Cet <pipcet <at> gmail.com>
> Date: Sun, 31 Oct 2021 09:41:58 +0000
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 46916 <at> debbugs.gnu.org
> 
> On Sun, Oct 31, 2021 at 2:56 AM Stefan Kangas <stefan <at> marxist.se> wrote:
> >
> > Eli Zaretskii <eliz <at> gnu.org> writes:
> >
> > >> From: Pip Cet <pipcet <at> gmail.com>
> > >> Date: Thu, 4 Mar 2021 12:44:49 +0000
> > >>
> > >> On Thu, Mar 4, 2021 at 12:26 PM Pip Cet <pipcet <at> gmail.com> wrote:
> > >> > Pure space should be removed. But until it is, it shouldn't exhaust
> > >> > all virtual memory, crashing the machines of the unwary ones who tried
> > >> > to build emacs without a ulimit...
> > >>
> > >> This minimal patch "fixes" the issue here, by throwing a fatal error
> > >> in this unusual and avoidable situation. No new test as not testable.
> > >
> > > Just enlarge the pure space amount, and be done with it.  That's what
> > > we always do when this happens.
> >
> > There's a patch here that was never installed.  Eli didn't sound to
> > enthusiastic.  Is there anything more left to do here, or should this
> > just be closed?  Or should purespace be enlarged?
> 
> I think Eli's suggestion was sarcastic, but I'm not sure.

It wasn't, FWIW.

Message #28 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Pip Cet <pipcet <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 46916 <at> debbugs.gnu.org
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Sun, 31 Oct 2021 07:02:47 -0700

Pip Cet <pipcet <at> gmail.com> writes:

> I have no objections to this report being closed, in case that's
> relevant: it's a clear bug but Eli appears to prefer not to fix it,
> and I can live with that. In fact, isn't that what WONTFIX is for?

Hmm, so I reviewed all this more carefully, and I think I agree that we
might as well fix this bug.

OT1H, the use-case you present is quite unusual, as almost no one will
call `purespace' outside of Emacs core development.

OTOH, it is a pretty bad behavior you observed, and the patch seems
minimal enough to me.  IIUC, outside of bootstrapping Emacs, we
won't/shouldn't call 'purespace' very often, so the code-path will not
be exercised in normal use (which means your patch won't hurt), and it
might save someones system from swapping to death.

Eli, I'm not sure if your previous reply was just intended to say "we
normally just increase the size of purespace", or if you also object to
installing this patch?

Message #31 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 46916 <at> debbugs.gnu.org, pipcet <at> gmail.com
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Sun, 31 Oct 2021 16:23:28 +0200

> From: Stefan Kangas <stefan <at> marxist.se>
> Date: Sun, 31 Oct 2021 07:02:47 -0700
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 46916 <at> debbugs.gnu.org
> 
> Eli, I'm not sure if your previous reply was just intended to say "we
> normally just increase the size of purespace", or if you also object to
> installing this patch?

I don't see a need for anything more complex than enlarging the pure
space, yes.  in fact, I don't even see a reason to enlarge the pure
space, since no one else reported a problem with the existing amount
of pure space.

Message #34 received at 46916 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 46916 <at> debbugs.gnu.org, pipcet <at> gmail.com
Subject: Re: bug#46916: 28.0.50; pure_alloc(10424, ...) fails badly when pure
 space is exhausted
Date: Fri, 5 Nov 2021 00:11:16 -0700

tags 46916 wontfix
close 46916
thanks

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Stefan Kangas <stefan <at> marxist.se>
>> Date: Sun, 31 Oct 2021 07:02:47 -0700
>> Cc: Eli Zaretskii <eliz <at> gnu.org>, 46916 <at> debbugs.gnu.org
>>
>> Eli, I'm not sure if your previous reply was just intended to say "we
>> normally just increase the size of purespace", or if you also object to
>> installing this patch?
>
> I don't see a need for anything more complex than enlarging the pure
> space, yes.  in fact, I don't even see a reason to enlarge the pure
> space, since no one else reported a problem with the existing amount
> of pure space.

OK, so I'm closing this bug as wontfix.  I somewhat agree with Pip Cet
that we could just fix this, but given that the plan is to eventually
remove purespace, we also might as well leave it all alone.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.