GNU bug report logs - #51110
Native-comp leaks fds open on /dev/ptmx

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 51110 in the body.
You can then email your comments to 51110 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stephen Gildea <stepheng+emacs <at> gildea.com>
To: bug-emacs <at> gnu.org
Subject: Native-comp leaks fds open on /dev/ptmx
Date: Sat, 09 Oct 2021 08:45:25 -0700

Version: 28.0.60
Severity: important

Native-comp leaks fds open on /dev/ptmx if you change 'invocation-name'
and then run a function that loads libraries and causes some compilation.
Emacs seems to compile each file multiple times (16 or more), and it soon
runs out of file descriptors.

To reproduce:

Use a newly built (configure --with-native-compilation) Emacs that has
never (successfully) run MH-E before; if your eln-cache has already been
populated, you will need to delete the cache to repeat the failure:

rm -r ~/.emacs.d/eln-cache

The Emacs must be interactive, and you can reproduce quickly as follows:

src/emacs -Q --eval '(setq invocation-name "emacs-leaks")' --eval '(mh-version)'

In a few seconds the echo area will say: "Creating pipe: Too many open files"

What should appear in the echo area: "nmh 1.7.1 installed as MH variant"
or maybe: "No MH variant found on the system"

Message #8 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stephen Gildea <stepheng+emacs <at> gildea.com>, Andrea Corallo <akrl <at> sdf.org>
Cc: 51110 <at> debbugs.gnu.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Sat, 09 Oct 2021 18:57:04 +0300

> From: Stephen Gildea <stepheng+emacs <at> gildea.com>
> Date: Sat, 09 Oct 2021 08:45:25 -0700
> 
> src/emacs -Q --eval '(setq invocation-name "emacs-leaks")' --eval '(mh-version)'

What real-life use case requires you to modify invocation-name?  That
variable is supposed to be read-only: it tells the program name used
to invoke Emacs of the running session.  If you change it, other
features will become confused.

IOW, I don't think we want to support this kind of usage.

Message #11 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Stephen Gildea <stepheng+emacs <at> gildea.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 51110 <at> debbugs.gnu.org, Andrea Corallo <akrl <at> sdf.org>
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Sat, 09 Oct 2021 10:01:43 -0700

The recipe I gave provides a simple way to show the unbounded resource
use of Emacs while compiling; I don't propose it as an actual use case.

Apparently what's happening is that Emacs forks off a child Emacs to do
the compile and allocates a pty to the child.  These ptys are not
closed until the compile completes.  It seems Emacs needs to better
manage the number of simultaneous open ptys.  Or at least be able to
wait and retry if an open fails.

Message #14 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stephen Gildea <stepheng+emacs <at> gildea.com>
Cc: 51110 <at> debbugs.gnu.org, akrl <at> sdf.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Sat, 09 Oct 2021 20:15:26 +0300

> From: Stephen Gildea <stepheng+emacs <at> gildea.com>
> cc: Andrea Corallo <akrl <at> sdf.org>, 51110 <at> debbugs.gnu.org
> Date: Sat, 09 Oct 2021 10:01:43 -0700
> 
> Apparently what's happening is that Emacs forks off a child Emacs to do
> the compile and allocates a pty to the child.  These ptys are not
> closed until the compile completes.  It seems Emacs needs to better
> manage the number of simultaneous open ptys.  Or at least be able to
> wait and retry if an open fails.

Emacs limits the number of these background processes, see
native-comp-async-jobs-number.  (The value zero means use half of the
number of execution units available in the system's processor.)  So we
generally shouldn't have more than that number of open pipes at any
given time.  For the obvious reasons they cannot be closed until the
sub-process exits.

Message #17 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Stephen Gildea <stepheng+emacs <at> gildea.com>, 51110 <at> debbugs.gnu.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Tue, 30 Nov 2021 20:13:07 +0000

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Stephen Gildea <stepheng+emacs <at> gildea.com>
>> cc: Andrea Corallo <akrl <at> sdf.org>, 51110 <at> debbugs.gnu.org
>> Date: Sat, 09 Oct 2021 10:01:43 -0700
>> 
>> Apparently what's happening is that Emacs forks off a child Emacs to do
>> the compile and allocates a pty to the child.  These ptys are not
>> closed until the compile completes.  It seems Emacs needs to better
>> manage the number of simultaneous open ptys.  Or at least be able to
>> wait and retry if an open fails.
>
> Emacs limits the number of these background processes, see
> native-comp-async-jobs-number.  (The value zero means use half of the
> number of execution units available in the system's processor.)  So we
> generally shouldn't have more than that number of open pipes at any
> given time.  For the obvious reasons they cannot be closed until the
> sub-process exits.

Is there any action we need/can take on this bug?

Thanks

  Andrea

Message #20 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: stepheng+emacs <at> gildea.com, 51110 <at> debbugs.gnu.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Tue, 30 Nov 2021 22:18:51 +0200

> From: Andrea Corallo <akrl <at> sdf.org>
> Cc: Stephen Gildea <stepheng+emacs <at> gildea.com>, 51110 <at> debbugs.gnu.org
> Date: Tue, 30 Nov 2021 20:13:07 +0000
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Stephen Gildea <stepheng+emacs <at> gildea.com>
> >> cc: Andrea Corallo <akrl <at> sdf.org>, 51110 <at> debbugs.gnu.org
> >> Date: Sat, 09 Oct 2021 10:01:43 -0700
> >> 
> >> Apparently what's happening is that Emacs forks off a child Emacs to do
> >> the compile and allocates a pty to the child.  These ptys are not
> >> closed until the compile completes.  It seems Emacs needs to better
> >> manage the number of simultaneous open ptys.  Or at least be able to
> >> wait and retry if an open fails.
> >
> > Emacs limits the number of these background processes, see
> > native-comp-async-jobs-number.  (The value zero means use half of the
> > number of execution units available in the system's processor.)  So we
> > generally shouldn't have more than that number of open pipes at any
> > given time.  For the obvious reasons they cannot be closed until the
> > sub-process exits.
> 
> Is there any action we need/can take on this bug?

I don't think there's anything we _can_ do here, and I tried to
explain why.

Message #23 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: stepheng+emacs <at> gildea.com, 51110 <at> debbugs.gnu.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Tue, 30 Nov 2021 20:24:59 +0000

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Andrea Corallo <akrl <at> sdf.org>
>> Cc: Stephen Gildea <stepheng+emacs <at> gildea.com>, 51110 <at> debbugs.gnu.org
>> Date: Tue, 30 Nov 2021 20:13:07 +0000
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> >> From: Stephen Gildea <stepheng+emacs <at> gildea.com>
>> >> cc: Andrea Corallo <akrl <at> sdf.org>, 51110 <at> debbugs.gnu.org
>> >> Date: Sat, 09 Oct 2021 10:01:43 -0700
>> >> 
>> >> Apparently what's happening is that Emacs forks off a child Emacs to do
>> >> the compile and allocates a pty to the child.  These ptys are not
>> >> closed until the compile completes.  It seems Emacs needs to better
>> >> manage the number of simultaneous open ptys.  Or at least be able to
>> >> wait and retry if an open fails.
>> >
>> > Emacs limits the number of these background processes, see
>> > native-comp-async-jobs-number.  (The value zero means use half of the
>> > number of execution units available in the system's processor.)  So we
>> > generally shouldn't have more than that number of open pipes at any
>> > given time.  For the obvious reasons they cannot be closed until the
>> > sub-process exits.
>> 
>> Is there any action we need/can take on this bug?
>
> I don't think there's anything we _can_ do here, and I tried to
> explain why.

I agree, I don't think we really want support `invocation-name'
modifications at runtime.

Shall we close this bug?

  Andrea

Message #26 received at 51110 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: stepheng+emacs <at> gildea.com, 51110 <at> debbugs.gnu.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Tue, 30 Nov 2021 22:29:36 +0200

> From: Andrea Corallo <akrl <at> sdf.org>
> Cc: stepheng+emacs <at> gildea.com, 51110 <at> debbugs.gnu.org
> Date: Tue, 30 Nov 2021 20:24:59 +0000
> 
> > I don't think there's anything we _can_ do here, and I tried to
> > explain why.
> 
> I agree, I don't think we really want support `invocation-name'
> modifications at runtime.
> 
> Shall we close this bug?

I think so.  But before we do, I'd like to let Stephen a chance to
convince us otherwise.

Message #31 received at 51110-done <at> debbugs.gnu.org (full text, mbox):

From: Stephen Gildea <stepheng+emacs <at> gildea.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Andrea Corallo <akrl <at> sdf.org>
Cc: 51110-done <at> debbugs.gnu.org
Subject: Re: bug#51110: Native-comp leaks fds open on /dev/ptmx
Date: Tue, 30 Nov 2021 18:28:36 -0800

Eli Zaretskii <eliz <at> gnu.org> wrote:

>   > From: Andrea Corallo <akrl <at> sdf.org>
>   > Cc: stepheng+emacs <at> gildea.com, 51110 <at> debbugs.gnu.org
>   > Date: Tue, 30 Nov 2021 20:24:59 +0000
>   > 
>   > > I don't think there's anything we _can_ do here, and I tried to
>   > > explain why.
>   > 
>   > I agree, I don't think we really want support `invocation-name'
>   > modifications at runtime.
>   > 
>   > Shall we close this bug?
>   
>   I think so.  But before we do, I'd like to let Stephen a chance to
>   convince us otherwise.

I'm happy to see this bug closed, and I hereby do so.
Thank you for the discussion.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.