GNU bug report logs - #75938
31.0.50; Temporary file-names overflowing MAX_PATH characters

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 75938 in the body.
You can then email your comments to 75938 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: "bug-gnu-emacs <at> gnu.org" <bug-gnu-emacs <at> gnu.org>
Subject: 31.0.50; Temporary file-names overflowing MAX_PATH characters
Date: Thu, 30 Jan 2025 03:45:43 +0000

[Message part 1 (text/plain, inline)]

I am not sure if this is a bug or not; so do what you want with this.

It is fully possibly to create names that are not valid as file names,
via make-temp-name and make-temp-file, which are frontend for the
make-temp-file-internal function. For example:

(make-temp-name (make-string 1024 ?a))

will succeed without problems, but of course:

(make-temp-file (make-string 1024 ?a))

will fail, later on when it actually tries to create the file.

Attached is a suggested fix; the idea is to save the unnecessary trip to
the OS; it signals error if the total file name length is longer than
MAX_PATH.

I don't know if it is a feature to have temp-names longer than certain
length, if it is than OK; this is more of a question/suggestion than a
bug report.

[Message part 2 (text/html, inline)]

[0001-Ensure-temporary-file-names-have-valid-lengths.patch (application/octet-stream, attachment)]

Message #8 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: arthur miller <arthur.miller <at> live.com>,
 Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50;
 Temporary file-names overflowing MAX_PATH characters
Date: Thu, 30 Jan 2025 09:41:18 +0200

> From: arthur miller <arthur.miller <at> live.com>
> Date: Thu, 30 Jan 2025 03:45:43 +0000
> 
> I am not sure if this is a bug or not; so do what you want with this.
> 
> It is fully possibly to create names that are not valid as file names,
> via make-temp-name and make-temp-file, which are frontend for the
> make-temp-file-internal function. For example:
> 
> (make-temp-name (make-string 1024 ?a))
> 
> will succeed without problems, but of course:
> 
> (make-temp-file (make-string 1024 ?a))
> 
> will fail, later on when it actually tries to create the file.
> 
> Attached is a suggested fix; the idea is to save the unnecessary trip to
> the OS; it signals error if the total file name length is longer than
> MAX_PATH.
> 
> I don't know if it is a feature to have temp-names longer than certain
> length, if it is than OK; this is more of a question/suggestion than a
> bug report.

I think the idea is that the call to make_uninit_string will signal an
error if its argument exceeds STRING_BYTES_MAX, and the relevant
system library functions will error with ENAMETOOLONG if they cannot
support such long file names.

Adding Paul to the discussion.

Message #11 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Thu, 30 Jan 2025 16:30:27 +0000

[Message part 1 (text/plain, inline)]

> I think the idea is that the call to make_uninit_string will signal an
> error if its argument exceeds STRING_BYTES_MAX, and the relevant
> system library functions will error with ENAMETOOLONG if they cannot
> support such long file names.

Yes, it does that. It is a large number though; but Windows does is not creating
it thugh:

Debugger entered--Lisp error: (file-missing "Creating file with prefix" "No such file or directory" "r:/Temp/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
  make-temp-file("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
  eval((make-temp-file (make-string 1024 97)) nil)
  ielm-eval-input(#("(make-temp-file  (make-string 1024 ?a))" 0 39 (fontified t)) nil)
  ielm-send-input(nil)
  ielm-return()
  funcall-interactively(ielm-return)
  command-execute(ielm-return)

That only dichotomy is that make-temp-name will produce a string, but that string
will not be suitable as a file name, but that perhaps is a feature? That was
also a part I was asking about. Perhaps, since the docs are already declaring
that using this function and later trying to create a file with that name asks
for troubles, perhaps it should be obsoleted?

By the way, while playing with this, I have discovered a very easy way to
blow-up stack: (make-temp-name (make-string (* 1024 1024 1024) ?a)). Is there
any way that byte-compiler or lisp-reader could detect such "innocent" code, or
 is it only code-review that can discover something like that?

Just a curious question, sorry for regressing.

[Message part 2 (text/html, inline)]

Message #14 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, arthur miller <arthur.miller <at> live.com>
Cc: 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Thu, 30 Jan 2025 11:58:06 -0800

On 1/29/25 23:41, Eli Zaretskii wrote:
> I think the idea is that the call to make_uninit_string will signal an
> error if its argument exceeds STRING_BYTES_MAX, and the relevant
> system library functions will error with ENAMETOOLONG if they cannot
> support such long file names.

Yes, that's the idea. There is at least one system (GNU/Hurd) which does 
not have a path limit.

On 1/30/25 08:30, arthur miller wrote:

>  make-temp-name will produce a string, but that string
> will not be suitable as a file name, but that perhaps is a feature?

Yes, because make-temp-name cannot always predict exactly which names 
are allowed. Some file systems don't allow encoding errors in file 
names, for example. When the manual says that make-temp-name "generates 
a string that might be a unique file name", it means that the string 
might not be unique and might not be a file name.


> since the docs are already declaring
> that using this function and later trying to create a file with that name asks
> for troubles, perhaps it should be obsoleted?

The first step in any such move would be to modify Emacs itself so that 
it never calls make-temp-name. Is that a project you could take on?



> By the way, while playing with this, I have discovered a very easy way to
> blow-up stack: (make-temp-name (make-string (* 1024 1024 1024) ?a)). Is there
> any way that byte-compiler or lisp-reader could detect such "innocent" code, or
>  is it only code-review that can discover something like that?

I don't see much point in trying to detect that particular problem 
statically. Not sure what you mean by "blow-up stack": what were the 
symptoms?

Message #17 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Eli Zaretskii <eliz <at> gnu.org>
Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 02:01:50 +0000

[Message part 1 (text/plain, inline)]

> >  make-temp-name will produce a string, but that string
> > will not be suitable as a file name, but that perhaps is a feature?
>
> Yes, because make-temp-name cannot always predict exactly which names
> are allowed. Some file systems don't allow encoding errors in file
> names, for example. When the manual says that make-temp-name "generates
> a string that might be a unique file name", it means that the string
> might not be unique and might not be a file name.
>
>
> > since the docs are already declaring
> > that using this function and later trying to create a file with that name asks
> > for troubles, perhaps it should be obsoleted?
>
> The first step in any such move would be to modify Emacs itself so that
> it never calls make-temp-name. Is that a project you could take on?
>

Probably not. Not because I don't want, but there is a lot of code in
different parts of Emacs, that would ask for hacking the code I am not
familiar with. Risk of introducing bug in something that does not have a
bug is big., and I am not sure I can even test all the code; so probably
for the best not to fix what ain't broken.

I did look at the usages; I didn't know this function was used that
much. I should have done that initially, tbh; wasn't thinking of it. I
see about 50 usages of that function, (65 in total, of which 15 are
changelog mentions). Some code does use this function to actually create
a temporary file, for example w32-direct-print-region-helper in
dos-w32.el. But some are just using it for a unique name, tramp for
example, without creating an actual file. That code would need some
replacement function; so I guess this one is as good as any.

> > By the way, while playing with this, I have discovered a very easy way to
> > blow-up stack: (make-temp-name (make-string (* 1024 1024 1024) ?a)). Is there
> > any way that byte-compiler or lisp-reader could detect such "innocent" code, or
> >  is it only code-review that can discover something like that?
>
> I don't see much point in trying to detect that particular problem
> statically. Not sure what you mean by "blow-up stack": what were the
> symptoms?

(make-temp-name (* 1024 1024 1024) ?a) <- asks for 1 gig of chars

After trying to execute that, I get message that Emacs has recovered
from the C stack overflow. Than in a short the system kills the Emacs
process. It is a bit random; sometimes the process survives and I can
repeat a couple times before Windows kills it, sometimes Emacs
process is killed after the first run. I think Emacs is trying to access
memory that does not belong to the process; segfault? Not sure what
is really hapening though.

[Message part 2 (text/html, inline)]

Message #20 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: arthur miller <arthur.miller <at> live.com>, Eli Zaretskii <eliz <at> gnu.org>
Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Re: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Thu, 30 Jan 2025 20:14:56 -0800

On 1/30/25 18:01, arthur miller wrote:
> (make-temp-name (* 1024 1024 1024) ?a) <- asks for 1 gig of chars
> 
> After trying to execute that, I get message that Emacs has recovered
> from the C stack overflow.

I assume you mean:

  (make-temp-name (make-string (* 1024 1024 1024) ?a))

and I don't observe the problem on bleeding-edge Emacs (commit 
a5965217fc1d7b56df60f8e798edd48ae52c8624) when built on Fedora 41 
x86-64. There is no C stack overflow, and make-temp-name signals 
(file-error "Creating file name with prefix" "File name too long" 
"aaa...") as expected.

If you're getting a C stack overflow on bleeding-edge Emacs, it might be 
helpful to see exactly why. This would likely require running Emacs 
under a debugger.

Message #23 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: arthur.miller <at> live.com, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 09:35:22 +0200

> Date: Thu, 30 Jan 2025 20:14:56 -0800
> Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> On 1/30/25 18:01, arthur miller wrote:
> > (make-temp-name (* 1024 1024 1024) ?a) <- asks for 1 gig of chars
> > 
> > After trying to execute that, I get message that Emacs has recovered
> > from the C stack overflow.
> 
> I assume you mean:
> 
>    (make-temp-name (make-string (* 1024 1024 1024) ?a))
> 
> and I don't observe the problem on bleeding-edge Emacs (commit 
> a5965217fc1d7b56df60f8e798edd48ae52c8624) when built on Fedora 41 
> x86-64. There is no C stack overflow, and make-temp-name signals 
> (file-error "Creating file name with prefix" "File name too long" 
> "aaa...") as expected.
> 
> If you're getting a C stack overflow on bleeding-edge Emacs, it might be 
> helpful to see exactly why. This would likely require running Emacs 
> under a debugger.

There are a few calls to literally 'alloca' in expand-file-name, at
least one of them specific to WINDOWSNT.  make-temp-name call
expand-file name inside make-temp-file-internal, so maybe this is the
reason?

Why don't all calls to 'alloca' in expand-file-name call SAFE_ALLOCA
instead?

Message #26 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: arthur.miller <at> live.com, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 00:25:43 -0800

On 2025-01-30 23:35, Eli Zaretskii wrote:
> There are a few calls to literally 'alloca' in expand-file-name, at
> least one of them specific to WINDOWSNT.  make-temp-name call
> expand-file name inside make-temp-file-internal, so maybe this is the
> reason?

Yes, that sounds right. From what I can see, these alloca calls are done 
only if DOS_NT so this potential stack overflow issue doesn't come up on 
my Fedora 41 platform.


> Why don't all calls to 'alloca' in expand-file-name call SAFE_ALLOCA
> instead?

I don't know.

Message #29 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: arthur.miller <at> live.com, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 10:43:17 +0200

> Date: Fri, 31 Jan 2025 00:25:43 -0800
> Cc: arthur.miller <at> live.com, 75938 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> On 2025-01-30 23:35, Eli Zaretskii wrote:
> > There are a few calls to literally 'alloca' in expand-file-name, at
> > least one of them specific to WINDOWSNT.  make-temp-name call
> > expand-file name inside make-temp-file-internal, so maybe this is the
> > reason?
> 
> Yes, that sounds right. From what I can see, these alloca calls are done 
> only if DOS_NT so this potential stack overflow issue doesn't come up on 
> my Fedora 41 platform.

I've replaced them with SAFE_ALLOCA now.  Arthur, please see if you
still get C stack overflow.

Message #32 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 13:56:01 +0000

[Message part 1 (text/plain, inline)]

NP, I'll rebuild and test; thnks. I already thought to rebild
 as debug build after Paul's mail last night.I'll come back
 when I am done. I just haven't had time, so I didn't
answered back.

[Message part 2 (text/html, inline)]

Message #35 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 15:57:35 +0000

[Message part 1 (text/plain, inline)]

> I've replaced them with SAFE_ALLOCA now.  Arthur, please see if you
still get C stack overflow.

Thus far, I pulled and compiled with -O2 -mtune=native; It does not end
in C stack overflow, but it does crash Emacs.

On first run, there was native compiler interfering, so I restarted Emacs, let
the native compiler do what it wants to do, and than restarted a new and
tried, and I get the same problem: memory exhaustion, when Emacs takes
somewhere around 12 gig. I have 32 gig ram in this machine; it is the same
computer from a couple of years (or more?) ago when Emacs was eating
all 32 gig on gnu/Linux).

Now, Emacs is taking about 12 gig when it reports memory exhaustion. You
can see from the screenhots, it is going up, and than ~12 or immidiately before
it reports memory exhausted in Messages buffer. Interested thing, the graphic
driver also crashed, so I couldn't take the screenshot of the last time, I don't know
if the changed resolution is visible on a screenshot. The process manager reports
only about 60% of memory usage, so I am not sure why it crashes, but felt like
back in time of Windows 98; it didn't help to restart explorer or change desktop
settings, I had to restart the system 🙂. On Arch, I was able to switch to
another tty, kill Emacs and restart X11, and everything was fine again.

I will rebuild also in debug mode.

[Message part 2 (text/html, inline)]

[emacs-make-temp-name-1.png (image/png, attachment)]

[emacs-make-temp-name-2.png (image/png, attachment)]

[emacs-make-temp-name-3.png (image/png, attachment)]

[emacs-make-temp-name-4.png (image/png, attachment)]

Message #38 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 15:58:34 +0000

[Message part 1 (text/plain, inline)]

A correction: it does not crash Emacs; but I have to restart it.

[Message part 2 (text/html, inline)]

Message #41 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: arthur miller <arthur.miller <at> live.com>
Cc: eggert <at> cs.ucla.edu, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 18:57:19 +0200

> From: arthur miller <arthur.miller <at> live.com>
> CC: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
> Date: Fri, 31 Jan 2025 15:57:35 +0000
> 
> > I've replaced them with SAFE_ALLOCA now.  Arthur, please see if you
> still get C stack overflow.
> 
> Thus far, I pulled and compiled with -O2 -mtune=native; It does not end
> in C stack overflow, but it does crash Emacs.
> 
> On first run, there was native compiler interfering, so I restarted Emacs, let
> the native compiler do what it wants to do, and than restarted a new and
> tried, and I get the same problem: memory exhaustion, when Emacs takes
> somewhere around 12 gig. I have 32 gig ram in this machine; it is the same
> computer from a couple of years (or more?) ago when Emacs was eating
> all 32 gig on gnu/Linux).
> 
> Now, Emacs is taking about 12 gig when it reports memory exhaustion. You
> can see from the screenhots, it is going up, and than ~12 or immidiately before
> it reports memory exhausted in Messages buffer. Interested thing, the graphic 
> driver also crashed, so I couldn't take the screenshot of the last time, I don't know
> if the changed resolution is visible on a screenshot. The process manager reports
> only about 60% of memory usage, so I am not sure why it crashes, but felt like
> back in time of Windows 98; it didn't help to restart explorer or change desktop
> settings, I had to restart the system 🙂. On Arch, I was able to switch to
> another tty, kill Emacs and restart X11, and everything was fine again.

When you say "Emacs crashed", do you mean memory exhaustion, or do you
mean an actual crash, with backtrace and the Abort dialog popped up?

Message #44 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: arthur miller <arthur.miller <at> live.com>
Cc: eggert <at> cs.ucla.edu, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 18:58:35 +0200

> From: arthur miller <arthur.miller <at> live.com>
> CC: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
> Date: Fri, 31 Jan 2025 15:58:34 +0000
> 
> A correction: it does not crash Emacs; but I have to restart it.

Restart Emacs? that's TRT when memory is exhausted..

Message #47 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>,
 "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Fri, 31 Jan 2025 20:30:17 +0000

[Message part 1 (text/plain, inline)]

I just have to restart it.

Question is why does it takes all the memory, and why only ~12 gig
 on 32 gig system. Is there some limit from the OS? I am not aware of
 such.

Anyway, regardless of the system, there is some bug, it is perhaps only
 limited to Windows platform? I'll rebuild with debug options, do you want
 me to do something special, how do I debug it, or you can reproduce it?
 This is on Windows 10.

________________________________
Från: Eli Zaretskii <eliz <at> gnu.org>
Skickat: den 31 januari 2025 17:58
Till: arthur miller <arthur.miller <at> live.com>
Kopia: eggert <at> cs.ucla.edu <eggert <at> cs.ucla.edu>; 75938 <at> debbugs.gnu.org <75938 <at> debbugs.gnu.org>
Ämne: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH characters

> From: arthur miller <arthur.miller <at> live.com>
> CC: "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
> Date: Fri, 31 Jan 2025 15:58:34 +0000
>
> A correction: it does not crash Emacs; but I have to restart it.

Restart Emacs? that's TRT when memory is exhausted..

[Message part 2 (text/html, inline)]

Message #50 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>,
 "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Sat, 1 Feb 2025 02:25:26 +0000

[Message part 1 (text/plain, inline)]

I have rebuilt with debug symbols on the laptop with Win 11.

Emacs does not crash, nor do I get memory exhaustion error.

However, it eats up all the RAM, and stays busy for minutes without
anything seemingly happen. It was responsive; I was able to C-g, and
Emacs continue to function normally (yesterday it was crashing after
a while), and also I was able to restart the operation again, and bail
out of it via C-g, with Emacs seemingly operating normally afterwards.

I have looke at make-string function, but it seems ok to me, so I am not
sure what is going on.

[Message part 2 (text/html, inline)]

[make-temp-name.png (image/png, attachment)]

Message #53 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: arthur miller <arthur.miller <at> live.com>
Cc: eggert <at> cs.ucla.edu, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Sat, 01 Feb 2025 10:08:37 +0200

> From: arthur miller <arthur.miller <at> live.com>
> CC: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>, "75938 <at> debbugs.gnu.org"
> 	<75938 <at> debbugs.gnu.org>
> Date: Fri, 31 Jan 2025 20:30:17 +0000
> 
> I just have to restart it. 
> 
> Question is why does it takes all the memory, and why only ~12 gig
>  on 32 gig system. Is there some limit from the OS? I am not aware of
>  such. 

No, it usually means your system doesn't have enough VM.  The fact
that you have 32 GB or physical memory installed is not enough to
judge this case, because on a system running for enough time quite a
bit of that could be used by processes other than Emacs.  The amount
of available VM is determined by the amount of available physical
memory and the amount of available swap ("pagefile" in Windows
parlance).  To see how much available VM you have, run msinfo32 and
look for "Available Virtual Memory".

OTOH, you seem to assume that creating a 1 GB string asks the system
for 1 GB of memory, but that is not so.  I think Emacs asks for more,
in particular because the string could grow, and because Emacs holds
strings in memory in a superset of UTF-8 representation, so each
character could use up to 5 bytes.

> Anyway, regardless of the system, there is some bug, it is perhaps only
>  limited to Windows platform?

Where do you see a bug?  From what you described, Emacs detects that
it will run out of memory and tells you to save your edits and
restart.  This is exactly what should happen in such cases, and not
only on Windows.

In any case, this is a separate issue, unrelated to the stack overflow
when asking to create files with very long names.  AFAIU, that bug is
now fixed on the master branch, right?

Message #56 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: arthur miller <arthur.miller <at> live.com>
Cc: eggert <at> cs.ucla.edu, 75938 <at> debbugs.gnu.org
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Sat, 01 Feb 2025 10:58:30 +0200

> From: arthur miller <arthur.miller <at> live.com>
> CC: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>, "75938 <at> debbugs.gnu.org"
> 	<75938 <at> debbugs.gnu.org>
> Date: Sat, 1 Feb 2025 02:25:26 +0000
> 
> I have rebuilt with debug symbols on the laptop with Win 11.
> 
> Emacs does not crash, nor do I get memory exhaustion error.

Good!  Is this a different machine from the one where you got "memory
exhausted"?  If so, it would mean this laptop has more free VM than
that other system.

> However, it eats up all the RAM, and stays busy for minutes without
> anything seemingly happen.

GC, perhaps?  Or maybe the system working on the pagefile (e.g.,
extending it, because what was there was exhausted)?

> It was responsive; I was able to C-g, and
> Emacs continue to function normally (yesterday it was crashing after
> a while), and also I was able to restart the operation again, and bail
> out of it via C-g, with Emacs seemingly operating normally afterwards.
> 
> I have looke at make-string function, but it seems ok to me, so I am not
> sure what is going on.

As I said earlier, I'm not sure I understand why you think there's any
problem in this scenario.

Message #59 received at 75938 <at> debbugs.gnu.org (full text, mbox):

From: arthur miller <arthur.miller <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>,
 "75938 <at> debbugs.gnu.org" <75938 <at> debbugs.gnu.org>
Subject: Sv: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Sat, 1 Feb 2025 10:02:23 +0000

[Message part 1 (text/plain, inline)]

> > Emacs does not crash, nor do I get memory exhaustion error.
>
> Good!  Is this a different machine from the one where you got "memory
> exhausted"?  If so, it would mean this laptop has more free VM than
> that other system.

Yes, this one is a laptop with 20 gig pagefile, 16 gig ram. I extended
pagefile just to see if will compute that string, but it didn't help.

> > However, it eats up all the RAM, and stays busy for minutes without
> > anything seemingly happen.
>
> GC, perhaps?  Or maybe the system working on the pagefile (e.g.,
> extending it, because what was there was exhausted)?

It happends quite immidiately as Emacs starts to execute the make-string
function.

It is enough to execute this: (make-string (* 1024 1024 1024) ?a). No
need for make-temp-name.

> > It was responsive; I was able to C-g, and
> > Emacs continue to function normally (yesterday it was crashing after
> > a while), and also I was able to restart the operation again, and bail
> > out of it via C-g, with Emacs seemingly operating normally afterwards.
> >
> > I have looke at make-string function, but it seems ok to me, so I am not
> > sure what is going on.
>
> As I said earlier, I'm not sure I understand why you think there's any
> problem in this scenario.

I am not sure what you think that I think that is a problem :).

I think the problem is that Emacs seems to ask for that much RAM/VM
whatever.

To answer the other mail:

> No, it usually means your system doesn't have enough VM.  The fact
> that you have 32 GB or physical memory installed is not enough to
> judge this case, because on a system running for enough time quite a
> bit of that could be used by processes other than Emacs.  The amount
> of available VM is determined by the amount of available physical
> memory and the amount of available swap ("pagefile" in Windows
> parlance).  To see how much available VM you have, run msinfo32 and
> look for "Available Virtual Memory".


Indeed, I did expected that it will start to page when physical RAM is
exhausted, but that obviously does not work that way, and yepp, on that
machine the swap file was disabled, since it never happened that I need
it. Yes, I understand that system also uses RAM for other processes, of
course.

> OTOH, you seem to assume that creating a 1 GB string asks the system
> for 1 GB of memory, but that is not so.  I think Emacs asks for more,
> in particular because the string could grow, and because Emacs holds
> strings in memory in a superset of UTF-8 representation, so each
> character could use up to 5 bytes.

I am aware that it can be more than 1 gig. I said gigabyte of
characters. In this case, utf8 internally, since I am askring for "a", I
guess it is one gigabyte of RAM too, it should fit into one byte since
according to utf8, but I don't know, perhaps Emacs does something
different? There is probably more storage needed for bookkeeping of this
data internally, and probably some intermediary buffers and so on when
function is executing.

Still even if it took 5 gigabyte for the string storage, does it not seem
strange to eat up more than double, almost tripple that ram? Perhaps
it's ok, I don't know; it is a question, just asking.

> AFAIU, that bug is now fixed on the master branch, right?

Yes, it seems so, so you can close the bug repport :). Thanks for fixing
it so fast!

[Message part 2 (text/html, inline)]

Message #64 received at 75938-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: arthur miller <arthur.miller <at> live.com>
Cc: 75938-done <at> debbugs.gnu.org, eggert <at> cs.ucla.edu
Subject: Re: bug#75938: 31.0.50; Temporary file-names overflowing MAX_PATH
 characters
Date: Sat, 01 Feb 2025 14:21:31 +0200

> From: arthur miller <arthur.miller <at> live.com>
> CC: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>, "75938 <at> debbugs.gnu.org"
> 	<75938 <at> debbugs.gnu.org>
> Date: Sat, 1 Feb 2025 10:02:23 +0000
> 
> > > However, it eats up all the RAM, and stays busy for minutes without
> > > anything seemingly happen.
> >
> > GC, perhaps?  Or maybe the system working on the pagefile (e.g.,
> > extending it, because what was there was exhausted)?
> 
> It happends quite immidiately as Emacs starts to execute the make-string
> function.

What is "it" in this case?

> > As I said earlier, I'm not sure I understand why you think there's any
> > problem in this scenario.
> 
> I am not sure what you think that I think that is a problem :).

Then what are we still discussing here?

> I think the problem is that Emacs seems to ask for that much RAM/VM
> whatever.

Why?  As I explained, Emacs asks for more than is immediately needed,
to avoid the need to ask the OS for more if you then enlarge the
string by 1 byte.

> > OTOH, you seem to assume that creating a 1 GB string asks the system
> > for 1 GB of memory, but that is not so.  I think Emacs asks for more,
> > in particular because the string could grow, and because Emacs holds
> > strings in memory in a superset of UTF-8 representation, so each
> > character could use up to 5 bytes.
> 
> I am aware that it can be more than 1 gig. I said gigabyte of
> characters. In this case, utf8 internally, since I am askring for "a", I
> guess it is one gigabyte of RAM too, it should fit into one byte since
> according to utf8, but I don't know, perhaps Emacs does something
> different?

See above.

> There is probably more storage needed for bookkeeping of this
> data internally, and probably some intermediary buffers and so on when
> function is executing.

Yes.  And also for printing the result, since evaluating a function
shows the result in the echo-area or in *scratch*.

> Still even if it took 5 gigabyte for the string storage, does it not seem
> strange to eat up more than double, almost tripple that ram? Perhaps
> it's ok, I don't know; it is a question, just asking.

Feel free to step with a debugger through the code, but I personally
am not surprised at all.

> > AFAIU, that bug is now fixed on the master branch, right?
> 
> Yes, it seems so, so you can close the bug repport :). Thanks for fixing
> it so fast!

Closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.