GNU bug report logs - #13079
24.3.50; Emacs cannot create subprocess

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 13079 in the body.
You can then email your comments to 13079 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Li Zhai <mrzhaili <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3.50; Emacs cannot create subprocess
Date: Tue, 4 Dec 2012 16:03:56 +0800

Hi,

I've noticed emacs cannot create subprocess in some cases:

 (progn
  (dotimes (i 25)
    (call-process-region (point-min) (point-min) "ddeclient" t 0 nil
"SUMATRA" "control")
    (message "%d" i)))

After executed above codes, emacs report:

"Spawning child process" "resource temporarily unavailable"

My emacs running environment as follows:

In GNU Emacs 24.3.50.1 (i386-mingw-nt6.1.7601)
 of 2012-11-30 on ALAN-NB
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --with-gcc (4.5) --cflags -I/ImageLib/giflib-4.1.6/lib
 -I/ImageLib/zlib-1.2.5 -I/ImageLib/libpng-1.4.5 -I/ImageLib/jpeg-8c
 -I/ImageLib/libxpm-3.5.8-w32-src/src
 -I/ImageLib/libxpm-3.5.8-w32-src/include -I/ImageLib/tiff-3.9.4/libtiff
 -I/ImageLib/libxml2-2.8.0/include
 -I/ImageLib/gnutls-3.0.20-w32/include'

Important settings:
  value of $LANG: zh_CN.GBK
  locale-coding-system: cp936
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  eldoc-mode: t
  paredit-mode: t
  TeX-PDF-mode: t
  savehist-mode: t
  desktop-save-mode: t
  anything-dired-mode: Enable anything completion in Dired functions.
Bindings affected are C, R, S, H.
This is deprecated for Emacs24+ users, use `ac-mode' instead.
  shell-dirtrack-mode: t
  show-paren-mode: t
  recentf-mode: t
  delete-selection-mode: t
  auto-image-file-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  hs-minor-mode: t

Recent input:
C-p C-p C-n M-m ; ; C-n C-n C-n C-w C-p C-p C-p C-y
C-p <tab> C-n C-n C-e C-x C-e C-x q q C-x C-e q C-x
C-e q M-x b u g <tab> C-g <f9> b u g . * r e p o r
t C-a C-e <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> e m a c s <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
C-a e m a c s . * C-n <return> <return> <return> C-x
1 C-c C-k C-g <wheel-down> <double-wheel-down> <triple-wheel-down>
<triple-wheel-down> <wheel-down> <double-wheel-down>
<triple-wheel-down> <triple-wheel-down> <triple-wheel-down>
<triple-wheel-down> <triple-wheel-down> <triple-wheel-down>
<wheel-up> <double-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <wheel-up> <double-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <wheel-up> <double-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <wheel-up> <double-wheel-up> <triple-wheel-up>
<triple-wheel-up> <help-echo> <down-mouse-1> <mouse-1>
<down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1>
<mouse-1> <down-mouse-1> <mouse-movement> <mouse-movement>
<drag-mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1>
<mouse-1> <down-mouse-1> <mouse-1> <wheel-down> <double-wheel-down>
<triple-wheel-down> <triple-wheel-down> <wheel-down>
<double-wheel-down> <triple-wheel-down> <triple-wheel-down>
<wheel-down> <double-wheel-down> <triple-wheel-down>
<triple-wheel-down> C-c C-c n <help-echo> <down-mouse-1>
<drag-mouse-1> C-x 1 <wheel-up> <double-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <wheel-up> <double-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<triple-wheel-up> <triple-wheel-up> <triple-wheel-up>
<down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1>
<mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1>
<down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1>
<mouse-1> <down-mouse-1> <mouse-1> C-/ C-/ C-/ C-/
C-x k <return> C-x C-b a a q M-x M-p <return> <down-mouse-1>
<mouse-1> X b <backspace> <backspace> <S-down> <S-down>
<S-down> C-g <up> <up> <up> C-x b s c r <return> C-x
b C-g C-h e M-> C-p C-p C-p C-p C-p C-p C-p C-p C-p
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p M-v C-s
s w a n C-s C-s C-g C-g C-x b <return> C-n C-n C-n
C-n C-n C-n C-n C-n C-n C-n C-n C-e C-x C-e <help-echo>
<down-mouse-1> <mouse-movement> <mouse-movement> <drag-mouse-1>
<help-echo> M-w q C-x b C-g M-x M-p <return>

Recent messages:
Mark set
byte-code: Beginning of buffer [6 times]
Auto-saving...done
Undo! [3 times]
user-error: No further undo information
Quit [2 times]
Mark set
Quit
Entering debugger...
Back to top level.
Quit

Load-path shadows:
None found.

Features:
(shadow sort mail-extr gnus-msg gnus-art mm-uu mml2015 epg-config
mm-view mml-smime smime dig gnus-sum nnoo gnus-group gnus-undo nnmail
mail-source gnus-start gnus-spec gnus-int gnus-range gnus-win gnus
gnus-ems nnheader emacsbug message rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mail-utils novice highlight-symbol
hi-lock misearch multi-isearch vc-git bm bs macros help-mode debug rect
hideshow eldoc paredit ind-util reftex-auc preview prv-emacs info
reporter tex-buf tex-fold cdlatex texmathp reftex-dcr reftex reftex-vars
font-latex latex derived tex-style tex savehist desktop saveplace
bbdb-autoloads bbdb timezone preview-latex tex-site auto-loads org-clock
org-exp ob-exp org-exp-blocks org-agenda w32-browser anything-config
browse-url imenu bookmark pp rx anything-match-plugin
semantic/util-modes semantic/util semantic semantic/tag semantic/lex
semantic/fw loaddefs mode-local cedet org ob-tangle ob-ref ob-lob
ob-table org-footnote org-src ob-comint ob-keys org-pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
org-compat org-macs ob-eval org-loaddefs xml grep compile tramp
tramp-compat tramp-loaddefs shell pcomplete comint format-spec dired-x
dired-aux anything smart-compile paren slime-autoloads cc-styles
cc-align cc-engine cc-vars cc-defs xcscope ring auto-install easy-mmode
ffap thingatpt find-func dired url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util url-parse
auth-source eieio gnus-util mm-util mail-prsvr password-cache url-vars
mailcap windmove cal-china-x cl-macs gv cl cl-lib cal-china lunar solar
cal-dst holidays hol-loaddefs cal-menu calendar cal-loaddefs window+
uniquify midnight filecache ido recentf tree-widget wid-edit easymenu
edmacro kmacro byte-opt warnings bytecomp byte-compile cconv nadvice
advice help-fns server ps-ccrypt delsel image-file avoid ansi-color
time-date china-util tooltip ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 ls-lisp w32-common-fns disp-table w32-win w32-vars tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process w32 multi-tty
emacs)

Message #8 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Li Zhai <mrzhaili <at> gmail.com>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Tue, 04 Dec 2012 22:23:31 +0200

> Date: Tue, 4 Dec 2012 16:03:56 +0800
> From: Li Zhai <mrzhaili <at> gmail.com>
> 
> I've noticed emacs cannot create subprocess in some cases:
> 
>  (progn
>   (dotimes (i 25)
>     (call-process-region (point-min) (point-min) "ddeclient" t 0 nil
> "SUMATRA" "control")
>     (message "%d" i)))
> 
> After executed above codes, emacs report:
> 
> "Spawning child process" "resource temporarily unavailable"

I cannot reproduce this here.  Does this happen in "emacs -Q"?
Also, what do you mean "after executing"?  Does it mean Emacs displays
this message before it ends the loop, i.e. before it launches 25
processes?  Or do you mean that after all the 25 processes are
launched, Emacs cannot start more subprocesses?  If the latter, does
the problem go away after you wait for all the 25 ddeclient
subprocesses to exit?

Message #11 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Li Zhai <mrzhaili <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Wed, 5 Dec 2012 10:26:51 +0800

Sorry. I'll explain more clear. I'm using emacs under windows 7.

I mean, emacs display the messages before the loop ends. And all the
processes are gone, emacs did not wait them to exit.

I've tried two different version from
http://ftp.gnu.org/gnu/emacs/windows/emacs-23.4-bin-i386.zip and
http://alpha.gnu.org/gnu/emacs/windows/emacs-20120917-r110062-bin-i386.zip

Here is how to reproduce the bug:

1. Open the emacs using ``emacs -Q''

2. paste the code into *scratch* buffer:

  (progn
   (dotimes (i 35)
     (call-process-region (point-min) (point-min) "ddeclient" t 0 nil
 "SUMATRA" "control")
     (message "%d" i)))

the *ONLY* difference between the previous code is I increase the
repeat times to call the function `call-process-region'. Here is 35
times, you could try to increase the number lager if the bug not
reproducing.

3. Eval the code using C-x C-e

4. You should see the emacs report an error:

Debugger entered--Lisp error: (file-error "Spawning child process"
"resource temporarily unavailable")

5. Quit the elisp debugger and execute the code again(press C-x C-e),
this time emacs should report:

Debugger entered--Lisp error: (file-error "Opening output file"
"permission denied" "c:/Users/iot003/Desktop/emacs-24.2.50/bin")
  call-process-region(1 1 "ddeclient" t 0 nil "SUMATRA" "control")

6. Quit the elisp debugger. This time, emacs cannot create any
sub-process any more. Try to execute a shell command by press `M-!
ls', you will find it out.

This problem occurred more often on my private build version. I guess
the problem is the default stack size assigned by compiler is too
small.

Thanks for your reply, Eli. Have a good day.


On Wed, Dec 5, 2012 at 4:23 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Tue, 4 Dec 2012 16:03:56 +0800
>> From: Li Zhai <mrzhaili <at> gmail.com>
>>
>> I've noticed emacs cannot create subprocess in some cases:
>>
>>  (progn
>>   (dotimes (i 25)
>>     (call-process-region (point-min) (point-min) "ddeclient" t 0 nil
>> "SUMATRA" "control")
>>     (message "%d" i)))
>>
>> After executed above codes, emacs report:
>>
>> "Spawning child process" "resource temporarily unavailable"
>
> I cannot reproduce this here.  Does this happen in "emacs -Q"?
> Also, what do you mean "after executing"?  Does it mean Emacs displays
> this message before it ends the loop, i.e. before it launches 25
> processes?  Or do you mean that after all the 25 processes are
> launched, Emacs cannot start more subprocesses?  If the latter, does
> the problem go away after you wait for all the 25 ddeclient
> subprocesses to exit?

Message #14 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Li Zhai <mrzhaili <at> gmail.com>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Wed, 05 Dec 2012 19:29:58 +0200

> Date: Wed, 5 Dec 2012 10:26:51 +0800
> From: Li Zhai <mrzhaili <at> gmail.com>
> Cc: 13079 <at> debbugs.gnu.org
> 
> I mean, emacs display the messages before the loop ends. And all the
> processes are gone, emacs did not wait them to exit.

Emacs doesn't wait because you told it so, by using zero as the 5th
argument to call-process-region.  This is normal.

> I've tried two different version from
> http://ftp.gnu.org/gnu/emacs/windows/emacs-23.4-bin-i386.zip and
> http://alpha.gnu.org/gnu/emacs/windows/emacs-20120917-r110062-bin-i386.zip
> 
> Here is how to reproduce the bug:
> 
> 1. Open the emacs using ``emacs -Q''
> 
> 2. paste the code into *scratch* buffer:
> 
>   (progn
>    (dotimes (i 35)
>      (call-process-region (point-min) (point-min) "ddeclient" t 0 nil
>  "SUMATRA" "control")
>      (message "%d" i)))
> 
> the *ONLY* difference between the previous code is I increase the
> repeat times to call the function `call-process-region'. Here is 35
> times, you could try to increase the number lager if the bug not
> reproducing.
> 
> 3. Eval the code using C-x C-e
> 
> 4. You should see the emacs report an error:
> 
> Debugger entered--Lisp error: (file-error "Spawning child process"
> "resource temporarily unavailable")

This is not a bug, but a known limitation of Emacs on Windows.  The
way management of subprocesses is implemented on Windows, Emacs can
only have 32 subprocesses simultaneously at any given time (because it
watches two separate handles for each subprocess, and uses an API that
supports maximum of 64 handles).  So this is also normal.

If you use 32 instead of 35, the above error message will not appear.
But that won't solve the _real_ problem here, see below.

> 5. Quit the elisp debugger and execute the code again(press C-x C-e),
> this time emacs should report:
> 
> Debugger entered--Lisp error: (file-error "Opening output file"
> "permission denied" "c:/Users/iot003/Desktop/emacs-24.2.50/bin")
>   call-process-region(1 1 "ddeclient" t 0 nil "SUMATRA" "control")
> 
> 6. Quit the elisp debugger. This time, emacs cannot create any
> sub-process any more. Try to execute a shell command by press `M-!
> ls', you will find it out.
> 
> This problem occurred more often on my private build version. I guess
> the problem is the default stack size assigned by compiler is too
> small.

No, this has nothing to do with something fancy like stack size, it's
a much more mundane problem.  It's because we fail to create temporary
files needed by call-process-region and commands that use it.  Did you
look in your TEMP directory?  I bet you have gobs of emXYZZY files
there, exactly 42 of them per each time you tried this stuff.

The immediate reason for the mysterious "permission denied" error
(about a directory!) is that call-process-region was not checking
whether the call to 'mktemp' failed.  When that function fails, it
produces an empty "file name", and call-process-region was boldly
using that as a file name, eventually trying to open a directory,
which on Windows fails with EACCES.  This part was easy to fix:
starting with revision 110996 on the emacs-24 branch, Emacs now
produces a more intelligent error message in this case.

The harder part of this bug is the reason _why_ 'mktemp' fails.  Your
command launches asynchronous subprocesses, creating a temporary file
for each one of them where Emacs puts the contents of the region.  In
this case, call-process-region arranges for the temporary files to be
removed when the call to call-process returns, and then calls
call-process.  However, since this is an async subprocess,
call-process returns immediately, without waiting for the subprocess
to exit, and Emacs deletes the temporary file.  This "cleanup" uses
the non-portable (outside of Posix filesystems) trick of deleting a
file while it is still open and used by the child process.  On
Windows, this attempt to delete the file fails, and the file is left
behind.

The other part of this puzzle is that 'mktemp' as implemented by the
Windows runtime can only have up to 42 simultaneous temporary files
per calling thread (the MSDN documentation says 26, but that's a lie).
Once there are 42 files in your TEMP directory created by a single
Emacs session, all the future calls to 'mktemp' from the same session
will fail, unless you manually delete those files.

This part of the bug is harder to solve, because some code needs to be
implemented that will defer deletion until the process exits.  So it
is not solved yet.  But at least you now have a work-around: manually
delete temporary em* files from your TEMP directory.

Message #17 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Li Zhai <mrzhaili <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Thu, 6 Dec 2012 09:58:19 +0800

I got it. The maximize number of calling mktemp is 42 if we don't
remove the temporary em* files.

I found a very interesting discuss about the limit of mktemp on windows.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47439

They implement a new mktemp function to walk around the limit on the
number of concurrently open files.

Can we implement our mktemp functions instead of using microsoft's stuff?

Message #20 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Li Zhai <mrzhaili <at> gmail.com>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Thu, 06 Dec 2012 05:54:28 +0200

> Date: Thu, 6 Dec 2012 09:58:19 +0800
> From: Li Zhai <mrzhaili <at> gmail.com>
> Cc: 13079 <at> debbugs.gnu.org
> 
> I got it. The maximize number of calling mktemp is 42 if we don't
> remove the temporary em* files.

Yes.

> I found a very interesting discuss about the limit of mktemp on windows.
> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47439
> 
> They implement a new mktemp function to walk around the limit on the
> number of concurrently open files.
> 
> Can we implement our mktemp functions instead of using microsoft's stuff?

But that will just push the limit farther.  It will not fix the
underlying fundamental problem of not deleting temporary files in this
case.  Since Emacs is typically run for long periods of time, sooner
or later the problem will strike anyway.

Message #23 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Li Zhai <mrzhaili <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Thu, 6 Dec 2012 16:18:10 +0800

I've tried another method, using the flag FILE_FLAG_DELETE_ON_CLOSE
while open the files.

Create a file by passed the flag FILE_FLAG_DELETE_ON_CLOSE to
CreateFile, if all the handles are closed, the file will be deleted
automatically. I wrote a very simple program to test that feature. It
seems works.

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <io.h>
#include <stdio.h>
#include <windows.h>

void main( void )
{
   HANDLE fh1, fh2;
   char buf[100];
   DWORD dwReadSize;
   DWORD dwWriteSize;

   fh1 = CreateFile("temp.txt", GENERIC_READ | GENERIC_WRITE,
                    FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                    NULL, CREATE_ALWAYS,
                    FILE_ATTRIBUTE_NORMAL | FILE_FLAG_DELETE_ON_CLOSE, NULL);

   WriteFile(fh1, "hello", 5, &dwWriteSize, NULL);

   fh2 = CreateFile("temp.txt", GENERIC_READ | GENERIC_WRITE,
                    FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                    NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);

   if (fh2 == INVALID_HANDLE_VALUE)
       perror( "Open failed on fh2" );

   printf("Close the first file handle...\n");
   CloseHandle(fh1);

   printf("Read file...\n");
   ReadFile(fh2, buf, 5, &dwReadSize, NULL);
   buf[5] = 0;
   printf("File content:\n %s\n", buf);

   _getch();

   printf("Close the second file handle...The file should be deleted\n");
   CloseHandle(fh2);
}


But this method has some limit. Firstly, all the files must
create/open by Win32 API, the codes are totally unportable. Secondly,
the file must be opened with a flag FILE_SHARE_DELETE, or the file
will not be deleted.

And I don't know how to integrate the Win32 API with the emacs codes.
Emacs codes use POSIX functions(_open, _close, _read...), and the
delete_on_close feature required WIN32 API. I've tried using
_O_TEMPORARY flags with _open function, but not succeeded.

On Thu, Dec 6, 2012 at 11:54 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Thu, 6 Dec 2012 09:58:19 +0800
>> From: Li Zhai <mrzhaili <at> gmail.com>
>> Cc: 13079 <at> debbugs.gnu.org
>>
>> I got it. The maximize number of calling mktemp is 42 if we don't
>> remove the temporary em* files.
>
> Yes.
>
>> I found a very interesting discuss about the limit of mktemp on windows.
>>
>> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47439
>>
>> They implement a new mktemp function to walk around the limit on the
>> number of concurrently open files.
>>
>> Can we implement our mktemp functions instead of using microsoft's stuff?
>
> But that will just push the limit farther.  It will not fix the
> underlying fundamental problem of not deleting temporary files in this
> case.  Since Emacs is typically run for long periods of time, sooner
> or later the problem will strike anyway.

Message #26 received at 13079 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Li Zhai <mrzhaili <at> gmail.com>
Cc: 13079 <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Thu, 06 Dec 2012 21:04:54 +0200

> Date: Thu, 6 Dec 2012 16:18:10 +0800
> From: Li Zhai <mrzhaili <at> gmail.com>
> Cc: 13079 <at> debbugs.gnu.org
> 
> I've tried another method, using the flag FILE_FLAG_DELETE_ON_CLOSE
> while open the files.
> 
> Create a file by passed the flag FILE_FLAG_DELETE_ON_CLOSE to
> CreateFile, if all the handles are closed, the file will be deleted
> automatically. I wrote a very simple program to test that feature. It
> seems works.
> 
> #include <fcntl.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <io.h>
> #include <stdio.h>
> #include <windows.h>
> 
> void main( void )
> {
>    HANDLE fh1, fh2;
>    char buf[100];
>    DWORD dwReadSize;
>    DWORD dwWriteSize;
> 
>    fh1 = CreateFile("temp.txt", GENERIC_READ | GENERIC_WRITE,
>                     FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
>                     NULL, CREATE_ALWAYS,
>                     FILE_ATTRIBUTE_NORMAL | FILE_FLAG_DELETE_ON_CLOSE, NULL);
> 
>    WriteFile(fh1, "hello", 5, &dwWriteSize, NULL);
> 
>    fh2 = CreateFile("temp.txt", GENERIC_READ | GENERIC_WRITE,
>                     FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
>                     NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
> 
>    if (fh2 == INVALID_HANDLE_VALUE)
>        perror( "Open failed on fh2" );
> 
>    printf("Close the first file handle...\n");
>    CloseHandle(fh1);
> 
>    printf("Read file...\n");
>    ReadFile(fh2, buf, 5, &dwReadSize, NULL);
>    buf[5] = 0;
>    printf("File content:\n %s\n", buf);
> 
>    _getch();
> 
>    printf("Close the second file handle...The file should be deleted\n");
>    CloseHandle(fh2);
> }
> 
> 
> But this method has some limit. Firstly, all the files must
> create/open by Win32 API, the codes are totally unportable. Secondly,
> the file must be opened with a flag FILE_SHARE_DELETE, or the file
> will not be deleted.
> 
> And I don't know how to integrate the Win32 API with the emacs codes.
> Emacs codes use POSIX functions(_open, _close, _read...), and the
> delete_on_close feature required WIN32 API. I've tried using
> _O_TEMPORARY flags with _open function, but not succeeded.

Yes, this technique will not help, for all the reasons you mention,
and then some.  For starters, in our case the problem is not with
Emacs holding the file open -- the call to write-region within
call-process-region closes the file before it returns -- but with the
child process that has the file connected to its standard input.  So
another process is involved here, not just Emacs.

Also, Emacs always uses emacs_open to open any files, which means it
doesn't know which files are temporary and which aren't.  Obviously,
we cannot open all files with FILE_FLAG_DELETE_ON_CLOSE or
_O_TEMPORARY, because most files Emacs writes should _not_ be deleted
when Emacs closes them.

Anyway, I think the solution to this should be to somehow retry the
deletion later, when the child process exits.  Nothing else will
resolve this bug.

Message #31 received at 13079-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: mrzhaili <at> gmail.com
Cc: 13079-done <at> debbugs.gnu.org
Subject: Re: bug#13079: 24.3.50; Emacs cannot create subprocess
Date: Sat, 15 Dec 2012 15:41:41 +0200

> Date: Wed, 05 Dec 2012 19:29:58 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 13079 <at> debbugs.gnu.org
> 
> The harder part of this bug is the reason _why_ 'mktemp' fails.  Your
> command launches asynchronous subprocesses, creating a temporary file
> for each one of them where Emacs puts the contents of the region.  In
> this case, call-process-region arranges for the temporary files to be
> removed when the call to call-process returns, and then calls
> call-process.  However, since this is an async subprocess,
> call-process returns immediately, without waiting for the subprocess
> to exit, and Emacs deletes the temporary file.  This "cleanup" uses
> the non-portable (outside of Posix filesystems) trick of deleting a
> file while it is still open and used by the child process.  On
> Windows, this attempt to delete the file fails, and the file is left
> behind.
> 
> The other part of this puzzle is that 'mktemp' as implemented by the
> Windows runtime can only have up to 42 simultaneous temporary files
> per calling thread (the MSDN documentation says 26, but that's a lie).
> Once there are 42 files in your TEMP directory created by a single
> Emacs session, all the future calls to 'mktemp' from the same session
> will fail, unless you manually delete those files.
> 
> This part of the bug is harder to solve, because some code needs to be
> implemented that will defer deletion until the process exits.  So it
> is not solved yet.

Should be fixed now (revision 11244 on the trunk).  I was able to run
your test case multiple times without any errors, and after that,
there were no left-over temporary files in the temporary directory.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.