GNU bug report logs - #77389
31.0.50; Restarting Emacs with (kill-emacs ... t) looses noninteractivity

Previous Next

To reply to this bug, email your comments to 77389 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; Restarting Emacs with (kill-emacs ... t) looses
 noninteractivity
Date: Sun, 30 Mar 2025 19:20:58 +0200

[ Disclaimer:

  - This is one of my more exotic bug reports, so I won't argue
    (at least not much) if you close this as "won't fix".

  - In particular since an Emacs restart, added by Lars in
    5be9a9ca, probably has only been meant to be used
    interactively.

  - But I do have a valid use case for a non-interactive restart,
    so I created this bug report ... ]


Write the following to a file named test.el:

------------------------- test.el -------------------------
(message "%S:%S" noninteractive command-line-args)
(when noninteractive
  (kill-emacs 0 t))
------------------------- test.el -------------------------

Then execute that file with

  ./src/emacs -Q --script test.el

I would expect that to inf-loop through the execvp from Fkill_emacs, but
instead it prints

  t:("./src/emacs" "-scriptload" "test.el")

to STDOUT and opens a GUI window with the message

  nil:("/home/jschmidt/work/emacs-master/src/emacs" "-scriptload" "test.el")

in its mode line.


Here is what I think happens (or at least a simplified version of
it, which does not cover e.g. the sorting of command line
arguments done by main):

1. The main function in emacs.c converts option "--script" to
   "-scriptload", setting the noninteractive flag while doing do:

     if (argmatch (argv, argc, "-script", "--script", 3, &junk, &skip_args))
       {
         noninteractive = 1;	/* Set batch mode.  */
         /* Convert --script to -scriptload, un-skip it, and sort again
            so that it will be handled in proper sequence.  */
         /* FIXME broken for --script=FILE - is that supposed to work?  */
         argv[skip_args - 1] = (char *) "-scriptload";
         skip_args -= 2;
         sort_args (argc, argv);
       }

2. Later in main, function init_cmdargs stores away the modified argv
   containing "-scriptload" into initial_argv.

3. When Fkill_emacs gets called with non-nil second argument, it calls
   function execvp on initial_argv, which contains "-scriptload" instead
   of "--script".

4. Therefore the next Emacs execution does not process the snippet
   quoted under 1. above and, hence, does not set the noninteractive
   flag.

As a work-around, one can put a "--batch" option before the "--script"
option, then above repro inf-loops as expected (by me).

Probably Fkill_emacs should call execvp not on the command line
arguments in initial_argv, which already got modified by main,
but on a copy of the *real* initial command line arguments as
they have been passed into main?  I have a patch to that effect,
and it also lets above repro inf-loop as expected by me.

Thanks as always for caring about exotic and non-exotic bugs and
Emacs as a whole!



In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.16.0) of 2025-03-30 built on sappc2
Repository revision: a72cfc52cc36b8233969dafb202915d36c4c5a9e
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12201009
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --with-native-compilation --with-mailutils'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM XINERAMA XINPUT2 XPM XRANDR GTK3
ZLIB

Important settings:
  value of $LC_COLLATE: POSIX
  value of $LC_TIME: POSIX
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr compile comint ansi-osc ansi-color ring comp-run
bytecomp byte-compile comp-common rx emacsbug lisp-mnt message mailcap
yank-media puny dired dired-loaddefs rfc822 mml mml-sec password-cache
epa derived epg rfc6068 epg-config gnus-util text-property-search
time-date subr-x mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd touch-screen
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo gtk x-toolkit xinput2 x multi-tty move-toolbar
make-network-process tty-child-frames native-compile emacs)

Memory information:
((conses 16 57068 9506) (symbols 48 6816 0) (strings 32 16707 3815)
 (string-bytes 1 439863) (vectors 16 11584)
 (vector-slots 8 134675 5392) (floats 8 21 14) (intervals 56 249 0)
 (buffers 984 11))

Message #8 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50;
 Restarting Emacs with (kill-emacs ... t) looses noninteractivity
Date: Sun, 30 Mar 2025 21:29:16 +0300

> Date: Sun, 30 Mar 2025 19:20:58 +0200
> From:  Jens Schmidt via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> [ Disclaimer:
> 
>   - This is one of my more exotic bug reports, so I won't argue
>     (at least not much) if you close this as "won't fix".
> 
>   - In particular since an Emacs restart, added by Lars in
>     5be9a9ca, probably has only been meant to be used
>     interactively.
> 
>   - But I do have a valid use case for a non-interactive restart,
>     so I created this bug report ... ]

Please describe your use case.

> Probably Fkill_emacs should call execvp not on the command line
> arguments in initial_argv, which already got modified by main,
> but on a copy of the *real* initial command line arguments as
> they have been passed into main?

Yes.  And on MS-Windows, it already does.

Message #11 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50; Restarting Emacs with (kill-emacs ... t)
 looses noninteractivity
Date: Tue, 1 Apr 2025 22:52:48 +0200

[Message part 1 (text/plain, inline)]

On 2025-03-30  20:29, Eli Zaretskii wrote:
>> Date: Sun, 30 Mar 2025 19:20:58 +0200
>> From:  Jens Schmidt via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>>
>> [ Disclaimer:
>>
>>   - This is one of my more exotic bug reports, so I won't argue
>>     (at least not much) if you close this as "won't fix".
>>
>>   - In particular since an Emacs restart, added by Lars in
>>     5be9a9ca, probably has only been meant to be used
>>     interactively.
>>
>>   - But I do have a valid use case for a non-interactive restart,
>>     so I created this bug report ... ]
> 
> Please describe your use case.

TL;DR:

1. Start foo.el in batch mode with "emacs --script foo.el".
2. foo.el reads the user init file and writes a modified copy of
   it to init-new.el.
3. Then foo.el calls (kill-emacs ... t), thus restarting itself,
   (hopefully) still in batch mode.
4. The newly started foo.el senses the presence of init-new.el
   and, hence, processes that in a final step.

In the above it is important that steps 2 and 4 are executed
from different, newly started Emacs processes which share STDOUT
and STDERR.

The bigger picture is my package bc-dot-emacs.el

  bc-dot-emacs.el --- Check Emacs init file syntax

  https://git.sr.ht/~jschmidt/bc-dot-emacs

which I sooner or later wanted to present on emacs-devel for
inclusion to ELPA.

>> Probably Fkill_emacs should call execvp not on the command line
>> arguments in initial_argv, which already got modified by main,
>> but on a copy of the *real* initial command line arguments as
>> they have been passed into main?
> 
> Yes.  And on MS-Windows, it already does.

Indeed, I missed that bit.  The only other non-Posix case seems
to be Android, which a) IIRC does not support non-interactive
operation and b) restarts Emacs with some mechanism in Java-land
that does not require Emacs itself to keep a copy of the initial
arguments.

So this leaves the Posix/execvp case, I think, which I hope to
have covered in the attached patch.

The only thing that bothers me a bit is that there is the following
memory-related initialization being executed in main *after*
copy_raw_args (or also the present sort_args) have already been
allocating memory:

#if !defined SYSTEM_MALLOC
  /* Arrange to get warning messages as memory fills up.  */
  memory_warnings (0, malloc_warning);

  /* Call malloc at least once, to run malloc_initialize_hook.
     Also call realloc and free for consistency.  */
  free (realloc (malloc (4), 4));

#endif	/* not SYSTEM_MALLOC */

Thanks.

[0001-Use-raw-argv-as-passed-into-main-to-restart-Emacs.patch (text/x-patch, attachment)]

Message #14 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50; Restarting Emacs with (kill-emacs ... t)
 looses noninteractivity
Date: Wed, 02 Apr 2025 14:43:47 +0300

> Date: Tue, 1 Apr 2025 22:52:48 +0200
> From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> Cc: 77389 <at> debbugs.gnu.org
> 
> >>   - But I do have a valid use case for a non-interactive restart,
> >>     so I created this bug report ... ]
> > 
> > Please describe your use case.
> 
> TL;DR:
> 
> 1. Start foo.el in batch mode with "emacs --script foo.el".
> 2. foo.el reads the user init file and writes a modified copy of
>    it to init-new.el.
> 3. Then foo.el calls (kill-emacs ... t), thus restarting itself,
>    (hopefully) still in batch mode.
> 4. The newly started foo.el senses the presence of init-new.el
>    and, hence, processes that in a final step.
> 
> In the above it is important that steps 2 and 4 are executed
> from different, newly started Emacs processes which share STDOUT
> and STDERR.

Why do you need to write Lisp to a file, only to have that file read
and executed? why not simply execute that Lisp directly in the
original session?

> The only thing that bothers me a bit is that there is the following
> memory-related initialization being executed in main *after*
> copy_raw_args (or also the present sort_args) have already been
> allocating memory:

Why does this bother you?  The old session is going down, and the new
one will re-execute that part anew, no?  What did I miss?

Message #17 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50; Restarting Emacs with (kill-emacs ... t)
 looses noninteractivity
Date: Wed, 2 Apr 2025 22:54:23 +0200

On 2025-04-02  13:43, Eli Zaretskii wrote:

>>> Please describe your use case.
>>
>> TL;DR:
>>
>> 1. Start foo.el in batch mode with "emacs --script foo.el".
>> 2. foo.el reads the user init file and writes a modified copy of
>>    it to init-new.el.
>> 3. Then foo.el calls (kill-emacs ... t), thus restarting itself,
>>    (hopefully) still in batch mode.
>> 4. The newly started foo.el senses the presence of init-new.el
>>    and, hence, processes that in a final step.
>>
>> In the above it is important that steps 2 and 4 are executed
>> from different, newly started Emacs processes which share STDOUT
>> and STDERR.
> 
> Why do you need to write Lisp to a file, only to have that file read
> and executed? why not simply execute that Lisp directly in the
> original session?

Because the state of the original session (most notably the loaded
features) influences (and gets influenced by) that Lisp.  As a
minimal reproducer, consider ~/test.el as:

----- ~/test.el -----
;; -*- lexical-binding: t -*-
(gnus-message "foo")
----- ~/test.el -----

Now the Lisp I want to execute in above foo.el is, grossly simplified,

  (byte-compile-file "~/test.el").

And the result of that depends on whether gnus-utils has been loaded
in the calling Emacs session or not.


>> The only thing that bothers me a bit is that there is the following
>> memory-related initialization being executed in main *after*
>> copy_raw_args (or also the present sort_args) have already been
>> allocating memory:
> 
> Why does this bother you?  The old session is going down, and the new
> one will re-execute that part anew, no?  What did I miss?

I'm not talking about old or new session here.  What bothers me is
also independent of this bug, I just came across it while staring at
the surrounding code.  Namely at the comment in this block:

  /* Call malloc at least once, to run malloc_initialize_hook.
     Also call realloc and free for consistency.  */
  free (realloc (malloc (4), 4));

This sounds like this is initialization that better should be
executed *before* any actual call to malloc is done.  Which is
currently not the case, since, for example, sort_args can call
(x)malloc and sort_args is called before the block quoted above.
But then I do not really understand what that block is about and
my concerns may be completely unsubstantiated.


But back to my patch: I did a cursory grep for initial_arg[cv]
through current master.  From the results I got the impression
that instead of setting up a second copy raw_initial_arg[cv]
like done in the first patch, one could also try to directly
use initial_arg[cv], like in the attached alternative patch.
Should I go that route instead or would that be too risky,
anyway?

Thanks!

Message #20 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50; Restarting Emacs with (kill-emacs ... t)
 looses noninteractivity
Date: Wed, 2 Apr 2025 23:20:36 +0200

[Message part 1 (text/plain, inline)]

Forgot to attach the alternative patch, sorry.

[0001-Use-raw-argv-as-passed-into-main-to-restart-Emacs-alternative.patch (text/x-patch, attachment)]

Message #23 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50; Restarting Emacs with (kill-emacs ... t)
 looses noninteractivity
Date: Thu, 03 Apr 2025 08:10:48 +0300

> Date: Wed, 2 Apr 2025 22:54:23 +0200
> Cc: 77389 <at> debbugs.gnu.org
> From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> 
> > Why do you need to write Lisp to a file, only to have that file read
> > and executed? why not simply execute that Lisp directly in the
> > original session?
> 
> Because the state of the original session (most notably the loaded
> features) influences (and gets influenced by) that Lisp.

I know the theory, but why is it a problem in practice?  We bump into
this issue every day, because doing stuff from an Emacs session that
runs for some time will always produce results that depend on what
happened in that session beforehand.

If you really do need to do something in a pristine Emacs, do that in
a separate sub-process, then read the results.  That's what we do with
native-compilation, for example.

> >> The only thing that bothers me a bit is that there is the following
> >> memory-related initialization being executed in main *after*
> >> copy_raw_args (or also the present sort_args) have already been
> >> allocating memory:
> > 
> > Why does this bother you?  The old session is going down, and the new
> > one will re-execute that part anew, no?  What did I miss?
> 
> I'm not talking about old or new session here.  What bothers me is
> also independent of this bug, I just came across it while staring at
> the surrounding code.  Namely at the comment in this block:
> 
>   /* Call malloc at least once, to run malloc_initialize_hook.
>      Also call realloc and free for consistency.  */
>   free (realloc (malloc (4), 4));
> 
> This sounds like this is initialization that better should be
> executed *before* any actual call to malloc is done.  Which is
> currently not the case, since, for example, sort_args can call
> (x)malloc and sort_args is called before the block quoted above.
> But then I do not really understand what that block is about and
> my concerns may be completely unsubstantiated.

Yes, we call heap allocation functions before that, so you don't need
to worry about that.  The purpose of that line is not to initialize
malloc, it's something else.

> But back to my patch: I did a cursory grep for initial_arg[cv]
> through current master.  From the results I got the impression
> that instead of setting up a second copy raw_initial_arg[cv]
> like done in the first patch, one could also try to directly
> use initial_arg[cv], like in the attached alternative patch.
> Should I go that route instead or would that be too risky,
> anyway?

That's a possibility, but please make all this copy_raw_args and its
calls #ifdef'ed away on WINDOWSNT, since we already do something like
that there, just better.  Also, these additions need comments to
explain why we do this.

Message #26 received at 77389 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77389 <at> debbugs.gnu.org
Subject: Re: bug#77389: 31.0.50; Restarting Emacs with (kill-emacs ... t)
 looses noninteractivity
Date: Thu, 3 Apr 2025 21:52:33 +0200

On 2025-04-03  07:10, Eli Zaretskii wrote:
>> Date: Wed, 2 Apr 2025 22:54:23 +0200
>> Cc: 77389 <at> debbugs.gnu.org
>> From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
>>
>>> Why do you need to write Lisp to a file, only to have that file read
>>> and executed? why not simply execute that Lisp directly in the
>>> original session?
>>
>> Because the state of the original session (most notably the loaded
>> features) influences (and gets influenced by) that Lisp.
> 
> If you really do need to do something in a pristine Emacs, do that in
> a separate sub-process, then read the results.  That's what we do with
> native-compilation, for example.

Thanks for the discussion, now we are basically at the point
where I failed to see a straightforward solution further on
and resorted to `(kill-emacs ... t)'.  Because:

The driving Elisp (foo.el) is only ever to be called as a script.
If I wanted to spawn a pristine and equally non-interactive Emacs
session from foo.el and present its STDOUT and STDERR together
with that of foo.el, then I would need to do so through an async
process, right?  With all the effort of setting up a sentinel and
a filter just to forward the child Emacs's STDOUT and STDERR to,
well, STDOUT.

Given the effort described above simply calling `kill-process'
plus a bit of recursion control seemed the better or at least
easier solution to transfer control to a pristine Emacs process.

>> But back to my patch: I did a cursory grep for initial_arg[cv]
>> through current master.  From the results I got the impression
>> that instead of setting up a second copy raw_initial_arg[cv]
>> like done in the first patch, one could also try to directly
>> use initial_arg[cv], like in the attached alternative patch.
>> Should I go that route instead or would that be too risky,
>> anyway?
> 
> That's a possibility, but please make all this copy_raw_args and its
> calls #ifdef'ed away on WINDOWSNT, since we already do something like
> that there, just better.  Also, these additions need comments to
> explain why we do this.

Agreed.  I checked the grep results in more detail, resulting in
the following follow-up questions:

- I guess the following files all don't get compiled on Windows
  and any references to initial_arg[vc] can be safely ignored in
  these: src/nsterm.m, src/xterm.c, src/xsmfns.c, src/pgtkterm.c

- sysdep.c refers to initial_argv[0] in function emacs_perror
  and it seems it does so for Windows as well.  What should I
  do about that one?  Use simply constant "emacs"?  Or the
  result of w32_my_exename (plus any required coding system
  conversion)?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.