GNU bug report logs - #34123
A patch to fix reading EOF characters in non-interactive mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 34123 in the body.
You can then email your comments to 34123 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Shawn Presser <shawnpresser <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: A patch to fix reading EOF characters in non-interactive mode
Date: Fri, 18 Jan 2019 04:35:34 -0600

[Message part 1 (text/plain, inline)]

Hello,

Here is a patch to allow batch mode scripts to handle EOF characters. This
solves the following problem:

$ git clone https://github.com/shawwn/y
$ cd y
$ bin/y
> (read t)
Lisp expression: 42
42
> (read t)
Lisp expression: ^Derror: Error reading from stdin
> Error reading from stdin
$

Notice that it’s currently impossible for non-interactive scripts to
recover from EOF when reading from tty. After the user sends an EOF
character by pressing C-d, calling read-from-minibuffer will always result
in an error.

This patch solves this by clearing stdin of errors and returning nil when
an EOF character is detected.

Here is the patch:

From 685452ca4a098bbd7c1062a35064755c1d09512a Mon Sep 17 00:00:00 2001
From: Shawn Presser <shawnpresser <at> gmail.com>
Date: Fri, 18 Jan 2019 03:57:37 -0600
Subject: [PATCH] Fix reading EOF characters in batch mode.

* src/minibuf.c (read_minibuf_noninteractive): Handle EOF characters by
clearing stdin and returning Qnil.
---
 src/minibuf.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/minibuf.c b/src/minibuf.c
index 321fda1ba8..8c88a316a7 100644
--- a/src/minibuf.c
+++ b/src/minibuf.c
@@ -250,7 +250,8 @@ read_minibuf_noninteractive (Lisp_Object prompt, bool
expflag,
   else
     {
       xfree (line);
-      error ("Error reading from stdin");
+      clearerr(stdin);
+      return Qnil;
     }

   /* If Lisp form desired instead of string, parse it.  */
-- 
2.19.1

Thanks!
— shawn

[Message part 2 (text/html, inline)]

[0001-Fix-reading-EOF-characters-in-batch-mode.patch (application/octet-stream, attachment)]

Message #8 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Shawn Presser <shawnpresser <at> gmail.com>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Fri, 18 Jan 2019 15:25:02 +0200

> From: Shawn Presser <shawnpresser <at> gmail.com>
> Date: Fri, 18 Jan 2019 04:35:34 -0600
> 
> $ git clone https://github.com/shawwn/y
> $ cd y
> $ bin/y
> > (read t)
> Lisp expression: 42
> 42
> > (read t)
> Lisp expression: ^Derror: Error reading from stdin
> > Error reading from stdin
> $ 
> 
> Notice that it’s currently impossible for non-interactive scripts to recover from EOF when reading from tty.

What would you want such non-interactive scripts to do after recovery?

Message #11 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Shawn Presser <shawnpresser <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Fri, 18 Jan 2019 08:22:31 -0600

[Message part 1 (text/plain, inline)]

I use Emacs Lisp as a runtime for Arc (https://github.com/shawwn/arcmacs) and
for Lumen (https://github.com/shawwn/y). The goal is to be able to run a
Lisp REPL from the terminal.

For example, in Arc:

  $ git clone https://github.com/laarc/laarc
  $ cd laarc
  $ make
  $ bin/arc
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> 1
  1
  arc> (def my-repl ()
         (pr "> ")
         (whilet expr (read)
           (write (eval expr))
           (prn) (pr "> ")))
  *** redefining repl
  #<procedure: repl>
  arc> (my-repl)
  > 1 ; we can exit this inner REPL with control-D, without exiting the
program.
  1
  > (+ 1 2)
  3
  > ^D
  arc> "If this were emacs in batch mode, the program would have exited
with errors here."
  "If this were emacs in batch mode, the program would have exited with
errors here."
  arc> "Instead, control-D returned us to the toplevel REPL. Let's type
another control-D to quit."
  "Instead, control-D returned us to the toplevel REPL. Let's type another
control-D to quit."
  arc> ^D
  $

Please compare this with arcmacs, which runs in Emacs Lisp:

  $ git clone https://github.com/shawwn/arcmacs
  $ cd arcmacs
  $ ./y-arc
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> 1
  1
  arc> (def my-repl ()
         (pr "> ")
         (whilet expr (read)
           (write (eval expr))
           (prn) (pr "> ")))
  arc> (my-repl)
  > 1  ; if we enter control-D, every REPL exits immediately and the
program quits
  1
  > 2
  2
  > ^D
  arc>
  $ echo We didn't intend to quit yet!

A single ^D exits the entire program, not just the inner REPL. This is
because after the user enters control-D, any attempt to call
`read-from-minibuffer` will always result in emacs throwing an error
with the message "Error reading from stdin".

The only way to gracefully handle this case in a script is to trap the
internal emacs error with (ignore-errors (read-from-minibuffer ""))
and to treat a nil result as if it were EOF. But that causes every
nested REPL to exit, since every attempt to (read-from-minibuffer "")
will throw "Error reading from stdin". In other words, emacs is
always failing to read from stdin after the user sends an EOF.

This patch fixes the problem by causing (read-from-minibuffer "") to
return nil if the user types control-D.


On Fri, Jan 18, 2019 at 7:25 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Shawn Presser <shawnpresser <at> gmail.com>
> > Date: Fri, 18 Jan 2019 04:35:34 -0600
> >
> > $ git clone https://github.com/shawwn/y
> > $ cd y
> > $ bin/y
> > > (read t)
> > Lisp expression: 42
> > 42
> > > (read t)
> > Lisp expression: ^Derror: Error reading from stdin
> > > Error reading from stdin
> > $
> >
> > Notice that it’s currently impossible for non-interactive scripts to
> recover from EOF when reading from tty.
>
> What would you want such non-interactive scripts to do after recovery?
>

[Message part 2 (text/html, inline)]

Message #14 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Shawn Presser <shawnpresser <at> gmail.com>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Fri, 18 Jan 2019 18:03:49 +0200

> From: Shawn Presser <shawnpresser <at> gmail.com>
> Date: Fri, 18 Jan 2019 08:22:31 -0600
> Cc: 34123 <at> debbugs.gnu.org
> 
> I use Emacs Lisp as a runtime for Arc (https://github.com/shawwn/arcmacs) and for Lumen
> (https://github.com/shawwn/y). The goal is to be able to run a Lisp REPL from the terminal.

So the usage you describe, and are interested in, is not
non-interactive, as far as Emacs is concerned, right?  Or am I missing
something?

Message #17 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Shawn Presser <shawnpresser <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Fri, 18 Jan 2019 10:20:54 -0600

[Message part 1 (text/plain, inline)]

It's non-interactive:

arc> |noninteractive|
t

(The expression |noninteractive| in arcmacs is equivalent to evaluating
'noninteractive in emacs lisp.)

The runtimes start by invoking emacs using `emacs -Q --script ...`, so it's
always noninteractive. And in general for writing shell scripts rather than
repls, it's important to be in noninteractive mode. But it's equally
important to have the ability to prompt the user from a shell script, and
for the user to be able to cancel by using ^D without causing stdin errors
on all subsequent prompts.



On Fri, Jan 18, 2019 at 10:04 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Shawn Presser <shawnpresser <at> gmail.com>
> > Date: Fri, 18 Jan 2019 08:22:31 -0600
> > Cc: 34123 <at> debbugs.gnu.org
> >
> > I use Emacs Lisp as a runtime for Arc (https://github.com/shawwn/arcmacs)
> and for Lumen
> > (https://github.com/shawwn/y). The goal is to be able to run a Lisp
> REPL from the terminal.
>
> So the usage you describe, and are interested in, is not
> non-interactive, as far as Emacs is concerned, right?  Or am I missing
> something?
>

[Message part 2 (text/html, inline)]

Message #20 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Shawn Presser <shawnpresser <at> gmail.com>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Fri, 18 Jan 2019 18:29:10 +0200

> From: Shawn Presser <shawnpresser <at> gmail.com>
> Date: Fri, 18 Jan 2019 10:20:54 -0600
> Cc: 34123 <at> debbugs.gnu.org
> 
> It's non-interactive:
> 
> arc> |noninteractive|
> t
> 
> (The expression |noninteractive| in arcmacs is equivalent to evaluating 'noninteractive in emacs lisp.)
> 
> The runtimes start by invoking emacs using `emacs -Q --script ...`, so it's always noninteractive. And in
> general for writing shell scripts rather than repls, it's important to be in noninteractive mode. But it's equally
> important to have the ability to prompt the user from a shell script, and for the user to be able to cancel by
> using ^D without causing stdin errors on all subsequent prompts.

I think we are miscommunicating, since your usage involves Emacs
reading from the terminal.  That is not noninteractive in my book.
What I mean by noninteractive is when stdin is not connected to a
terminal.  Your proposed patch affects that case as well, doesn't it?

Message #23 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Shawn Presser <shawnpresser <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Fri, 18 Jan 2019 11:23:11 -0600

[Message part 1 (text/plain, inline)]

> What I mean by noninteractive is when stdin is not connected to a
terminal.  Your proposed patch affects that case as well, doesn't it?

I don't think so, because the only case that this patch affects is the case
where emacs would signal "error reading from stdin". I don't think I've
ever seen emacs throw that error during normal emacs usage, even when
reading from stdin not connected to a terminal.

But admittedly, I am not certain in all cases. It's probably better to
phrase this as a question: Has any elisp code been written to handle "error
reading from stdin"? For example, (ignore-error ...) would mask the error
message, if it could somehow be generated during normal emacs operation.

However, even in that case, the operation of emacs would end up exactly the
same, since `nil` is now returned, which is exactly what the `(ignore-error
...)` expression would return (if such code already exists).

That said, it would be surprising if this were expected behavior, because
the only way for user code to detect and handle it would be to parse the
error message, since it's a regular 'error type rather than a type that can
be handled via condition-case. I don't think this situation has been
covered till now, because emacs users typically don't care about reading
from stdin in batch mode.

Actually, I just found a blog post from 2014 that complains about this very
case:
http://joelmccracken.github.io/entries/reading-writing-data-in-emacs-batch-via-stdin-stdout/

> When EOF is read, an error is thrown, which is useless to us. Instead of
worrying about the error, we can wrap the function call in an ignore-errors
macro, which will return nil when an EOF occurs.

They don't seem to care about being able to call read-from-minibuffer
multiple times. But if they did care, they would find it quite impossible
after the first EOF.


On Fri, Jan 18, 2019 at 10:29 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Shawn Presser <shawnpresser <at> gmail.com>
> > Date: Fri, 18 Jan 2019 10:20:54 -0600
> > Cc: 34123 <at> debbugs.gnu.org
> >
> > It's non-interactive:
> >
> > arc> |noninteractive|
> > t
> >
> > (The expression |noninteractive| in arcmacs is equivalent to evaluating
> 'noninteractive in emacs lisp.)
> >
> > The runtimes start by invoking emacs using `emacs -Q --script ...`, so
> it's always noninteractive. And in
> > general for writing shell scripts rather than repls, it's important to
> be in noninteractive mode. But it's equally
> > important to have the ability to prompt the user from a shell script,
> and for the user to be able to cancel by
> > using ^D without causing stdin errors on all subsequent prompts.
>
> I think we are miscommunicating, since your usage involves Emacs
> reading from the terminal.  That is not noninteractive in my book.
> What I mean by noninteractive is when stdin is not connected to a
> terminal.  Your proposed patch affects that case as well, doesn't it?
>

[Message part 2 (text/html, inline)]

Message #26 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Shawn Presser <shawnpresser <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Sat, 19 Jan 2019 19:22:06 -0600

[Message part 1 (text/plain, inline)]

Is there another way I can address the concern here? I'm willing to put in
the work to get this in.

To put it more simply, it seems like an error that emacs doesn't call
clearerr(stdin) after reading EOF. Without that, it's impossible to ever
read from stdin after the first EOF. Should that be changed?

On Fri, Jan 18, 2019 at 10:29 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Shawn Presser <shawnpresser <at> gmail.com>
> > Date: Fri, 18 Jan 2019 10:20:54 -0600
> > Cc: 34123 <at> debbugs.gnu.org
> >
> > It's non-interactive:
> >
> > arc> |noninteractive|
> > t
> >
> > (The expression |noninteractive| in arcmacs is equivalent to evaluating
> 'noninteractive in emacs lisp.)
> >
> > The runtimes start by invoking emacs using `emacs -Q --script ...`, so
> it's always noninteractive. And in
> > general for writing shell scripts rather than repls, it's important to
> be in noninteractive mode. But it's equally
> > important to have the ability to prompt the user from a shell script,
> and for the user to be able to cancel by
> > using ^D without causing stdin errors on all subsequent prompts.
>
> I think we are miscommunicating, since your usage involves Emacs
> reading from the terminal.  That is not noninteractive in my book.
> What I mean by noninteractive is when stdin is not connected to a
> terminal.  Your proposed patch affects that case as well, doesn't it?
>

[Message part 2 (text/html, inline)]

Message #29 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Shawn Presser <shawnpresser <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Mon, 21 Jan 2019 02:59:36 -0600

[Message part 1 (text/plain, inline)]

Hi, just checking in. I wanted to thank you for taking the time to reply
and ask about this bug. It's my first emacs patch and I'm not entirely sure
if I'm following the process correctly here. I'd like to try answering your
questions more succinctly.

Is there a "next step" at this point, or something that I can do to help
with emacs in general?




On Fri, Jan 18, 2019 at 10:29 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Shawn Presser <shawnpresser <at> gmail.com>
> > Date: Fri, 18 Jan 2019 10:20:54 -0600
> > Cc: 34123 <at> debbugs.gnu.org
> >
> > It's non-interactive:
> >
> > arc> |noninteractive|
> > t
> >
> > (The expression |noninteractive| in arcmacs is equivalent to evaluating
> 'noninteractive in emacs lisp.)
> >
> > The runtimes start by invoking emacs using `emacs -Q --script ...`, so
> it's always noninteractive. And in
> > general for writing shell scripts rather than repls, it's important to
> be in noninteractive mode. But it's equally
> > important to have the ability to prompt the user from a shell script,
> and for the user to be able to cancel by
> > using ^D without causing stdin errors on all subsequent prompts.
>
> I think we are miscommunicating, since your usage involves Emacs
> reading from the terminal.  That is not noninteractive in my book.
> What I mean by noninteractive is when stdin is not connected to a
> terminal.  Your proposed patch affects that case as well, doesn't it?
>

[Message part 2 (text/html, inline)]

Message #32 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Shawn Presser <shawnpresser <at> gmail.com>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Mon, 21 Jan 2019 17:44:20 +0200

> From: Shawn Presser <shawnpresser <at> gmail.com>
> Date: Mon, 21 Jan 2019 02:59:36 -0600
> Cc: 34123 <at> debbugs.gnu.org
> 
> Hi, just checking in. I wanted to thank you for taking the time to reply and ask about this bug. It's my first
> emacs patch and I'm not entirely sure if I'm following the process correctly here. I'd like to try answering your
> questions more succinctly.
> 
> Is there a "next step" at this point, or something that I can do to help with emacs in general?

I'm still considering the implications of your suggestion on the other
use cases.  One possible alternative would be to add a function that
will clear the error condition, and let users/Lisp programs call it.
This way, the default response to EOF will remain intact.

Perhaps someone else will chime in with opinions and ideas.

Thanks.

Message #35 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Shawn Presser <shawnpresser <at> gmail.com>
Cc: 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Sat, 22 Jun 2019 20:18:36 -0400

[Message part 1 (text/plain, inline)]

Shawn Presser <shawnpresser <at> gmail.com> writes:

> Notice that it’s currently impossible for non-interactive scripts to
> recover from EOF when reading from tty. After the user sends an EOF
> character by pressing C-d, calling read-from-minibuffer will always result
> in an error.

This isn't quite true, see the attached script which catches the error
and only exits after two EOFs.

[bug-34123-read-script.el (text/plain, attachment)]

[Message part 3 (text/plain, inline)]

A sample usage:

    $ emacs -Q --script ../bug-34123-read-script.el 
    > one
    line0: "one"
    > 
    EOF1
    > two
    line2: "two"
    > 
    EOF2

>        xfree (line);
> -      error ("Error reading from stdin");
> +      clearerr(stdin);
> +      return Qnil;

I think it would make sense to change that error to be an end-of-file
error, to make it easier to catch (having to examine the error message,
as I did in my script is not so nice).

Message #38 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Shawn Presser <shawnpresser <at> gmail.com>, 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Mon, 10 Aug 2020 13:56:11 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> I'm still considering the implications of your suggestion on the other
> use cases.  One possible alternative would be to add a function that
> will clear the error condition, and let users/Lisp programs call it.
> This way, the default response to EOF will remain intact.
>
> Perhaps someone else will chime in with opinions and ideas.

And then Noam answered a few months later:

Noam Postavsky <npostavs <at> gmail.com> writes:

> Shawn Presser <shawnpresser <at> gmail.com> writes:
>
>> Notice that it’s currently impossible for non-interactive scripts to
>> recover from EOF when reading from tty. After the user sends an EOF
>> character by pressing C-d, calling read-from-minibuffer will always result
>> in an error.
>
> This isn't quite true, see the attached script which catches the error
> and only exits after two EOFs.

Shawn, does Noam's solution work for you?

Noam also had this suggestion:

>>        xfree (line);
>> -      error ("Error reading from stdin");
>> +      clearerr(stdin);
>> +      return Qnil;
>
> I think it would make sense to change that error to be an end-of-file
> error, to make it easier to catch (having to examine the error message,
> as I did in my script is not so nice).

And I think that's true?  It happens here in read_minibuf_noninteractive:

  if (len || c == '\n' || c == '\r')
    {
      val = make_string (line, len);
      xfree (line);
    }
  else
    {
      xfree (line);
      error ("Error reading from stdin");
    }

So I think that error can only happen on an end-of-file?  Anybody
objecting to changing that to and end-of-file error?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #41 received at 34123 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Shawn Presser <shawnpresser <at> gmail.com>, 34123 <at> debbugs.gnu.org
Subject: Re: bug#34123: A patch to fix reading EOF characters in
 non-interactive mode
Date: Wed, 19 Aug 2020 12:57:18 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> And I think that's true?  It happens here in read_minibuf_noninteractive:
>
>   if (len || c == '\n' || c == '\r')
>     {
>       val = make_string (line, len);
>       xfree (line);
>     }
>   else
>     {
>       xfree (line);
>       error ("Error reading from stdin");
>     }
>
> So I think that error can only happen on an end-of-file?  Anybody
> objecting to changing that to and end-of-file error?

There were no objections, so I've now done this in Emacs 28.  (It didn't
seem NEWS-worthy, so I didn't add a NEWS item.)

Witch that change, I think Noam's approach to handling EOF characters in
non-interactive Emacsen should work fine, so I'm closing this bug
report.  If more work needs to be done here, please respond and we'll
reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.