GNU bug report logs -
#59545
29.0.50; Eshell fails to redirect output of sourced eshell file
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 59545 in the body.
You can then email your comments to 59545 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Thu, 24 Nov 2022 15:50:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Milan Zimmermann <milan.zimmermann <at> gmail.com>
:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org
.
(Thu, 24 Nov 2022 15:50:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Emacs 29 Eshell Bug: Sourcing 'redirect-echo.esh' and redirecting output
to a file, results in the first echo string ('hello') showing in eshell,
only the second ('there')(presumably because it is last) showing in the
output file.
To reproduce: Create a eshell script named 'redirect-echo.esh' with two
echo commands then source the script, redirecting the stdout to another
file or buffer,
Actual result:
tmp $ cat redirect-echo.esh
echo hello
echo there
tmp $ source redirect-echo.esh > redirect-echo.out
hello
tmp $ cat redirect-echo.out
theretmp $
Expected result:
tmp $ cat redirect-echo.esh
echo hello
echo there
tmp $ source redirect-echo.esh > redirect-echo.out
tmp $ cat redirect-echo.out
hello
there
tmp $
As a note, the same behavior if elisp "print" is used instead of
echo. Also the same behavior if I redirect output to an Emacs buffer
instead of the file.
In GNU Emacs 29.0.50 (build 1, x86_64-suse-linux-gnu, GTK+ Version
3.24.34, cairo version 1.17.6)
System Description: openSUSE Tumbleweed
Configured using:
'configure --host=x86_64-suse-linux-gnu --build=x86_64-suse-linux-gnu
--program-prefix= --disable-dependency-tracking --prefix=/usr
--exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
--sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include
--libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var
--sharedstatedir=/var/lib --mandir=/usr/share/man
--infodir=/usr/share/info --disable-build-details --without-pop
--with-mailutils --without-hesiod --with-gameuser=:games
--with-kerberos --with-kerberos5 --with-file-notification=inotify
--with-modules --enable-autodepend --prefix=/usr
--mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share
--localstatedir=/var --sharedstatedir=/var/lib
--libexecdir=/usr/libexec --with-file-notification=yes
--enable-locallisppath=/usr/share/emacs/29.0.50/site-lisp:/usr/share/emacs/site-lisp
--without-x --with-json --without-xim --with-sound --with-xpm
--with-jpeg --with-tiff --with-gif --with-png --with-rsvg --with-dbus
--without-xft --without-gpm --with-pgtk --without-native-compilation
--with-toolkit-scroll-bars --with-libotf --with-m17n-flt --with-cairo
--without-xwidgets --with-dumping=pdumper 'CFLAGS=-O2 -Wall
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong
-funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection
-Werror=return-type -flto=auto -D_GNU_SOURCE
-DGDK_DISABLE_DEPRECATION_WARNINGS -DGLIB_DISABLE_DEPRECATION_WARNINGS'
LDFLAGS=-flto=auto'
Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP XIM GTK3 ZLIB
Important settings:
value of $LANG: en_CA.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
Major mode: Eshell
Minor modes in effect:
shell-dirtrack-mode: t
eshell-prompt-mode: t
eshell-hist-mode: t
eshell-pred-mode: t
eshell-cmpl-mode: t
eshell-proc-mode: t
eshell-arg-mode: t
tooltip-mode: t
global-eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
line-number-mode: t
indent-tabs-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils pcmpl-unix
em-unix em-term term disp-table shell subr-x ehelp em-script em-prompt
em-ls em-hist em-pred em-glob em-extpipe em-cmpl em-dirs esh-var
pcomplete comint ansi-osc ansi-color ring em-basic em-banner em-alias
esh-mode eshell esh-cmd generator esh-ext esh-opt esh-proc esh-io
esh-arg esh-module esh-groups esh-util cus-edit pp cus-start cus-load
icons wid-edit cl-loaddefs cl-lib files-x rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/pgtk-win pgtk-win term/common-win pgtk-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
gtk pgtk lcms2 multi-tty make-network-process emacs)
Memory information:
((conses 16 77374 9505)
(symbols 48 8750 0)
(strings 32 24407 2625)
(string-bytes 1 719677)
(vectors 16 15061)
(vector-slots 8 203823 6855)
(floats 8 34 37)
(intervals 56 583 0)
(buffers 984 12))
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Wed, 21 Dec 2022 00:20:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 59545 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 11/24/2022 7:49 AM, Milan Zimmermann wrote:
> Emacs 29 Eshell Bug: Sourcing 'redirect-echo.esh' and redirecting output
> to a file, results in the first echo string ('hello') showing in eshell,
> only the second ('there')(presumably because it is last) showing in the
> output file.
It turns out there's an even simpler way to reproduce this:
~ $ {echo hi; echo bye} > #<buf>
hi ;; Buffer "buf" now contains the string "bye".
Initially[1], I said that this was an issue with the implementation of
'eshell-protect', but it turns out that it's actually an issue in an
adjacent part of the Eshell I/O code. Specifically, every statement in
Eshell gets its own set of default I/O handles, when it should actually
inherit the handles from its parent. So in the example above, "echo hi"
has the default I/O handles (pointing to the terminal), when its stdout
handle should point to the buffer "buf".
Attached is a patch series to fix this, with a bunch of new tests. I
also fixed a related issue where redirecting to /dev/null could clobber
your other redirects. (There's *also* an issue that should be fixed for
the release branch; I'll send that in a separate message.)
[1] https://lists.gnu.org/archive/html/emacs-devel/2022-11/msg01504.html
[0001-Add-eshell-duplicate-handles-to-return-a-copy-of-fil.patch (text/plain, attachment)]
[0002-Fix-handling-of-output-handles-in-nested-Eshell-form.patch (text/plain, attachment)]
[0003-Simplify-handling-of-dev-null-redirection-in-Eshell.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Wed, 21 Dec 2022 00:30:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 59545 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 12/20/2022 4:18 PM, Jim Porter wrote:
> Attached is a patch series to fix this, with a bunch of new tests. I
> also fixed a related issue where redirecting to /dev/null could clobber
> your other redirects. (There's *also* an issue that should be fixed for
> the release branch; I'll send that in a separate message.)
Eli, this is the patch for the release branch (it corresponds to part
0003 of the patch series for master). Is this ok to merge? It's a
regression that was introduced in Emacs 28.1, and the fix is pretty simple.
[0001-When-redirecting-to-the-null-device-in-Eshell-use-de.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Wed, 21 Dec 2022 09:55:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 59545 <at> debbugs.gnu.org (full text, mbox):
Jim Porter <jporterbugs <at> gmail.com> writes:
Hi Jim,
> Eli, this is the patch for the release branch (it corresponds to part
> 0003 of the patch series for master). Is this ok to merge? It's a
> regression that was introduced in Emacs 28.1, and the fix is pretty
> simple.
>
> From b04f42cca272b9a0f3b5e3167ce956523b161a7e Mon Sep 17 00:00:00 2001
> From: Jim Porter <jporterbugs <at> gmail.com>
> Date: Tue, 20 Dec 2022 16:20:50 -0800
> Subject: [PATCH] When redirecting to the null device in Eshell, use
> "/dev/null"
>
> This is so that users can type "cmd ... > /dev/null" in Eshell no
> matter what their system's null device is called. This partially
> reverts 67a8bdb90c9b5865b7f17290c7135b1a5458c36d.
And when they want to use another value? (null-device) returns
"/dev/null" for local default-directory's if you're not on MS
Windows. On MS Windows, it returns "NUL".
With a remote default-directory, the value is configurable (as
connection-local variable). Per default it is also "/dev/null", but it
could be changed.
Do you want to suppress this mechanism in Eshell? Why? I guess it is
more appropriate to install a handler for the actual value of
(null-device), instead just a handler for "/dev/null" only. And if you
want to make "/dev/null" a system-independant default, add the same
handler for this in parallel. Then both would be equivalent on MS Windows:
--8<---------------cut here---------------start------------->8---
cmd ... > /dev/null
cmd ... > NUL
--8<---------------cut here---------------end--------------->8---
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Wed, 21 Dec 2022 12:02:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 59545 <at> debbugs.gnu.org (full text, mbox):
> Date: Tue, 20 Dec 2022 16:29:31 -0800
> From: Jim Porter <jporterbugs <at> gmail.com>
> Cc: eliz <at> gnu.org
>
> On 12/20/2022 4:18 PM, Jim Porter wrote:
> > Attached is a patch series to fix this, with a bunch of new tests. I
> > also fixed a related issue where redirecting to /dev/null could clobber
> > your other redirects. (There's *also* an issue that should be fixed for
> > the release branch; I'll send that in a separate message.)
>
> Eli, this is the patch for the release branch (it corresponds to part
> 0003 of the patch series for master). Is this ok to merge? It's a
> regression that was introduced in Emacs 28.1, and the fix is pretty simple.
Yes, OK to install this on the release branch.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Wed, 21 Dec 2022 18:49:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 59545 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 12/21/2022 1:54 AM, Michael Albinus wrote:
> And when they want to use another value? (null-device) returns
> "/dev/null" for local default-directory's if you're not on MS
> Windows. On MS Windows, it returns "NUL".
Right. The intent is for the virtual name for the null device to be the
same in Eshell no matter what the system really calls it. On non-MS
systems, this shouldn't actually be necessary, since you could just
write to the *real* /dev/null. The virtual target in Eshell is just so
that /dev/null also works on MS Windows/DOS.
However, I would have thought that you could write to NUL on MS Windows
without any special handling. The Emacs manual has this to say:
"[On MS Windows,] referencing any file whose name matches a DOS
character device, such as NUL or LPT1 or PRN or CON, with or without any
file-name extension, will always resolve to those character devices, in
any directory. Therefore, only use such file names when you want to use
the corresponding character device."
I'd expect that to mean that if you opened a buffer and tried to save it
as "NUL", it would just work, but instead I get:
Write error: Bad file descriptor, c:/NUL
With that in mind, here are two patches (one for 29 and one for master)
to let Eshell handle both "/dev/null" and (on MS systems) "NUL". That
way, users get the best of both worlds.
[29--0001-When-redirecting-to-the-null-device-in-Eshell-allow-.patch (text/plain, attachment)]
[master--0003-Simplify-handling-of-dev-null-redirection-in-Eshell.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Wed, 21 Dec 2022 19:28:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 59545 <at> debbugs.gnu.org (full text, mbox):
> Date: Wed, 21 Dec 2022 10:48:18 -0800
> Cc: 59545 <at> debbugs.gnu.org, eliz <at> gnu.org,
> Milan Zimmermann <milan.zimmermann <at> gmail.com>
> From: Jim Porter <jporterbugs <at> gmail.com>
>
> However, I would have thought that you could write to NUL on MS Windows
> without any special handling. The Emacs manual has this to say:
>
> "[On MS Windows,] referencing any file whose name matches a DOS
> character device, such as NUL or LPT1 or PRN or CON, with or without any
> file-name extension, will always resolve to those character devices, in
> any directory. Therefore, only use such file names when you want to use
> the corresponding character device."
>
> I'd expect that to mean that if you opened a buffer and tried to save it
> as "NUL", it would just work, but instead I get:
>
> Write error: Bad file descriptor, c:/NUL
It's a bug, unrelated to the actual writing. I fixed it now on the
emacs-29 branch. You should be able now to save "NUL" on MS-Windows.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Thu, 22 Dec 2022 01:22:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 59545 <at> debbugs.gnu.org (full text, mbox):
On 12/21/2022 11:26 AM, Eli Zaretskii wrote:
> It's a bug, unrelated to the actual writing. I fixed it now on the
> emacs-29 branch. You should be able now to save "NUL" on MS-Windows.
Ah, good that we caught that bug then. With that change, my original
patches should be sufficient. I've merged my patch for the emacs-29
branch as d6c8d5dbc9fc4786e91b76654058e904c96f0e11 (with some additional
improvements to the comment/commit message).
I'll wait to merge my changes to master for a couple days to give others
time to comment though, since those changes are more extensive.
Reply sent
to
Jim Porter <jporterbugs <at> gmail.com>
:
You have taken responsibility.
(Thu, 22 Dec 2022 20:03:01 GMT)
Full text and
rfc822 format available.
Notification sent
to
Milan Zimmermann <milan.zimmermann <at> gmail.com>
:
bug acknowledged by developer.
(Thu, 22 Dec 2022 20:03:02 GMT)
Full text and
rfc822 format available.
Message #31 received at 59545-done <at> debbugs.gnu.org (full text, mbox):
On 12/21/2022 5:20 PM, Jim Porter wrote:
> I'll wait to merge my changes to master for a couple days to give others
> time to comment though, since those changes are more extensive.
Merged to master as 17bf6a829ca2fd2920c01e1aee30ab16b9c672eb. (I guess
that was only a day and not "a couple days", but I think it would be
good to get this on the master branch so that a wider audience can test
it to verify that everything is ok.)
Closing this now.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Sat, 24 Dec 2022 07:30:02 GMT)
Full text and
rfc822 format available.
Message #34 received at 59545 <at> debbugs.gnu.org (full text, mbox):
reopen 59545
thanks
On 12/22/2022 12:02 PM, Jim Porter wrote:
> Merged to master as 17bf6a829ca2fd2920c01e1aee30ab16b9c672eb. (I guess
> that was only a day and not "a couple days", but I think it would be
> good to get this on the master branch so that a wider audience can test
> it to verify that everything is ok.)
I found a problem with the patch on master:
~ $ {echo hello; echo world} | rev
olleh ;; "dlrow" is missing!
This happens because the way I'm copying output handles around results
in EOF being sent to "rev" after "echo hello". To be fair, this didn't
work correctly before either, but the problem was different. Prior to my
patch:
~ $ {echo hello; echo world} | rev
hello
dlrow
(Only "echo world" is actually piped through "rev".) I'll try to come up
with a fix in the next couple days.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Sun, 25 Dec 2022 01:38:02 GMT)
Full text and
rfc822 format available.
Message #37 received at 59545 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 12/23/2022 11:29 PM, Jim Porter wrote:
> I found a problem with the patch on master:
>
> ~ $ {echo hello; echo world} | rev
> olleh ;; "dlrow" is missing!
>
> This happens because the way I'm copying output handles around results
> in EOF being sent to "rev" after "echo hello".
Attached is a patch to fix this. I'm going to look into adding more test
cases if I can think of any before merging this.
I'll also see if I can fix the FIXME comment I added, but this is a part
of Eshell that's fairly brittle, and I think the *real* fix for that is
moving to running Eshell commands in a separate thread, as discussed on
emacs-devel. (I have a very WIP patch for this that already works
surprisingly well, but it's going to require a lot more work before it's
even worth making a feature branch.)
[0001-Fix-reference-counting-of-Eshell-I-O-handles.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Sun, 25 Dec 2022 21:50:01 GMT)
Full text and
rfc822 format available.
Message #40 received at 59545 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 12/24/2022 5:36 PM, Jim Porter wrote:
> I'll also see if I can fix the FIXME comment I added, but this is a part
> of Eshell that's fairly brittle, and I think the *real* fix for that is
> moving to running Eshell commands in a separate thread, as discussed on
> emacs-devel.
Ok, it turns out that the regression test that was failing
(eshell-tests/queue-input) wasn't testing the right thing, so I've fixed
that and also found a real bug in the queued-input code. The FIXME
comment is now resolved, although I admit I'm not 100% sure why it
helped improve things. I still don't entirely understand
'eshell-do-eval's inner workings...
I also added an assertion to make sure we're not trying to close I/O
handles more times than we should, which revealed another bug (this time
with my patch), so I've fixed that too.
I think this should resolve all the issues now, so unless anyone has
objections, I'll merge this to the master branch in a few days.
[0001-Fix-reference-counting-of-Eshell-I-O-handles.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Mon, 26 Dec 2022 19:51:01 GMT)
Full text and
rfc822 format available.
Message #43 received at 59545 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 12/25/2022 1:49 PM, Jim Porter wrote:
> I think this should resolve all the issues now, so unless anyone has
> objections, I'll merge this to the master branch in a few days.
Attached is the same patch as before, but I added a few new tests to
ensure that I/O handles are properly closed after using control flow
forms like 'while' or 'if' in Eshell.
[0001-Fix-reference-counting-of-Eshell-I-O-handles.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#59545
; Package
emacs
.
(Fri, 30 Dec 2022 06:41:02 GMT)
Full text and
rfc822 format available.
Message #46 received at 59545-done <at> debbugs.gnu.org (full text, mbox):
On 12/26/2022 11:50 AM, Jim Porter wrote:
> On 12/25/2022 1:49 PM, Jim Porter wrote:
>> I think this should resolve all the issues now, so unless anyone has
>> objections, I'll merge this to the master branch in a few days.
Merged as 073da412a139e317959f56e359ed12de726a0a35 to master. Closing
this again.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Fri, 27 Jan 2023 12:24:09 GMT)
Full text and
rfc822 format available.
This bug report was last modified 1 year and 87 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.