GNU bug report logs - #69937
29.1; Eshell ${} expansion includes stderr

Previous Next

To reply to this bug, email your comments to 69937 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.1; Eshell ${} expansion includes stderr
Date: Fri, 22 Mar 2024 00:20:14 -0400

Hi,

When performing expansion with ${} in Eshell, the expansion will include
both stdin and stderr.

--8<---------------cut here---------------start------------->8---
~ $ echo ${ruby -e 'puts 0' -e 'STDERR.puts "1"' -e 'puts 2'}
(0 1 2)
--8<---------------cut here---------------end--------------->8---

This behavior is undocumented, different from normal shells and can
introduce surprising behavior where the resulting output list can vary.
(Multithreaded programs are less likely to worry about the ordering of
stdout and stderr statements, stderr may not be printed every run,
different things may be printed to stderr, etc.)

If this behavior is desired, it should be explicitly documented in the
Eshell manual.

Personally I feel ${} expansion should just use stdout, not stderr.

A third alternative may be to introduce another $ expansion
that only uses stdout.

Confirmed with emacs -Q.


In GNU Emacs 29.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37,
cairo version 1.16.0)
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Guix System

Configured using:
 'configure
 CONFIG_SHELL=/gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16/bin/bash
 SHELL=/gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16/bin/bash
 --prefix=/gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1
 --enable-fast-install --with-cairo --with-modules
 --with-native-compilation=aot --disable-build-details'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2 XPM
GTK3 ZLIB

Important settings:
  value of $EMACSLOADPATH: /home/richard/.guix-home/profile/share/emacs/site-lisp:/gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1/share/emacs/29.1/lisp
  value of $EMACSNATIVELOADPATH: /home/richard/.guix-home/profile/lib/emacs/native-site-lisp
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8-unix

Major mode: Message

Minor modes in effect:
  hexl-follow-ascii: t
  eshell-syntax-highlighting-global-mode: t
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-services-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  erc-networks-mode: t
  beacon-mode: t
  all-the-icons-completion-mode: t
  which-key-mode: t
  display-time-mode: t
  marginalia-mode: t
  savehist-mode: t
  vertico-mode: t
  global-git-commit-mode: t
  mml-mode: t
  magit-auto-revert-mode: t
  eat-eshell-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: message-do-auto-fill
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  abbrev-mode: t

Load-path shadows:
/gnu/store/x3hyyjg169fqvy0pw2lcvfiswvi607k2-emacs-transient-0.5.3/share/emacs/site-lisp/transient-0.5.3/transient hides /gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1/share/emacs/29.1/lisp/transient
/gnu/store/6pzp2k94wb6ck9ra4p45v45fx1kbyn1y-emacs-project-0.10.0/share/emacs/site-lisp/project-0.10.0/project hides /gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1/share/emacs/29.1/lisp/progmodes/project
/gnu/store/rqnkbk05rfwsy2h5cd1cif30sdr5bcr8-emacs-xref-1.6.3/share/emacs/site-lisp/xref-1.6.3/xref hides /gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1/share/emacs/29.1/lisp/progmodes/xref
/gnu/store/78zv37kfy77gdj0jswhs81wg9s10s29l-emacs-soap-client-3.2.3/share/emacs/site-lisp/soap-client-3.2.3/soap-client hides /gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1/share/emacs/29.1/lisp/net/soap-client
/gnu/store/78zv37kfy77gdj0jswhs81wg9s10s29l-emacs-soap-client-3.2.3/share/emacs/site-lisp/soap-client-3.2.3/soap-inspect hides /gnu/store/ni49ash6niyqny9h8wmp71z8hmpmx80k-emacs-29.1/share/emacs/29.1/lisp/net/soap-inspect

Features:
(shadow emacsbug ibuf-ext ibuffer ibuffer-loaddefs debug backtrace
shortdoc ruler-mode hexl pcmpl-unix align tramp-cmds tramp cl-print
em-rebind em-smart tramp-loaddefs trampver tramp-integration
tramp-compat latexenc two-column macrostep-c cmacexp macrostep
smartparens-c cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs texinfo texinfo-loaddefs pulse
consult-xref grep mm-archive sort smiley gnus-cite mail-extr textsec
uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check qp
gnus-async gnus-bcklg gnus-ml nndraft nnmh utf-7 nnfolder nnnil
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg nntp
gnus-cache goto-addr vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs
log-view vc bug-reference make-mode eshell-syntax-highlighting em-unix
em-term em-script em-prompt em-ls em-hist em-pred em-glob em-extpipe
em-cmpl em-dirs em-basic em-banner em-alias eshell dired-aux sh-script
smie treesit executable misearch multi-isearch dabbrev ace-window avy
geiser-mode geiser-xref geiser-compile vc-git vc-dispatcher mule-util
embark-consult consult magit-bookmark bookmark network-stream epa-file
erc-list erc-menu erc-join erc-ring erc-pcomplete erc-track erc-match
erc-button erc-fill erc-stamp erc-netsplit erc-services erc-goodies erc
erc-backend erc-networks erc-common erc-compat erc-loaddefs embark-org
embark ffap diary-lib diary-loaddefs cal-iso oc-basic ol-eww eww
url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum shr
pixel-fill kinsoku url-file svg dom gnus-group gnus-undo gnus-start
gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo parse-time
gnus-spec gnus-int gnus-range gnus-win ol-docview doc-view jka-compr
image-mode exif ol-bibtex bibtex iso8601 ol-bbdb ol-w3m ol-doi
org-link-doi face-remap org-agenda org-element org-persist xdg org-id
avl-tree org-refile smartparens-org ob-plantuml org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-src ob-comint org-pcomplete
org-list org-footnote org-faces org-entities ob-emacs-lisp ob-core
ob-eval org-cycle org-table org-keys oc org-loaddefs find-func cal-menu
calendar cal-loaddefs ol org-fold org-fold-core org-compat org-version
org-macs smartparens-config smartparens-text smartparens advice
rainbow-delimiters hl-line rs-ui beacon all-the-icons-completion
all-the-icons all-the-icons-faces data-material data-weathericons
data-octicons data-fileicons data-faicons data-alltheicons which-key
diminish doom-dracula-theme doom-themes doom-themes-base moody time
rs-tools rs-smtp smtpmail rs-skeleton skeleton rs-project rs-navigation
marginalia orderless savehist vertico rs-media rs-magit magit-extras
magit-submodule magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
package url-handlers magit-repos magit-apply magit-wip magit-log
which-func imenu magit-diff smerge-mode diff diff-mode git-commit
log-edit message sendmail yank-media rfc822 mml mml-sec epa derived
mailabbrev gmm-utils mailheader pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
magit-mode magit-git magit-base magit-section cursor-sensor crm
rs-integrations debbugs soap-client mm-decode mm-bodies mm-encode
url-http url-auth mail-parse rfc2231 rfc2047 rfc2045 ietf-drums url-gw
nsm puny rng-xsd rng-dt rng-util xsd-regexp rs-ibuffer rs-eshell esh-var
esh-mode esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util files-x eat term disp-table ehelp shell pcomplete
color rs-erc rs-epg epg rfc6068 epg-config rs-elfeed rs-editing rs-dired
ls-lisp dired dired-loaddefs rs-core server autorevert filenotify delsel
comp comp-cstr warnings rx exec-path-from-shell rs-auth-source
auth-source-pass rs-scheme geiser-guile info-look info transient
format-spec geiser geiser-debug geiser-repl geiser-image geiser-capf
geiser-doc geiser-menu geiser-autodoc geiser-edit geiser-completion
geiser-eval geiser-connection tq geiser-syntax scheme geiser-impl
help-fns radix-tree geiser-log geiser-popup view geiser-custom
geiser-base rs-rust rs-password-store rs-org plantuml-mode dash edmacro
kmacro use-package-bind-key bind-key rs-docker rs-csharp rs-cl slime
easy-mmode apropos compile etags fileloop generator xref project
arc-mode archive-mode noutline outline icons pp comint ansi-osc
ansi-color ring hyperspec thingatpt browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie generate-lisp-file
url-domsuf url-util url-parse auth-source eieio eieio-core
password-cache json map byte-opt url-vars mailcap rs-nnrss rs-gnus gnus
nnheader gnus-util text-property-search time-date mail-utils range
mm-util mail-prsvr wid-edit cus-load rs-utils rs-constants battery dbus
xml subr-x no-littering compat cl-macs gv cl-extra help-mode cl-seq
use-package-core cl-loaddefs cl-lib bytecomp byte-compile
lorem-ipsum-autoloads geiser-autoloads geiser-guile-autoloads
xterm-color-autoloads rust-mode-autoloads xref-autoloads
project-autoloads spinner-autoloads hydra-autoloads ht-autoloads
lsp-mode-autoloads flycheck-autoloads rustic-autoloads
password-store-autoloads pass-autoloads plantuml-mode-autoloads
yaml-mode-autoloads docker-compose-mode-autoloads
dockerfile-mode-autoloads web-mode-autoloads macrostep-autoloads
slime-autoloads which-key-autoloads moody-autoloads
doom-themes-autoloads shrink-path-autoloads nerd-icons-autoloads
doom-modeline-autoloads dirvish-autoloads diminish-autoloads
beacon-autoloads all-the-icons-completion-autoloads memoize-autoloads
all-the-icons-autoloads embark-autoloads consult-autoloads
marginalia-autoloads orderless-autoloads vertico-autoloads
popup-autoloads s-autoloads f-autoloads dumb-jump-autoloads
avy-autoloads ace-window-autoloads tablist-autoloads pdf-tools-autoloads
async-autoloads with-editor-autoloads transient-autoloads
magit-autoloads soap-client-autoloads debbugs-autoloads
eshell-syntax-highlighting-autoloads eat-autoloads elfeed-autoloads
rainbow-delimiters-autoloads markdown-mode-autoloads dash-autoloads
smartparens-autoloads exec-path-from-shell-autoloads compat-autoloads
no-littering-autoloads guix-emacs rmc iso-transl tooltip cconv eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 3008426 209077)
 (symbols 48 52597 24)
 (strings 32 262936 38973)
 (string-bytes 1 9305503)
 (vectors 16 149078)
 (vector-slots 8 3199779 256497)
 (floats 8 2383 4879)
 (intervals 56 233797 8887)
 (buffers 984 124))

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Message #8 received at 69937 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Richard Sent <richard <at> freakingpenguin.com>, 69937 <at> debbugs.gnu.org
Subject: Re: bug#69937: 29.1; Eshell ${} expansion includes stderr
Date: Thu, 21 Mar 2024 22:56:48 -0700

On 3/21/2024 9:20 PM, Richard Sent wrote:
> When performing expansion with ${} in Eshell, the expansion will include
> both stdin and stderr.
> 
> --8<---------------cut here---------------start------------->8---
> ~ $ echo ${ruby -e 'puts 0' -e 'STDERR.puts "1"' -e 'puts 2'}
> (0 1 2)
> --8<---------------cut here---------------end--------------->8---

Thanks for noticing this. I think this is simply a bug (although it may 
originally have been intentional due to Eshell's more-limited 
redirection operators in Emacs 28 and earlier). It would probably be 
good to follow other shells here, and people who *want* to capture both 
stdout and stderr can use 2>&1 as usual.

I'll try and put together a patch for this over the weekend.

Message #11 received at 69937 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Richard Sent <richard <at> freakingpenguin.com>, 69937 <at> debbugs.gnu.org
Subject: Re: bug#69937: 29.1; Eshell ${} expansion includes stderr
Date: Sat, 23 Mar 2024 12:31:37 -0700

On 3/21/2024 10:56 PM, Jim Porter wrote:
> I'll try and put together a patch for this over the weekend.

I looked into this some more and found that it's not quite so easy to do 
this in a way that preserves functionality. Normally, you should be able 
to use the "2>&1" syntax to redirect (and thus capture) *all* the 
output. However, that fails for the $<command> form, since it will 
mis-parse this:

  $<command 2>&1>

Escaping the inner ">" won't work, since the parser doesn't unescape 
that character. Unescaping that would be tricky to get anyway, so I'm 
not sure it's the best route. Another hypothetical syntax would be 
(whitespace just for readability):

  $< { command 2>&1 } >

That *also* doesn't work today, but reworking 'eshell-find-delimiter' 
should make that possible, and likely fix some other surprising cases in 
the Eshell syntax. I have some ideas for how to do this, but it's a 
pretty invasive change, so it'll take quite a bit of thought. I don't 
have time to do this right now, but I'll take a look in the coming month 
or so.

Thankfully, aside from this wrinkle, all the *other* parts to fix this 
bug are pretty straightforward, and in fact I already have a WIP patch 
for them.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.