GNU bug report logs - #59612
29.0.50; Eshell: The behavior of conditionals depends on whitespace

Previous Next

To reply to this bug, email your comments to 59612 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Milan Zimmermann <milan.zimmermann <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; Eshell: The behavior of conditionals depends on whitespace
Date: Sat, 26 Nov 2022 10:52:34 -0500

[Message part 1 (text/plain, inline)]

Create a script from this block
################################################################
# The behavior of conditionals depends on whitespace
# Source this script to test it

# This formatting works as expected; evaluates the "true" block.
# Result:
#  "It is NOT 2"
#

if { = 2 2 } {
  echo "It is 2"
} {
   echo "It is NOT 2"
}

# This formatting causes a BUG: both the "true" block and the "false" block
are evaluated.
# I tried several combinations, it appears it the "false" block starts on
it's own line,
# it is no longer treated a part of the "if" expression. Which could be
argued
# is expected because there is no "else" that would define whether the
second block
# is not part of the "if" expression. BUT see next test
#
# Result:
#  "It is 3"
#  "It is NOT 3"
#

if { = 3 3 } {
   echo "It is 3"
}
{
   echo "It is NOT 3"
}

# BUT we get the same incorrect result if we place the whole if expression
into {}
{
  if { = 4 4 } {
     echo "It is 4"
  }
  {
     echo "It is NOT 4"
  }
}
######################################################################

Actual result:
~/tmp $ source ./conditionals-bug.esh
It is 2
It is 3
It is NOT 3
It is 4
It is NOT 4

Expected result (although 3 may be questionable with eshell syntax):
~/tmp $ source ./conditionals-bug.esh
It is 2
It is 3
It is 4




In GNU Emacs 29.0.50 (build 1, x86_64-suse-linux-gnu, GTK+ Version
3.24.34, cairo version 1.17.6)
System Description: openSUSE Tumbleweed

Configured using:
 'configure --host=x86_64-suse-linux-gnu --build=x86_64-suse-linux-gnu
 --program-prefix= --disable-dependency-tracking --prefix=/usr
 --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
 --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include
 --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --disable-build-details --without-pop
 --with-mailutils --without-hesiod --with-gameuser=:games
 --with-kerberos --with-kerberos5 --with-file-notification=inotify
 --with-modules --enable-autodepend --prefix=/usr
 --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share
 --localstatedir=/var --sharedstatedir=/var/lib
 --libexecdir=/usr/libexec --with-file-notification=yes
 --enable-locallisppath=/usr/share/emacs/29.0.50/site-lisp:/usr/share/emacs/site-lisp
 --without-x --with-json --without-xim --with-sound --with-xpm
 --with-jpeg --with-tiff --with-gif --with-png --with-rsvg --with-dbus
 --without-xft --without-gpm --with-pgtk --without-native-compilation
 --with-toolkit-scroll-bars --with-libotf --with-m17n-flt --with-cairo
 --without-xwidgets --with-dumping=pdumper 'CFLAGS=-O2 -Wall
 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong
 -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection
 -Werror=return-type -flto=auto -D_GNU_SOURCE
 -DGDK_DISABLE_DEPRECATION_WARNINGS -DGLIB_DISABLE_DEPRECATION_WARNINGS'
 LDFLAGS=-flto=auto'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_CA.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Eshell

Minor modes in effect:
  shell-dirtrack-mode: t
  eshell-prompt-mode: t
  eshell-hist-mode: t
  eshell-pred-mode: t
  eshell-cmpl-mode: t
  eshell-proc-mode: t
  eshell-arg-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date rx em-unix
em-term term disp-table shell subr-x ehelp em-script em-prompt em-ls
em-hist em-pred em-glob em-extpipe em-cmpl em-dirs esh-var pcomplete
comint ansi-osc ansi-color ring em-basic em-banner em-alias esh-mode
eshell esh-cmd generator esh-ext esh-opt esh-proc esh-io esh-arg
esh-module esh-groups esh-util cus-edit pp cus-start cus-load icons
wid-edit cl-loaddefs cl-lib files-x rmc iso-transl tooltip cconv eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 79793 9821)
 (symbols 48 8916 0)
 (strings 32 24608 1742)
 (string-bytes 1 726991)
 (vectors 16 15021)
 (vector-slots 8 205006 15892)
 (floats 8 51 33)
 (intervals 56 542 0)
 (buffers 984 13))

[Message part 2 (text/html, inline)]

Message #8 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Milan Zimmermann <milan.zimmermann <at> gmail.com>
To: 59612 <at> debbugs.gnu.org
Subject: A comment line in example change
Date: Sat, 26 Nov 2022 10:59:13 -0500

[Message part 1 (text/plain, inline)]

The script I provided has a mis-typed comment on line 6. The script should
be:

################################################################
# The behavior of conditionals depends on whitespace
# Source this script to test it

# This formatting works as expected; evaluates the "true" block.
# Result:
#  "It is 2" # <=== The original had a typo here, included NOT
#

if { = 2 2 } {
  echo "It is 2"
} {
   echo "It is NOT 2"
}

# This formatting causes a BUG: both the "true" block and the "false" block
are evaluated.
# I tried several combinations, it appears it the "false" block starts on
it's own line,
# it is no longer treated a part of the "if" expression. Which could be
argued
# is expected because there is no "else" that would define whether the
second block
# is not part of the "if" expression. BUT see next test
#
# Result:
#  "It is 3"
#  "It is NOT 3"
#

if { = 3 3 } {
   echo "It is 3"
}
{
   echo "It is NOT 3"
}

# BUT we get the same incorrect result if we place the whole if expression
into {}
{
  if { = 4 4 } {
     echo "It is 4"
  }
  {
     echo "It is NOT 4"
  }
}
######################################################################

[Message part 2 (text/html, inline)]

Message #11 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Milan Zimmermann <milan.zimmermann <at> gmail.com>
To: 59612 <at> debbugs.gnu.org
Subject: Incorrect behavior also with ${CONDITION}
Date: Sat, 26 Nov 2022 11:19:08 -0500

[Message part 1 (text/plain, inline)]

I should add that I realized using {CONDITION} alone is not a reasonable
thing in my examples, as it uses exist status.

In a real code, one would use  ${CONDITION}. The modified script is
attached below #####. However, the behavior is still not as expected:

~/tmp $ source ./conditionals-bug.esh
It is 2
It is 3
It is NOT 3
It is 4
It is NOT 4


######################################
# The behavior of conditionals depends on whitespace
# Source this script to test it

# This formatting works as expected; evaluates the "true" block.
# Result:
#  "It is 2"
#

if ${= 2 2} {
  echo "It is 2"
} {
   echo "It is NOT 2"
}

# This formatting causes a BUG: both the "true" block and the "false" block
are evaluated.
# I tried several combinations, it appears it the "false" block starts on
it's own line,
# it is no longer treated a part of the "if" expression. Which could be
argued
# is expected because there is no "else" that would define whether the
second block
# is not part of the "if" expression. BUT see next test
#
# Result:
#  "It is 3"
#  "It is NOT 3"
#

if ${ = 3 3 } {
   echo "It is 3"
}
{
   echo "It is NOT 3"
}

# BUT we get the same incorrect result if we place the whole if expression
into {}
{
  if ${ = 4 4 } {
     echo "It is 4"
  }
  {
     echo "It is NOT 4"
  }
}

[Message part 2 (text/html, inline)]

Message #14 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Milan Zimmermann <milan.zimmermann <at> gmail.com>, 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: Incorrect behavior also with ${CONDITION}
Date: Sat, 26 Nov 2022 10:28:20 -0800

On 11/26/2022 8:19 AM, Milan Zimmermann wrote:
> I should add that I realized using {CONDITION} alone is not a reasonable 
> thing in my examples, as it uses exist status.
> 
> In a real code, one would use  ${CONDITION}. The modified script is 
> attached below #####. However, the behavior is still not as expected:

In this case, they should actually be the same thing. When the command 
in question is a Lisp function, a nil result sets the exit status to 2. 
This doesn't hold for external commands though; the output and the exit 
status are independent of one another. In that case, you'd want to spell 
it like so:

  if {[ 2 -eq 2 ]} { echo "good" }

So in conclusion, I think it's better to prefer {CONDITION} for 'if' forms.

Message #17 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Milan Zimmermann <milan.zimmermann <at> gmail.com>, 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: 29.0.50; Eshell: The behavior of conditionals depends
 on whitespace
Date: Sat, 26 Nov 2022 10:42:42 -0800

On 11/26/2022 7:52 AM, Milan Zimmermann wrote:
> # Result:
> #  "It is 3"
> #  "It is NOT 3"
> #
> 
> if { = 3 3 } {
>     echo "It is 3"
> }
> {
>     echo "It is NOT 3"
> }

According to Eshell's logic, I think this is correct (though 
inconvenient). Because Eshell treats a newline as the end of a command 
whenever possible, it just sees these as two separate commands.

> # BUT we get the same incorrect result if we place the whole if 
> expression into {}
> {
>    if { = 4 4 } {
>       echo "It is 4"
>    }
>    {
>       echo "It is NOT 4"
>    }
> }

This is really the same as the above: {...} allows multiple commands, so 
it sees this as two separate commands nested inside the {}.

Ultimately, I think this is closer to a feature request: adding an 
"else" token would disambiguate this:

  if { = 2 2 } {
    echo "good"
  }
  else {
    echo "bad"
  }

Actually making this work in Eshell's internals might be painful though...

I do also see a potential bug. I'd expect this to work, but it doesn't:

  if { = 2 2 } \
  { echo "good" } \
  { echo "bad" }

Message #20 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Milan Zimmermann <milan.zimmermann <at> gmail.com>, 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: Incorrect behavior also with ${CONDITION}
Date: Sat, 26 Nov 2022 11:32:10 -0800

On 11/26/2022 10:28 AM, Jim Porter wrote:
> On 11/26/2022 8:19 AM, Milan Zimmermann wrote:
>> I should add that I realized using {CONDITION} alone is not a 
>> reasonable thing in my examples, as it uses exist status.
>>
>> In a real code, one would use  ${CONDITION}. The modified script is 
>> attached below #####. However, the behavior is still not as expected:
> 
> In this case, they should actually be the same thing. When the command 
> in question is a Lisp function, a nil result sets the exit status to 2. 
> This doesn't hold for external commands though; the output and the exit 
> status are independent of one another. In that case, you'd want to spell 
> it like so:
> 
>    if {[ 2 -eq 2 ]} { echo "good" }
> 
> So in conclusion, I think it's better to prefer {CONDITION} for 'if' forms.

Whoops, disregard this. For Lisp functions, I'm thinking of (CONDITION), 
not {CONDITION}.

Message #23 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Milan Zimmermann <milan.zimmermann <at> gmail.com>, 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: 29.0.50; Eshell: The behavior of conditionals depends
 on whitespace
Date: Sat, 26 Nov 2022 16:39:56 -0800

On 11/26/2022 10:42 AM, Jim Porter wrote:
> I do also see a potential bug. I'd expect this to work, but it doesn't:
> 
>    if { = 2 2 } \
>    { echo "good" } \
>    { echo "bad" }

Worse than just a bug, this (well, a similar example) is actually a 
regression from Emacs 28. I've filed bug#59622 for it.

Message #26 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Milan Zimmermann <milan.zimmermann <at> gmail.com>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: 29.0.50; Eshell: The behavior of conditionals depends
 on whitespace
Date: Sat, 26 Nov 2022 21:16:42 -0500

[Message part 1 (text/plain, inline)]

Jim, thanks for the follow-up. Please feel free to close this.  A few
comments inline though:


On Sat, Nov 26, 2022 at 1:42 PM Jim Porter <jporterbugs <at> gmail.com> wrote:

> On 11/26/2022 7:52 AM, Milan Zimmermann wrote:
> > # Result:
> > #  "It is 3"
> > #  "It is NOT 3"
> > #
> >
> > if { = 3 3 } {
> >     echo "It is 3"
> > }
> > {
> >     echo "It is NOT 3"
> > }
>
> According to Eshell's logic, I think this is correct (though
> inconvenient). Because Eshell treats a newline as the end of a command
> whenever possible, it just sees these as two separate commands.
>

Yes, I agree. From the way of thinking "whitespace should not matter" it is
a surprising behavior though.

>
> > # BUT we get the same incorrect result if we place the whole if
> > expression into {}
> > {
> >    if { = 4 4 } {
> >       echo "It is 4"
> >    }
> >    {
> >       echo "It is NOT 4"
> >    }
> > }
>
> This is really the same as the above: {...} allows multiple commands, so
> it sees this as two separate commands nested inside the {}.
>

Yeah, I realized the same during my hike today. The wrapping {..} should
not make a difference, asking for that would make things worse.


>
> Ultimately, I think this is closer to a feature request: adding an
> "else" token would disambiguate this:
>
>    if { = 2 2 } {
>      echo "good"
>    }
>    else {
>      echo "bad"
>    }
>
> Actually making this work in Eshell's internals might be painful though...
>

Yes. Well, personally I feel the current behavior means that "lisp-iness"
seeps through too much into the scripting behavior. So a feature request
for adding and "else" would work for me.  I will abstain from doing that at
the moment, as I feel asking something like that requires a deeper review
of how close eshell should behave like lisp syntax-wise. But if someone
files such a request, I will not complain :)

BTW, a slightly related question if I may: A further diversion of
lisp-iness, I do not suppose there is a way to do a "return"? In bash, the
ability to "return" from sourced bash scripts or functions allows us to
deal with errors at the beginning, then process the main logic. In Eshell ,
I am doing things like:

=============
if ${ not { = {length $*} 3 } } {
   echo "Invalid arguments: $*"
   echo "Usage: $0  file-to-generate.dart  snippet.dart  template.dart"
} {
   if ${ not { file-exists-p $2 || file-exists-p $3 } } {
      echo "File $2 or $3 does not exist."
   } {
      ##### The main logic here is indented 2 levels deep
      export file-to-generate=$1
      export snippet="${cat $2}"
      export template="${cat $3}"

      echo $file-to-generate
      echo $snippet
      echo $template

      # todo : check if template contains string %s
      # format string template, substitute snippet, place to generated-code
      # echo generated-code to file-to-generate
   }
}
=============

>
> I do also see a potential bug. I'd expect this to work, but it doesn't:
>
>    if { = 2 2 } \
>    { echo "good" } \
>    { echo "bad" }
>

Yeah, you are right. I just tried that.

Thanks,
Milan

[Message part 2 (text/html, inline)]

Message #29 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Milan Zimmermann <milan.zimmermann <at> gmail.com>
Cc: 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: 29.0.50; Eshell: The behavior of conditionals depends
 on whitespace
Date: Sat, 26 Nov 2022 19:13:31 -0800

On 11/26/2022 6:16 PM, Milan Zimmermann wrote:
> Jim, thanks for the follow-up. Please feel free to close this.

I think it would be reasonable to leave this open to track adding 
support for some kind of "command re-joining" logic that things like 
if/else forms could use. Then something like this would just work:

  if { condition }
  { true-case }
  else
  { false-case }

At a high level, I'm thinking something like this:

1. Enhance 'eshell-rewrite-if-command' to support "if"/"else if"/"else" 
forms.

2. Add some top-level command-rewriting logic that lets you join 
multiple separate commands back into one. I think Eshell splits the 
commands up line-by-line pretty early in the process, so re-joining them 
later might be the least-invasive way to do this. It'll take some 
further diagnosis though.

> Yes, I agree. From the way of thinking "whitespace should not matter" it 
> is a surprising behavior though.

Yeah, it's a strange result, and possibly a sign that the syntax for 
Eshell conditionals wasn't the ideal way to do things. But it is what it 
is now, and hopefully there are ways to make it less surprising without 
making a major incompatible change to syntax.

> BTW, a slightly related question if I may: A further diversion of 
> lisp-iness, I do not suppose there is a way to do a "return"? In bash, 
> the ability to "return" from sourced bash scripts or functions allows us 
> to deal with errors at the beginning, then process the main logic.

I think this is related to a TODO in the Eshell manual to add a 
Bash-like "function" command, which would let you write whole functions 
in Eshell command form. I've also thought about the idea of adding 
syntax in Eshell so you can write stuff in Lisp forms but then go back 
out to writing command forms. Something like:

  (defun some-function ()
    (do-stuff)
    ($ "echo $foobar") ;; Invoke an Eshell command.
    )

That might be tricky to get all the plumbing working though.

Message #32 received at 59612 <at> debbugs.gnu.org (full text, mbox):

From: Milan Zimmermann <milan.zimmermann <at> gmail.com>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: 59612 <at> debbugs.gnu.org
Subject: Re: bug#59612: 29.0.50; Eshell: The behavior of conditionals depends
 on whitespace
Date: Sun, 27 Nov 2022 00:07:40 -0500

[Message part 1 (text/plain, inline)]

These are all great ideas (implementing else on if as a kind of joining
commands, adding eshell-functions, adding ability to invoke Eshell
commands/functions from the elisp blocks).

Thanks for sharing the details. I would love to offer help to dig into some
of it, but I need to restrain myself from overpromising.

In the meantime, I will continue to report bugs or things that do not feel
right, hope that is ok. (I decided this third time to use eshell as my main
shell at least for personal projects will succeed at least in a limited
way.)



On Sat, Nov 26, 2022 at 10:13 PM Jim Porter <jporterbugs <at> gmail.com> wrote:

> On 11/26/2022 6:16 PM, Milan Zimmermann wrote:
> > Jim, thanks for the follow-up. Please feel free to close this.
>
> I think it would be reasonable to leave this open to track adding
> support for some kind of "command re-joining" logic that things like
> if/else forms could use. Then something like this would just work:
>
>    if { condition }
>    { true-case }
>    else
>    { false-case }
>
> At a high level, I'm thinking something like this:
>
> 1. Enhance 'eshell-rewrite-if-command' to support "if"/"else if"/"else"
> forms.
>
> 2. Add some top-level command-rewriting logic that lets you join
> multiple separate commands back into one. I think Eshell splits the
> commands up line-by-line pretty early in the process, so re-joining them
> later might be the least-invasive way to do this. It'll take some
> further diagnosis though.
>
> > Yes, I agree. From the way of thinking "whitespace should not matter" it
> > is a surprising behavior though.
>
> Yeah, it's a strange result, and possibly a sign that the syntax for
> Eshell conditionals wasn't the ideal way to do things. But it is what it
> is now, and hopefully there are ways to make it less surprising without
> making a major incompatible change to syntax.
>
> > BTW, a slightly related question if I may: A further diversion of
> > lisp-iness, I do not suppose there is a way to do a "return"? In bash,
> > the ability to "return" from sourced bash scripts or functions allows us
> > to deal with errors at the beginning, then process the main logic.
>
> I think this is related to a TODO in the Eshell manual to add a
> Bash-like "function" command, which would let you write whole functions
> in Eshell command form. I've also thought about the idea of adding
> syntax in Eshell so you can write stuff in Lisp forms but then go back
> out to writing command forms. Something like:
>
>    (defun some-function ()
>      (do-stuff)
>      ($ "echo $foobar") ;; Invoke an Eshell command.
>      )
>
> That might be tricky to get all the plumbing working though.
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.