GNU bug report logs - #33113
ls: incorrect quoting of "="

Previous Next

To reply to this bug, email your comments to 33113 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vincent Lefevre <vincent <at> vinc17.net>
To: bug-coreutils <at> gnu.org
Subject: incorrect and inconsistent quoting in ls output
Date: Mon, 22 Oct 2018 01:34:40 +0200

I get the following with ls (GNU coreutils) 8.30.

zira% touch a=b a=b\&c
zira% ls a=b*
'a=b'  'a=b&c'
zira% ls -b a=b*
a=b  a=b&c
zira% ls -F a=b*
'a=b'  'a=b&c'
zira% ls -bF a=b*
a\=b  a\=b&c

AFAIK, the = character is not a shell metacharacter (except with zsh
but only in the first position), thus does not need to be quoted.

Moreover, while & is not a metacharacter, it is a special character
that should be quoted for practical reasons.

In particular, this inconsistency with -bF makes a\=b&c unusable by
copy-paste, as a\=b&c can't be used directly in a shell command, and
'a\=b&c' is not OK either.

Note: Such filenames with = and & can be produced by "wget -r".

-- 
Vincent Lefèvre <vincent <at> vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #8 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Vincent Lefevre <vincent <at> vinc17.net>
Cc: 33113 <at> debbugs.gnu.org
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Mon, 22 Oct 2018 01:21:40 -0700

Vincent Lefevre wrote:
> I get the following with ls (GNU coreutils) 8.30.
> 
> zira% touch a=b a=b\&c
> zira% ls a=b*
> 'a=b'  'a=b&c'
> zira% ls -b a=b*
> a=b  a=b&c
> zira% ls -F a=b*
> 'a=b'  'a=b&c'
> zira% ls -bF a=b*
> a\=b  a\=b&c
> 
> AFAIK, the = character is not a shell metacharacter (except with zsh
> but only in the first position), thus does not need to be quoted.

'=' is a shell metacharacter after 'set -k' in Bash.

> Moreover, while & is not a metacharacter, it is a special character
> that should be quoted for practical reasons.

As far as I can see '&' is being quoted correctly in the above examples. -b 
means to quote for C strings, not for the shell. If you want to quote for the 
shell, try --quoting-style='shell-escape'.

Message #11 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Vincent Lefevre <vincent <at> vinc17.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 33113 <at> debbugs.gnu.org
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Mon, 22 Oct 2018 10:44:57 +0200

On 2018-10-22 01:21:40 -0700, Paul Eggert wrote:
> Vincent Lefevre wrote:
> > I get the following with ls (GNU coreutils) 8.30.
> > 
> > zira% touch a=b a=b\&c
> > zira% ls a=b*
> > 'a=b'  'a=b&c'
> > zira% ls -b a=b*
> > a=b  a=b&c
> > zira% ls -F a=b*
> > 'a=b'  'a=b&c'
> > zira% ls -bF a=b*
> > a\=b  a\=b&c
> > 
> > AFAIK, the = character is not a shell metacharacter (except with zsh
> > but only in the first position), thus does not need to be quoted.

Actually it seems that the meaning of "metacharacter" is ambiguous.
In bash, & is regarded as a metacharacter:

  metacharacter
      A  character that, when unquoted, separates words.  One of the
      following:
      |  & ; ( ) < > space tab newline

> '=' is a shell metacharacter after 'set -k' in Bash.

It's a special character, but not a metacharacter (see above).
But then I agree on the need for quoting, if particular shells
are taking into account.

> > Moreover, while & is not a metacharacter, it is a special character
> > that should be quoted for practical reasons.
> 
> As far as I can see '&' is being quoted correctly in the above examples.

Not with "ls -bF": the quoting in a\=b&c is inconsistent.
Just like =, & should be quoted. Or = shouldn't be quoted
(see below).

> -b means to quote for C strings, not for the shell.

Hmm, yes... The issue here is that -b was actually changing the
quoting style, hence my confusion on this point. That's a bug in
the ls man page, which does not say so:

       -b, --escape
              print C-style escapes for nongraphic characters

The info manual is OK:

‘-b’
‘--escape’
‘--quoting-style=escape’
     Quote nongraphic characters in file names using alphabetic and
     octal backslash sequences like those used in C.

> If you want to quote for the shell, try
> --quoting-style='shell-escape'.

That's already the default. Actually, the confusion comes from the
ls man page.

On the behavior, there's still the issue concerning = and &.

-- 
Vincent Lefèvre <vincent <at> vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #14 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Vincent Lefevre <vincent <at> vinc17.net>
Cc: 33113 <at> debbugs.gnu.org
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Tue, 23 Oct 2018 17:54:24 -0700

On 10/22/18 1:44 AM, Vincent Lefevre wrote:
> On the behavior, there's still the issue concerning = and &.

Yes, you're right, there's no need to quote = with -b.

Message #17 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Vincent Lefevre <vincent <at> vinc17.net>
Cc: 33113 <at> debbugs.gnu.org
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Sat, 27 Oct 2018 07:28:26 -0700

On 23/10/18 17:54, Paul Eggert wrote:
> On 10/22/18 1:44 AM, Vincent Lefevre wrote:
>> On the behavior, there's still the issue concerning = and &.
> 
> Yes, you're right, there's no need to quote = with -b.

The reason \= is quoted with -bF is to distinguish socket
names that end in =. An edge case indeed, though the issue
is not particular to '=', but all of the classifier chars.

Now we could quote with -F only if these chars are at the end,
though I'm not sure that complexity is warranted.

cheers,
Pádraig

Message #20 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Vincent Lefevre <vincent <at> vinc17.net>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 33113 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Sun, 28 Oct 2018 14:21:02 +0100

On 2018-10-27 07:28:26 -0700, Pádraig Brady wrote:
> On 23/10/18 17:54, Paul Eggert wrote:
> > On 10/22/18 1:44 AM, Vincent Lefevre wrote:
> >> On the behavior, there's still the issue concerning = and &.
> > 
> > Yes, you're right, there's no need to quote = with -b.
> 
> The reason \= is quoted with -bF is to distinguish socket
> names that end in =. An edge case indeed, though the issue
> is not particular to '=', but all of the classifier chars.
> 
> Now we could quote with -F only if these chars are at the end,
> though I'm not sure that complexity is warranted.

In any case, this form of quoting incorrect with -b, as \= is invalid
in ISO C.

-- 
Vincent Lefèvre <vincent <at> vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #23 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Vincent Lefevre <vincent <at> vinc17.net>
Cc: 33113 <at> debbugs.gnu.org, Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Sun, 28 Oct 2018 13:11:33 -0700

Vincent Lefevre wrote:
>> Now we could quote with -F only if these chars are at the end,
>> though I'm not sure that complexity is warranted.
> In any case, this form of quoting incorrect with -b, as \= is invalid
> in ISO C.

That's right, we need another way to escape classifier characters with -bF, 
since the current method is clearly wrong.

Let's use ""= instead, as it's valid ISO C. In other words, where we currently 
do this:

$ python -c "import socket as s; sock = s.socket(s.AF_UNIX); sock.bind('b')"
$ touch b= b=x
$ ls -bF
b=  b\=  b\=x

the last command should output this instead:

b=  b""=  b=x

This works because in ISO C "b""=" is equivalent to "b=". We should do this only 
with characters at the end, because it's not needed elsewhere and the "" is 
annoying.

Message #26 received at 33113 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Vincent Lefevre <vincent <at> vinc17.net>
Cc: 33113 <at> debbugs.gnu.org, Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#33113: incorrect and inconsistent quoting in ls output
Date: Sun, 28 Oct 2018 21:48:08 -0600

Hello,

On 2018-10-28 2:11 p.m., Paul Eggert wrote:
> 
> That's right, we need another way to escape classifier characters with 
> -bF, since the current method is clearly wrong.
[...]
> This works because in ISO C "b""=" is equivalent to "b=". We should do 
> this only with characters at the end, because it's not needed elsewhere 
> and the "" is annoying.

Not sure if this is relevant,
but while going over old bugs I noticed this:

  Bug in 'ls -FQ': incorrectly quoted characters
  http://bugs.gnu.org/29832

Which reports incorrect quoting of "@" as "\@"
and also mentions "ls -b", and had a pending patch which was
never committed.

regards,
 -assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.