GNU bug report logs - #33942
ls directly uses filename as option parameter

Previous Next

Package: coreutils;

Reported by: westlake <westlake2012 <at> videotron.ca>

Date: Tue, 1 Jan 2019 08:08:01 UTC

Severity: normal

Tags: notabug

Merged with 33943

Done: Assaf Gordon <assafgordon <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 33942 in the body.
You can then email your comments to 33942 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-coreutils <at> gnu.org:
bug#33942; Package coreutils. (Tue, 01 Jan 2019 08:08:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to westlake <westlake2012 <at> videotron.ca>:
New bug report received and forwarded. Copy sent to bug-coreutils <at> gnu.org. (Tue, 01 Jan 2019 08:08:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: westlake <westlake2012 <at> videotron.ca>
To: bug-coreutils <at> gnu.org
Subject: ls directly uses filename as option parameter
Date: Mon, 31 Dec 2018 21:52:28 -0500
Hi coreutils team,

 I have known long time about certain commands that use "--" as a 
specially reserved parameter. However, I find the behaviour of it with 
ls showing a little confusing results and believe this surmounts to a bug,

by example, ls exhibits unpredictable behaviour when there's also a file 
called "--".

it took me a bit of time today to look into after I revisited the 
problems of having "-" at the beginning of filenames, but decided with a 
little extra time into the holidays to look at it.. and found out that 
having a file called "./--" creates unpredictable behaviour.. cut to the 
chase, I pretty much summarize that the filename "./--" is definitely 
being consumed and utilized as a "program parameter" which is not a very 
sane thing for a program to do (so this is why I am filing this as a bug 
report here first rather than going directly with developers responsible 
for getopt)

$ mkdir tmp
$ cd tmp   (so we start with an empty folder)

$touch 0 ./--a ./-a ./-_a ./--

$ ls -lad  -* [^-]*
-rw-r--r-- 1 user user 0 Dec 31 18:37 0
-rw-r--r-- 1 user user 0 Dec 31 18:33 -_a
-rw-r--r-- 1 user user 0 Dec 31 18:34 --a

ls -lad  -* [^-]*
-rw-r--r-- 1 user user 0 Dec 31 18:37 0
-rw-r--r-- 1 user user 0 Dec 31 18:33 -_a
-rw-r--r-- 1 user user 0 Dec 31 18:34 --a

$ ls -lad --  -* [^-]*
-rw-r--r-- 1 user user 0 Dec 31 19:10 --
-rw-r--r-- 1 user user 0 Dec 31 18:37 0
-rw-r--r-- 1 user user 0 Dec 31 18:33 -_a
-rw-r--r-- 1 user user 0 Dec 31 18:34 --a

$ ls -lad -- *
-rw-r--r-- 1 user user 0 Dec 31 18:35 --
-rw-r--r-- 1 user user 0 Dec 31 18:37 0
-rw-r--r-- 1 user user 0 Dec 31 18:33 -_a
-rw-r--r-- 1 user user 0 Dec 31 18:34 --a


Notice that example 2, "-*" is being parsed with "--a" ,   is "./--" not 
showing up due to "design" or is this a "bug" ?  If "--a" is getting 
listed, then so should the file called "./--",

"$ ls -lad -- *"

 this command shows all files.

Delete file ./-- with "rm ./--", and now we try the command "ls -lad *", 
the output now is ls complains.

Okay I understand ls complaining:
"
ls -lad *
ls: invalid option -- '_'
Try 'ls --help' for more information.
"

^ but it is counter-intuitive because having a file called "./--", and 
this command passes.

If there is detectable filenames as the next example would show, then 
why can't "-*" list file ./--  ? Is this a bug or a design to comply 
with similar behaviour of ls on *nix?


See this example,
"
ls -lAd -- -* [^-]* --
"

Here the first instance of "--" is taken as a special option for ls, and 
the second "--" is being taken as a filename literal.  There is really 
only one file called ./-- , but here as expected it is intentionally 
being listed twice.

So here ls works as expected, but why I am mentioning this..
"
$ ls -* [^-]* -- -lad
ls: cannot access '-lad': No such file or directory
--  0  -_a  --a
"

Here we can see that ls is performing filenaming parsing after it 
detected filenaming arguments after the "-*" ,  evidently it is also 
taking "-lad" as a filename literal, and this is understood.

So essentially here we have "--" being taken as a filename literal, and 
ls uses this correctly.


Question: so why can't ls expand "-*" to "./--"  ? (fwiw, it does when 
using "ls -lad -- -*, and when using "--" as a special parameter)

What about file "---" or file "----" ?

These files show up properly,

$ touch ./--- ./----

$ ls -lAd -* [^-]*
-rw-r--r-- 1 user user 0 Dec 31 21:10 ---
-rw-r--r-- 1 user user 0 Dec 31 21:10 ----
-rw-r--r-- 1 user user 0 Dec 31 21:02 0
-rw-r--r-- 1 user user 0 Dec 31 21:02 -a
-rw-r--r-- 1 user user 0 Dec 31 21:02 -_a
-rw-r--r-- 1 user user 0 Dec 31 21:02 --a

^  file "./--" is missing, why? (getopt?)

I'm curious as to why ls would take the file "./--" and then use that as 
a "special parameter" rather than a regular file, and then on top of 
this completely omit it from output ( ls -lad * )... It uses it as a 
special parameter because it then does not fail with that message " ls: 
invalid option -- '_' " ...

I think it is a bug, because if something is read from disk, it should 
not directly be treated as a special parameter to ls, .. so this is why 
I am filing this.

thanks








Information forwarded to bug-coreutils <at> gnu.org:
bug#33942; Package coreutils. (Wed, 02 Jan 2019 09:22:02 GMT) Full text and rfc822 format available.

Message #8 received at 33942 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: westlake <westlake2012 <at> videotron.ca>, 33942 <at> debbugs.gnu.org
Subject: Re: bug#33942: ls directly uses filename as option parameter
Date: Wed, 2 Jan 2019 02:21:22 -0700
tags 33942 notabug
close 33942
stop

Hello,

On 2018-12-31 7:52 p.m., westlake wrote:
>   I have known long time about certain commands that use "--" as a 
> specially reserved parameter. However, I find the behaviour of it with 
> ls showing a little confusing results and believe this surmounts to a bug,

What you describe below is not a bug, but the indented behavior.

>[...] and found out that 
> having a file called "./--" creates unpredictable behaviour [...]

First,
It's important to understand the filename is not "./--" (4 characters),
but "--" (2 characters) in the current directory ("./").

This is similar to typing "/home/user/--".
You wouldn't say the filename is "/home/user/--" - the filename is "--"
and it is in the directory "/home/user/".

While it might sound like splitting hairs, this is critical to
understand what's going on here. The filename starts with (and contains
only) two minus characters.


> $ touch 0 ./--a ./-a ./-_a ./--
> 
> $ ls -lad  -* [^-]*

Second,
It is important to understand that it is the shell which does
the filename completion (e.g. "*" and "-*" etc.).
The filename expiation happens before "ls" is even executed.

With this in mind, it'll be easier to understand what's
going on if you add "echo" to every command, and look
at what's printed:

  $ echo ls -lad  -* [^-]*
  ls -lad -- -a -_a --a 0

  $ echo ls -lad --  -* [^-]*
  ls -lad -- -- -a -_a --a 0

  $ echo ls -lad -- *
  ls -lad -- -- 0 -a -_a --a


> Notice that example 2, "-*" is being parsed with "--a" ,   is "./--" not 
> showing up due to "design" or is this a "bug" ?  If "--a" is getting 
> listed, then so should the file called "./--",

Not a bug.

Using "echo", you can see what are the exact parameters that are passed
to "ls".

And then follow these simple steps:
1. The first "--" tells ls to stop processing options.
2. anything that starts with a dash BEFORE the first "--" marker
is taken to be an option.
3. anything  AFTER the first "--" marker is taken to be a file name 
(strings starting with a single dash, and including additional "--" 
strings).

That's it - and it works consistently for most GNU programs, not just ls.

> Okay I understand ls complaining:
> "
> ls -lad *
> ls: invalid option -- '_'
> Try 'ls --help' for more information.

using "echo" will show what's happening:

  $ rm ./--
  $ echo ls -lad *
  ls -lad 0 -a -_a --a

With the file "--" gone (which was seen be "ls" as the "--" marker),
ls now sees "-a" as an option (a valid option),
then it sees "-_a" (and "_" is not a valid option) - hence the error 
message.

> ^ but it is counter-intuitive because having a file called "./--", and 
> this command passes.
> 

If you follow the simple steps above (and use "echo" to see how the
shell performs filename expansion), I hope all the other issues you've
listed in your email will become clear.

If not - please do write with specific cases.

I'm closing this as "not a bug", but discussion can continue
by replying to this thread.

-assaf





Added tag(s) notabug. Request was from Assaf Gordon <assafgordon <at> gmail.com> to control <at> debbugs.gnu.org. (Wed, 02 Jan 2019 09:22:02 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 33942 <at> debbugs.gnu.org and westlake <westlake2012 <at> videotron.ca> Request was from Assaf Gordon <assafgordon <at> gmail.com> to control <at> debbugs.gnu.org. (Wed, 02 Jan 2019 09:22:02 GMT) Full text and rfc822 format available.

Forcibly Merged 33942 33943. Request was from Glenn Morris <rgm <at> gnu.org> to control <at> debbugs.gnu.org. (Wed, 02 Jan 2019 18:10:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Thu, 31 Jan 2019 12:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 5 years and 59 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.