GNU bug report logs - #36416
realpath doesn't preserve '//' in -m mode

Previous Next

Package: coreutils;

Reported by: L A Walsh <coreutils <at> tlinx.org>

Date: Fri, 28 Jun 2019 12:54:02 UTC

Severity: normal

Tags: notabug

Done: Eric Blake <eblake <at> redhat.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36416 in the body.
You can then email your comments to 36416 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-coreutils <at> gnu.org:
bug#36416; Package coreutils. (Fri, 28 Jun 2019 12:54:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to L A Walsh <coreutils <at> tlinx.org>:
New bug report received and forwarded. Copy sent to bug-coreutils <at> gnu.org. (Fri, 28 Jun 2019 12:54:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Coreutils <bug-coreutils <at> gnu.org>
Subject: realpath doesn't preserve '//' in -m mode
Date: Fri, 28 Jun 2019 04:11:29 -0700
realpath -m //sysname/rootdir/.
returns "/sysname/rootdir"  This is incorrect, according to POSIX.

and #3-4 have specific meanings in various applications that  in usage.

1) leading double slashes are to be preserved
   My Notes:  in combining paths like pathcat(//, sys, /a) =>
    should get //sys/a
              in combining paths like pathcat(//, /sys, /a) => /sys sys
specifies it wants to be directory, not a system., so at root, '//' refer to
no system, and we just get: '/sys/a'

2) final slash indicates end node is a directory

3) a final . in a source specifies something under the directory name.

~4?)  I don't think /.. at end of a pathname can be safely deleted
and have same meaning as without it, but brain is too tired to detail
right now.






Information forwarded to bug-coreutils <at> gnu.org:
bug#36416; Package coreutils. (Fri, 28 Jun 2019 17:49:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Coreutils <bug-coreutils <at> gnu.org>
Cc: Eric Blake <eblake <at> redhat.com>
Subject: Re: realpath doesn't preserve '..' consistently as per POSIX, nor '/.'
Date: Fri, 28 Jun 2019 10:47:53 -0700
I don't recall reading anything about posix allowing the elimination
of such paths.

If I have an application layer on top of linux that processes those
double slashes, it may or may not work depending on the underlying OS?

That wouldn't be very portable w/r/t a Portable OS Information eXchange
standard.

I also specified the elimination of the single dots after a slash
which in reality don't have the same meaning as the slash.

It would useful if bugs were not closed before asking for more
information.  I've seen many bugs over the years that were closed
when they were real bugs that came back and bit the project hard,
including bugs that were closed out because I hadn't sent them through
cygwin's list first when I knew they were not bugs in cygwin.



On 2019/06/28 08:38, Eric Blake wrote:
> tag ### notabug
> thanks
> 
> On 6/28/19 6:11 AM, L A Walsh wrote:
>> realpath -m //sysname/rootdir/.
>> returns "/sysname/rootdir"  This is incorrect, according to POSIX.
> 
> No, it is implementation-defined if it is correct.  You failed to say
> what platform you are on.  If you are on Cygwin, where // is distinct
> from /, then it is incorrect - but that's not the behavior of realpath
> on Cygwin.  If you are Linux, where // is an alias for /, then the
> behavior is correct.
> 
> Unless you can provide more details about which system you saw this
> behavior on, and which version of realpath you tested, I'm marking this
> as not a bug.
> 




Information forwarded to bug-coreutils <at> gnu.org:
bug#36416; Package coreutils. (Fri, 28 Jun 2019 19:04:02 GMT) Full text and rfc822 format available.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: L A Walsh <coreutils <at> tlinx.org>, Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: realpath doesn't preserve '..' consistently as per POSIX, nor '/.'
Date: Fri, 28 Jun 2019 14:02:39 -0500
[Message part 1 (text/plain, inline)]
On 6/28/19 12:47 PM, L A Walsh wrote:
> I don't recall reading anything about posix allowing the elimination
> of such paths.

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html

4.13:
"If a pathname begins with two successive <slash> characters, the first
component following the leading <slash> characters may be interpreted in
an implementation-defined manner, "

Linux has an implementation-defined manner: "// is an alias for /, and
therefore may be shortened"

Cygwin has an implementation-defined manner: "// is distinct from /, and
therefore must be preserved"

Note that "may be shortened" is not synonymous with "must be shortened"
nor with "must be preserved".  So on Linux, preserving "//" is
POSIX-compliant, AND shortening "//" to "/" is POSIX-compliant (since
both choices comply with the implementation definition).

> 
> If I have an application layer on top of linux that processes those
> double slashes, it may or may not work depending on the underlying OS?

If your application layer is on top of Linux, then yes, the underlying
OS guarantees that your application processing a path beginning with
"//" is correct whether it preserves or shortens that path to "/".

If you port the source code of an application originally built on Linux
to then operate on some other operating system, then the source code is
buggy if it shortens the path to "/". But you have to actually check the
source code to see if it has that bug - just because 'realpath' in
coreutils shortens '//' to '/' on Linux does NOT mean that it is buggy,
because the code for realpath has an explicit check (run at
configure-time) for whether "//" is distinct from "/" - and on platforms
where that check is answered differently than on Linux, the code behaves
differently.  That is, the coreutils code is self-adapting to the
implementation definition of the implementation it is compiled on.


> It would useful if bugs were not closed before asking for more
> information. 

They can always be reopened if you provide enough relevant information
to show it was closed prematurely.  But so far in this case, you have
not done so, but merely confirmed my suspicions - you noticed a
difference in behavior between Linux and Cygwin, but both behaviors were
POSIX-compliant when factoring in the implementation-defined nature of
the underlying implementation.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Added tag(s) notabug. Request was from Eric Blake <eblake <at> redhat.com> to control <at> debbugs.gnu.org. (Fri, 28 Jun 2019 19:04:03 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 36416 <at> debbugs.gnu.org and L A Walsh <coreutils <at> tlinx.org> Request was from Eric Blake <eblake <at> redhat.com> to control <at> debbugs.gnu.org. (Fri, 28 Jun 2019 19:04:04 GMT) Full text and rfc822 format available.

Information forwarded to bug-coreutils <at> gnu.org:
bug#36416; Package coreutils. (Sat, 29 Jun 2019 19:22:02 GMT) Full text and rfc822 format available.

Message #18 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Eric Blake <eblake <at> redhat.com>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: realpath doesn't preserve '..' consistently as per POSIX, nor '/.'
Date: Sat, 29 Jun 2019 12:21:23 -0700

On 2019/06/28 12:02, Eric Blake wrote:
> If your application layer is on top of Linux, then yes, the underlying
> OS guarantees that your application processing a path beginning with
> "//" is correct whether it preserves or shortens that path to "/".
----
	Linux doesn't forever guarantee that it will or won't
use '//'.


> If you port the source code of an application originally built on Linux
> to then operate on some other operating system, then the source code is
> buggy if it shortens the path to "/". But you have to actually check the
> source code to see if it has that bug - just because 'realpath' in
> coreutils shortens '//' to '/' on Linux does NOT mean that it is buggy,
> because the code for realpath has an explicit check (run at
> configure-time) for whether "//" is distinct from "/" - and on platforms
> where that check is answered differently than on Linux, the code behaves
> differently.  That is, the coreutils code is self-adapting to the
> implementation definition of the implementation it is compiled on.
---
	As long as realpath guarantees that the generated binary 
won't work on any other version of linux, that would be true.  However
the run-time binary isn't recompiled with each version of linux.

	
> 
> 
>> It would useful if bugs were not closed before asking for more
>> information. 
> 
> They can always be reopened if you provide enough relevant information
> to show it was closed prematurely.  But so far in this case, you have
> not done so, but merely confirmed my suspicions - you noticed a
> difference in behavior between Linux and Cygwin, but both behaviors were
> POSIX-compliant when factoring in the implementation-defined nature of
> the underlying implementation.
----
	And you confirm that there there is a design flaw in realpath.

	Realpath is not part of the OS and is not regenerated as part
of the kernel build process. If a patch is applied to specifically 
to enable or assign meaning to '//' separately that to '/', 'realpath'
will continue to run as it was configured to run on the earlier version.

	Realpath can't assume either that it was configured properly
for the current linux binary -- neither that the user knows which is
correct, nor that some behavior that was true at compile time is
still true at run time.

	
	Realpath also trims '.' off of "[dir]/." when these do not have
identical meaning within the linux or cygwin ecospheres.  If I process
source and destination paths through realpath before passing them
to another user app, like 'cp', realpath will alter the behavior.  

	If you look at the historic behavior of cp w/relation to 
to *nix, it behaves differently if the source comes from '<dir>' than
if it comes from '<dir>/.'.

	Similarly if you look at the historic behavior of *nix, you
can't claim '//' and '/' are identical.  Realpath fixes it's
path-modifying behaviors at compile time but doesn't disallow its 
binary being run on differently compiled copies of linux.

	FWIW, for device paths in /etc/fstab, the notation for CIFS remote
mounts is '//', as in:
 
mount -t cifs //Win_Desktop/C/ /Athenae -o rw,cifsacl,nocase,serverino,vers=2.1

It seems useful for '//' to made available on linux as an automounter path, 
though with nothing mounted, refering to the local system.

I could see concatenating paths '//' & '/usr' being reduced to '/usr',
but concatenating '//' & svrname reducing to '//svrname'.


Other realpath problem:
----- -------- -------

But for a real case now where processing path-args with realpath resulting
in different and unwanted behavior:

using a function for shorthand:

rp() { declare p=(); for e in "$@"; do p+=($(realpath -m "$e")); done ; set "${p[@]}"; printf "%s\n" "$@"; }; export rp

  # && sample files && cd to /tmp: 

mkdir /tmp/{a,b}; touch /tmp/a{1,2}; cd /tmp;

cp -a $(rp a/.) $(rp b/.)
  #versus
cp -a a/. b/.

produce very different results.  I would assert that reducing
'dir/.' to simply 'dir/'

is also incorrect.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 28 Jul 2019 11:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 245 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.