GNU bug report logs - #62385
POSIX behavior of readlink, realpath

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 62385 in the body.
You can then email your comments to 62385 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: bug-coreutils <at> gnu.org
Subject: POSIX behavior of readlink, realpath
Date: Wed, 22 Mar 2023 09:19:02 -0500

POSIX will be standardizing readlink(1) and realpath(1):

https://www.austingroupbugs.net/view.php?id=1457

Among other things, I can see the following changes that coreutils
will need to make to become compliant, or else we need to push back on
the POSIX folks if we have strong reasons to complain that their
specification will break things:

POSIX wants 'readlink non-symlink' to output a diagnostic; that is, it
looks like POSIX wants us to behave like '-v' is enabled by default
(our current behavior of -q by default will be non-compliant).

POSIX wants us to support 'realpath -E file'.  I'm not quite sure if
it is matches our existing behavior when -e is omitted (in which case,
all the more we have to do is have -E coming later than -e disable the
earlier -e).  In particular, it gives a convincing example:

"The behavior with the -E option when file does not resolve (with
symbolic links followed) to an existing file is not the same as simply
calling realpath() with the path prefix of the file operand and
writing the resulting pathname, a <slash>, and the last component of
file to standard output. For example, if /tmp/nofile does not exist,
and file is A/B where A is an existing directory and B is a symbolic
link to /tmp/nofile, realpath with -E will output /tmp/nofile, but if
B is a symbolic link to /tmp/nofile/foo, realpath with -E will treat
this as an error. In both cases <tt>realpath("A/B")</tt> would fail
with errno set to [ENOENT]. Even though <tt>realpath("A")</tt> would
succeed, in neither case is anything ending /B the result."

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Message #8 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: 62385 <at> debbugs.gnu.org
Subject: bug#62385: POSIX behavior of readlink, realpath
Date: Sat, 02 Aug 2025 21:33:15 -0700

[Message part 1 (text/plain, inline)]

Hi Eric,

You said:

> Among other things, I can see the following changes that coreutils
> will need to make to become compliant, or else we need to push back on
> the POSIX folks if we have strong reasons to complain that their
> specification will break things:
>
> POSIX wants 'readlink non-symlink' to output a diagnostic; that is, it
> looks like POSIX wants us to behave like '-v' is enabled by default
> (our current behavior of -q by default will be non-compliant).

I have attached a patch that I think is correct for readlink.

It is a bit more complex than just behaving the same as 'readlink -v'
previously would. This is because the behavior of non-POSIX
options. Here are some commands:

    $ touch file
    $ ln -s symlink file
    
    # Example 1
    $ readlink -v file
    readlink: file: Invalid argument

    # Example 2
    $ readlink -v -f file 
    /home/collin/file

If 'readlink -v' where 'POSIXLY_CORRECT=1 readlink' then example 1 would
conform. I think that example 2 would violate POSIX.

So, when POSIXLY_CORRECT is enabled we can call 'readlink', check for
errors, and then canonicalize if it is successful.

Collin

[0001-readlink-emit-errors-when-POSIXLY_CORRECT-is-set.patch (text/x-patch, attachment)]

Message #11 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Collin Funk <collin.funk1 <at> gmail.com>, 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 3 Aug 2025 10:25:02 +0100

On 03/08/2025 05:33, Collin Funk wrote:
> Hi Eric,
> 
> You said:
> 
>> Among other things, I can see the following changes that coreutils
>> will need to make to become compliant, or else we need to push back on
>> the POSIX folks if we have strong reasons to complain that their
>> specification will break things:
>>
>> POSIX wants 'readlink non-symlink' to output a diagnostic; that is, it
>> looks like POSIX wants us to behave like '-v' is enabled by default
>> (our current behavior of -q by default will be non-compliant).
> 
> I have attached a patch that I think is correct for readlink.
> 
> It is a bit more complex than just behaving the same as 'readlink -v'
> previously would. This is because the behavior of non-POSIX
> options. Here are some commands:
> 
>      $ touch file
>      $ ln -s symlink file
>      
>      # Example 1
>      $ readlink -v file
>      readlink: file: Invalid argument
> 
>      # Example 2
>      $ readlink -v -f file
>      /home/collin/file
> 
> If 'readlink -v' where 'POSIXLY_CORRECT=1 readlink' then example 1 would
> conform. I think that example 2 would violate POSIX.
> 
> So, when POSIXLY_CORRECT is enabled we can call 'readlink', check for
> errors, and then canonicalize if it is successful.

I'd say any option apart from -n implicitly disables strict POSIX conformance,
since those options are not in POSIX at all. So all I'd do is enable -v
if POSIXLY_CORRECT is set.  There is the argument for defaulting to -v,
but I see FreeBSD does not output an error either if file is not a symlink,
so it's probably best to only enable -v if POSIXLY_CORRECT is set.

In NEWS this should be under "Changes in behavior" rather than "New Features".

Thanks for following up on this.

Padraig.

Message #14 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 03 Aug 2025 11:08:01 -0700

[Message part 1 (text/plain, inline)]

Hi Pádraig,

Pádraig Brady <P <at> draigBrady.com> writes:

> I'd say any option apart from -n implicitly disables strict POSIX conformance,
> since those options are not in POSIX at all.

It seems like this was my misunderstanding. I thought non-POSIX options
couldn't change:

    If file does not name a symbolic link, readlink shall write a
    diagnostic message to standard error and exit with non-zero status.

But I like the behavior you and Dmitry suggest better.

> So all I'd do is enable -v if POSIXLY_CORRECT is set. There is the
> argument for defaulting to -v, but I see FreeBSD does not output an
> error either if file is not a symlink, so it's probably best to only
> enable -v if POSIXLY_CORRECT is set.
>
> In NEWS this should be under "Changes in behavior" rather than "New Features".

Done, v2 attached.

Collin

[v2-0001-readlink-emit-errors-when-POSIXLY_CORRECT-is-set.patch (text/x-patch, attachment)]

Message #17 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Collin Funk <collin.funk1 <at> gmail.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 3 Aug 2025 20:05:45 +0100

On 03/08/2025 19:08, Collin Funk wrote:
> Hi Pádraig,
> 
> Pádraig Brady <P <at> draigBrady.com> writes:
> 
>> I'd say any option apart from -n implicitly disables strict POSIX conformance,
>> since those options are not in POSIX at all.
> 
> It seems like this was my misunderstanding. I thought non-POSIX options
> couldn't change:
> 
>      If file does not name a symbolic link, readlink shall write a
>      diagnostic message to standard error and exit with non-zero status.
> 
> But I like the behavior you and Dmitry suggest better.
> 
>> So all I'd do is enable -v if POSIXLY_CORRECT is set. There is the
>> argument for defaulting to -v, but I see FreeBSD does not output an
>> error either if file is not a symlink, so it's probably best to only
>> enable -v if POSIXLY_CORRECT is set.
>>
>> In NEWS this should be under "Changes in behavior" rather than "New Features".
> 
> Done, v2 attached.

Looks good.

thanks!
Padraig

Message #20 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Collin Funk <collin.funk1 <at> gmail.com>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 3 Aug 2025 12:09:13 -0700

On 2025-08-03 11:08, Collin Funk wrote:

> +  readlink will behave as if the -v option is used if the
> +  POSIXLY_CORRECT environment variable is defined.

This isn't true if -q/--quiet/-s/--silent is specified. I would reword 
this to something like "readlink now defaults to being verbose if 
POSIXLY_CORRECT is set."

The --usage message and doc/coreutils.texi should be updated to match 
the new behavior.

There's a spurious empty line inserted into readlink.c.

Message #23 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 03 Aug 2025 12:11:26 -0700

Pádraig Brady <P <at> draigBrady.com> writes:

>> Done, v2 attached.
>
> Looks good.

Pushed, thanks to you and Dmitry for the review.

Leaving this bug open for realpath. I'll have a look at that as well,
just requires some more reading than readlink did. :)

Collin

Message #26 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 62385 <at> debbugs.gnu.org, Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 03 Aug 2025 12:47:57 -0700

[Message part 1 (text/plain, inline)]

Hi Paul,

I should have waited a few minutes before pushing. Oops...

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> On 2025-08-03 11:08, Collin Funk wrote:
>
>> +  readlink will behave as if the -v option is used if the
>> +  POSIXLY_CORRECT environment variable is defined.
>
> This isn't true if -q/--quiet/-s/--silent is specified. I would reword
> this to something like "readlink now defaults to being verbose if
> POSIXLY_CORRECT is set."

True, that is more accurate.

> The --usage message and doc/coreutils.texi should be updated to match
> the new behavior.

I added a brief mention in doc/coreutils.texi. I guess it could be made
more explicit by mentioning under --silent and --quiet as well.

I only saw POSIXLY_CORRECT mentioned in the --help message for 'df' and
'du' for the block size difference. So I assumed it was not nessecary.

How about the wording and formatting of the attatched patch?

> There's a spurious empty line inserted into readlink.c.

I rewrote v2 based on v1 and left that. Oops.

Collin

[0001-readlink-improve-documentation-for-POSIXLY_CORRECT-i.patch (text/x-patch, attachment)]

Message #29 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Collin Funk <collin.funk1 <at> gmail.com>
Cc: 62385 <at> debbugs.gnu.org, Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 3 Aug 2025 13:07:40 -0700

On 2025-08-03 12:47, Collin Funk wrote:
> How about the wording and formatting of the attatched patch?

Thanks, looks good.

Message #32 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: "Dmitry V. Levin" <ldv <at> strace.io>
To: Collin Funk <collin.funk1 <at> gmail.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 3 Aug 2025 10:42:15 +0300

Hi,

On Sat, Aug 02, 2025 at 09:33:15PM -0700, Collin Funk wrote:
> Hi Eric,
> 
> You said:
> 
> > Among other things, I can see the following changes that coreutils
> > will need to make to become compliant, or else we need to push back on
> > the POSIX folks if we have strong reasons to complain that their
> > specification will break things:
> >
> > POSIX wants 'readlink non-symlink' to output a diagnostic; that is, it
> > looks like POSIX wants us to behave like '-v' is enabled by default
> > (our current behavior of -q by default will be non-compliant).
> 
> I have attached a patch that I think is correct for readlink.
> 
> It is a bit more complex than just behaving the same as 'readlink -v'
> previously would. This is because the behavior of non-POSIX
> options. Here are some commands:
> 
>     $ touch file
>     $ ln -s symlink file
>     
>     # Example 1
>     $ readlink -v file
>     readlink: file: Invalid argument
> 
>     # Example 2
>     $ readlink -v -f file 
>     /home/collin/file
> 
> If 'readlink -v' where 'POSIXLY_CORRECT=1 readlink' then example 1 would
> conform. I think that example 2 would violate POSIX.

Given that -f is not in POSIX, I don't see why the example 2 would violate
it.

> So, when POSIXLY_CORRECT is enabled we can call 'readlink', check for
> errors, and then canonicalize if it is successful.

I'd argue that such a change of readlink -e/-f/-m behaviour makes little
sense as it defeats the purpose of these options.


-- 
ldv

Message #35 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sat, 09 Aug 2025 18:04:44 -0700

[Message part 1 (text/plain, inline)]

Collin Funk <collin.funk1 <at> gmail.com> writes:

> Pádraig Brady <P <at> draigBrady.com> writes:
>
>>> Done, v2 attached.
>>
>> Looks good.
>
> Pushed, thanks to you and Dmitry for the review.
>
> Leaving this bug open for realpath. I'll have a look at that as well,
> just requires some more reading than readlink did. :)

It looks like 'realpath -E' is the same as Coreutils 'readlink -f', and
default behavior of Coreutils 'realpath'.  POSIX us to keep our default
behavior [1]:

    If no options are specified, realpath shall canonicalize the specified
    pathname in an unspecified manner such that the resulting absolute
    pathname does not contain any components that refer to files of type
    symbolic link and does not contain any components that are dot or
    dot-dot.

Here is a proposed patch.

I don't love having 'realpath -E' and 'readlink -f' with the same long
option name, but having the same functionality with a different long
option name seems worse.

Collin

[1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/realpath.html

[0001-realpath-support-the-E-option-required-by-POSIX.patch (text/x-patch, attachment)]

Message #38 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Collin Funk <collin.funk1 <at> gmail.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 10 Aug 2025 10:06:08 +0100

On 10/08/2025 02:04, Collin Funk wrote:
> Collin Funk <collin.funk1 <at> gmail.com> writes:
> 
>> Pádraig Brady <P <at> draigBrady.com> writes:
>>
>>>> Done, v2 attached.
>>>
>>> Looks good.
>>
>> Pushed, thanks to you and Dmitry for the review.
>>
>> Leaving this bug open for realpath. I'll have a look at that as well,
>> just requires some more reading than readlink did. :)
> 
> It looks like 'realpath -E' is the same as Coreutils 'readlink -f', and
> default behavior of Coreutils 'realpath'.  POSIX us to keep our default
> behavior [1]:
> 
>      If no options are specified, realpath shall canonicalize the specified
>      pathname in an unspecified manner such that the resulting absolute
>      pathname does not contain any components that refer to files of type
>      symbolic link and does not contain any components that are dot or
>      dot-dot.
> 
> Here is a proposed patch.
> 
> I don't love having 'realpath -E' and 'readlink -f' with the same long
> option name, but having the same functionality with a different long
> option name seems worse.
> 
> Collin
> 
> [1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/realpath.html
> 

Logic looks good.
The --help for -E is too wordy I think.
I would squash this in instead.

thanks!
Padraig


diff --git a/src/realpath.c b/src/realpath.c
index b06e5e845..1f7882b49 100644
--- a/src/realpath.c
+++ b/src/realpath.c
@@ -72,11 +72,8 @@ usage (int status)
 Print the resolved absolute file name.\n\
 "), stdout);
       fputs (_("\
-  -E, --canonicalize           canonicalize by following every symlink in\n\
-                               every component of the given name recursively;\
+  -E, --canonicalize           all but the last component must exist (default)\
 \n\
-                               all but the last component must exist\n\
-                               (this is the default)\n\
   -e, --canonicalize-existing  all components of the path must exist\n\
   -m, --canonicalize-missing   no path components need exist or be a directory\
 \n\

Message #41 received at 62385 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 62385 <at> debbugs.gnu.org
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sun, 10 Aug 2025 11:43:29 -0700

Pádraig Brady <P <at> draigBrady.com> writes:


> Logic looks good.
> The --help for -E is too wordy I think.
> I would squash this in instead.
>
> thanks!
> Padraig
>
> diff --git a/src/realpath.c b/src/realpath.c
> index b06e5e845..1f7882b49 100644
> --- a/src/realpath.c
> +++ b/src/realpath.c
> @@ -72,11 +72,8 @@ usage (int status)
>   Print the resolved absolute file name.\n\
>   "), stdout);
>         fputs (_("\
> -  -E, --canonicalize           canonicalize by following every symlink in\n\
> -                               every component of the given name recursively;\
> +  -E, --canonicalize           all but the last component must exist (default)\
>   \n\
> -                               all but the last component must exist\n\
> -                               (this is the default)\n\
>     -e, --canonicalize-existing  all components of the path must exist\n\
>     -m, --canonicalize-missing   no path components need exist or be a directory\
>   \n\

Thanks, I pushed it with that change.

I lazily copied the long description from 'readlink'. The longer
description probably makes sense there because canonicalizing file names
isn't its primary purpose.

Collin

Message #46 received at 62385-done <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: 62385-done <at> debbugs.gnu.org
Cc: Paul Eggert <eggert <at> cs.ucla.edu>,
 Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#62385: POSIX behavior of readlink, realpath
Date: Sat, 16 Aug 2025 18:43:34 -0700

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> On 2025-08-03 12:47, Collin Funk wrote:
>> How about the wording and formatting of the attatched patch?
>
> Thanks, looks good.

Marking this bug as done since readlink and realpath should now comply
with POSIX 2024. If not, we can treat it as a new bug.

Thanks Paul for the last patch improving the documentation regarding
using the command portably.

Collin

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.