GNU bug report logs -
#78910
tail does not support -r added by POSIX.1-2024
Previous Next
To reply to this bug, email your comments to 78910 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 05:37:03 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Collin Funk <collin.funk1 <at> gmail.com>:
New bug report received and forwarded. Copy sent to
bug-coreutils <at> gnu.org.
(Fri, 27 Jun 2025 05:37:03 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Creating this bug report since I have not been able to create a working
implementation yet, and perhaps my looking into this will inspire others
thoughts...
POSIX-1.2024 adds the following SYNOPSIS for 'tail' [1]:
tail [-f] [-c number|-n number] [file]
tail -r [-n number] [file]
Where the description for '-r' is:
Copy the lines in reverse order (last line first). If -n is
specified, that many lines of the file, starting with the last line,
shall be copied. If -n is not specified, every line of the input
file shall be copied.
In a simple implementation of 'tail', one that allocates an array to
store each line in memory, this would be easy to implement. However,
the Coreutils version reads the file in BUFSIZE/getpagesize() chunks
and then prints, repeating until completed. Therefore, we cannot just
modify the existing code to swap lines in memory.
Collin
[1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 09:11:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On 27/06/2025 06:36, Collin Funk wrote:
> Creating this bug report since I have not been able to create a working
> implementation yet, and perhaps my looking into this will inspire others
> thoughts...
>
> POSIX-1.2024 adds the following SYNOPSIS for 'tail' [1]:
>
> tail [-f] [-c number|-n number] [file]
> tail -r [-n number] [file]
>
> Where the description for '-r' is:
>
> Copy the lines in reverse order (last line first). If -n is
> specified, that many lines of the file, starting with the last line,
> shall be copied. If -n is not specified, every line of the input
> file shall be copied.
>
> In a simple implementation of 'tail', one that allocates an array to
> store each line in memory, this would be easy to implement. However,
> the Coreutils version reads the file in BUFSIZE/getpagesize() chunks
> and then prints, repeating until completed. Therefore, we cannot just
> modify the existing code to swap lines in memory.
>
> Collin
>
> [1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
tail -r comes from the BSDs.
Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
I'm a bit surprised -r was added by POSIX, but fair enough.
Thanks for flagging this.
Padraig
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 09:32:04 GMT)
Full text and
rfc822 format available.
Message #11 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On 27/06/2025 10:10, Pádraig Brady wrote:
> On 27/06/2025 06:36, Collin Funk wrote:
>> Creating this bug report since I have not been able to create a working
>> implementation yet, and perhaps my looking into this will inspire others
>> thoughts...
>>
>> POSIX-1.2024 adds the following SYNOPSIS for 'tail' [1]:
>>
>> tail [-f] [-c number|-n number] [file]
>> tail -r [-n number] [file]
>>
>> Where the description for '-r' is:
>>
>> Copy the lines in reverse order (last line first). If -n is
>> specified, that many lines of the file, starting with the last line,
>> shall be copied. If -n is not specified, every line of the input
>> file shall be copied.
>>
>> In a simple implementation of 'tail', one that allocates an array to
>> store each line in memory, this would be easy to implement. However,
>> the Coreutils version reads the file in BUFSIZE/getpagesize() chunks
>> and then prints, repeating until completed. Therefore, we cannot just
>> modify the existing code to swap lines in memory.
>>
>> Collin
>>
>> [1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
>
> tail -r comes from the BSDs.
> Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
> I'm a bit surprised -r was added by POSIX, but fair enough.
>
> Thanks for flagging this.
Note the GNU info doc on tail says:
"GNU ‘tail’ can output any amount of data (some other versions of
‘tail’ cannot). It also has no ‘-r’ option (print in reverse), sincereversing a file is really a different job from printing the end of a
file; BSD ‘tail’ (which is the one with ‘-r’) can only reverse files
that are at most as large as its buffer, which is typically 32 KiB. A
more reliable and versatile way to reverse files is the GNU ‘tac’
command.
"
BTW the POSIX tracker on this is:
https://www.austingroupbugs.net/view.php?id=877
cheers,
Padraig
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 15:52:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On Fri, Jun 27, 2025 at 2:16 AM Pádraig Brady <P <at> draigbrady.com> wrote:
> On 27/06/2025 06:36, Collin Funk wrote:
> > Creating this bug report since I have not been able to create a working
> > implementation yet, and perhaps my looking into this will inspire others
> > thoughts...
> >
> > POSIX-1.2024 adds the following SYNOPSIS for 'tail' [1]:
> >
> > tail [-f] [-c number|-n number] [file]
> > tail -r [-n number] [file]
> >
> > Where the description for '-r' is:
> >
> > Copy the lines in reverse order (last line first). If -n is
> > specified, that many lines of the file, starting with the last line,
> > shall be copied. If -n is not specified, every line of the input
> > file shall be copied.
> >
> > In a simple implementation of 'tail', one that allocates an array to
> > store each line in memory, this would be easy to implement. However,
> > the Coreutils version reads the file in BUFSIZE/getpagesize() chunks
> > and then prints, repeating until completed. Therefore, we cannot just
> > modify the existing code to swap lines in memory.
> >
> > Collin
> >
> > [1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
>
> tail -r comes from the BSDs.
> Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
> I'm a bit surprised -r was added by POSIX, but fair enough.
"Surprised" is putting it lightly. I am disappointed and am tempted to
push back and to delay encumbering GNU tail with -r.
That is an option no GNU system needs, since they've all had tac since
before 1992-era textutils.
I've Cc'd Eric Blake, in case someone wants to propose adding tac to
POSIX in spite of the fact that the BSDs still lack it.
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 19:17:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 78910 <at> debbugs.gnu.org (full text, mbox):
Hi Jim and Pádraig,
Jim Meyering <jim <at> meyering.net> writes:
>> tail -r comes from the BSDs.
>> Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
>> I'm a bit surprised -r was added by POSIX, but fair enough.
>
> "Surprised" is putting it lightly. I am disappointed and am tempted to
> push back and to delay encumbering GNU tail with -r.
> That is an option no GNU system needs, since they've all had tac since
> before 1992-era textutils.
>
> I've Cc'd Eric Blake, in case someone wants to propose adding tac to
> POSIX in spite of the fact that the BSDs still lack it.
Thanks for reminding me about 'tac'. For some reason I forgot about it.
That solves my question about how to implement 'tail -r' while avoiding
arbitrary limits. But I agree that duplicating parts of 'tac' into
'tail' does not seem ideal.
Collin
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 22:22:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On 27/06/2025 20:16, Collin Funk wrote:
> Hi Jim and Pádraig,
>
> Jim Meyering <jim <at> meyering.net> writes:
>
>>> tail -r comes from the BSDs.
>>> Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
>>> I'm a bit surprised -r was added by POSIX, but fair enough.
>>
>> "Surprised" is putting it lightly. I am disappointed and am tempted to
>> push back and to delay encumbering GNU tail with -r.
>> That is an option no GNU system needs, since they've all had tac since
>> before 1992-era textutils.
>>
>> I've Cc'd Eric Blake, in case someone wants to propose adding tac to
>> POSIX in spite of the fact that the BSDs still lack it.
>
> Thanks for reminding me about 'tac'. For some reason I forgot about it.
>
> That solves my question about how to implement 'tail -r' while avoiding
> arbitrary limits. But I agree that duplicating parts of 'tac' into
> 'tail' does not seem ideal.
Yes not ideal.
I wouldn't be rushing to implement it TBH.
Note the POSIX specified functionality is
limited to -n and a single file,
so we might be able to implement the POSIX subset
by just piping the output (of one file) to tac internally?
BSD functionality is more involved:
$ seq 10 > a; cp a b $ tail -n2 -r a b ==> a <==
10
9
==> b <==
10
9
cheers,
Pádraig
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Fri, 27 Jun 2025 22:43:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 78910 <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady <P <at> draigBrady.com> writes:
> On 27/06/2025 20:16, Collin Funk wrote:
>> Hi Jim and Pádraig,
>> Jim Meyering <jim <at> meyering.net> writes:
>>
>>>> tail -r comes from the BSDs.
>>>> Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
>>>> I'm a bit surprised -r was added by POSIX, but fair enough.
>>>
>>> "Surprised" is putting it lightly. I am disappointed and am tempted to
>>> push back and to delay encumbering GNU tail with -r.
>>> That is an option no GNU system needs, since they've all had tac since
>>> before 1992-era textutils.
>>>
>>> I've Cc'd Eric Blake, in case someone wants to propose adding tac to
>>> POSIX in spite of the fact that the BSDs still lack it.
>> Thanks for reminding me about 'tac'. For some reason I forgot about
>> it.
>> That solves my question about how to implement 'tail -r' while
>> avoiding
>> arbitrary limits. But I agree that duplicating parts of 'tac' into
>> 'tail' does not seem ideal.
>
> Yes not ideal.
>
> I wouldn't be rushing to implement it TBH.
I didn't get far in implementing it. So I am not in any rush.
> Note the POSIX specified functionality is
> limited to -n and a single file,
> so we might be able to implement the POSIX subset
> by just piping the output (of one file) to tac internally?
This would probably require adding a '--tac-program=' option to tail
in order to handle
'./configure --program-(prefix|suffix|transform-name)=' before and after
install.
See a similar fix I did for diffutils a while ago [1]. In that case
'--diff-program=' was already supported so no big deal. But I don't
really want to add '--tac-program='. Seems a bit awkward to me.
Collin
[1] https://lists.gnu.org/archive/html/bug-diffutils/2024-06/msg00016.html
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Sun, 29 Jun 2025 23:43:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 78910 <at> debbugs.gnu.org (full text, mbox):
Jim Meyering wrote:
> That is an option no GNU system needs, since they've all had tac since
> before 1992-era textutils.
But 'tac' does not have a line-number-limit argument.
The POSIX rationale [1] has
"While both
tail -n$n | tac
and
tac | head -n$n
can be used to output a fixed length of reversed line output, the
standard developers decided that it was preferable to have a single
utility tail -r -n$n for the same purpose."
The second of these alternatives, 'tac | head -n$n' will not work well
with non-seekable files: it requires 'tac' to buffer the *entire* input
(as huge as it may be), before extracting a few lines of it.
The first alternative looks better: 'tail -n$n | tac'. But thinking
through it, it seems the logic that 'tail' uses for 'tail -n$n' is
also nearly suitable for 'tail -r -n$n':
- In function file_lines(), instead of calling dump_remainder at the
end, the loop would call xwrite_stdout once for each line (with
special considerations for lines that span more than 1 buffer).
- In function pipe_lines(), all the relevant data is in memory at
the end. It's only a question of doing the xwrite_stdout calls
on smaller pieces and in reverse order.
When implemented this way, this will be more efficient than to spawn
'tac' as a separate subprocess.
Collin Funk wrote:
> But I agree that duplicating parts of 'tac' into
> 'tail' does not seem ideal.
Which parts of the 'tac' program would need to be reused in 'tail'?
Maybe the output() function?
In any case, the technique for avoiding code duplication is well-known.
The programs 'cp', 'mv', 'install' have overlapping functionality
without necessitating duplicated code.
Bruno
[1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Mon, 30 Jun 2025 11:49:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On 30/06/2025 00:42, Bruno Haible via GNU coreutils Bug Reports wrote:
> Jim Meyering wrote:
>> That is an option no GNU system needs, since they've all had tac since
>> before 1992-era textutils.
>
> But 'tac' does not have a line-number-limit argument.
>
> The POSIX rationale [1] has
>
> "While both
> tail -n$n | tac
> and
> tac | head -n$n
> can be used to output a fixed length of reversed line output, the
> standard developers decided that it was preferable to have a single
> utility tail -r -n$n for the same purpose."
Right these are equivalent, so it's only worth considering
the more efficient tail -n$n | tac
> The second of these alternatives, 'tac | head -n$n' will not work well
> with non-seekable files: it requires 'tac' to buffer the *entire* input
> (as huge as it may be), before extracting a few lines of it.
>
> The first alternative looks better: 'tail -n$n | tac'. But thinking
> through it, it seems the logic that 'tail' uses for 'tail -n$n' is
> also nearly suitable for 'tail -r -n$n':
> - In function file_lines(), instead of calling dump_remainder at the
> end, the loop would call xwrite_stdout once for each line (with
> special considerations for lines that span more than 1 buffer).
> - In function pipe_lines(), all the relevant data is in memory at
> the end. It's only a question of doing the xwrite_stdout calls
> on smaller pieces and in reverse order.
>
> When implemented this way, this will be more efficient than to spawn
> 'tac' as a separate subprocess.
That's not really the unix model though.
Having separate processes also implicitly leverages multiple processors
so you'd have to account for that.
Saying all that I'm not strongly against it,
especially since POSIX standardised it,
but I'm just surprised they standardised it.
Note there are cases where merging functionality can have algorithmic advantages,
in which case there is a much stronger argument for merging.
For example we have previously mentioned sort --tail=$n or --head=$n
would be useful (and more commonly required) functionality.
See: https://lists.gnu.org/archive/html/bug-coreutils/2004-04/msg00157.html
It would be especially useful if implemented in O(n log n) complexity.
cheers,
Padraig
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Tue, 01 Jul 2025 17:09:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On 6/28/25 00:21, Pádraig Brady wrote:
> I wouldn't be rushing to implement it TBH.
Furthermore, -r clashes with the existing -f,--follow ... hence it's even more
surprising that POSIX does not specify what should happen for those logically
conflicting modes.
Have a nice day,
Berny
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Tue, 01 Jul 2025 19:30:03 GMT)
Full text and
rfc822 format available.
Message #35 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On Fri, Jun 27, 2025 at 08:51:10AM -0700, Jim Meyering wrote:
> > >
> > > [1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
> >
> > tail -r comes from the BSDs.
> > Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
> > I'm a bit surprised -r was added by POSIX, but fair enough.
>
> "Surprised" is putting it lightly. I am disappointed and am tempted to
> push back and to delay encumbering GNU tail with -r.
> That is an option no GNU system needs, since they've all had tac since
> before 1992-era textutils.
>
> I've Cc'd Eric Blake, in case someone wants to propose adding tac to
> POSIX in spite of the fact that the BSDs still lack it.
I'll bring up this topic at the next Austin Group meeting.
At any rate, it looks like tail -r was added due to this bug report in
2014 (sadly, it is no longer an option to ask Jorg Schilling why he
wanted it standardized)
https://www.austingroupbugs.net/view.php?id=877
But I did ask at the time (with no answer) why we went with 'tail -r'
instead of 'tac'.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Tue, 01 Jul 2025 19:34:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On Tue, Jul 01, 2025 at 02:28:48PM -0500, Eric Blake wrote:
> On Fri, Jun 27, 2025 at 08:51:10AM -0700, Jim Meyering wrote:
> > > >
> > > > [1] https://pubs.opengroup.org/onlinepubs/9799919799/utilities/tail.html
> > >
> > > tail -r comes from the BSDs.
> > > Also the BSDs don't have tac(1) which overlaps in functionality quite a bit.
> > > I'm a bit surprised -r was added by POSIX, but fair enough.
> >
> > "Surprised" is putting it lightly. I am disappointed and am tempted to
> > push back and to delay encumbering GNU tail with -r.
> > That is an option no GNU system needs, since they've all had tac since
> > before 1992-era textutils.
> >
> > I've Cc'd Eric Blake, in case someone wants to propose adding tac to
> > POSIX in spite of the fact that the BSDs still lack it.
>
> I'll bring up this topic at the next Austin Group meeting.
>
> At any rate, it looks like tail -r was added due to this bug report in
> 2014 (sadly, it is no longer an option to ask Jorg Schilling why he
> wanted it standardized)
> https://www.austingroupbugs.net/view.php?id=877
>
> But I did ask at the time (with no answer) why we went with 'tail -r'
> instead of 'tac'.
Correction - my comment asking why we didn't go with tac was dated
2014-10-23 at 22:44,
https://www.austingroupbugs.net/view.php?id=877#c2428
but the wording to move forward with 'tail -r' was amended in place,
so even though the proposed wording was in place before my question at
2014-10-23 15:18, it was later amended on 2014-10-30 to add the
Rationale on why 'tail -r' provides a potential algorithmic
improvement over 'tac | head -n' when it comes to amount of memory
consumed.
https://www.austingroupbugs.net/view.php?id=877#c2425
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Tue, 01 Jul 2025 19:38:02 GMT)
Full text and
rfc822 format available.
Message #41 received at 78910 <at> debbugs.gnu.org (full text, mbox):
On Tue, Jul 01, 2025 at 07:07:45PM +0200, Bernhard Voelker wrote:
> On 6/28/25 00:21, Pádraig Brady wrote:
> > I wouldn't be rushing to implement it TBH.
>
> Furthermore, -r clashes with the existing -f,--follow ... hence it's even more
> surprising that POSIX does not specify what should happen for those logically
> conflicting modes.
Remember, the POSIX synopsis lines lists two distinct invocation
styles:
tail [-f] [-c number|-n number] [file]
tail -r [-n number] [file]
which means undefined behavior from the POSIX point of view if -f and
-r are ever attempted at the same time. Therefore, whatever GNU
decides to do on that combo (preferably, diagnose it early with an
error and exit, since I can't figure out anything else useful it could
do) will not violate POSIX.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
Information forwarded
to
bug-coreutils <at> gnu.org:
bug#78910; Package
coreutils.
(Tue, 01 Jul 2025 19:46:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 78910 <at> debbugs.gnu.org (full text, mbox):
Hi Eric,
Eric Blake <eblake <at> redhat.com> writes:
> Remember, the POSIX synopsis lines lists two distinct invocation
> styles:
>
> tail [-f] [-c number|-n number] [file]
> tail -r [-n number] [file]
Yep, I was going to mention this too. But I was not sure if it should
have a mention in the description to make it more clear.
> which means undefined behavior from the POSIX point of view if -f and
> -r are ever attempted at the same time. Therefore, whatever GNU
> decides to do on that combo (preferably, diagnose it early with an
> error and exit, since I can't figure out anything else useful it could
> do) will not violate POSIX.
I think it should be standardized to fail when -f and -r are used
together. That seems like the only reasonable behavior to me.
Thanks,
Collin
This bug report was last modified today.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.