GNU bug report logs - #24495
tail -F does not terminate when running out of names to watch

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24495 in the body.
You can then email your comments to 24495 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Julian Büning <julian.buening <at> rwth-aachen.de>
To: <bug-coreutils <at> gnu.org>
Cc: Oscar Soria Dustmann <Oscar.SoriaDustmann <at> comsys.rwth-aachen.de>,
 Daniel Schemmel <daniel.schemmel <at> comsys.rwth-aachen.de>
Subject: tail -F does not terminate when running out of names to watch
Date: Wed, 21 Sep 2016 16:22:07 +0200

observed behavior:

$ mkdir foo
$ tail -F foo &
[1] 1000
tail: error reading 'foo': Is a directory
tail: foo: cannot follow end of this type of file; giving up on this name
$ rmdir foo ; echo moo > foo
$ jobs
[1]+  Running                 tail -F foo &

expected behavior option 1:

$ mkdir foo
$ tail -F foo &
tail: error reading 'foo': Is a directory
tail: foo: cannot follow end of this type of file; giving up on this name
$ rmdir foo ; echo moo > foo
[1]+  Done                 tail -F foo &

expected behavior option 2:

$ mkdir foo
$ tail -F foo &
[1] 1000
tail: error reading 'foo': Is a directory
tail: foo: cannot follow end of this type of file
$ rmdir foo ; echo moo > foo
tail: 'foo' has appeared;  following new file
moo
$ jobs
[1]+  Running                 tail -F foo &

tail does neither terminate nor display any contents of the file foo.
We would assume that either tail terminates after displaying the error
message (because there is no other file left) or that tail would
infinitely retry to read a file with the same name and if such a file
is created.

We could reproduce this behavior with versions 8.25 on ArchLinux and
8.25 and 8.25.71-1db94 on Fedora (package and compiled from source)
with and without ---disable-inotify. At least in version 8.25,
tail_forever_inotify is not executed because any_non_remote_file
returns false, which disables inotify in line 2361 (tail.c).

We can trace the cause of this behavior for the non-inotify version of
tail_forever:
For a directory, the ignore flag in the corresponding struct File_spec
is set to true. Thus, tail_forever skips this file in line 1130 (tail.c)
and does not modify this flag during any iteration of the enclosing
while(1). Since no check is being done if any valid target remains,
this leads to the observed non-terminating behavior.

This behavior was found using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen
University.

Regards,
Julian Büning

Message #8 received at 24495 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Julian Büning <julian.buening <at> rwth-aachen.de>,
 24495 <at> debbugs.gnu.org
Cc: Oscar Soria Dustmann <Oscar.SoriaDustmann <at> comsys.rwth-aachen.de>,
 Daniel Schemmel <daniel.schemmel <at> comsys.rwth-aachen.de>
Subject: Re: bug#24495: tail -F does not terminate when running out of names
 to watch
Date: Wed, 21 Sep 2016 19:15:59 +0100

[Message part 1 (text/plain, inline)]

On 21/09/16 15:22, Julian Büning wrote:
> observed behavior:
> 
> $ mkdir foo
> $ tail -F foo &
> [1] 1000
> tail: error reading 'foo': Is a directory
> tail: foo: cannot follow end of this type of file; giving up on this name
> $ rmdir foo ; echo moo > foo
> $ jobs
> [1]+  Running                 tail -F foo &
> 
> expected behavior option 1:
> 
> $ mkdir foo
> $ tail -F foo &
> tail: error reading 'foo': Is a directory
> tail: foo: cannot follow end of this type of file; giving up on this name
> $ rmdir foo ; echo moo > foo
> [1]+  Done                 tail -F foo &
> 
> expected behavior option 2:
> 
> $ mkdir foo
> $ tail -F foo &
> [1] 1000
> tail: error reading 'foo': Is a directory
> tail: foo: cannot follow end of this type of file
> $ rmdir foo ; echo moo > foo
> tail: 'foo' has appeared;  following new file
> moo
> $ jobs
> [1]+  Running                 tail -F foo &
> 
> tail does neither terminate nor display any contents of the file foo.
> We would assume that either tail terminates after displaying the error
> message (because there is no other file left) or that tail would
> infinitely retry to read a file with the same name and if such a file
> is created.
> 
> We could reproduce this behavior with versions 8.25 on ArchLinux and
> 8.25 and 8.25.71-1db94 on Fedora (package and compiled from source)
> with and without ---disable-inotify. At least in version 8.25,
> tail_forever_inotify is not executed because any_non_remote_file
> returns false, which disables inotify in line 2361 (tail.c).
> 
> We can trace the cause of this behavior for the non-inotify version of
> tail_forever:
> For a directory, the ignore flag in the corresponding struct File_spec
> is set to true. Thus, tail_forever skips this file in line 1130 (tail.c)
> and does not modify this flag during any iteration of the enclosing
> while(1). Since no check is being done if any valid target remains,
> this leads to the observed non-terminating behavior.
> 
> This behavior was found using Symbolic Execution techniques developed in
> the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen
> University.

We can get expected behavior option 1 with the attached patch.
Note that's inconsistent with current inotify behavior which does
_not_ actually give up on the name, as can be seen when starting
with a (non existent) file:

$ touch foo
$ tail -F foo&
[1] 13624
$ rm foo; mkdir foo
tail: ‘foo’ has become inaccessible: No such file or directory
tail: ‘foo’ has been replaced with an untailable file; giving up on this name
$ rmdir foo; echo foo > foo
tail: ‘foo’ has become inaccessible: No such file or directory
tail: ‘foo’ has appeared;  following new file
foo

The attached patch also removes the "; giving up on this name"
message in the inotify case as that's not the case.

Ideally we'd have expected behavior option 2
both with and without inotify.
I'll need to look a bit more as to why we have that
limitation without inotify.

I'll add a test case before applying anything.

thanks,
Pádraig

[tail-retry-exit.patch (text/x-patch, attachment)]

Message #11 received at 24495 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>,
 Julian Büning <julian.buening <at> rwth-aachen.de>,
 24495 <at> debbugs.gnu.org
Cc: Oscar Soria Dustmann <Oscar.SoriaDustmann <at> comsys.rwth-aachen.de>,
 Daniel Schemmel <daniel.schemmel <at> comsys.rwth-aachen.de>
Subject: Re: bug#24495: tail -F does not terminate when running out of names
 to watch
Date: Tue, 27 Sep 2016 22:16:56 +0200

On 09/21/2016 08:15 PM, Pádraig Brady wrote:
> We can get expected behavior option 1 with the attached patch.
> Note that's inconsistent with current inotify behavior which does
> _not_ actually give up on the name, as can be seen when starting
> with a (non existent) file:
> 
> $ touch foo
> $ tail -F foo&
> [1] 13624
> $ rm foo; mkdir foo
> tail: ‘foo’ has become inaccessible: No such file or directory
> tail: ‘foo’ has been replaced with an untailable file; giving up on this name
> $ rmdir foo; echo foo > foo
> tail: ‘foo’ has become inaccessible: No such file or directory
> tail: ‘foo’ has appeared;  following new file
> foo
> 
> The attached patch also removes the "; giving up on this name"
> message in the inotify case as that's not the case.
> 
> Ideally we'd have expected behavior option 2
> both with and without inotify.
> I'll need to look a bit more as to why we have that
> limitation without inotify.

The new behavior is nice, but it would really be better to have
consistent behavior in inotify and polling mode.

Thanks & have a nice day,
Berny

Message #14 received at 24495 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Julian Büning <julian.buening <at> rwth-aachen.de>,
 24495 <at> debbugs.gnu.org
Cc: Oscar Soria Dustmann <Oscar.SoriaDustmann <at> comsys.rwth-aachen.de>,
 Daniel Schemmel <daniel.schemmel <at> comsys.rwth-aachen.de>
Subject: Re: bug#24495: tail -F does not terminate when running out of names
 to watch
Date: Wed, 28 Sep 2016 14:27:42 +0100

[Message part 1 (text/plain, inline)]

On 27/09/16 21:16, Bernhard Voelker wrote:
> On 09/21/2016 08:15 PM, Pádraig Brady wrote:
>> We can get expected behavior option 1 with the attached patch.
>> Note that's inconsistent with current inotify behavior which does
>> _not_ actually give up on the name, as can be seen when starting
>> with a (non existent) file:
>>
>> $ touch foo
>> $ tail -F foo&
>> [1] 13624
>> $ rm foo; mkdir foo
>> tail: ‘foo’ has become inaccessible: No such file or directory
>> tail: ‘foo’ has been replaced with an untailable file; giving up on this name
>> $ rmdir foo; echo foo > foo
>> tail: ‘foo’ has become inaccessible: No such file or directory
>> tail: ‘foo’ has appeared;  following new file
>> foo
>>
>> The attached patch also removes the "; giving up on this name"
>> message in the inotify case as that's not the case.
>>
>> Ideally we'd have expected behavior option 2
>> both with and without inotify.
>> I'll need to look a bit more as to why we have that
>> limitation without inotify.
> 
> The new behavior is nice, but it would really be better to have
> consistent behavior in inotify and polling mode.

Yes consistency would be nice. Looking at the polling behavior I see it was
not fundamental to the initial implementation and only changed later:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=FILEUTILS-4_0q-68-g54d12f7

The attached patch how has the preferred behavior option 2
both with and without inotify.

thanks,
Pádraig

[tail-retry-exit.patch (text/x-patch, attachment)]

Message #17 received at 24495 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>,
 Julian Büning <julian.buening <at> rwth-aachen.de>,
 24495 <at> debbugs.gnu.org
Cc: Oscar Soria Dustmann <Oscar.SoriaDustmann <at> comsys.rwth-aachen.de>,
 Daniel Schemmel <daniel.schemmel <at> comsys.rwth-aachen.de>
Subject: Re: bug#24495: tail -F does not terminate when running out of names
 to watch
Date: Wed, 28 Sep 2016 23:37:51 +0200

On 09/28/2016 03:27 PM, Pádraig Brady wrote:
> The attached patch how has the preferred behavior option 2
> both with and without inotify.

Great, this looks good to me.
+1

Thanks & have a nice day,
Berny

Message #22 received at 24495-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Julian Büning <julian.buening <at> rwth-aachen.de>,
 24495-done <at> debbugs.gnu.org
Cc: Oscar Soria Dustmann <Oscar.SoriaDustmann <at> comsys.rwth-aachen.de>,
 Daniel Schemmel <daniel.schemmel <at> comsys.rwth-aachen.de>
Subject: Re: bug#24495: tail -F does not terminate when running out of names
 to watch
Date: Wed, 28 Sep 2016 23:43:26 +0100

On 28/09/16 22:37, Bernhard Voelker wrote:
> On 09/28/2016 03:27 PM, Pádraig Brady wrote:
>> The attached patch how has the preferred behavior option 2
>> both with and without inotify.
> 
> Great, this looks good to me.
> +1

Thanks for the reviews.
Pushed at:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=v8.25-73-gf04daf5

Marking this as done

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.