GNU bug report logs - #41700
grep -v always exiting with 1 for empty file

Previous Next

Package: grep;

Reported by: Andi Kleen <andi <at> firstfloor.org>

Date: Thu, 4 Jun 2020 04:33:01 UTC

Severity: normal

Tags: notabug

Done: Paul Eggert <eggert <at> cs.ucla.edu>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41700 in the body.
You can then email your comments to 41700 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 04:33:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Andi Kleen <andi <at> firstfloor.org>:
New bug report received and forwarded. Copy sent to bug-grep <at> gnu.org. (Thu, 04 Jun 2020 04:33:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andi Kleen <andi <at> firstfloor.org>
To: bug-grep <at> gnu.org
Subject: grep -v always exiting with 1 for empty file
Date: Wed, 3 Jun 2020 20:26:41 -0700
% grep --version
grep (GNU grep) 3.4
...
% echo -n > foo
% grep -v foo foo ; echo $?
1

Would expect it to exit with zero in this case, since foo is not in the
file.

When the file is one byte it works as expected:

% echo > foo
% grep -v foo foo ; echo $?

0
%





Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 05:22:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Paul Jackson" <pj <at> usa.net>
To: bug-grep <at> gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 04 Jun 2020 00:21:15 -0500
If I am reading the grep.c code correctly,
asking grep to read a zero length file,
such as:

    grep foo /dev/null

causes the code to quit, with a non-zero
exit status, fairly early on, as soon as the
first call to "fillbuf()" returns with 0 bytes read.

This seems to apply except in the case of
grep -L (list files not matching) which special
cases the exit status to enable listing an
empty file as non-matching.

Being a crusty old codger, I'd recommend
changing the grep(1) man page to match
the current behavior, as observed by Andi
and myself, rather than changing the code
to be consistent with what Andi, reasonably
enough, expected.

Every time any little detail like this changes,
somebody's code breaks quietly in the night.

However since I'm not volunteering to submit
the documentation (or code) patch, my vote
counts for little :).

Carry on.

-- 
                Paul Jackson
                pj <at> usa.net




Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 10:27:02 GMT) Full text and rfc822 format available.

Message #11 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: Andi Kleen <andi <at> firstfloor.org>
Cc: 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 04 Jun 2020 19:26:34 +0900
On Wed, 3 Jun 2020 20:26:41 -0700
Andi Kleen <andi <at> firstfloor.org> wrote:

> 
> % grep --version
> grep (GNU grep) 3.4
> ...
> % echo -n > foo
> % grep -v foo foo ; echo $?
> 1
> 
> Would expect it to exit with zero in this case, since foo is not in the
> file.
> 
> When the file is one byte it works as expected:
> 
> % echo > foo
> % grep -v foo foo ; echo $?
> 
> 0
> %

`0 if a line is selected, 1 if no lines were selected,' in manual.
`0: One or more lines were selected. 1: No lines were selected.' in
POSIX.

> % echo -n > foo
> % grep -v foo foo ; echo $?

It returns no lines, so it seems reasonable to return 1.





Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 15:09:01 GMT) Full text and rfc822 format available.

Message #14 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Andi Kleen <andi <at> firstfloor.org>
To: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Cc: Andi Kleen <andi <at> firstfloor.org>, 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 07:06:55 -0700
On Thu, Jun 04, 2020 at 07:26:34PM +0900, Norihiro Tanaka wrote:
> 
> On Wed, 3 Jun 2020 20:26:41 -0700
> Andi Kleen <andi <at> firstfloor.org> wrote:
> 
> > 
> > % grep --version
> > grep (GNU grep) 3.4
> > ...
> > % echo -n > foo
> > % grep -v foo foo ; echo $?
> > 1
> > 
> > Would expect it to exit with zero in this case, since foo is not in the
> > file.
> > 
> > When the file is one byte it works as expected:
> > 
> > % echo > foo
> > % grep -v foo foo ; echo $?
> > 
> > 0
> > %
> 
> `0 if a line is selected, 1 if no lines were selected,' in manual.
> `0: One or more lines were selected. 1: No lines were selected.' in
> POSIX.
> 
> > % echo -n > foo
> > % grep -v foo foo ; echo $?
> 
> It returns no lines, so it seems reasonable to return 1.

I would argue that a empty file has a single empty line.

Besides it completely breaks the "is foo not in file" functionality.

In normal terminology that's a off by one bug. Why should 
an empty file be different than any other files?

I just had to rewrite a test script of mine and add lots of 

echo >> outputfile

statements to work around it.

-Andi





Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 19:20:02 GMT) Full text and rfc822 format available.

Message #17 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Andi Kleen <andi <at> firstfloor.org>, Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Cc: 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 12:19:14 -0700
On 6/4/20 7:06 AM, Andi Kleen wrote:
> I would argue that a empty file has a single empty line.

No, an empty file has no lines. A single empty line would be a file of size 1,
containing just a newline byte.

> Besides it completely breaks the "is foo not in file" functionality.

That's not what -v is for. -v asks "is not-FOO in file". If you want "is FOO not
in file", use -L.




Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 19:51:01 GMT) Full text and rfc822 format available.

Message #20 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Andi Kleen <andi <at> firstfloor.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Andi Kleen <andi <at> firstfloor.org>, Norihiro Tanaka <noritnk <at> kcn.ne.jp>,
 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 12:30:02 -0700
On Thu, Jun 04, 2020 at 12:19:14PM -0700, Paul Eggert wrote:
> On 6/4/20 7:06 AM, Andi Kleen wrote:
> > I would argue that a empty file has a single empty line.
> 
> No, an empty file has no lines. A single empty line would be a file of size 1,
> containing just a newline byte.

Just out of curiosity, in your definition would 

echo -n foo > file

(so no newline, but non zero length)

have one or zero lines?

> 
> > Besides it completely breaks the "is foo not in file" functionality.
> 
> That's not what -v is for. -v asks "is not-FOO in file". If you want "is FOO not
> in file", use -L.

Ok.

-Andi




Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 20:15:01 GMT) Full text and rfc822 format available.

Message #23 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Andi Kleen <andi <at> firstfloor.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 15:13:44 -0500
On 6/4/20 2:30 PM, Andi Kleen wrote:
> Just out of curiosity, in your definition would
> 
> echo -n foo > file

[side point: echo -n is not portable, printf is better]

> 
> (so no newline, but non zero length)
> 
> have one or zero lines?

Neither. Per POSIX, a non-empty file that does not end in newline is not 
a text file, and therefore POSIX says that tools like grep that are only 
documented to work on text files have unspecified behavior on such input.

The most intuitive behavior is that grep behaves as if the file included 
the trailing newline, in which case the output (if there is a match) 
actually displays a newline that was not present in the input, but there 
have also been historical grep that behave as if the file ended at the 
last newline, and ignore the trailing garbage even if it would have 
matched were a trailing newline present.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org





Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 20:28:01 GMT) Full text and rfc822 format available.

Message #26 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Andi Kleen <andi <at> firstfloor.org>
Cc: Norihiro Tanaka <noritnk <at> kcn.ne.jp>, 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 13:27:39 -0700
On 6/4/20 12:30 PM, Andi Kleen wrote:

> in your definition would 
> 
> echo -n foo > file
> 
> (so no newline, but non zero length)

No, the file has zero length:

$ echo -n >file
$ ls -l file
-rw-r--r-- 1 eggert eggert 0 Jun  4 13:24 file
> have one or zero lines?

Empty files have no lines.

On 6/4/20 1:13 PM, Eric Blake wrote:
> The most intuitive behavior is that grep behaves as if the file included the
> trailing newline

That's what grep does with files that end in a non-newline byte; such files are
also not text files so POSIX does not specify the behavior. But grep, like other
GNU tools, treats empty files as if they contain no lines; this matches most
people's intuition.




Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 20:35:01 GMT) Full text and rfc822 format available.

Message #29 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Andi Kleen <andi <at> firstfloor.org>
Cc: 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 15:34:18 -0500
On 6/4/20 3:27 PM, Paul Eggert wrote:
> On 6/4/20 12:30 PM, Andi Kleen wrote:
> 
>> in your definition would
>>
>> echo -n foo > file
>>
>> (so no newline, but non zero length)
> 
> No, the file has zero length:
> 
> $ echo -n >file

You missed the 'echo -n foo' that prints 3 bytes to file.

> $ ls -l file
> -rw-r--r-- 1 eggert eggert 0 Jun  4 13:24 file
>> have one or zero lines?
> 
> Empty files have no lines.
> 
> On 6/4/20 1:13 PM, Eric Blake wrote:
>> The most intuitive behavior is that grep behaves as if the file included the
>> trailing newline
> 
> That's what grep does with files that end in a non-newline byte; such files are
> also not text files so POSIX does not specify the behavior.

I answered the question about the 3-byte file.  The 0-byte file _IS_ a 
text file, consisting of zero lines.

> But grep, like other
> GNU tools, treats empty files as if they contain no lines; this matches most
> people's intuition.

POSIX actually requires this behavior for an empty file.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org





Information forwarded to bug-grep <at> gnu.org:
bug#41700; Package grep. (Thu, 04 Jun 2020 20:43:01 GMT) Full text and rfc822 format available.

Message #32 received at 41700 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eric Blake <eblake <at> redhat.com>, Andi Kleen <andi <at> firstfloor.org>
Cc: 41700 <at> debbugs.gnu.org
Subject: Re: bug#41700: grep -v always exiting with 1 for empty file
Date: Thu, 4 Jun 2020 13:42:12 -0700
On 6/4/20 1:34 PM, Eric Blake wrote:
>>> in your definition would
>>>
>>> echo -n foo > file
>>>
>>> (so no newline, but non zero length)
>>
>> No, the file has zero length:
>>
>> $ echo -n >file
> 
> You missed the 'echo -n foo' that prints 3 bytes to file.

Ah sorry, indeed I did. In that case my interpretation indeed agrees with yours.




Added tag(s) notabug. Request was from Paul Eggert <eggert <at> cs.ucla.edu> to control <at> debbugs.gnu.org. (Mon, 21 Sep 2020 19:33:02 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 41700 <at> debbugs.gnu.org and Andi Kleen <andi <at> firstfloor.org> Request was from Paul Eggert <eggert <at> cs.ucla.edu> to control <at> debbugs.gnu.org. (Mon, 21 Sep 2020 19:33:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Tue, 20 Oct 2020 11:24:03 GMT) Full text and rfc822 format available.

This bug report was last modified 3 years and 160 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.