GNU bug report logs - #32943
grep pattern < fifo fails on Cygwin (and MinGW), but not on Linux

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32943 in the body.
You can then email your comments to 32943 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: bug-grep <at> gnu.org
Subject: grep pattern < fifo fails on Cygwin (and MinGW), but not on Linux
Date: Fri, 05 Oct 2018 12:02:47 +0200

Hi,

Yes, I might not be following "procedure" (I do not know), but I decided
to report this failure, because I canNOT ascertain whether "the list" is
aware of this failure or not.

grep fails on Cygwin (and apparently also on MinGW), as follows:

@@ mkfifo fifo
@@ echo X > fifo & grep . < fifo # X may be omitted
[1] <some procid>
grep: (standard input): Invalid argument
[1]+  Done                    echo > fifo

This failure does not occur on Linux.

As far as I can tell, the maintainer of grep on Cygwin, hardly modifies
the source code of grep. Therefore I surmise? that the original tarball
must be in error.

I applied strace to "grep . < fifo", and as far as I can tell "an lseek
is applied to the fifo", which obviously fails.

The "lseek" occurs on Cygwin (and MinGW?), but not on Linux (as far as
I can tell, the code that is executed on Cygwin is not the same as the
one that is executed on Linux).

The failure does not occur if e.g. cut is substituted for grep.

@@ mkfifo fifo
@@ echo X > fifo & cut -f1 < fifo # X may be omitted
No error ...

Q: are you aware of this failure?

Regards,
Henri

Message #8 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: 32943 <at> debbugs.gnu.org
Subject: Output from strace on Cygwin
Date: Fri, 05 Oct 2018 16:39:48 +0200

[Message part 1 (text/plain, inline)]

Hi,

See top of file (strace.attach2pid.grep) to learn how I applied
strace.

Regards,
Henri

[strace.attach2pid.grep (text/plain, attachment)]

Message #11 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Houder <houder <at> xs4all.nl>, 32943 <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Sat, 6 Oct 2018 13:12:38 -0700

Houder wrote:
> I applied strace to "grep . < fifo", and as far as I can tell "an lseek
> is applied to the fifo", which obviously fails.
> 
> The "lseek" occurs on Cygwin (and MinGW?), but not on Linux

That's expected, as the code needs to run lseek in some places on Cygwin where 
it's not needed on GNU/Linux.

My guess is that you've got an old version of grep on Cygwin, and need to 
upgrade to the current version. If you still have problems with the latest 
version, please investigate via GDB and/or other means exactly why grep is 
failing for you. The strace output isn't enough, unfortunately.

Message #14 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: 32943 <at> debbugs.gnu.org
Cc: Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Sat, 06 Oct 2018 23:03:04 +0200

On 2018-10-06 22:12, Paul Eggert wrote:
> Houder wrote:
>> I applied strace to "grep . < fifo", and as far as I can tell "an 
>> lseek
>> is applied to the fifo", which obviously fails.
>> 
>> The "lseek" occurs on Cygwin (and MinGW?), but not on Linux
> 
> That's expected, as the code needs to run lseek in some places on
> Cygwin where it's not needed on GNU/Linux.
> 
> My guess is that you've got an old version of grep on Cygwin, and need
> to upgrade to the current version. If you still have problems with the
> latest version, please investigate via GDB and/or other means exactly
> why grep is failing for you. The strace output isn't enough,
> unfortunately.

Hi Paul,

Thanks for the reply! (I know now that "the list" is aware my post :-)

(and yes, in the mean time I have found plenty of URLs pointing to the
 project and the bug-tracker)

My reply is to the list, CC to you. (or should I only reply to the 
list?)

Old version? The version used by Cygwin is based on 3.0. As far as I can
tell, the most recent tarball is 3.1 ?????

I did investigate using gdb and had a hard time getting a useful result
because of the indirection (grep . < fifo).

After searching the internet, I proceeded as follows:

@@ gdb /usr/bin/grep

This also loads the symbol table on Cygwin.

(gdb) set args .
(gdb) br main
(gdb) run

GDB stops at main()

(gdb) p dup2(open("fifo", 0), 0)

Next I execute "echo aaa > fifo" from another terminal; this makes GDB
return to the prompt (the prompt did not return after the above dup2()).

Subsequently I put a breakpoint on suppressible_error().

When the excution hits the breakpoint, I got this stack trace:

(gdb) bt
#0  suppressible_error (errnum=22) at 
/usr/src/debug/grep-3.0-2/src/grep.c:595
#1  0x0000000100403f05 in grep (ineof=<synthetic pointer>, 
st=0xffffc890, fd=0) at /usr/src/debug/grep-3.0-2/src/grep.c:1480
#2  grepdesc (desc=desc <at> entry=0, command_line=command_line <at> entry=true) 
at /usr/src/debug/grep-3.0-2/src/grep.c:1875
#3  0x0000000100428f74 in grep_command_line_arg (arg=<optimized out>) at 
/usr/src/debug/grep-3.0-2/src/grep.c:1915
#4  main (argc=<optimized out>, argv=<optimized out>) at 
/usr/src/debug/grep-3.0-2/src/grep.c:294

This "translates" to the following callflow:

main
  grep_command_line_arg
    grepdesc
      grep
        reset # appears to return true ...
        fillbuf # (1st call in grep() ) appears to return false
          suppressible_error

However I am not sure of this result; when I repeat the procedure and 
put a
breakpoint on fillbuf(), the execution does not break on fillbuf().

(trouble is I am not "set up" for building on Cygwin; otherwise I would 
build
 my "own" (not optimized) version of grep)

Regards,
Henri

Message #17 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Houder <houder <at> xs4all.nl>, 32943 <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Sat, 6 Oct 2018 16:45:46 -0700

Houder wrote:

> Old version? The version used by Cygwin is based on 3.0. As far as I can
> tell, the most recent tarball is 3.1 ?????

Yes, that's the most recent release.

When building grep 3.1 for use with GDB, I suggest turning off optimization. 
Something like this:

make clean
make CFLAGS='-g3'

> I did investigate using gdb and had a hard time getting a useful result
> because of the indirection (grep . < fifo).

Try running grep as follows

$ gdb src/grep
...
(gdb) run . < fifo

> (gdb) bt
> #0  suppressible_error (errnum=22) at /usr/src/debug/grep-3.0-2/src/grep.c:595
...
That stack trace is coming from grep 3.0. Please debug grep 3.1 instead.

Message #20 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: 32943 <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Sun, 07 Oct 2018 10:23:41 +0200

On 2018-10-07 01:45, Paul Eggert wrote:
[snip]

> Try running grep as follows
> 
> $ gdb src/grep
> ...
> (gdb) run . < fifo

For those reading this thread:

 - 
https://stackoverflow.com/questions/9031554/gdb-input-redirection-using-cygwin
   ( gdb input redirection using cygwin )

    "It seems that input redirection in gdb does not work in Cygwin e.g

     (gdb) run < input.txt

     Is there other way to redirect input in gdb of Cygwin?"

-----

Message #23 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: 32943 <at> debbugs.gnu.org
Cc: Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Mon, 08 Oct 2018 09:35:40 +0200

[Message part 1 (text/plain, inline)]

On 2018-10-06 22:12, Paul Eggert wrote:
[snip}

> My guess is that you've got an old version of grep on Cygwin, and need
> to upgrade to the current version. If you still have problems with the
> latest version, please investigate via GDB and/or other means exactly
> why grep is failing for you ...

Hi Paul,

I have found what causes grep to fail. (after building v3.1 of grep)

But in order not to mislead others reading this thread, I must correct 
what I
wrote in my initial post. There I wrote: "as far as I can tell, the code 
that
is executed on Cygwin is not the same as the one that is executed on 
Linux".

Wrong. My "strace-ing" misled me.

Returning to grep's failure on Gygwin:

grep terminates with an error on Cygwin because lseek() on Cygwin fails 
to
ascertain that it is applied to a FIFO.

As result Cygwin's executive returns EINVAL in errno, in stead of 
ESPIPE, as
required (my interpretation after reading "man 2 lseek").

ESPIPE is expected in reset() (src/grep.c), after lseek() has been 
applied to
stdin (which points to a FIFO here).

Until v2.26 of grep, a call to S_ISREG(st->st_mode) circumvented the 
call to
lseek() (and the call to suppressible_error()) and made reset() return 
true.

Version v2.27 of grep removed the call to S_ISREG(st->st_mode). As 
result of
that lseek() is invoked.

lseek() fails, but an exception is made for errno == ESPIPE. For this 
reason
grep does not fail on Linux (reset() returns true).

However on Cygwin, suppressible_error() is called and reset() returns 
false,
which makes grep fail on Cygwin.

You can see for yourself in the attached logfiles (GDB sessions).

-----
To what did I apply GDB?

 - downloaded the tarball (grep-3.1.tar.xz) from 
http://ftp.gnu.org/gnu/grep/
 - build grep using "make CFLAG='-O -g3'", both on Linux and Cygwin

Regards,
Henri

[gdb-cygwin.log (text/plain, attachment)]

[gdb-linux.log (text/plain, attachment)]

Message #26 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Houder <houder <at> xs4all.nl>, 32943 <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Mon, 8 Oct 2018 08:49:24 -0700

On 10/8/18 12:35 AM, Houder wrote:
> grep terminates with an error on Cygwin because lseek() on Cygwin 
> fails to
> ascertain that it is applied to a FIFO. 


That sounds like a bug in Cygwin, then. Can you find out exactly why it 
occurs, by using GDB within Cygwin? Then you can file a bug report with 
the Cygwin folks.

Possibly the bug is something like the following: grep calls lseek (fd, 
0, SEEK_CUR). The Cygwin lseek implementation asks MS-Windows what the 
file offset is, gets an error because the file descriptor is that of a 
pipe, and somehow turns that into EINVAL. Cygwin should turn it into 
ESPIPE instead.

Message #29 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: 32943 <at> debbugs.gnu.org
Cc: Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Mon, 08 Oct 2018 20:10:35 +0200

On 2018-10-08 17:49, Paul Eggert wrote:
> On 10/8/18 12:35 AM, Houder wrote:
>> grep terminates with an error on Cygwin because lseek() on Cygwin 
>> fails to
>> ascertain that it is applied to a FIFO.
> 
> 
> That sounds like a bug in Cygwin, then. Can you find out exactly why
> it occurs, by using GDB within Cygwin? Then you can file a bug report
> with the Cygwin folks.

Yes, it _is_ a bug in cywin1.dll (i.e. the executive). No, I cannot find
out _exactly_ why it occurs (see below). A "bug report" has been filed,

Cygwin has no official bug tracker. A bug is reported on the 
mailinglist.

I have reported my findings here:

    https://cygwin.com/ml/cygwin/2018-10/msg00062.html

E. Blake, one of members on your list, has already acknowledged that the
Cygwin executive must be fixed.

> Possibly the bug is something like the following: grep calls lseek
> (fd, 0, SEEK_CUR). The Cygwin lseek implementation asks MS-Windows
> what the file offset is, gets an error because the file descriptor is
> that of a pipe, and somehow turns that into EINVAL. Cygwin should turn
> it into ESPIPE instead.

Cygwin has been written in C++ (it would take me a very long time to get
familiar with the implementation).

lseek() is implemented in winsup/cygwin/syscalls.cc. The "real" lseek()
however is implemented in one of the many "handlers", depending on the
device type on which lseek() is applied (I am guessing).

Apparently, "nothing" has been implemented for a fifo, and as result
of that the basic handler fhandler_base::lseek() in

    winsup/cygwin/fhandler.cc (1031) is invoked.

Here NtQueryInformationFile() is called, which fails with

ERROR_INVALID_PARAMETER (87), which Cygwin turns into EINVAL (22).

We have to wait until Corinna Vinschen returns from holidays (end of
october), because Corinna Vinschen is the only one who can deal with
the executive.

Regards,
Henri

Message #32 received at 32943 <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: 32943 <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Tue, 09 Oct 2018 13:29:34 +0200

On 2018-10-08 20:10, Houder wrote:
[snip]

> Here NtQueryInformationFile() is called, which fails with

s/ NtQueryInformationFile / NtSetInformationFile /

Set! Not Query.

> ERROR_INVALID_PARAMETER (87), which Cygwin turns into EINVAL (22).
> 
> We have to wait until Corinna Vinschen returns from holidays (end of
> october), because Corinna Vinschen is the only one who can deal with
> the executive.

correcting myself in order NOT to submit INcorrect info to the list.

-----

Message #37 received at 32943-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Houder <houder <at> xs4all.nl>
Cc: 32943-done <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Wed, 1 Jan 2020 17:09:45 -0800

On 10/8/18 11:10 AM, Houder wrote:
> We have to wait until Corinna Vinschen returns from holidays (end of
> october), because Corinna Vinschen is the only one who can deal with
> the executive.

Corinna later pushed a patch for this Cygwin bug <https://cygwin.com/ml/cygwin/2018-10/msg00087.html> so I'm closing this old grep bug report.

Message #40 received at 32943-done <at> debbugs.gnu.org (full text, mbox):

From: Houder <houder <at> xs4all.nl>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 32943-done <at> debbugs.gnu.org
Subject: Re: bug#32943: grep pattern < fifo fails on Cygwin (and MinGW), but
 not on Linux
Date: Thu, 02 Jan 2020 16:09:11 +0100

On 2020-01-02 02:09, Paul Eggert wrote:
> On 10/8/18 11:10 AM, Houder wrote:
>> We have to wait until Corinna Vinschen returns from holidays (end of
>> october), because Corinna Vinschen is the only one who can deal with
>> the executive.
> 
> Corinna later pushed a patch for this Cygwin bug
> <https://cygwin.com/ml/cygwin/2018-10/msg00087.html> so I'm closing
> this old grep bug report.

Agreed! Correct! Thank you.

(if I am not supposed to respond to this e-mail, then my apologies)

Regards,
Henri

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.