GNU bug report logs - #43782
in egrep (grep 3.0), the order of include & exclude is surprisingly significant

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43782 in the body.
You can then email your comments to 43782 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Ruckstuhl <John.Ruckstuhl <at> intusurg.com>
To: "bug-grep <at> gnu.org" <bug-grep <at> gnu.org>
Subject: in egrep (grep 3.0), the order of include & exclude is surprisingly
 significant
Date: Sat, 3 Oct 2020 19:00:49 +0000

[Message part 1 (text/plain, inline)]

In egrep (grep 3.0), the order of options (multiple --include and --exclude), is surprisingly significant.
I was puzzled by 5 extra output lines in my first egrep,
so I did another egrep, with the --include moved up front.
Known bug?
Yes, I realize that the include by itself would have been sufficient, the exclude in this case is superfluous.
Thx,
John R.

Report bugs to: bug-grep <at> gnu.org
Report Cygwin bugs to: cygwin <at> cygwin.com
GNU grep home page: <http://www.gnu.org/software/grep/>
General help using GNU software: <http://www.gnu.org/gethelp/>

      johnru <at> ISIMFGDEV400 /cygdrive/c/Program Files (x86)/foxtrot/EC Test SW (750390-mango)
      $ egrep -r -i -n -l --exclude="*.bak" --include="*.py" "\.name\b|testname" ISITestFramework | cat -n
           1  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule.py
           2  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule_0.py
           3  ISITestFramework/DUT/EC/EC.py
           4  ISITestFramework/DUT/Endoscope/Endoscope.py
           5  ISITestFramework/DUT/GenericDUT.py
           6  ISITestFramework/DUT/LightEngine/LightEngine/LightEngine.py
      !!   7  ISITestFramework/HTML/BatFileUtils.html
      !!   8  ISITestFramework/HTML/Libs/jquery/1.12.2/jquery.min.js
           9  ISITestFramework/IQOQDQUtils.py
      !!  10  ISITestFramework/Reports/ReportTemplates/Libraries/GoogleChartsJsapi.js
      !!  11  ISITestFramework/Reports/ReportTemplates/Libraries/jquery-latest.min.js
      !!  12  ISITestFramework/Reports/ReportTemplates/TestResultsSimple.html
          13  ISITestFramework/Reports/ReportTools.py
          14  ISITestFramework/Reports/ReportUtils.py
          15  ISITestFramework/Reports/SummaryMetricWriter.py
          16  ISITestFramework/Test.py
          17  ISITestFramework/TestTools.py

      johnru <at> ISIMFGDEV400 /cygdrive/c/Program Files (x86)/foxtrot/EC Test SW (750390-mango)
      $ egrep -r -i -n -l --include="*.py" --exclude="*.bak" "\.name\b|testname" ISITestFramework | cat -n
           1  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule.py
           2  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule_0.py
           3  ISITestFramework/DUT/EC/EC.py
           4  ISITestFramework/DUT/Endoscope/Endoscope.py
           5  ISITestFramework/DUT/GenericDUT.py
           6  ISITestFramework/DUT/LightEngine/LightEngine/LightEngine.py
           7  ISITestFramework/IQOQDQUtils.py
           8  ISITestFramework/Reports/ReportTools.py
           9  ISITestFramework/Reports/ReportUtils.py
          10  ISITestFramework/Reports/SummaryMetricWriter.py
          11  ISITestFramework/Test.py
          12  ISITestFramework/TestTools.py

      johnru <at> ISIMFGDEV400 /cygdrive/c/Program Files (x86)/foxtrot/EC Test SW (750390-mango)
      $ egrep --version
      grep (GNU grep) 3.0
      Packaged by Cygwin (3.0-2)
      Copyright (C) 2017 Free Software Foundation, Inc.
      License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.

      Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

[Message part 2 (text/html, inline)]

Message #10 received at 43782-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: John Ruckstuhl <John.Ruckstuhl <at> intusurg.com>
Cc: 43782-done <at> debbugs.gnu.org
Subject: Re: bug#43782: in egrep (grep 3.0), the order of include & exclude is
 surprisingly significant
Date: Sat, 3 Oct 2020 12:44:22 -0700

[Message part 1 (text/plain, inline)]

On 10/3/20 12:00 PM, John Ruckstuhl wrote:
> In egrep (grep 3.0), the order of options (multiple --include and --exclude), is surprisingly significant.
> I was puzzled by 5 extra output lines in my first egrep,
> so I did another egrep, with the --include moved up front.

That's expected behavior, but it's not documented and should be. Thanks for 
bringing it up. I installed the attached doc patch.

[0001-doc-document-include-exclude-better.patch (text/x-patch, attachment)]

Message #13 received at 43782-done <at> debbugs.gnu.org (full text, mbox):

From: John Ruckstuhl <John.Ruckstuhl <at> intusurg.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "43782-done <at> debbugs.gnu.org" <43782-done <at> debbugs.gnu.org>
Subject: Re: bug#43782: in egrep (grep 3.0), the order of include & exclude is
 surprisingly significant
Date: Mon, 5 Oct 2020 15:18:17 +0000

> On 10/3/20 12:00 PM, John Ruckstuhl wrote:
> > In egrep (grep 3.0), the order of options (multiple --include and --exclude), is surprisingly significant.
> > I was puzzled by 5 extra output lines in my first egrep, so I did 
> > another egrep, with the --include moved up front.
>
> That's expected behavior, but it's not documented and should be. Thanks for bringing it up. I installed the attached > doc patch.

Thank you for the quick response.
I looked at the patch, and for several minutes I was still not comprehending it...
But I think I've got it now.

This
    $ egrep -r -i -n -l --exclude="*.bak" --include="*.py" "$PAT" "$DIR"
does not map to this
    find "$DIR" -type f ! -name "*.bak" -name "*.py" |
    while read F; do egrep -i -n -l "$PAT" "$F"; done

It maps to this (with INCFIRST representing "-true" or "-false")
     find "$DIR" -type f \( \
         \( ! -name "*.bak" ! -name "*.py" \) \( \
             ! "$INCFIRST" \( -true \) \
             -o \
             "$INCFIRST" \( ! -name "*.bak" -name "*.py" \) \
         \) \
         -o \
             \( ! -name "*.bak" -name "*.py" \) \
     \) |
    while read F; do egrep -i -n -l "$PAT" "$F"; done

Yikes.
As it is not yet documented,
I'll happily venmo you $50 to consider it a bug instead of an undocumented behavior.
Warm regards, Paul.  :)
John Ruckstuhl


johnru <at> ISIMFGDEV400 /cygdrive/c/Program Files (x86)/foxtrot/EC Test SW (750390-mango)
$ egrep -r -i -n -l --exclude="*.bak" --include="*.py" "\.name\b|testname" ISITestFramework | cat -n
     1  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule.py
     2  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule_0.py
     3  ISITestFramework/DUT/EC/EC.py
     4  ISITestFramework/DUT/Endoscope/Endoscope.py
     5  ISITestFramework/DUT/GenericDUT.py
     6  ISITestFramework/DUT/LightEngine/LightEngine/LightEngine.py
!!   7  ISITestFramework/HTML/BatFileUtils.html
!!   8  ISITestFramework/HTML/Libs/jquery/1.12.2/jquery.min.js
     9  ISITestFramework/IQOQDQUtils.py
!!  10  ISITestFramework/Reports/ReportTemplates/Libraries/GoogleChartsJsapi.js
!!  11  ISITestFramework/Reports/ReportTemplates/Libraries/jquery-latest.min.js
!!  12  ISITestFramework/Reports/ReportTemplates/TestResultsSimple.html
    13  ISITestFramework/Reports/ReportTools.py
    14  ISITestFramework/Reports/ReportUtils.py
    15  ISITestFramework/Reports/SummaryMetricWriter.py
    16  ISITestFramework/Test.py
    17  ISITestFramework/TestTools.py

johnru <at> ISIMFGDEV400 /cygdrive/c/Program Files (x86)/foxtrot/EC Test SW (750390-mango)
$ egrep -r -i -n -l --include="*.py" --exclude="*.bak" "\.name\b|testname" ISITestFramework | cat -n
     1  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule.py
     2  ISITestFramework/DUT/CameraModule/CameraModule/CameraModule_0.py
     3  ISITestFramework/DUT/EC/EC.py
     4  ISITestFramework/DUT/Endoscope/Endoscope.py
     5  ISITestFramework/DUT/GenericDUT.py
     6  ISITestFramework/DUT/LightEngine/LightEngine/LightEngine.py
     7  ISITestFramework/IQOQDQUtils.py
     8  ISITestFramework/Reports/ReportTools.py
     9  ISITestFramework/Reports/ReportUtils.py
    10  ISITestFramework/Reports/SummaryMetricWriter.py
    11  ISITestFramework/Test.py
    12  ISITestFramework/TestTools.py

this is the same logic:
$ for INCFIRST in -true -false; do
>     printf "%s: %s\n" "\$INCFIRST" "$INCFIRST"
>
>     # if no --incude or --exclude options match,
>     #   if not INCFIRST then file is included
>     #   if INCFIRST then file is not included
>     find "$DIR" -type f \( \
>         \( ! -name "*.bak" ! -name "*.py" \) \( \
>             ! "$INCFIRST" -true \
>             -o \
>             "$INCFIRST" \( ! -name "*.bak" -name "*.py" \) \
>         \) \
>         -o \
>             \( ! -name "*.bak" -name "*.py" \) \
>     \) |
>     # egrep "\.(html|js)$"
>     G | wc -l
> done
$INCFIRST: -true
12
$INCFIRST: -false
17

Message #16 received at 43782-done <at> debbugs.gnu.org (full text, mbox):

From: John Ruckstuhl <John.Ruckstuhl <at> intusurg.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "43782-done <at> debbugs.gnu.org" <43782-done <at> debbugs.gnu.org>
Subject: RE: bug#43782: in egrep (grep 3.0), the order of include & exclude is
 surprisingly significant
Date: Mon, 5 Oct 2020 15:57:32 +0000

> This
>     $ egrep -r -i -n -l --exclude="*.bak" --include="*.py" "$PAT" "$DIR"
> does not map to this
>     find "$DIR" -type f ! -name "*.bak" -name "*.py" |
>     while read F; do egrep -i -n -l "$PAT" "$F"; done
>
> It maps to this (with INCFIRST representing "-true" or "-false")
>      find "$DIR" -type f \( \
>          \( ! -name "*.bak" ! -name "*.py" \) \( \
>              ! "$INCFIRST" \( -true \) \
>              -o \
>              "$INCFIRST" \( ! -name "*.bak" -name "*.py" \) \
>          \) \
>          -o \
>              \( ! -name "*.bak" -name "*.py" \) \
>      \) |
>     while read F; do egrep -i -n -l "$PAT" "$F"; done
>
> Yikes.
> As it is not yet documented,
> I'll happily venmo you $50 to consider it a bug instead of an undocumented behavior.
> Warm regards, Paul.  :)
> John Ruckstuhl

okay, the above mess simplifies of course to something less yikesy,
    find "$DIR" -type f \( \
        \( ! -name "*.bak" -name "*.py" \) \
        -o ! "$INCFIRST" \( ! -name "*.bak" ! -name "*.py" \) \
    \) |
    while read F; do egrep -i -n -l "$PAT" "$F"; done

but the $50 offer still stands... :)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.