GNU bug report logs - #18892
few test failure with 'grep-2.20.72-d512'

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18892 in the body.
You can then email your comments to 18892 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: bug-grep <at> gnu.org
Subject: few test failure with 'grep-2.20.72-d512'
Date: Wed, 29 Oct 2014 16:51:51 -0400

[Message part 1 (text/plain, inline)]

Hello,

On 10/29/2014 02:29 PM, Jim Meyering wrote:
> Thanks to many fixes and improvements by Paul Eggert and Norihiro Tanaka,
> here is a pre-release snapshot:
>
> grep snapshot:
>    http://meyering.net/grep/grep-ss.tar.xz      1.2 MB
>    http://meyering.net/grep/grep-ss.tar.xz.sig
>    http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>

On Debian 7.6 (amd64, gcc 4.7.2) and Ubuntu 14.04.1 (amd64, gcc 4.8.2), these fail:
  XFAIL: triple-backref
  FAIL: word-multibyte
Attached is the debian76.test-suite.log from Debian (Ubuntu seemed identical).

---

On OpenBSD5.5 (amd64, gcc 4.2.1) these failed:
  XFAIL: equiv-classes
  FAIL: word-multibyte
Attached is the openbsd55.test-suite.log.

---

On CentOS 7 (amd64, gcc 4.8.2),
   GNU Hurd (x86, gcc 4.8.2),
   OpenSUSE 13.1 (amd64, gcc 4.8.1)
this fails:
  XFAIL: triple-backref

Attached are the logs.

---

On FreeBSD 9.3 (amd64, gcc 4.2.1), this fails:
===
XFAIL: equiv-classes
====================

<C3><A9>
XFAIL equiv-classes (exit status: 1)
===

---

On NetBSD 6.1.4, these fail:
  XFAIL: equiv-classes
  FAIL: symlink
  FAIL: word-multibyte
Attached is netbsd614.test-suite.log

---

No failures on:
  FreeBSD 10 (amd64, clang 3.3)
  CentOS 6.5 (amd64, gcc 4.4.7)
  Fedora 20 (amd64, gcc 4.8.3)


Regards,
 - Assaf

[debian76.test-suite2.log (text/x-log, attachment)]

[openbsd55.test-suite.log (text/x-log, attachment)]

[netbsd614.test-suite.log (text/x-log, attachment)]

[centos7.test-suite.log (text/x-log, attachment)]

[hurd05.test-suite.log (text/x-log, attachment)]

[opensuse131.test-suite.log (text/x-log, attachment)]

Message #8 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Bruce Dubbs <bruce.dubbs <at> gmail.com>
To: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Wed, 29 Oct 2014 16:56:14 -0500

Assaf Gordon wrote:
> Hello,
>
> On 10/29/2014 02:29 PM, Jim Meyering wrote:
>> Thanks to many fixes and improvements by Paul Eggert and Norihiro Tanaka,
>> here is a pre-release snapshot:
>>
>> grep snapshot:
>>    http://meyering.net/grep/grep-ss.tar.xz      1.2 MB
>>    http://meyering.net/grep/grep-ss.tar.xz.sig
>>    http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>>
>
> On Debian 7.6 (amd64, gcc 4.7.2) and Ubuntu 14.04.1 (amd64, gcc 4.8.2),
> these fail:
>    XFAIL: triple-backref
>    FAIL: word-multibyte
> Attached is the debian76.test-suite.log from Debian (Ubuntu seemed
> identical).

Linux From Scratch 7.6

configure and make were clean.

$ env RUN_EXPENSIVE_TESTS=yes make -k check

big-match: skipped test: not enough main memory to run the test
sjis-mb: skipped test: SJIS locale not found

I do have 8G of memory.

XFAIL: triple-backref

SKIP: test-vc-list-files-cvs.sh  (cvs not present)
SKIP: test-wcrtomb-w32-1.sh      (windows only?)
SKIP: test-wcrtomb-w32-2.sh
SKIP: test-wcrtomb-w32-3.sh
SKIP: test-wcrtomb-w32-4.sh
SKIP: test-wcrtomb-w32-5.sh
FAIL: word-multibyte

Looking at below, the FAIL may be because I have the locale 
zh_CN.gb18030, but not a plain zh_CN locale.

And confirming after installing a plain zh_CN locale, word-multibyte passes.

  -- Bruce

====================

++ initial_cwd_=/usr/src/grep/grep-2.20.72-d512/tests
++ fail=0
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /usr/src/grep/grep-2.20.72-d512/tests gt-word-multibyte.XXXX
+++ case $# in
+++ destdir_=/usr/src/grep/grep-2.20.72-d512/tests
+++ template_=gt-word-multibyte.XXXX
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ case $template_ in
++++ unset TMPDIR
+++ d=/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
+++ case $d in
+++ test -d /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
++++ ls -dgo /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
++++ tr S -
+++ perms='drwx------ 2 4096 Oct 29 16:24 
/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7'
+++ case $perms in
+++ test 0 = 0
+++ echo /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
+++ return
++ test_dir_=/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
++ cd /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
++ gl_init_sh_nl_='
'
++ IFS='
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ ../src
+ test 1 '!=' 0
+ path_dir_=../src
+ case $path_dir_ in
+ abs_path_dir_=/usr/src/grep/grep-2.20.72-d512/tests/../src
+ case $abs_path_dir_ in
+ 
PATH=/usr/src/grep/grep-2.20.72-d512/tests/../src:/usr/src/grep/grep-2.20.72-d512/src:/usr/local/bin:/bin:/usr/bin:/opt/texlive/2014/bin/x86_64-linux:/opt/fop:/opt/libreoffice-4.3.2.2/bin:/opt/OpenJDK-1.7.0.65-bin/bin:/opt/ant/bin:/opt/kde/bin:/opt/qt4/bin:/opt/qt5/bin:/opt/xorg/bin:/home/bdubbs/bin:/sbin:/usr/sbin
+ create_exe_shims_ /usr/src/grep/grep-2.20.72-d512/tests/../src
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ require_en_utf8_locale_
+ path_prepend_ .
+ test 1 '!=' 0
+ path_dir_=.
+ case $path_dir_ in
+ abs_path_dir_=/usr/src/grep/grep-2.20.72-d512/tests/.
+ case $abs_path_dir_ in
+ 
PATH=/usr/src/grep/grep-2.20.72-d512/tests/.:/usr/src/grep/grep-2.20.72-d512/tests/../src:/usr/src/grep/grep-2.20.72-d512/src:/usr/local/bin:/bin:/usr/bin:/opt/texlive/2014/bin/x86_64-linux:/opt/fop:/opt/libreoffice-4.3.2.2/bin:/opt/OpenJDK-1.7.0.65-bin/bin:/opt/ant/bin:/opt/kde/bin:/opt/qt4/bin:/opt/qt5/bin:/opt/xorg/bin:/home/bdubbs/bin:/sbin:/usr/sbin
+ create_exe_shims_ /usr/src/grep/grep-2.20.72-d512/tests/.
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ case $(get-mb-cur-max en_US.UTF-8) in
++ get-mb-cur-max en_US.UTF-8
+ printf '\xc3\xa1\n'
+ LC_ALL=en_US.UTF-8
+ export LC_ALL
+ fail=0
+ for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
+ out=out1-en_US.UTF-8
+ LC_ALL=en_US.UTF-8
+ grep '\w' in
+ compare in out1-en_US.UTF-8
+ compare_dev_null_ in out1-en_US.UTF-8
+ test 2 = 2
+ test xin = x/dev/null
+ test xout1-en_US.UTF-8 = x/dev/null
+ return 2
+ case $? in
+ compare_ in out1-en_US.UTF-8
+ diff -u in out1-en_US.UTF-8
+ out=out2-en_US.UTF-8
+ LC_ALL=en_US.UTF-8
+ grep '\W' in
+ compare /dev/null out2-en_US.UTF-8
+ compare_dev_null_ /dev/null out2-en_US.UTF-8
+ test 2 = 2
+ test x/dev/null = x/dev/null
+ test -s out2-en_US.UTF-8
+ return 0
+ return 0
+ for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
+ out=out1-zh_CN
+ LC_ALL=zh_CN
+ grep '\w' in
+ fail=1
+ compare in out1-zh_CN
+ compare_dev_null_ in out1-zh_CN
+ test 2 = 2
+ test xin = x/dev/null
+ test xout1-zh_CN = x/dev/null
+ return 2
+ case $? in
+ compare_ in out1-zh_CN
 diff -u in out1-zh_CN
--- in   2014-10-29 16:24:46.775542037 -0500
+++ out1-zh_CN 2014-10-29 16:24:46.783541998 -0500
@@ -1 +0,0 @@
-á
+ fail=1
+ out=out2-zh_CN
+ LC_ALL=zh_CN
+ grep '\W' in
+ fail=1
+ compare /dev/null out2-zh_CN
+ compare_dev_null_ /dev/null out2-zh_CN
+ test 2 = 2
+ test x/dev/null = x/dev/null
+ test -s out2-zh_CN
+ emit_diff_u_header_ /dev/null out2-zh_CN
+ printf '%s\n' 'diff -u /dev/null out2-zh_CN' '--- /dev/null 
1970-01-01' '+++ out2-zh_CN   1970-01-01'
diff -u /dev/null out2-zh_CN
--- /dev/null  1970-01-01
+++ out2-zh_CN 1970-01-01
+ sed 's/^/+/' out2-zh_CN
+á
+ return 1
+ case $? in
+ return 1
+ fail=1
+ for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
+ out=out1-fr_FR.UTF-8
+ LC_ALL=fr_FR.UTF-8
+ grep '\w' in
+ compare in out1-fr_FR.UTF-8
+ compare_dev_null_ in out1-fr_FR.UTF-8
+ test 2 = 2
+ test xin = x/dev/null
+ test xout1-fr_FR.UTF-8 = x/dev/null
+ return 2
+ case $? in
+ compare_ in out1-fr_FR.UTF-8
+ diff -u in out1-fr_FR.UTF-8
+ out=out2-fr_FR.UTF-8
+ LC_ALL=fr_FR.UTF-8
+ grep '\W' in
+ compare /dev/null out2-fr_FR.UTF-8
+ compare_dev_null_ /dev/null out2-fr_FR.UTF-8
+ test 2 = 2
+ test x/dev/null = x/dev/null
+ test -s out2-fr_FR.UTF-8
+ return 0
+ return 0
+ Exit 1
+ set +e
+ exit 1
+ exit 1
+ remove_tmp_
+ __st=1
+ cleanup_
+ :
+ cd /usr/src/grep/grep-2.20.72-d512/tests
+ chmod -R u+rwx 
/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
+ rm -rf /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
+ exit 1
FAIL word-multibyte (exit status: 1)

Message #11 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruce Dubbs <bruce.dubbs <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Thu, 30 Oct 2014 11:55:54 -0700

[Message part 1 (text/plain, inline)]

Thank you both for all the testing and prompt feedback!
I've looked into the failures of the brand new word-multibyte test,
and am hoping the attached will fix it by requiring use of
the zh_CN.UTF-8 locale, rather than letting the system
choose a locale matching "zh_CN".  Does this patch solve
the problem for that test?

On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
> Assaf Gordon wrote:
>>
>> Hello,
>>
>> On 10/29/2014 02:29 PM, Jim Meyering wrote:
>>>
>>> Thanks to many fixes and improvements by Paul Eggert and Norihiro Tanaka,
>>> here is a pre-release snapshot:
>>>
>>> grep snapshot:
>>>    http://meyering.net/grep/grep-ss.tar.xz      1.2 MB
>>>    http://meyering.net/grep/grep-ss.tar.xz.sig
>>>    http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>>>
>>
>> On Debian 7.6 (amd64, gcc 4.7.2) and Ubuntu 14.04.1 (amd64, gcc 4.8.2),
>> these fail:
>>    XFAIL: triple-backref
>>    FAIL: word-multibyte
>> Attached is the debian76.test-suite.log from Debian (Ubuntu seemed
>> identical).
>
>
> Linux From Scratch 7.6
>
> configure and make were clean.
>
> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>
> big-match: skipped test: not enough main memory to run the test
> sjis-mb: skipped test: SJIS locale not found
>
> I do have 8G of memory.
>
> XFAIL: triple-backref
>
> SKIP: test-vc-list-files-cvs.sh  (cvs not present)
> SKIP: test-wcrtomb-w32-1.sh      (windows only?)
> SKIP: test-wcrtomb-w32-2.sh
> SKIP: test-wcrtomb-w32-3.sh
> SKIP: test-wcrtomb-w32-4.sh
> SKIP: test-wcrtomb-w32-5.sh
> FAIL: word-multibyte
>
> Looking at below, the FAIL may be because I have the locale zh_CN.gb18030,
> but not a plain zh_CN locale.
>
> And confirming after installing a plain zh_CN locale, word-multibyte passes.
>
>   -- Bruce
>
> ====================
>
> ++ initial_cwd_=/usr/src/grep/grep-2.20.72-d512/tests
> ++ fail=0
> +++ testdir_prefix_
> +++ printf gt
> ++ pfx_=gt
> +++ mktempd_ /usr/src/grep/grep-2.20.72-d512/tests gt-word-multibyte.XXXX
> +++ case $# in
> +++ destdir_=/usr/src/grep/grep-2.20.72-d512/tests
> +++ template_=gt-word-multibyte.XXXX
> +++ MAX_TRIES_=4
> +++ case $destdir_ in
> +++ case $template_ in
> ++++ unset TMPDIR
> +++ d=/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> +++ case $d in
> +++ test -d /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> ++++ ls -dgo /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> ++++ tr S -
> +++ perms='drwx------ 2 4096 Oct 29 16:24
> /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7'
> +++ case $perms in
> +++ test 0 = 0
> +++ echo /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> +++ return
> ++ test_dir_=/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> ++ cd /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> ++ gl_init_sh_nl_='
> '
> ++ IFS='
> '
> ++ for sig_ in 1 2 3 13 15
> +++ expr 1 + 128
> ++ eval 'trap '\''Exit 129'\'' 1'
> +++ trap 'Exit 129' 1
> ++ for sig_ in 1 2 3 13 15
> +++ expr 2 + 128
> ++ eval 'trap '\''Exit 130'\'' 2'
> +++ trap 'Exit 130' 2
> ++ for sig_ in 1 2 3 13 15
> +++ expr 3 + 128
> ++ eval 'trap '\''Exit 131'\'' 3'
> +++ trap 'Exit 131' 3
> ++ for sig_ in 1 2 3 13 15
> +++ expr 13 + 128
> ++ eval 'trap '\''Exit 141'\'' 13'
> +++ trap 'Exit 141' 13
> ++ for sig_ in 1 2 3 13 15
> +++ expr 15 + 128
> ++ eval 'trap '\''Exit 143'\'' 15'
> +++ trap 'Exit 143' 15
> ++ trap remove_tmp_ 0
> + path_prepend_ ../src
> + test 1 '!=' 0
> + path_dir_=../src
> + case $path_dir_ in
> + abs_path_dir_=/usr/src/grep/grep-2.20.72-d512/tests/../src
> + case $abs_path_dir_ in
> +
> PATH=/usr/src/grep/grep-2.20.72-d512/tests/../src:/usr/src/grep/grep-2.20.72-d512/src:/usr/local/bin:/bin:/usr/bin:/opt/texlive/2014/bin/x86_64-linux:/opt/fop:/opt/libreoffice-4.3.2.2/bin:/opt/OpenJDK-1.7.0.65-bin/bin:/opt/ant/bin:/opt/kde/bin:/opt/qt4/bin:/opt/qt5/bin:/opt/xorg/bin:/home/bdubbs/bin:/sbin:/usr/sbin
> + create_exe_shims_ /usr/src/grep/grep-2.20.72-d512/tests/../src
> + case $EXEEXT in
> + return 0
> + shift
> + test 0 '!=' 0
> + export PATH
> + require_en_utf8_locale_
> + path_prepend_ .
> + test 1 '!=' 0
> + path_dir_=.
> + case $path_dir_ in
> + abs_path_dir_=/usr/src/grep/grep-2.20.72-d512/tests/.
> + case $abs_path_dir_ in
> +
> PATH=/usr/src/grep/grep-2.20.72-d512/tests/.:/usr/src/grep/grep-2.20.72-d512/tests/../src:/usr/src/grep/grep-2.20.72-d512/src:/usr/local/bin:/bin:/usr/bin:/opt/texlive/2014/bin/x86_64-linux:/opt/fop:/opt/libreoffice-4.3.2.2/bin:/opt/OpenJDK-1.7.0.65-bin/bin:/opt/ant/bin:/opt/kde/bin:/opt/qt4/bin:/opt/qt5/bin:/opt/xorg/bin:/home/bdubbs/bin:/sbin:/usr/sbin
> + create_exe_shims_ /usr/src/grep/grep-2.20.72-d512/tests/.
> + case $EXEEXT in
> + return 0
> + shift
> + test 0 '!=' 0
> + export PATH
> + case $(get-mb-cur-max en_US.UTF-8) in
> ++ get-mb-cur-max en_US.UTF-8
> + printf '\xc3\xa1\n'
> + LC_ALL=en_US.UTF-8
> + export LC_ALL
> + fail=0
> + for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
> + out=out1-en_US.UTF-8
> + LC_ALL=en_US.UTF-8
> + grep '\w' in
> + compare in out1-en_US.UTF-8
> + compare_dev_null_ in out1-en_US.UTF-8
> + test 2 = 2
> + test xin = x/dev/null
> + test xout1-en_US.UTF-8 = x/dev/null
> + return 2
> + case $? in
> + compare_ in out1-en_US.UTF-8
> + diff -u in out1-en_US.UTF-8
> + out=out2-en_US.UTF-8
> + LC_ALL=en_US.UTF-8
> + grep '\W' in
> + compare /dev/null out2-en_US.UTF-8
> + compare_dev_null_ /dev/null out2-en_US.UTF-8
> + test 2 = 2
> + test x/dev/null = x/dev/null
> + test -s out2-en_US.UTF-8
> + return 0
> + return 0
> + for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
> + out=out1-zh_CN
> + LC_ALL=zh_CN
> + grep '\w' in
> + fail=1
> + compare in out1-zh_CN
> + compare_dev_null_ in out1-zh_CN
> + test 2 = 2
> + test xin = x/dev/null
> + test xout1-zh_CN = x/dev/null
> + return 2
> + case $? in
> + compare_ in out1-zh_CN
>  diff -u in out1-zh_CN
> --- in   2014-10-29 16:24:46.775542037 -0500
> +++ out1-zh_CN 2014-10-29 16:24:46.783541998 -0500
> @@ -1 +0,0 @@
> -á
> + fail=1
> + out=out2-zh_CN
> + LC_ALL=zh_CN
> + grep '\W' in
> + fail=1
> + compare /dev/null out2-zh_CN
> + compare_dev_null_ /dev/null out2-zh_CN
> + test 2 = 2
> + test x/dev/null = x/dev/null
> + test -s out2-zh_CN
> + emit_diff_u_header_ /dev/null out2-zh_CN
> + printf '%s\n' 'diff -u /dev/null out2-zh_CN' '--- /dev/null 1970-01-01'
> '+++ out2-zh_CN   1970-01-01'
> diff -u /dev/null out2-zh_CN
> --- /dev/null  1970-01-01
> +++ out2-zh_CN 1970-01-01
> + sed 's/^/+/' out2-zh_CN
> +á
> + return 1
> + case $? in
> + return 1
> + fail=1
> + for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
> + out=out1-fr_FR.UTF-8
> + LC_ALL=fr_FR.UTF-8
> + grep '\w' in
> + compare in out1-fr_FR.UTF-8
> + compare_dev_null_ in out1-fr_FR.UTF-8
> + test 2 = 2
> + test xin = x/dev/null
> + test xout1-fr_FR.UTF-8 = x/dev/null
> + return 2
> + case $? in
> + compare_ in out1-fr_FR.UTF-8
> + diff -u in out1-fr_FR.UTF-8
> + out=out2-fr_FR.UTF-8
> + LC_ALL=fr_FR.UTF-8
> + grep '\W' in
> + compare /dev/null out2-fr_FR.UTF-8
> + compare_dev_null_ /dev/null out2-fr_FR.UTF-8
> + test 2 = 2
> + test x/dev/null = x/dev/null
> + test -s out2-fr_FR.UTF-8
> + return 0
> + return 0
> + Exit 1
> + set +e
> + exit 1
> + exit 1
> + remove_tmp_
> + __st=1
> + cleanup_
> + :
> + cd /usr/src/grep/grep-2.20.72-d512/tests
> + chmod -R u+rwx
> /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> + rm -rf /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
> + exit 1
> FAIL word-multibyte (exit status: 1)
>
>
>

[0001-tests-avoid-false-positive-failure-due-to-some-zh_CN.patch (application/octet-stream, attachment)]

Message #14 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruce Dubbs <bruce.dubbs <at> gmail.com>, Assaf Gordon <agordon <at> wi.mit.edu>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Thu, 30 Oct 2014 12:07:36 -0700

For the record, you may want to run something like this to survey
whether \w matches E-acute across all  installed zh_CN locales:

(here's output from an os x 10.9.5 system -- on debian unstable and
fedora rawhide, they're all "0"s)
$ for i in $(locale -a|grep -i cn); do printf '\303\251'|LC_ALL=$i
src/grep -q '\w'; echo $?: $i;done
0: zh_CN
1: zh_CN.eucCN
1: zh_CN.GB18030
1: zh_CN.GB2312
1: zh_CN.GBK
0: zh_CN.UTF-8

On Thu, Oct 30, 2014 at 11:55 AM, Jim Meyering <jim <at> meyering.net> wrote:
> Thank you both for all the testing and prompt feedback!
> I've looked into the failures of the brand new word-multibyte test,
> and am hoping the attached will fix it by requiring use of
> the zh_CN.UTF-8 locale, rather than letting the system
> choose a locale matching "zh_CN".  Does this patch solve
> the problem for that test?
>
> On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
>> Assaf Gordon wrote:
>>>
>>> Hello,
>>>
>>> On 10/29/2014 02:29 PM, Jim Meyering wrote:
>>>>
>>>> Thanks to many fixes and improvements by Paul Eggert and Norihiro Tanaka,
>>>> here is a pre-release snapshot:
>>>>
>>>> grep snapshot:
>>>>    http://meyering.net/grep/grep-ss.tar.xz      1.2 MB
>>>>    http://meyering.net/grep/grep-ss.tar.xz.sig
>>>>    http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>>>>
>>>
>>> On Debian 7.6 (amd64, gcc 4.7.2) and Ubuntu 14.04.1 (amd64, gcc 4.8.2),
>>> these fail:
>>>    XFAIL: triple-backref
>>>    FAIL: word-multibyte
>>> Attached is the debian76.test-suite.log from Debian (Ubuntu seemed
>>> identical).
>>
>>
>> Linux From Scratch 7.6
>>
>> configure and make were clean.
>>
>> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>>
>> big-match: skipped test: not enough main memory to run the test
>> sjis-mb: skipped test: SJIS locale not found
>>
>> I do have 8G of memory.
>>
>> XFAIL: triple-backref
>>
>> SKIP: test-vc-list-files-cvs.sh  (cvs not present)
>> SKIP: test-wcrtomb-w32-1.sh      (windows only?)
>> SKIP: test-wcrtomb-w32-2.sh
>> SKIP: test-wcrtomb-w32-3.sh
>> SKIP: test-wcrtomb-w32-4.sh
>> SKIP: test-wcrtomb-w32-5.sh
>> FAIL: word-multibyte
>>
>> Looking at below, the FAIL may be because I have the locale zh_CN.gb18030,
>> but not a plain zh_CN locale.
>>
>> And confirming after installing a plain zh_CN locale, word-multibyte passes.
>>
>>   -- Bruce
>>
>> ====================
>>
>> ++ initial_cwd_=/usr/src/grep/grep-2.20.72-d512/tests
>> ++ fail=0
>> +++ testdir_prefix_
>> +++ printf gt
>> ++ pfx_=gt
>> +++ mktempd_ /usr/src/grep/grep-2.20.72-d512/tests gt-word-multibyte.XXXX
>> +++ case $# in
>> +++ destdir_=/usr/src/grep/grep-2.20.72-d512/tests
>> +++ template_=gt-word-multibyte.XXXX
>> +++ MAX_TRIES_=4
>> +++ case $destdir_ in
>> +++ case $template_ in
>> ++++ unset TMPDIR
>> +++ d=/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> +++ case $d in
>> +++ test -d /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> ++++ ls -dgo /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> ++++ tr S -
>> +++ perms='drwx------ 2 4096 Oct 29 16:24
>> /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7'
>> +++ case $perms in
>> +++ test 0 = 0
>> +++ echo /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> +++ return
>> ++ test_dir_=/usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> ++ cd /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> ++ gl_init_sh_nl_='
>> '
>> ++ IFS='
>> '
>> ++ for sig_ in 1 2 3 13 15
>> +++ expr 1 + 128
>> ++ eval 'trap '\''Exit 129'\'' 1'
>> +++ trap 'Exit 129' 1
>> ++ for sig_ in 1 2 3 13 15
>> +++ expr 2 + 128
>> ++ eval 'trap '\''Exit 130'\'' 2'
>> +++ trap 'Exit 130' 2
>> ++ for sig_ in 1 2 3 13 15
>> +++ expr 3 + 128
>> ++ eval 'trap '\''Exit 131'\'' 3'
>> +++ trap 'Exit 131' 3
>> ++ for sig_ in 1 2 3 13 15
>> +++ expr 13 + 128
>> ++ eval 'trap '\''Exit 141'\'' 13'
>> +++ trap 'Exit 141' 13
>> ++ for sig_ in 1 2 3 13 15
>> +++ expr 15 + 128
>> ++ eval 'trap '\''Exit 143'\'' 15'
>> +++ trap 'Exit 143' 15
>> ++ trap remove_tmp_ 0
>> + path_prepend_ ../src
>> + test 1 '!=' 0
>> + path_dir_=../src
>> + case $path_dir_ in
>> + abs_path_dir_=/usr/src/grep/grep-2.20.72-d512/tests/../src
>> + case $abs_path_dir_ in
>> +
>> PATH=/usr/src/grep/grep-2.20.72-d512/tests/../src:/usr/src/grep/grep-2.20.72-d512/src:/usr/local/bin:/bin:/usr/bin:/opt/texlive/2014/bin/x86_64-linux:/opt/fop:/opt/libreoffice-4.3.2.2/bin:/opt/OpenJDK-1.7.0.65-bin/bin:/opt/ant/bin:/opt/kde/bin:/opt/qt4/bin:/opt/qt5/bin:/opt/xorg/bin:/home/bdubbs/bin:/sbin:/usr/sbin
>> + create_exe_shims_ /usr/src/grep/grep-2.20.72-d512/tests/../src
>> + case $EXEEXT in
>> + return 0
>> + shift
>> + test 0 '!=' 0
>> + export PATH
>> + require_en_utf8_locale_
>> + path_prepend_ .
>> + test 1 '!=' 0
>> + path_dir_=.
>> + case $path_dir_ in
>> + abs_path_dir_=/usr/src/grep/grep-2.20.72-d512/tests/.
>> + case $abs_path_dir_ in
>> +
>> PATH=/usr/src/grep/grep-2.20.72-d512/tests/.:/usr/src/grep/grep-2.20.72-d512/tests/../src:/usr/src/grep/grep-2.20.72-d512/src:/usr/local/bin:/bin:/usr/bin:/opt/texlive/2014/bin/x86_64-linux:/opt/fop:/opt/libreoffice-4.3.2.2/bin:/opt/OpenJDK-1.7.0.65-bin/bin:/opt/ant/bin:/opt/kde/bin:/opt/qt4/bin:/opt/qt5/bin:/opt/xorg/bin:/home/bdubbs/bin:/sbin:/usr/sbin
>> + create_exe_shims_ /usr/src/grep/grep-2.20.72-d512/tests/.
>> + case $EXEEXT in
>> + return 0
>> + shift
>> + test 0 '!=' 0
>> + export PATH
>> + case $(get-mb-cur-max en_US.UTF-8) in
>> ++ get-mb-cur-max en_US.UTF-8
>> + printf '\xc3\xa1\n'
>> + LC_ALL=en_US.UTF-8
>> + export LC_ALL
>> + fail=0
>> + for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
>> + out=out1-en_US.UTF-8
>> + LC_ALL=en_US.UTF-8
>> + grep '\w' in
>> + compare in out1-en_US.UTF-8
>> + compare_dev_null_ in out1-en_US.UTF-8
>> + test 2 = 2
>> + test xin = x/dev/null
>> + test xout1-en_US.UTF-8 = x/dev/null
>> + return 2
>> + case $? in
>> + compare_ in out1-en_US.UTF-8
>> + diff -u in out1-en_US.UTF-8
>> + out=out2-en_US.UTF-8
>> + LC_ALL=en_US.UTF-8
>> + grep '\W' in
>> + compare /dev/null out2-en_US.UTF-8
>> + compare_dev_null_ /dev/null out2-en_US.UTF-8
>> + test 2 = 2
>> + test x/dev/null = x/dev/null
>> + test -s out2-en_US.UTF-8
>> + return 0
>> + return 0
>> + for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
>> + out=out1-zh_CN
>> + LC_ALL=zh_CN
>> + grep '\w' in
>> + fail=1
>> + compare in out1-zh_CN
>> + compare_dev_null_ in out1-zh_CN
>> + test 2 = 2
>> + test xin = x/dev/null
>> + test xout1-zh_CN = x/dev/null
>> + return 2
>> + case $? in
>> + compare_ in out1-zh_CN
>>  diff -u in out1-zh_CN
>> --- in   2014-10-29 16:24:46.775542037 -0500
>> +++ out1-zh_CN 2014-10-29 16:24:46.783541998 -0500
>> @@ -1 +0,0 @@
>> -á
>> + fail=1
>> + out=out2-zh_CN
>> + LC_ALL=zh_CN
>> + grep '\W' in
>> + fail=1
>> + compare /dev/null out2-zh_CN
>> + compare_dev_null_ /dev/null out2-zh_CN
>> + test 2 = 2
>> + test x/dev/null = x/dev/null
>> + test -s out2-zh_CN
>> + emit_diff_u_header_ /dev/null out2-zh_CN
>> + printf '%s\n' 'diff -u /dev/null out2-zh_CN' '--- /dev/null 1970-01-01'
>> '+++ out2-zh_CN   1970-01-01'
>> diff -u /dev/null out2-zh_CN
>> --- /dev/null  1970-01-01
>> +++ out2-zh_CN 1970-01-01
>> + sed 's/^/+/' out2-zh_CN
>> +á
>> + return 1
>> + case $? in
>> + return 1
>> + fail=1
>> + for LOC in en_US.UTF-8 zh_CN '$LOCALE_FR_UTF8'
>> + out=out1-fr_FR.UTF-8
>> + LC_ALL=fr_FR.UTF-8
>> + grep '\w' in
>> + compare in out1-fr_FR.UTF-8
>> + compare_dev_null_ in out1-fr_FR.UTF-8
>> + test 2 = 2
>> + test xin = x/dev/null
>> + test xout1-fr_FR.UTF-8 = x/dev/null
>> + return 2
>> + case $? in
>> + compare_ in out1-fr_FR.UTF-8
>> + diff -u in out1-fr_FR.UTF-8
>> + out=out2-fr_FR.UTF-8
>> + LC_ALL=fr_FR.UTF-8
>> + grep '\W' in
>> + compare /dev/null out2-fr_FR.UTF-8
>> + compare_dev_null_ /dev/null out2-fr_FR.UTF-8
>> + test 2 = 2
>> + test x/dev/null = x/dev/null
>> + test -s out2-fr_FR.UTF-8
>> + return 0
>> + return 0
>> + Exit 1
>> + set +e
>> + exit 1
>> + exit 1
>> + remove_tmp_
>> + __st=1
>> + cleanup_
>> + :
>> + cd /usr/src/grep/grep-2.20.72-d512/tests
>> + chmod -R u+rwx
>> /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> + rm -rf /usr/src/grep/grep-2.20.72-d512/tests/gt-word-multibyte.LTZ7
>> + exit 1
>> FAIL word-multibyte (exit status: 1)
>>
>>
>>

Message #17 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Thu, 30 Oct 2014 16:45:23 -0400

On 10/30/2014 03:07 PM, Jim Meyering wrote:
> For the record, you may want to run something like this to survey
> whether \w matches E-acute across all  installed zh_CN locales:
>
> (here's output from an os x 10.9.5 system -- on debian unstable and
> fedora rawhide, they're all "0"s)
> $ for i in $(locale -a|grep -i cn); do printf '\303\251'|LC_ALL=$i
> src/grep -q '\w'; echo $?: $i;done
> 0: zh_CN
> 1: zh_CN.eucCN
> 1: zh_CN.GB18030
> 1: zh_CN.GB2312
> 1: zh_CN.GBK
> 0: zh_CN.UTF-8
>

===
Debian 7.6
(no output, installed locales are: C  C.UTF-8  en_US.utf8 POSIX)
====
Ubuntu 14:
(no output, installed locales are C C.UTF-8 POSIX and en_* , en_*.utf8)
====
FreeBSD 10.0
1: zh_CN.GB18030
1: zh_CN.GB2312
1: zh_CN.GBK
0: zh_CN.UTF-8
1: zh_CN.eucCN
====
FreeBSD 9.3
1: zh_CN.GB18030
1: zh_CN.GB2312
1: zh_CN.GBK
0: zh_CN.UTF-8
1: zh_CN.eucCN
===
OpenBSD 5.5
0: zh_CN.UTF-8
===
NetBSD  6.1.4
1: zh_CN
1: zh_CN.GB18030
1: zh_CN.GB2312
0: zh_CN.UTF-8
1: zh_CN.eucCN
===
CentOS 7
0: bo_CN
0: bo_CN.utf8
0: ug_CN
0: ug_CN.utf8
0: zh_CN
0: zh_CN.gb18030
0: zh_CN.gb2312
0: zh_CN.gbk
0: zh_CN.utf8
====
CentOS 6.5
0: bo_CN
0: bo_CN.utf8
0: ug_CN
0: ug_CN.utf8
0: zh_CN
0: zh_CN.gb18030
0: zh_CN.gb2312
0: zh_CN.gbk
0: zh_CN.utf8
=====
OpenSUSE 13.1
0: bo_CN
0: ug_CN
0: zh_CN
0: zh_CN.gb18030
0: zh_CN.gbk
0: zh_CN.utf8
=====

Message #20 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Bruce Dubbs <bruce.dubbs <at> gmail.com>
To: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Thu, 30 Oct 2014 17:05:44 -0500

FYI, LFS only has zh_CN.GB18030 for Chinese by default.  We only create 
the locales needed for the glibc regression tests.  We can add others 
easily enough but perhaps the test should only check for those present.

  -- Bruce

Jim Meyering wrote:
> For the record, you may want to run something like this to survey
> whether \w matches E-acute across all  installed zh_CN locales:
>
> (here's output from an os x 10.9.5 system -- on debian unstable and
> fedora rawhide, they're all "0"s)
> $ for i in $(locale -a|grep -i cn); do printf '\303\251'|LC_ALL=$i
> src/grep -q '\w'; echo $?: $i;done
> 0: zh_CN
> 1: zh_CN.eucCN
> 1: zh_CN.GB18030
> 1: zh_CN.GB2312
> 1: zh_CN.GBK
> 0: zh_CN.UTF-8
>
> On Thu, Oct 30, 2014 at 11:55 AM, Jim Meyering <jim <at> meyering.net> wrote:
>> Thank you both for all the testing and prompt feedback!
>> I've looked into the failures of the brand new word-multibyte test,
>> and am hoping the attached will fix it by requiring use of
>> the zh_CN.UTF-8 locale, rather than letting the system
>> choose a locale matching "zh_CN".  Does this patch solve
>> the problem for that test?
>>
>> On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
>>> Assaf Gordon wrote:
>>>>
>>>> Hello,
>>>>
>>>> On 10/29/2014 02:29 PM, Jim Meyering wrote:
>>>>>
>>>>> Thanks to many fixes and improvements by Paul Eggert and Norihiro Tanaka,
>>>>> here is a pre-release snapshot:
>>>>>
>>>>> grep snapshot:
>>>>>     http://meyering.net/grep/grep-ss.tar.xz      1.2 MB
>>>>>     http://meyering.net/grep/grep-ss.tar.xz.sig
>>>>>     http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>>>>>
>>>>
>>>> On Debian 7.6 (amd64, gcc 4.7.2) and Ubuntu 14.04.1 (amd64, gcc 4.8.2),
>>>> these fail:
>>>>     XFAIL: triple-backref
>>>>     FAIL: word-multibyte
>>>> Attached is the debian76.test-suite.log from Debian (Ubuntu seemed
>>>> identical).
>>>
>>>
>>> Linux From Scratch 7.6
>>>
>>> configure and make were clean.
>>>
>>> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>>>
>>> big-match: skipped test: not enough main memory to run the test
>>> sjis-mb: skipped test: SJIS locale not found
>>>
>>> I do have 8G of memory.
>>>
>>> XFAIL: triple-backref
>>>
>>> SKIP: test-vc-list-files-cvs.sh  (cvs not present)
>>> SKIP: test-wcrtomb-w32-1.sh      (windows only?)
>>> SKIP: test-wcrtomb-w32-2.sh
>>> SKIP: test-wcrtomb-w32-3.sh
>>> SKIP: test-wcrtomb-w32-4.sh
>>> SKIP: test-wcrtomb-w32-5.sh
>>> FAIL: word-multibyte
>>>
>>> Looking at below, the FAIL may be because I have the locale zh_CN.gb18030,
>>> but not a plain zh_CN locale.
>>>
>>> And confirming after installing a plain zh_CN locale, word-multibyte passes.
>>>
>>>    -- Bruce

Message #23 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sat, 1 Nov 2014 16:06:54 -0700

On Wed, Oct 29, 2014 at 1:51 PM, Assaf Gordon <assafgordon <at> gmail.com> wrote:
> Hello,
>
> On 10/29/2014 02:29 PM, Jim Meyering wrote:
>>
>> Thanks to many fixes and improvements by Paul Eggert and Norihiro Tanaka,
>> here is a pre-release snapshot:
>>
>> grep snapshot:
>>    http://meyering.net/grep/grep-ss.tar.xz      1.2 MB
>>    http://meyering.net/grep/grep-ss.tar.xz.sig
>>    http://meyering.net/grep/grep-2.20.72-d512.tar.xz
...
> On NetBSD 6.1.4, these fail:
>   XFAIL: equiv-classes
>   FAIL: symlink
>   FAIL: word-multibyte
> Attached is netbsd614.test-suite.log

Hi Assaf,

First, that "XFAIL" is an expected failure, not a test failure, per se.
The "word-multibyte" failure should be fixed by the patch I've already pushed.

However, I'll need more information to understand the "symlink" failure.
Normally the log file contains verbose output information (resulting
from init.sh's "set -x"), but on your NetBSD system, it appears not to
have been enabled, perhaps because that system's /bin/sh is
inadequate.  Can you rerun the tests using bash as your shell?  That
may be enough to evoke the log information that will highlight which
part(s) of the "symlink" test actually fail.

Message #26 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sat, 01 Nov 2014 21:26:45 -0400

[Message part 1 (text/plain, inline)]

Hello,

On 11/01/2014 07:06 PM, Jim Meyering wrote:
>> On NetBSD 6.1.4, these fail:
>>    XFAIL: equiv-classes
>>    FAIL: symlink
>>    FAIL: word-multibyte
>
<...>

> However, I'll need more information to understand the "symlink" failure.
> Normally the log file contains verbose output information (resulting
> from init.sh's "set -x"), but on your NetBSD system, it appears not to
> have been enabled, perhaps because that system's /bin/sh is
> inadequate.  Can you rerun the tests using bash as your shell?  That
> may be enough to evoke the log information that will highlight which
> part(s) of the "symlink" test actually fail.

It turned out not to be directly a shell issue (I tried with forcing shell to bash), but a BSD make vs GNU make.
using "gmake" did use the correct shell, and produced a detailed log, which is attached.

If I understand correctly, the failure is triggered in line 177 in the attached log.
running "grep -r '^'" was supposed to return zero, but it returns 2.

I disabled the removal of the temporary directory, and so was able to reproduce manually:
====
$ uname -a
NetBSD  6.1.4 NetBSD 6.1.4 (GENERIC) amd64

$ pwd
/tmp/grep-2.20.72-d512/tests/gt-symlink.1F01
$ ls -lR
total 20
drwxr-xr-x  2 miles  wheel  512 Nov  2 00:55 dir
-rw-r--r--  1 miles  wheel   30 Nov  2 00:55 exp
-rw-r--r--  1 miles  wheel   30 Nov  2 00:55 grepout
-rw-r--r--  1 miles  wheel   30 Nov  2 00:55 out
-rw-r--r--  1 miles  wheel   30 Nov  2 00:55 out-t

./dir:
total 8
-rw-r--r--  1 miles  wheel  2 Nov  2 00:55 a
-rw-r--r--  1 miles  wheel  2 Nov  2 00:55 b
lrwxr-xr-x  1 miles  wheel  1 Nov  2 00:55 c -> a
lrwxr-xr-x  1 miles  wheel  1 Nov  2 00:55 d -> .
lrwxr-xr-x  1 miles  wheel  8 Nov  2 00:55 e -> dangling

$ cd dir
$ pwd
/tmp/grep-2.20.72-d512/tests/gt-symlink.1F01/dir

$ ../../../src/grep -r '^' ; echo $?
a:a
b:b
../../../src/grep: c: Inappropriate file type or format
../../../src/grep: d: Inappropriate file type or format
../../../src/grep: e: Inappropriate file type or format
2
====

For comparison,
On Linux (Ubuntu 14.04.1 amd64), the exit code is zero, somehow c,d,e are ignored:

===
$ uname -a
Linux XXXX 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 GNU/Linux
$ pwd
/tmp/grep/grep-2.20.72-d512/tests/gt-symlink.3Zym
$ ls -lR
.:
total 20
drwxrwxr-x 2 gordon gordon 4096 Nov  1 21:18 dir
-rw-rw-r-- 1 gordon gordon   30 Nov  1 21:18 exp
-rw-rw-r-- 1 gordon gordon   30 Nov  1 21:18 grepout
-rw-rw-r-- 1 gordon gordon   30 Nov  1 21:18 out
-rw-rw-r-- 1 gordon gordon   30 Nov  1 21:18 out-t

./dir:
total 8
-rw-rw-r-- 1 gordon gordon 2 Nov  1 21:18 a
-rw-rw-r-- 1 gordon gordon 2 Nov  1 21:18 b
lrwxrwxrwx 1 gordon gordon 1 Nov  1 21:18 c -> a
lrwxrwxrwx 1 gordon gordon 1 Nov  1 21:18 d -> .
lrwxrwxrwx 1 gordon gordon 8 Nov  1 21:18 e -> dangling

$ cd dir
$ pwd
/tmp/grep/grep-2.20.72-d512/tests/gt-symlink.3Zym/dir

$ ../../../src/grep -r '^' ; echo $?
b:b
a:a
0
===


 - Assaf

[netbsd614.test-suite.log (text/x-log, attachment)]

Message #29 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruce Dubbs <bruce.dubbs <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sat, 1 Nov 2014 18:48:53 -0700

On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
...
>>>    http://meyering.net/grep/grep-2.20.72-d512.tar.xz
...
> Linux From Scratch 7.6
>
> configure and make were clean.
>
> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>
> big-match: skipped test: not enough main memory to run the test
> sjis-mb: skipped test: SJIS locale not found
>
> I do have 8G of memory.

That test fails when grep fails to allocate enough space to hold a 2GiB line.
No big deal, and not hard to imaging happening with a few applications
in 8GiB of RAM.

Message #32 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sat, 01 Nov 2014 19:39:12 -0700

Assaf Gordon wrote:
> ./dir:
> total 8
> -rw-r--r--  1 miles  wheel  2 Nov  2 00:55 a
> -rw-r--r--  1 miles  wheel  2 Nov  2 00:55 b
> lrwxr-xr-x  1 miles  wheel  1 Nov  2 00:55 c -> a
> lrwxr-xr-x  1 miles  wheel  1 Nov  2 00:55 d -> .
> lrwxr-xr-x  1 miles  wheel  8 Nov  2 00:55 e -> dangling
>
> $ cd dir
> $ pwd
> /tmp/grep-2.20.72-d512/tests/gt-symlink.1F01/dir
>
> $ ../../../src/grep -r '^' ; echo $?
> a:a
> b:b
> ../../../src/grep: c: Inappropriate file type or format
> ../../../src/grep: d: Inappropriate file type or format
> ../../../src/grep: e: Inappropriate file type or format
> 2

What system calls is this 'grep' executing?  What's the output of ktrace and kdump?

Message #35 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Bruce Dubbs <bruce.dubbs <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sat, 01 Nov 2014 21:56:11 -0500

Jim Meyering wrote:
> On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
> ...
>>>>     http://meyering.net/grep/grep-2.20.72-d512.tar.xz
> ...
>> Linux From Scratch 7.6
>>
>> configure and make were clean.
>>
>> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>>
>> big-match: skipped test: not enough main memory to run the test
>> sjis-mb: skipped test: SJIS locale not found
>>
>> I do have 8G of memory.
>
> That test fails when grep fails to allocate enough space to hold a 2GiB line.
> No big deal, and not hard to imaging happening with a few applications
> in 8GiB of RAM.

Curious though.  It should have been OK.  I did not have any gui running 
at the time.  Even if RAM was full, swap should have handled it.

$ free
            total       used       free     shared    buffers     cached
Mem:      8105972    2380012    5725960        880     610460     326820
-/+ buffers/cache:   1442732    6663240
Swap:    10485756       3216   10482540

  -- Bruce

Message #38 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruce Dubbs <bruce.dubbs <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sun, 2 Nov 2014 06:38:10 -0800

Your total free memory is obviously larger than 2GiB, but remember
that what's listed there is the total, while the offending usage
required a *contiguous* 2GiB region.

On Sat, Nov 1, 2014 at 7:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
> Jim Meyering wrote:
>>
>> On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com>
>> wrote:
>> ...
>>>>>
>>>>>     http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>>
>> ...
>>>
>>> Linux From Scratch 7.6
>>>
>>> configure and make were clean.
>>>
>>> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>>>
>>> big-match: skipped test: not enough main memory to run the test
>>> sjis-mb: skipped test: SJIS locale not found
>>>
>>> I do have 8G of memory.
>>
>>
>> That test fails when grep fails to allocate enough space to hold a 2GiB
>> line.
>> No big deal, and not hard to imaging happening with a few applications
>> in 8GiB of RAM.
>
>
> Curious though.  It should have been OK.  I did not have any gui running at
> the time.  Even if RAM was full, swap should have handled it.
>
> $ free
>             total       used       free     shared    buffers     cached
> Mem:      8105972    2380012    5725960        880     610460     326820
> -/+ buffers/cache:   1442732    6663240
> Swap:    10485756       3216   10482540
>
>   -- Bruce
>
>

Message #41 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sun, 02 Nov 2014 13:07:57 -0500

[Message part 1 (text/plain, inline)]

Hello,

On 11/01/2014 10:39 PM, Paul Eggert wrote:
> Assaf Gordon wrote:

>> $ ../../../src/grep -r '^' ; echo $?
>> a:a
>> b:b
>> ../../../src/grep: c: Inappropriate file type or format
>> ../../../src/grep: d: Inappropriate file type or format
>> ../../../src/grep: e: Inappropriate file type or format
>> 2
>
> What system calls is this 'grep' executing?  What's the output of ktrace and kdump?

Attached is the ktrace+tdump output of running the command.

The error appears on line 217 (open() returns -1 for file "c"),
though similar errors stemming from 'fstat' for files a,b are ignored (lines 150,191).

- Assaf

[netbsd.grp.out.txt.xz (application/x-xz, attachment)]

Message #44 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Bruce Dubbs <bruce.dubbs <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sun, 02 Nov 2014 12:59:34 -0600

Jim Meyering wrote:
> Your total free memory is obviously larger than 2GiB, but remember
> that what's listed there is the total, while the offending usage
> required a *contiguous* 2GiB region.

My understanding of the kernel is that it allocates in 4K blocks and 
maps them to user space to appear to be contiguous.

If I write a program that does:
  size_t size = 0xA0000000;  // 2,684,354,560
  char*  mem  = (char*) malloc( size );
  size_t i;

  for ( i = 0; i < size; i++, mem++ ) *mem='A';

There are no problems, but it does take about 9 seconds on my system.
The start address is 7ffa5405d010.
The end   address is 7ffaf405d010.

What am I missing?

  -- Bruce


> On Sat, Nov 1, 2014 at 7:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com> wrote:
>> Jim Meyering wrote:
>>>
>>> On Wed, Oct 29, 2014 at 2:56 PM, Bruce Dubbs <bruce.dubbs <at> gmail.com>
>>> wrote:
>>> ...
>>>>>>
>>>>>>      http://meyering.net/grep/grep-2.20.72-d512.tar.xz
>>>
>>> ...
>>>>
>>>> Linux From Scratch 7.6
>>>>
>>>> configure and make were clean.
>>>>
>>>> $ env RUN_EXPENSIVE_TESTS=yes make -k check
>>>>
>>>> big-match: skipped test: not enough main memory to run the test
>>>> sjis-mb: skipped test: SJIS locale not found
>>>>
>>>> I do have 8G of memory.
>>>
>>>
>>> That test fails when grep fails to allocate enough space to hold a 2GiB
>>> line.
>>> No big deal, and not hard to imaging happening with a few applications
>>> in 8GiB of RAM.
>>
>>
>> Curious though.  It should have been OK.  I did not have any gui running at
>> the time.  Even if RAM was full, swap should have handled it.
>>
>> $ free
>>              total       used       free     shared    buffers     cached
>> Mem:      8105972    2380012    5725960        880     610460     326820
>> -/+ buffers/cache:   1442732    6663240
>> Swap:    10485756       3216   10482540
>>
>>    -- Bruce
>>
>>
>

Message #47 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Mon, 03 Nov 2014 15:14:53 +0900

[Message part 1 (text/plain, inline)]

Assaf Gordon <assafgordon <at> gmail.com> wrote:
> On 11/01/2014 10:39 PM, Paul Eggert wrote:
> > Assaf Gordon wrote:
> 
> >> $ ../../../src/grep -r '^' ; echo $?
> >> a:a
> >> b:b
> >> ../../../src/grep: c: Inappropriate file type or format
> >> ../../../src/grep: d: Inappropriate file type or format
> >> ../../../src/grep: e: Inappropriate file type or format
> >> 2
> >
> > What system calls is this 'grep' executing?  What's the output of ktrace and kdump?
> 
> Attached is the ktrace+tdump output of running the command.
> 
> The error appears on line 217 (open() returns -1 for file "c"),
> though similar errors stemming from 'fstat' for files a,b are ignored (lines 150,191).
> 
> - Assaf

NetBSD returns errno == EFTYPE in open with O_NOFOLLOW for symlink
instead of errno == ELOOP which POSIX says.  Similarly, Tru64 returns
errno == ENOTSUP.  So I added them to ignorable list.

[0001-grep-fix-symlink-non-conforming-to-POSIX.patch (text/plain, attachment)]

Message #50 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sun, 02 Nov 2014 22:16:04 -0800

[Message part 1 (text/plain, inline)]

Assaf Gordon wrote:
> Attached is the ktrace+tdump output of running the command.

Thanks, I see the problem now: NetBSD uses a nonstandard errno value when openat 
(fd, "symlink", ... O_NOFOLLOW ...) fails.  I installed the attached, which 
should fix the NetBSD problem.

[0001-grep-port-O_NOFOLLOW-errno-checking-to-NetBSD.patch (text/plain, attachment)]

Message #53 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Norihiro Tanaka <noritnk <at> kcn.ne.jp>, Assaf Gordon <assafgordon <at> gmail.com>
Cc: 18892 <at> debbugs.gnu.org
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Sun, 02 Nov 2014 22:30:33 -0800

Norihiro Tanaka wrote:
> /* Tru64 5.1B uses errno == ENOTSUP.  */

Ah, I see that you were working on the same bug I was.  I don't think we need to 
worry about Tru64, partly because ENOTSUP would be troublesome anyway (does it 
mean that O_NOFOLLOW isn't supported? if so, 'grep' has other problems), partly 
because HP stopped supporting Tru64 in December 2012 and typically we can stop 
supporting a platform once its supplier stops.

Message #56 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 18892 <at> debbugs.gnu.org, Assaf Gordon <assafgordon <at> gmail.com>
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Mon, 03 Nov 2014 15:37:58 +0900

> Norihiro Tanaka wrote:
> > /* Tru64 5.1B uses errno == ENOTSUP.  */
> 
> Ah, I see that you were working on the same bug I was.  I don't think
> we need to worry about Tru64, partly because ENOTSUP would be troublesome
> anyway (does it mean that O_NOFOLLOW isn't supported? if so, 'grep' has
> other problems), partly because HP stopped supporting Tru64 in December
> 2012 and typically we can stop supporting a platform once its supplier
> stops.

Thanks, I agree you.  And I feel your work is more elegant than my. (^_^)

Message #59 received at 18892 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 18892 <at> debbugs.gnu.org, Assaf Gordon <assafgordon <at> gmail.com>
Subject: Re: bug#18892: few test failure with 'grep-2.20.72-d512'
Date: Mon, 3 Nov 2014 06:49:45 -0800

Nice patch.  Thank you.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.