GNU bug report logs -
#8846
coreutils-8.12 on HP-UX 11.31: 3 of 365 tests failed
Previous Next
Reported by: Bruno Haible <bruno <at> clisp.org>
Date: Mon, 13 Jun 2011 09:17:01 UTC
Severity: normal
Tags: moreinfo
Done: Jim Meyering <jim <at> meyering.net>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8846 in the body.
You can then email your comments to 8846 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 09:17:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Bruno Haible <bruno <at> clisp.org>:
New bug report received and forwarded. Copy sent to
bug-coreutils <at> gnu.org.
(Mon, 13 Jun 2011 09:17:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On HP-UX 11.31, built with cc, coreutils-8.12 gives 3 test suite failures:
FAIL: misc/printf-surprise (exit: 1)
FAIL: dd/nocache (exit: 1)
FAIL: du/inaccessible-cwd (exit: 1)
Find attached the log file.
--
In memoriam Anna Göldi <http://en.wikipedia.org/wiki/Anna_Göldi>
[test-suite.log (text/x-log, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 09:33:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On 13/06/11 10:15, Bruno Haible wrote:
> On HP-UX 11.31, built with cc, coreutils-8.12 gives 3 test suite failures:
>
> FAIL: misc/printf-surprise (exit: 1)
> FAIL: dd/nocache (exit: 1)
> FAIL: du/inaccessible-cwd (exit: 1)
>
> Find attached the log file.
Re dd/nocache, I'm suspecting an incorrect match
against the error message returned.
It should skip the test for 'Operation not supported'
Could you give the error returned for:
dd if=/dev/zero of=ifile conv=fdatasync count=100
dd if=ifile iflag=nocache count=0
cheers,
Pádraig.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 09:56:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady wrote:
> Could you give the error returned for:
>
> dd if=/dev/zero of=ifile conv=fdatasync count=100
> dd if=ifile iflag=nocache count=0
Here are the results:
$ ./dd if=/dev/zero of=ifile conv=fdatasync count=100
100+0 records in
100+0 records out
51200 bytes (51 kB) copied, 0.14338 s, 357 kB/s
$ ./dd if=ifile iflag=nocache count=0
./dd: failed to discard cache for: `ifile': Not a typewriter
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000375367 s, 0.0 kB/s
Bruno
--
In memoriam Anna Göldi <http://en.wikipedia.org/wiki/Anna_Göldi>
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:12:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Bruno Haible wrote:
> On HP-UX 11.31, built with cc, coreutils-8.12 gives 3 test suite failures:
>
> FAIL: misc/printf-surprise (exit: 1)
> FAIL: dd/nocache (exit: 1)
> FAIL: du/inaccessible-cwd (exit: 1)
>
> Find attached the log file.
>
...
> FAIL: misc/printf-surprise (exit: 1)
> ====================================
>
> ./init.sh: line 77: 1: Bad file number
> printf (GNU coreutils) 8.12
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
...
Hi Bruno
Thank you for the testing and report.
That "bad file number" error comes from this code in init.sh:
: ${stderr_fileno_=2}
warn_ () { echo "$@" 1>&$stderr_fileno_; }
Because of that, the log contains less information than usual.
If you can easily apply this patch and rerun the failing tests,
I'd appreciate it. It should avoid the error and let us see
more details about what is failing.
From 25e7bded3f2abff58540b0fcead2ba110d344bb0 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Mon, 13 Jun 2011 12:07:14 +0200
Subject: [PATCH] init.sh: accommodate shells for which 1>&$stderr_fileno_
fails
* tests/init.sh (warn_): Use eval to work around a bug in some shells,
like those of Solaris 10 and HP-UX 11.11.
---
tests/init.sh | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tests/init.sh b/tests/init.sh
index 14f2e26..7a701f6 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
# the reason for skip/failure to console, rather than to the .log files.
: ${stderr_fileno_=2}
-warn_ () { echo "$@" 1>&$stderr_fileno_; }
+warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
--
1.7.6.rc0.293.g40857
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:31:02 GMT)
Full text and
rfc822 format available.
Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):
Hello Jim.
Just a minor nit, since your fix will have to be ported to automake's
`tests/defs' too ...
On Monday 13 June 2011, Jim Meyering wrote:
> Bruno Haible wrote:
> > On HP-UX 11.31, built with cc, coreutils-8.12 gives 3 test suite failures:
> >
> > FAIL: misc/printf-surprise (exit: 1)
> > FAIL: dd/nocache (exit: 1)
> > FAIL: du/inaccessible-cwd (exit: 1)
> >
> > Find attached the log file.
> >
> ...
> > FAIL: misc/printf-surprise (exit: 1)
> > ====================================
> >
> > ./init.sh: line 77: 1: Bad file number
> > printf (GNU coreutils) 8.12
> > Copyright (C) 2011 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law.
> ...
>
> Hi Bruno
>
> Thank you for the testing and report.
> That "bad file number" error comes from this code in init.sh:
>
> : ${stderr_fileno_=2}
> warn_ () { echo "$@" 1>&$stderr_fileno_; }
>
> Because of that, the log contains less information than usual.
> If you can easily apply this patch and rerun the failing tests,
> I'd appreciate it. It should avoid the error and let us see
> more details about what is failing.
>
>
> From 25e7bded3f2abff58540b0fcead2ba110d344bb0 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering <at> redhat.com>
> Date: Mon, 13 Jun 2011 12:07:14 +0200
> Subject: [PATCH] init.sh: accommodate shells for which 1>&$stderr_fileno_
> fails
>
> * tests/init.sh (warn_): Use eval to work around a bug in some shells,
> like those of Solaris 10 and HP-UX 11.11.
> ---
> tests/init.sh | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/tests/init.sh b/tests/init.sh
> index 14f2e26..7a701f6 100644
> --- a/tests/init.sh
> +++ b/tests/init.sh
> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
> # the reason for skip/failure to console, rather than to the .log files.
> : ${stderr_fileno_=2}
>
> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
> +warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
>
Wouldn't it be simpler, and potentially more correct, to use simply:
warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
With your new implementation, I see this on Debian with bash and dash:
$ stderr_fileno_=2; warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
$ $ warn_ '$HOME'
/home/stefano
which is not intended behaviour IMHO.
> fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
> skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
> framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
> --
> 1.7.6.rc0.293.g40857
>
>
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:31:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:36:02 GMT)
Full text and
rfc822 format available.
Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):
Stefano Lattarini wrote:
> Hello Jim.
>
> Just a minor nit, since your fix will have to be ported to automake's
> `tests/defs' too ...
...
>> diff --git a/tests/init.sh b/tests/init.sh
>> index 14f2e26..7a701f6 100644
>> --- a/tests/init.sh
>> +++ b/tests/init.sh
>> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
>> # the reason for skip/failure to console, rather than to the .log files.
>> : ${stderr_fileno_=2}
>>
>> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
>> +warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
>>
> Wouldn't it be simpler, and potentially more correct, to use simply:
>
> warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
>
> With your new implementation, I see this on Debian with bash and dash:
>
> $ stderr_fileno_=2; warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
> $ $ warn_ '$HOME'
> /home/stefano
>
> which is not intended behaviour IMHO.
Hah! Thank you for the prompt fix.
That's more than a minor nit ;-)
I'm glad you caught my error.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:36:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:47:02 GMT)
Full text and
rfc822 format available.
Message #29 received at submit <at> debbugs.gnu.org (full text, mbox):
Stefano Lattarini wrote:
> Hello Jim.
...
> Wouldn't it be simpler, and potentially more correct, to use simply:
>
> warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
>
> With your new implementation, I see this on Debian with bash and dash:
>
> $ stderr_fileno_=2; warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
> $ $ warn_ '$HOME'
> /home/stefano
>
> which is not intended behaviour IMHO.
Thanks again. Here's what I've just pushed.
I'll do the same for gnulib shortly.
From 6fb9aeedd1b858a61d5cbf7f15782adf29ff733a Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Mon, 13 Jun 2011 12:07:14 +0200
Subject: [PATCH] init.sh: accommodate shells for which 1>&$stderr_fileno_
fails
* tests/init.sh (warn_): Use eval to work around a bug in some shells,
like those of Solaris 10 and HP-UX 11.11.
Improved by Stefano Lattarini.
---
tests/init.sh | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tests/init.sh b/tests/init.sh
index 14f2e26..25850af 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
# the reason for skip/failure to console, rather than to the .log files.
: ${stderr_fileno_=2}
-warn_ () { echo "$@" 1>&$stderr_fileno_; }
+warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
--
1.7.6.rc0.293.g40857
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:47:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 10:54:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On 13/06/11 10:55, Bruno Haible wrote:
> Pádraig Brady wrote:
>> Could you give the error returned for:
>>
>> dd if=/dev/zero of=ifile conv=fdatasync count=100
>> dd if=ifile iflag=nocache count=0
>
> Here are the results:
>
> $ ./dd if=/dev/zero of=ifile conv=fdatasync count=100
> 100+0 records in
> 100+0 records out
> 51200 bytes (51 kB) copied, 0.14338 s, 357 kB/s
> $ ./dd if=ifile iflag=nocache count=0
> ./dd: failed to discard cache for: `ifile': Not a typewriter
Weird. I've relaxed the test with the following.
cheers,
Pádraig.
--- a/tests/dd/nocache
+++ b/tests/dd/nocache
@@ -36,12 +36,12 @@ done
# Advise to drop cache for whole file
if ! dd if=ifile iflag=nocache count=0 2>err; then
- if grep -F 'Operation not supported' err >/dev/null; then
- warn_ 'skipping part; this file system lacks support for posix_fadvise()'
- skip=1
- else
- fail=1
- fi
+ # We could check for 'Operation not supported' in err here,
+ # but that was seen to be brittle. HPUX returns ENOTTY for example.
+ # So assume that if this basic operation fails, it's due to lack
+ # of support by the system.
+ warn_ 'skipping part; this file system lacks support for posix_fadvise()'
+ skip=1
fi
if test "$skip" != 1; then
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 11:04:02 GMT)
Full text and
rfc822 format available.
Message #38 received at submit <at> debbugs.gnu.org (full text, mbox):
On Monday 13 June 2011, Jim Meyering wrote:
> Stefano Lattarini wrote:
> > Hello Jim.
> ...
> > Wouldn't it be simpler, and potentially more correct, to use simply:
> >
> > warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
> >
> > With your new implementation, I see this on Debian with bash and dash:
> >
> > $ stderr_fileno_=2; warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
> > $ $ warn_ '$HOME'
> > /home/stefano
> >
> > which is not intended behaviour IMHO.
>
> Thanks again. Here's what I've just pushed.
> I'll do the same for gnulib shortly.
>
> From 6fb9aeedd1b858a61d5cbf7f15782adf29ff733a Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering <at> redhat.com>
> Date: Mon, 13 Jun 2011 12:07:14 +0200
> Subject: [PATCH] init.sh: accommodate shells for which 1>&$stderr_fileno_
> fails
>
> * tests/init.sh (warn_): Use eval to work around a bug in some shells,
> like those of Solaris 10 and HP-UX 11.11.
> Improved by Stefano Lattarini.
> ---
> tests/init.sh | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/tests/init.sh b/tests/init.sh
> index 14f2e26..25850af 100644
> --- a/tests/init.sh
> +++ b/tests/init.sh
> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
> # the reason for skip/failure to console, rather than to the .log files.
> : ${stderr_fileno_=2}
>
> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
> +warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
> fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
> skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
> framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
> --
> 1.7.6.rc0.293.g40857
>
Thanks. I'll port this to automake tests/defs soonish; I assume you are
ok to be credited as lead author in ChangeLog and git commit, right?
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 11:04:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 12:08:02 GMT)
Full text and
rfc822 format available.
Message #44 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Jim Meyering wrote:
> --- a/tests/init.sh
> +++ b/tests/init.sh
> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
> # the reason for skip/failure to console, rather than to the .log files.
> : ${stderr_fileno_=2}
>
> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
> +warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
> fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
> skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
> framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
After applying this patch, here's the test-suite.log that I now get.
--
In memoriam Anna Göldi <http://en.wikipedia.org/wiki/Anna_Göldi>
[test-suite.log (text/x-log, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 12:09:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 13:38:02 GMT)
Full text and
rfc822 format available.
Message #50 received at submit <at> debbugs.gnu.org (full text, mbox):
Bruno Haible wrote:
> Jim Meyering wrote:
>> --- a/tests/init.sh
>> +++ b/tests/init.sh
>> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
>> # the reason for skip/failure to console, rather than to the .log files.
>> : ${stderr_fileno_=2}
>>
>> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
>> +warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
>> fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
>> skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
>> framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
>
> After applying this patch, here's the test-suite.log that I now get.
Thank you.
However, it looks like using "eval" didn't help at all.
What does this do on that system?
sh -c 'e=2; warn_ () { echo "$@" 1>&$e; }; warn_ x'
This is what happens via tests/check.mk:
printf '#!/bin/sh\ne=9; warn_ () { echo "$@" 1>&$e; }; warn_ x\n' > k
sh -c 'exec 9>&2; /bin/sh k'
Does that evoke a warning for you?
If so, does hard-coding the "9" help?
printf '#!/bin/sh\ne=9; warn_ () { echo "$@" 1>&9; }; warn_ x\n' > k
sh -c 'exec 9>&2; /bin/sh k'
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 13:38:03 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:00:03 GMT)
Full text and
rfc822 format available.
Message #56 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 04:11 AM, Jim Meyering wrote:
> Thank you for the testing and report.
> That "bad file number" error comes from this code in init.sh:
>
> : ${stderr_fileno_=2}
> warn_ () { echo "$@" 1>&$stderr_fileno_; }
>
> Because of that, the log contains less information than usual.
I'm having problems summarizing this shell bug into a succinct
description that would be worth including in the autoconf manual. What
exactly is the bug here?
That is, my attempt to provoke the problem on HP-UX 11.11 with this
snippet is not failing with a "bad file number":
$ /bin/sh -c 'n=3; f(){ echo "$@" 1>&$n; }; exec 3>&2; f hi'
hi
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:04:01 GMT)
Full text and
rfc822 format available.
Message #59 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Eric Blake wrote:
> On 06/13/2011 04:11 AM, Jim Meyering wrote:
>> Thank you for the testing and report.
>> That "bad file number" error comes from this code in init.sh:
>>
>> : ${stderr_fileno_=2}
>> warn_ () { echo "$@" 1>&$stderr_fileno_; }
>>
>> Because of that, the log contains less information than usual.
>
> I'm having problems summarizing this shell bug into a succinct
> description that would be worth including in the autoconf manual. What
> exactly is the bug here?
>
> That is, my attempt to provoke the problem on HP-UX 11.11 with this
> snippet is not failing with a "bad file number":
>
> $ /bin/sh -c 'n=3; f(){ echo "$@" 1>&$n; }; exec 3>&2; f hi'
> hi
Per Bruno's report, running "make check" on HP-UX 11.31 resulted
in one "Bad file number" diagnostic per use of init.sh.
Perhaps it's a problem in 11.31 that is not in 11.1.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:20:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 07:36 AM, Jim Meyering wrote:
> However, it looks like using "eval" didn't help at all.
> What does this do on that system?
>
> sh -c 'e=2; warn_ () { echo "$@" 1>&$e; }; warn_ x'
$ sh -c 'e=2; warn_ () { echo "$@" 1>&$e; }; warn_ x'
x
>
> This is what happens via tests/check.mk:
>
> printf '#!/bin/sh\ne=9; warn_ () { echo "$@" 1>&$e; }; warn_ x\n' > k
> sh -c 'exec 9>&2; /bin/sh k'
>
> Does that evoke a warning for you?
Yes:
% sh -c 'exec 9>&2; /bin/sh k'
k: 9: Generated or received a file descriptor number that is not valid.
But I think the reason that it invokes a problem is not because of the
1>&$e construct, but because HP-UX opens secondary file descriptors as
cloexec or otherwise closing them at some point, so the '/bin/sh k'
child process is not inheriting fd 9 from the parent process.
Does anyone know the HP-UX counterpart to Linux' strace in order to see
what syscalls are in use by the HP-UX shell, and why/where fd 9 is
getting closed?
>
> If so, does hard-coding the "9" help?
No.
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:39:02 GMT)
Full text and
rfc822 format available.
Message #65 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Eric Blake wrote:
> On 06/13/2011 07:36 AM, Jim Meyering wrote:
>> However, it looks like using "eval" didn't help at all.
>> What does this do on that system?
>>
>> sh -c 'e=2; warn_ () { echo "$@" 1>&$e; }; warn_ x'
>
> $ sh -c 'e=2; warn_ () { echo "$@" 1>&$e; }; warn_ x'
> x
>
>>
>> This is what happens via tests/check.mk:
>>
>> printf '#!/bin/sh\ne=9; warn_ () { echo "$@" 1>&$e; }; warn_ x\n' > k
>> sh -c 'exec 9>&2; /bin/sh k'
>>
>> Does that evoke a warning for you?
>
> Yes:
>
> % sh -c 'exec 9>&2; /bin/sh k'
> k: 9: Generated or received a file descriptor number that is not valid.
>
> But I think the reason that it invokes a problem is not because of the
> 1>&$e construct, but because HP-UX opens secondary file descriptors as
> cloexec or otherwise closing them at some point, so the '/bin/sh k'
> child process is not inheriting fd 9 from the parent process.
Ah ha!
That sounds like a syscall that needs to be gnulib-replaced in bash/zsh.
Of course, first we'd need to gnulib-enable a shell.
> Does anyone know the HP-UX counterpart to Linux' strace in order to see
> what syscalls are in use by the HP-UX shell, and why/where fd 9 is
> getting closed?
http://bhami.com/rosetta.html suggests the following:
caliper fprof
caliper ktrace
trace (freeware)
tusc (11+, freeware)
>> If so, does hard-coding the "9" help?
>
> No.
Thanks for investigating.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:39:02 GMT)
Full text and
rfc822 format available.
Message #68 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Stefano Lattarini wrote:
> On Monday 13 June 2011, Jim Meyering wrote:
>> Stefano Lattarini wrote:
>> > Hello Jim.
>> ...
>> > Wouldn't it be simpler, and potentially more correct, to use simply:
>> >
>> > warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
>> >
>> > With your new implementation, I see this on Debian with bash and dash:
>> >
>> > $ stderr_fileno_=2; warn_ () { eval 'echo "'"$@"'" 1>&'"$stderr_fileno_"; }
>> > $ $ warn_ '$HOME'
>> > /home/stefano
>> >
>> > which is not intended behaviour IMHO.
>>
>> Thanks again. Here's what I've just pushed.
>> I'll do the same for gnulib shortly.
>>
>> From 6fb9aeedd1b858a61d5cbf7f15782adf29ff733a Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <meyering <at> redhat.com>
>> Date: Mon, 13 Jun 2011 12:07:14 +0200
>> Subject: [PATCH] init.sh: accommodate shells for which 1>&$stderr_fileno_
>> fails
>>
>> * tests/init.sh (warn_): Use eval to work around a bug in some shells,
>> like those of Solaris 10 and HP-UX 11.11.
>> Improved by Stefano Lattarini.
>> ---
>> tests/init.sh | 2 +-
>> 1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> diff --git a/tests/init.sh b/tests/init.sh
>> index 14f2e26..25850af 100644
>> --- a/tests/init.sh
>> +++ b/tests/init.sh
>> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
>> # the reason for skip/failure to console, rather than to the .log files.
>> : ${stderr_fileno_=2}
>>
>> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
>> +warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
>> fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
>> skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
>> framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
>> --
>> 1.7.6.rc0.293.g40857
>>
> Thanks. I'll port this to automake tests/defs soonish; I assume you are
> ok to be credited as lead author in ChangeLog and git commit, right?
Looks like that change is not useful after all.
I'll revert it in coreutils.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:40:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 15:48:01 GMT)
Full text and
rfc822 format available.
Message #74 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 09:37 AM, Jim Meyering wrote:
>> But I think the reason that it invokes a problem is not because of the
>> 1>&$e construct, but because HP-UX opens secondary file descriptors as
>> cloexec or otherwise closing them at some point, so the '/bin/sh k'
>> child process is not inheriting fd 9 from the parent process.
>
> Ah ha!
> That sounds like a syscall that needs to be gnulib-replaced in bash/zsh.
> Of course, first we'd need to gnulib-enable a shell.
Yep, I've confirmed that both HP-UX and Solaris 10 ksh use CLOEXEC fds
for intermediate fd values created by exec; that POSIX permits (but
fortunately does not require) this behavior, and that autoconf already
documents the pitfall of expecting fds > 2 to be preserved into child
processes:
http://www.gnu.org/software/autoconf/manual/autoconf.html#File-Descriptors
> Don't rely on open file descriptors being open in child processes. In ksh, file descriptors above 2 which are opened using ‘exec n>file’ are closed by a subsequent ‘exec’ (such as that involved in the fork-and-exec which runs a program or script). Thus, using sh, we have:
>
> $ cat ./descrips
> #!/bin/sh -
> echo hello >&5
> $ exec 5>t
> $ ./descrips
> $ cat t
> hello
> $
>
> But using ksh:
>
> $ exec 5>t
> $ ./descrips
> hello
> $ cat t
> $
On Solaris 10, truss shows this for 'exec 9>&2':
dup(2) = 3
close(9) Err#9 EBADF
fcntl(3, F_DUPFD, 0x00000009) = 9
close(3) = 0
fcntl(9, F_SETFD, 0x00000001) = 0
> tusc (11+, freeware)
Likewise on HP-UX 11.31, tusc shows:
dup(2) ................................................... = 3
close(9) ................................................. ERR#9 EBADF
fcntl(3, F_DUPFD, 9) ..................................... = 9
close(3) ................................................. = 0
fcntl(9, F_SETFD, 1) ..................................... = 0
About the only thing I can think of is to pass the details about the
desired redirection as a command-line or env-var setting, to be done by
the script itself, rather than relying on the parent shell (the one in
make) being able to do a redirection that persists into the child (the
one sourcing init.sh).
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 17:02:02 GMT)
Full text and
rfc822 format available.
Message #77 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Eric Blake wrote:
> On 06/13/2011 09:37 AM, Jim Meyering wrote:
>>> But I think the reason that it invokes a problem is not because of the
>>> 1>&$e construct, but because HP-UX opens secondary file descriptors as
>>> cloexec or otherwise closing them at some point, so the '/bin/sh k'
>>> child process is not inheriting fd 9 from the parent process.
>>
>> Ah ha!
>> That sounds like a syscall that needs to be gnulib-replaced in bash/zsh.
>> Of course, first we'd need to gnulib-enable a shell.
>
> Yep, I've confirmed that both HP-UX and Solaris 10 ksh use CLOEXEC fds
> for intermediate fd values created by exec; that POSIX permits (but
> fortunately does not require) this behavior, and that autoconf already
> documents the pitfall of expecting fds > 2 to be preserved into child
> processes:
>
> http://www.gnu.org/software/autoconf/manual/autoconf.html#File-Descriptors
>
>> Don't rely on open file descriptors being open in child
>> processes. In ksh, file descriptors above 2 which are opened using
>> ‘exec n>file’ are closed by a subsequent ‘exec’ (such as that
>> involved in the fork-and-exec which runs a program or script). Thus,
>> using sh, we have:
>>
>> $ cat ./descrips
>> #!/bin/sh -
>> echo hello >&5
>> $ exec 5>t
>> $ ./descrips
>> $ cat t
>> hello
>> $
>>
>> But using ksh:
>>
>> $ exec 5>t
>> $ ./descrips
>> hello
>> $ cat t
>> $
>
> On Solaris 10, truss shows this for 'exec 9>&2':
>
> dup(2) = 3
> close(9) Err#9 EBADF
> fcntl(3, F_DUPFD, 0x00000009) = 9
> close(3) = 0
> fcntl(9, F_SETFD, 0x00000001) = 0
>
>> tusc (11+, freeware)
>
> Likewise on HP-UX 11.31, tusc shows:
>
> dup(2) ................................................... = 3
> close(9) ................................................. ERR#9 EBADF
> fcntl(3, F_DUPFD, 9) ..................................... = 9
> close(3) ................................................. = 0
> fcntl(9, F_SETFD, 1) ..................................... = 0
>
> About the only thing I can think of is to pass the details about the
> desired redirection as a command-line or env-var setting, to be done by
> the script itself, rather than relying on the parent shell (the one in
> make) being able to do a redirection that persists into the child (the
> one sourcing init.sh).
Thanks for confirming.
Actually the fix (at least for bourne shell scripts) is even easier
than that: perform the redirection from init.sh's setup_ function
I'm going to revert (in coreutils) my eval-adding change,
and then apply this instead:
From d987cf87de5e7e597e295914c536bd332c24cc63 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Mon, 13 Jun 2011 18:54:53 +0200
Subject: [PATCH] init.sh: redirect FD 9 to stderr again, for Solaris 10 and
HP-UX
* tests/init.sh (setup_): When $stderr_fileno_ is not 2, redirect it.
Prior to this change, we would redirect before the shell fork-and-exec
performed via automake's TESTS_ENVIRONMENT, but that redirection was
ineffective on Solaris 10 and HP-UX 11.31, due to the fact that those
systems set the CLOEXEC bit on FDs larger than 2. Thus our redirection
of FD 9 would not survive the fork-and-exec of running each test script.
---
tests/init.sh | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/tests/init.sh b/tests/init.sh
index 60d1bc1..d101643 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -317,6 +317,11 @@ path_prepend_ ()
setup_ ()
{
+ # If we're redirecting a file descriptor larger than 2, say via automake's
+ # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
+ # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
+ test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
+
if test "$VERBOSE" = yes; then
# Test whether set -x may cause the selected shell to corrupt an
# application's stderr. Many do, including zsh-4.3.10 and the /bin/sh
--
1.7.6.rc0.293.g40857
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 19:22:01 GMT)
Full text and
rfc822 format available.
Message #80 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Jim Meyering wrote:
> --- a/tests/init.sh
> +++ b/tests/init.sh
> @@ -317,6 +317,11 @@ path_prepend_ ()
>
> setup_ ()
> {
> + # If we're redirecting a file descriptor larger than 2, say via automake's
> + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
> + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
> + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
> +
> if test "$VERBOSE" = yes; then
> # Test whether set -x may cause the selected shell to corrupt an
> # application's stderr. Many do, including zsh-4.3.10 and the /bin/sh
With this change, I get the attached log on HP-UX 11.31.
--
In memoriam Anna Göldi <http://en.wikipedia.org/wiki/Anna_Göldi>
[test-suite.log (text/x-log, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 19:46:02 GMT)
Full text and
rfc822 format available.
Message #83 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Bruno Haible wrote:
> Jim Meyering wrote:
>> --- a/tests/init.sh
>> +++ b/tests/init.sh
>> @@ -317,6 +317,11 @@ path_prepend_ ()
>>
>> setup_ ()
>> {
>> + # If we're redirecting a file descriptor larger than 2, say via automake's
>> + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
>> + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
>> + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
>> +
>> if test "$VERBOSE" = yes; then
>> # Test whether set -x may cause the selected shell to corrupt an
>> # application's stderr. Many do, including zsh-4.3.10 and the /bin/sh
>
> With this change, I get the attached log on HP-UX 11.31.
>
> ==============================================
> GNU coreutils 8.12: tests/test-suite.log
> ==============================================
>
> 3 of 365 tests failed. (88 tests were not run).
...
Hi Bruno,
Thanks for persevering.
That confirms the init.sh change solves one problem.
However, HP's /bin/sh has the bug that set -x corrupts stderr,
and in that case, init.sh decides not to honor VERBOSE=yes
because that led inevitably to numerous false positives due to the bug.
The easiest way to work around that is to use some other shell.
If you have bash, you might want to run this (but see below):
sh=/usr/local/bin/bash
make CONFIG_SHELL=$sh CONFIG_SHELL=$sh VERBOSE=yes \
check -C tests TESTS='misc/printf-surprise du/inaccessible-cwd'
> FAIL: dd/nocache (exit: 1)
...
Pádraig's patch should have resolved this.
> FAIL: du/inaccessible-cwd (exit: 1)
This failure may be expected on HP-UX, assuming it lacks the *at functions
and its getcwd doesn't have the feature that Linux and others do
of working even in an inaccessible directory.
That leaves this one:
> FAIL: misc/printf-surprise (exit: 1)
> ====================================
Which may be failing because the ulimit of 10MiB is too low:
head -c 10 fifo > out &
( ulimit -v 10000; env $prog %20000000f 0 2>err-msg > fifo )
exit=$?
Can you try it manually?
E.g., run these commands and report the output?
mkfifo fifo
head -c 10 fifo > out &
( ulimit -v 10000; env printf %20000000f 0 2>err-msg > fifo )
exit=$?
n_out=$(wc -c < out)
echo "$n_out::$exit"
cat err-msg
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 20:33:02 GMT)
Full text and
rfc822 format available.
Message #86 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Hi Jim. Probably you're totally going to hate me today, but ...
On Monday 13 June 2011, Jim Meyering wrote:
>
> From d987cf87de5e7e597e295914c536bd332c24cc63 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering <at> redhat.com>
> Date: Mon, 13 Jun 2011 18:54:53 +0200
> Subject: [PATCH] init.sh: redirect FD 9 to stderr again, for Solaris 10 and HP-UX
>
> * tests/init.sh (setup_): When $stderr_fileno_ is not 2, redirect it.
> Prior to this change, we would redirect before the shell fork-and-exec
> performed via automake's TESTS_ENVIRONMENT, but that redirection was
> ineffective on Solaris 10 and HP-UX 11.31, due to the fact that those
> systems set the CLOEXEC bit on FDs larger than 2. Thus our redirection
> of FD 9 would not survive the fork-and-exec of running each test script.
> ---
> tests/init.sh | 5 +++++
> 1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/tests/init.sh b/tests/init.sh
> index 60d1bc1..d101643 100644
> --- a/tests/init.sh
> +++ b/tests/init.sh
> @@ -317,6 +317,11 @@ path_prepend_ ()
>
> setup_ ()
> {
> + # If we're redirecting a file descriptor larger than 2, say via automake's
> + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
> + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
> + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
> +
>
... isn't this equivalent to just using ">&2" unconditionally in 'warn_()'?
IMHO, the right fix is to to modify the code in TESTS_ENVIRONMENT to avoid the
definition of $stderr_fileno_ the shell performs closed-on-exec; e.g.,
TESTS_ENVIRONMENT = ...; \
if test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'; then
stderr_fileno_=9; export stderr_fileno_;
else
unset stderr_fileno_ || :
fi
If we know that bash and zsh are well behaved, we can even avoid a couple of
forks (Cygwin users won't hate us too much then):
TESTS_ENVIRONMENT = ...; \
if test -n "$${ZSH_VERSION}$${BASH_VERSION}" || \
test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'
then
stderr_fileno_=9; export stderr_fileno_;
else
unset stderr_fileno_ || :
fi
A better fix would be to do the redirect $stderr_fileno_>&2 in tess/init.sh
iff $stderr_fileno_ is closed, but how can that be portably determined
without printing trash on the user screen (and for *each* test)?
> if test "$VERBOSE" = yes; then
> # Test whether set -x may cause the selected shell to corrupt an
> # application's stderr. Many do, including zsh-4.3.10 and the /bin/sh
> --
> 1.7.6.rc0.293.g40857
>
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 20:42:02 GMT)
Full text and
rfc822 format available.
Message #89 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On Monday 13 June 2011, Stefano Lattarini wrote:
> Hi Jim. Probably you're totally going to hate me today, but ...
>
> On Monday 13 June 2011, Jim Meyering wrote:
> >
> > From d987cf87de5e7e597e295914c536bd332c24cc63 Mon Sep 17 00:00:00 2001
> > From: Jim Meyering <meyering <at> redhat.com>
> > Date: Mon, 13 Jun 2011 18:54:53 +0200
> > Subject: [PATCH] init.sh: redirect FD 9 to stderr again, for Solaris 10 and HP-UX
> >
> > * tests/init.sh (setup_): When $stderr_fileno_ is not 2, redirect it.
> > Prior to this change, we would redirect before the shell fork-and-exec
> > performed via automake's TESTS_ENVIRONMENT, but that redirection was
> > ineffective on Solaris 10 and HP-UX 11.31, due to the fact that those
> > systems set the CLOEXEC bit on FDs larger than 2. Thus our redirection
> > of FD 9 would not survive the fork-and-exec of running each test script.
> > ---
> > tests/init.sh | 5 +++++
> > 1 files changed, 5 insertions(+), 0 deletions(-)
> >
> > diff --git a/tests/init.sh b/tests/init.sh
> > index 60d1bc1..d101643 100644
> > --- a/tests/init.sh
> > +++ b/tests/init.sh
> > @@ -317,6 +317,11 @@ path_prepend_ ()
> >
> > setup_ ()
> > {
> > + # If we're redirecting a file descriptor larger than 2, say via automake's
> > + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
> > + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
> > + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
> > +
> >
> ... isn't this equivalent to just using ">&2" unconditionally in 'warn_()'?
>
> IMHO, the right fix is to to modify the code in TESTS_ENVIRONMENT to avoid the
> definition of $stderr_fileno_ the shell performs closed-on-exec; e.g.,
>
> TESTS_ENVIRONMENT = ...; \
> if test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'; then
> stderr_fileno_=9; export stderr_fileno_;
> else
> unset stderr_fileno_ || :
> fi
>
> If we know that bash and zsh are well behaved, we can even avoid a couple of
> forks (Cygwin users won't hate us too much then):
>
> TESTS_ENVIRONMENT = ...; \
> if test -n "$${ZSH_VERSION}$${BASH_VERSION}" || \
> test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'
> then
> stderr_fileno_=9; export stderr_fileno_;
> else
> unset stderr_fileno_ || :
> fi
>
> A better fix would be to do the redirect $stderr_fileno_>&2 in tess/init.sh
> iff $stderr_fileno_ is closed, but how can that be portably determined
> without printing trash on the user screen (and for *each* test)?
>
But this last observaton makes me think. The only purpose of $stderr_fileno_
is to allow the test to print diagnostic on the user's tty, instead of burying
it in the test logs; at this point, we might do the redirection only if the
fd 2 is a tty, so that we will know that, in `tests/init.sh', either:
[1] $stderr_fileno_ refers to a tty, even after the automake parallel-tests
driver has made its own redirections; or:
[2] $stderr_fileno_ is simply the file descriptor 2, which is expected to be
open for writing in any remotely sane setup.
Then we can test, from within tests/init.sh, whether $stderr_fileno_ has been
closed or not by doing:
test 2 -eq "$stderr_fileno_" || test -t "$stderr_fileno_"
and if this is not the case, we eval "exec $stderr_fileno_ >&2" and live
happily. All without extra forks or overly complex `TESTS_ENVIRONMENT'
definitions.
WDYT?
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 20:47:02 GMT)
Full text and
rfc822 format available.
Message #92 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 02:32 PM, Stefano Lattarini wrote:
>> +++ b/tests/init.sh
>> @@ -317,6 +317,11 @@ path_prepend_ ()
>>
>> setup_ ()
>> {
>> + # If we're redirecting a file descriptor larger than 2, say via automake's
>> + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
>> + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
>> + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
>> +
>>
> ... isn't this equivalent to just using ">&2" unconditionally in 'warn_()'?
Not quite, in coreutils' current usage.
Coreutils sets stderr_fileno_=9 in init.cfg, not TESTS_ENVIRONMENT;
which means that stderr can be changed after a call to setup_ but before
a call to warn_. Thus, we want to output to $stderr_fileno_ (which is
9, and is always the original stderr), even during a test where stderr
has been redirected somewhere else.
>
> IMHO, the right fix is to to modify the code in TESTS_ENVIRONMENT to avoid the
> definition of $stderr_fileno_ the shell performs closed-on-exec; e.g.,
>
> TESTS_ENVIRONMENT = ...; \
> if test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'; then
> stderr_fileno_=9; export stderr_fileno_;
> else
> unset stderr_fileno_ || :
> fi
But setting things during TESTS_ENVIRONMENT does seem a bit more robust.
Or rather, I think that the best action is to set stderr_fileno_
alongside the code that does the redirection; if it is TESTS_ENVIRONMENT
that does the exec 9>&2, then TESTS_ENVIRONMENT does seem like the saner
place (rather than init.cfg) for setting stderr_fileno_.
>
> If we know that bash and zsh are well behaved, we can even avoid a couple of
> forks (Cygwin users won't hate us too much then):
>
> TESTS_ENVIRONMENT = ...; \
> if test -n "$${ZSH_VERSION}$${BASH_VERSION}" || \
> test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'
> then
> stderr_fileno_=9; export stderr_fileno_;
> else
> unset stderr_fileno_ || :
> fi
>
> A better fix would be to do the redirect $stderr_fileno_>&2 in tess/init.sh
> iff $stderr_fileno_ is closed, but how can that be portably determined
> without printing trash on the user screen (and for *each* test)?
Not possible to portably sniff out closed fds; quoting the autoconf manual:
> Don't rely on duplicating a closed file descriptor to cause an
> error. With Solaris @command{/bin/sh}, when the redirection fails, the
> output goes to the original file descriptor.
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 21:41:02 GMT)
Full text and
rfc822 format available.
Message #95 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On Monday 13 June 2011, Eric Blake wrote:
> On 06/13/2011 02:32 PM, Stefano Lattarini wrote:
> >> +++ b/tests/init.sh
> >> @@ -317,6 +317,11 @@ path_prepend_ ()
> >>
> >> setup_ ()
> >> {
> >> + # If we're redirecting a file descriptor larger than 2, say via automake's
> >> + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
> >> + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
> >> + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
> >> +
> >>
> > ... isn't this equivalent to just using ">&2" unconditionally in 'warn_()'?
>
> Not quite, in coreutils' current usage.
>
> Coreutils sets stderr_fileno_=9 in init.cfg, not TESTS_ENVIRONMENT;
> which means that stderr can be changed after a call to setup_ but before
> a call to warn_.
>
True. I expressed myself badly; my point is, now coreutils will behave
as if `$stderr_fileno_' wouldn't have been set in `init.cfg' -- which I
reckon is hardly what we want.
> Thus, we want to output to $stderr_fileno_ (which is 9, and is always the
> original stderr), even during a test where stderr has been redirected
> somewhere else.
>
Then I'm missing something crucial here; my mental picture of what's going
on here is the following:
[1] TESTS_ENVIRONMENT redirects fd 9 to fd 2, which is the original stderr
of the make process (usually a user tty);
[2] The automake parallel-tests driver redirect fds 1 and 2 to the .log
file associated with the test; but fd 9 still points to the orginal
stderr, not to the log file (which is what we want);
[3] The $(SHELL), which here we assume is ksh with close-on-exec, does a
fork+exec to execute the test script; thus, when that script begins
its execution, it has the fd 9 closed (d'oh!, but expected now);
[4] the scripts sources init.sh, which in turn:
[4.1] sources init.cfg, which defines $stderr_fileno_ to 9 (which,
mind you, is a closed fd at this point);
[4.2] executes setup_, which sees that $stderr_fileno_ is *not* set to
`2', and thus re-define it to `2', thus directing it to the log
file.
The net outcome is that the original stderr is lost, and that, by default,
warn_ and company will output their messages into the log file. If this
is the effect we want, we can obtain it much more simply by removing the
`exec 9>&2' redirection from TESTS_ENVIRONMENT, and removing the code
snippet `test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"' from
tests/init.sh.
> >
> > IMHO, the right fix is to to modify the code in TESTS_ENVIRONMENT to avoid the
> > definition of $stderr_fileno_ the shell performs closed-on-exec; e.g.,
> >
> > TESTS_ENVIRONMENT = ...; \
> > if test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'; then
> > stderr_fileno_=9; export stderr_fileno_;
> > else
> > unset stderr_fileno_ || :
> > fi
>
> But setting things during TESTS_ENVIRONMENT does seem a bit more robust.
> Or rather, I think that the best action is to set stderr_fileno_
> alongside the code that does the redirection; if it is TESTS_ENVIRONMENT
> that does the exec 9>&2, then TESTS_ENVIRONMENT does seem like the saner
> place (rather than init.cfg) for setting stderr_fileno_.
>
I fully agree with this (even if it won't help in our particualr situation
IMHO).
> >
> > If we know that bash and zsh are well behaved, we can even avoid a couple of
> > forks (Cygwin users won't hate us too much then):
> >
> > TESTS_ENVIRONMENT = ...; \
> > if test -n "$${ZSH_VERSION}$${BASH_VERSION}" || \
> > test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'
> > then
> > stderr_fileno_=9; export stderr_fileno_;
> > else
> > unset stderr_fileno_ || :
> > fi
> >
> > A better fix would be to do the redirect $stderr_fileno_>&2 in tess/init.sh
> > iff $stderr_fileno_ is closed, but how can that be portably determined
> > without printing trash on the user screen (and for *each* test)?
>
> Not possible to portably sniff out closed fds; quoting the autoconf manual:
>
> > Don't rely on duplicating a closed file descriptor to cause an
> > error. With Solaris @command{/bin/sh}, when the redirection fails, the
> > output goes to the original file descriptor.
>
>
I suspected that; see my later mail:
<http://lists.gnu.org/archive/html/bug-coreutils/2011-06/msg00074.html>
for an alternative, simpler proposal.
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 22:03:02 GMT)
Full text and
rfc822 format available.
Message #98 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On Monday 13 June 2011, Stefano Lattarini wrote:
> On Monday 13 June 2011, Stefano Lattarini wrote:
> > Hi Jim. Probably you're totally going to hate me today, but ...
> >
> > On Monday 13 June 2011, Jim Meyering wrote:
> > >
> > > From d987cf87de5e7e597e295914c536bd332c24cc63 Mon Sep 17 00:00:00 2001
> > > From: Jim Meyering <meyering <at> redhat.com>
> > > Date: Mon, 13 Jun 2011 18:54:53 +0200
> > > Subject: [PATCH] init.sh: redirect FD 9 to stderr again, for Solaris 10 and HP-UX
> > >
> > > * tests/init.sh (setup_): When $stderr_fileno_ is not 2, redirect it.
> > > Prior to this change, we would redirect before the shell fork-and-exec
> > > performed via automake's TESTS_ENVIRONMENT, but that redirection was
> > > ineffective on Solaris 10 and HP-UX 11.31, due to the fact that those
> > > systems set the CLOEXEC bit on FDs larger than 2. Thus our redirection
> > > of FD 9 would not survive the fork-and-exec of running each test script.
> > > ---
> > > tests/init.sh | 5 +++++
> > > 1 files changed, 5 insertions(+), 0 deletions(-)
> > >
> > > diff --git a/tests/init.sh b/tests/init.sh
> > > index 60d1bc1..d101643 100644
> > > --- a/tests/init.sh
> > > +++ b/tests/init.sh
> > > @@ -317,6 +317,11 @@ path_prepend_ ()
> > >
> > > setup_ ()
> > > {
> > > + # If we're redirecting a file descriptor larger than 2, say via automake's
> > > + # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some systems
> > > + # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
> > > + test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
> > > +
> > >
> > ... isn't this equivalent to just using ">&2" unconditionally in 'warn_()'?
> >
> > IMHO, the right fix is to to modify the code in TESTS_ENVIRONMENT to avoid the
> > definition of $stderr_fileno_ the shell performs closed-on-exec; e.g.,
> >
> > TESTS_ENVIRONMENT = ...; \
> > if test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'; then
> > stderr_fileno_=9; export stderr_fileno_;
> > else
> > unset stderr_fileno_ || :
> > fi
> >
> > If we know that bash and zsh are well behaved, we can even avoid a couple of
> > forks (Cygwin users won't hate us too much then):
> >
> > TESTS_ENVIRONMENT = ...; \
> > if test -n "$${ZSH_VERSION}$${BASH_VERSION}" || \
> > test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = x'foo'
> > then
> > stderr_fileno_=9; export stderr_fileno_;
> > else
> > unset stderr_fileno_ || :
> > fi
> >
> > A better fix would be to do the redirect $stderr_fileno_>&2 in tests/init.sh
> > iff $stderr_fileno_ is closed, but how can that be portably determined
> > without printing trash on the user screen (and for *each* test)?
> >
> But this last observaton makes me think. The only purpose of $stderr_fileno_
> is to allow the test to print diagnostic on the user's tty, instead of burying
> it in the test logs; at this point, we might do the redirection only if the
> fd 2 is a tty, so that we will know that, in `tests/init.sh', either:
> [1] $stderr_fileno_ refers to a tty, even after the automake parallel-tests
> driver has made its own redirections; or:
> [2] $stderr_fileno_ is simply the file descriptor 2, which is expected to be
> open for writing in any remotely sane setup.
> Then we can test, from within tests/init.sh, whether $stderr_fileno_ has been
> closed or not by doing:
> test 2 -eq "$stderr_fileno_" || test -t "$stderr_fileno_"
> and if this is not the case, we eval "exec $stderr_fileno_ >&2" and live
> happily. All without extra forks or overly complex `TESTS_ENVIRONMENT'
> definitions.
>
Only that BSD make and Solaris dmake, when running in concurrent mode,
redirect stdout/stderr to temporary files or to named or anonymous pipes
(for output serialization); so that my proposed idiom wouldn't work with
them. Sigh.
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 22:06:02 GMT)
Full text and
rfc822 format available.
Message #101 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 03:39 PM, Stefano Lattarini wrote:
>
> [3] The $(SHELL), which here we assume is ksh with close-on-exec, does a
> fork+exec to execute the test script; thus, when that script begins
> its execution, it has the fd 9 closed (d'oh!, but expected now);
Oddly enough, it looks like it is _only_ 'exec 9>&2' that suffers from
close-on-exec semantics; using '(blah) 9>&2' leaves fd 9 inheritable.
So maybe the real fix is to do the redirection to a subshell or grouping
{} rather than via exec, at which point even ksh and HP-UX sh will allow
the original stderr to be inherited as fd 9 to the child script process:
$ printf '#!/bin/sh\nexec >&9\n' > k
$ sh k
k[2]: 9: Generated or received a file descriptor number that is not valid.
$ sh k 9>&2
$ sh -c 'exec 9>&2; /bin/sh k'
k[2]: 9: Generated or received a file descriptor number that is not valid.
$ sh -c 'sh k 9>&2'
$ sh -c '(sh k) 9>&2'
$ sh -c '{ sh k;} 9>&2'
$
but I don't know if that will require some changes in automake's
parallel-test driver setup code.
And Autoconf should probably document that the close-on-exec semantics
of some shells applies only to the exec builtin, since they do not
happen to other redirections.
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 22:26:02 GMT)
Full text and
rfc822 format available.
Message #104 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On Monday 13 June 2011, Eric Blake wrote:
>
> Not possible to portably sniff out closed fds; quoting the autoconf manual:
>
> > Don't rely on duplicating a closed file descriptor to cause an
> > error. With Solaris @command{/bin/sh}, when the redirection fails, the
> > output goes to the original file descriptor.
>
Do the shells with the close-on-exec issue also suffer of the issue with
closed fds you've reported? If not, the following could be enough to
solve our situation without having to change automake:
if (exec 3>&-; exec 4>&3) >/dev/null 2>&1; then
# Cannot determine whether a file descriptor is closed, fall back
# to inferior hack.
if test 2 -ne "$stderr_fileno_" && test ! -t "$stderr_fileno_"; then
eval "exec $stderr_fileno_>&2" # Or is `stderr_fileno_=2' enough?
fi
else
if (exec 3>&9) >/dev/null 2>&1; then
:
else
eval "exec $stderr_fileno_>&2" # Or is `stderr_fileno_=2' enough?
fi
fi
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 22:30:02 GMT)
Full text and
rfc822 format available.
Message #107 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On Tuesday 14 June 2011, Eric Blake wrote:
> On 06/13/2011 03:39 PM, Stefano Lattarini wrote:
> >
> > [3] The $(SHELL), which here we assume is ksh with close-on-exec, does a
> > fork+exec to execute the test script; thus, when that script begins
> > its execution, it has the fd 9 closed (d'oh!, but expected now);
>
> Oddly enough, it looks like it is _only_ 'exec 9>&2' that suffers from
> close-on-exec semantics; using '(blah) 9>&2' leaves fd 9 inheritable.
>
> So maybe the real fix is to do the redirection to a subshell or grouping
> {} rather than via exec, at which point even ksh and HP-UX sh will allow
> the original stderr to be inherited as fd 9 to the child script process:
>
> $ printf '#!/bin/sh\nexec >&9\n' > k
> $ sh k
> k[2]: 9: Generated or received a file descriptor number that is not valid.
> $ sh k 9>&2
> $ sh -c 'exec 9>&2; /bin/sh k'
> k[2]: 9: Generated or received a file descriptor number that is not valid.
> $ sh -c 'sh k 9>&2'
>
If this work, then using a bare `>&2' *at the end of TESTS_ENVIRONMENT* and
*without a following semicolon* might give a portable workaround, as if I'm
not mistaken POSIX mandates that redirections can be specified anywere on
the command line, and are to be evaluated from left to right.
UPDATE: Yes, it seems to work. I'll add a testcase to the 'maint' branch in
case you and Jim decide to go with this solution (and you can confirm that it
really works).
> $ sh -c '(sh k) 9>&2'
> $ sh -c '{ sh k;} 9>&2'
> $
>
> but I don't know if that will require some changes in automake's
> parallel-test driver setup code.
>
> And Autoconf should probably document that the close-on-exec semantics
> of some shells applies only to the exec builtin, since they do not
> happen to other redirections.
>
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 22:39:02 GMT)
Full text and
rfc822 format available.
Message #110 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 04:24 PM, Stefano Lattarini wrote:
> On Monday 13 June 2011, Eric Blake wrote:
>>
>> Not possible to portably sniff out closed fds; quoting the autoconf manual:
>>
>>> Don't rely on duplicating a closed file descriptor to cause an
>>> error. With Solaris @command{/bin/sh}, when the redirection fails, the
>>> output goes to the original file descriptor.
>>
> Do the shells with the close-on-exec issue also suffer of the issue with
> closed fds you've reported? If not, the following could be enough to
> solve our situation without having to change automake:
Where are you proposing this hack? In init.sh (after fd 9 has already
been closed if the calling shell used exec instead of some other form of
redirection)? That is, why are we trying to fix init.sh to deal with a
closed fd after the fact, when it seems like the better approach is to
guarantee that fd is never closed on exec in the first place? Which
necessarily implies altering the caller.
>
> if (exec 3>&-; exec 4>&3) >/dev/null 2>&1; then
> # Cannot determine whether a file descriptor is closed, fall back
> # to inferior hack.
> if test 2 -ne "$stderr_fileno_" && test ! -t "$stderr_fileno_"; then
> eval "exec $stderr_fileno_>&2" # Or is `stderr_fileno_=2' enough?
> fi
Or is the whole point of this hack to init.sh to avoid the spurious
warning about EBADF and to instead redirect errors to the
(possibly-altered-by-make) fd 2 when fd 9 was lost, so that at least the
warning messages appear in the logs even though we lost the chance to
display them on the tty?
At any rate, in answer to your question:
Solaris /bin/sh (the shell where failed redirections are silently
ignored) does this:
$ /bin/sh -c '(exec 3>&-; exec 4>&3) >/dev/null 2>&1; echo $?'
0
$ cat k
#!/bin/sh
e=9; warn_ () { echo "$@" 1>&$e; }; warn_ x
$ /bin/sh k
x
$ /bin/sh k >/dev/null
$ /bin/sh k 9>&2
x
$ /bin/sh k 9>&2 >/dev/null
x
That is, if fd 9 is closed, then 1>&$e (aka >&9) is a no-op on that
shell, and output goes to the original fd 1; but if fd 9 is open, then
output goes to fd 9.
Whereas ksh and HP-UX sh are both vocal on failed redirections, for
reliable detection:
$ ksh -c '(exec 3>&-; exec 4>&3) >/dev/null 2>&1; echo $?'
1
$ ksh k
k[2]: 9: cannot open [Bad file number]
$ ksh k >/dev/null
k[2]: 9: cannot open [Bad file number]
$ ksh k 9>&2
x
$ ksh k 9>&2 >/dev/null
x
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Mon, 13 Jun 2011 22:48:03 GMT)
Full text and
rfc822 format available.
Message #113 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 06/13/2011 04:29 PM, Stefano Lattarini wrote:
> If this work, then using a bare `>&2' *at the end of TESTS_ENVIRONMENT* and
You meant a bare `9>&2', but yes that does seem to be workable for what
we want!
> *without a following semicolon* might give a portable workaround, as if I'm
> not mistaken POSIX mandates that redirections can be specified anywere on
> the command line, and are to be evaluated from left to right.
Yes, all shells support these as equivalent:
9>&2 sh k
sh k 9>&2
>
> UPDATE: Yes, it seems to work. I'll add a testcase to the 'maint' branch in
> case you and Jim decide to go with this solution (and you can confirm that it
> really works).
Cool! Definitely worth documenting in the automake manual, as owner of
TESTS_ENVIRONMENT and as a client of init.sh functionality, as well as
your proposed automake testcase addition to ensure we don't break it.
--
Eric Blake eblake <at> redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Tue, 14 Jun 2011 07:35:01 GMT)
Full text and
rfc822 format available.
Message #116 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Notice: this is a JFTR answer; for the final solution of the issue, see
the better working solution at:
<http://lists.gnu.org/archive/html/bug-coreutils/2011-06/msg00080.html>
<http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00016.html>
On Tuesday 14 June 2011, Eric Blake wrote:
> On 06/13/2011 04:24 PM, Stefano Lattarini wrote:
> > On Monday 13 June 2011, Eric Blake wrote:
> >>
> >> Not possible to portably sniff out closed fds; quoting the autoconf manual:
> >>
> >>> Don't rely on duplicating a closed file descriptor to cause an
> >>> error. With Solaris @command{/bin/sh}, when the redirection fails, the
> >>> output goes to the original file descriptor.
> >>
> > Do the shells with the close-on-exec issue also suffer of the issue with
> > closed fds you've reported? If not, the following could be enough to
> > solve our situation without having to change automake:
>
> Where are you proposing this hack? In init.sh (after fd 9 has already
> been closed if the calling shell used exec instead of some other form of
> redirection)? That is, why are we trying to fix init.sh to deal with a
> closed fd after the fact, when it seems like the better approach is to
> guarantee that fd is never closed on exec in the first place? Which
> necessarily implies altering the caller.
>
> >
> > if (exec 3>&-; exec 4>&3) >/dev/null 2>&1; then
> > # Cannot determine whether a file descriptor is closed, fall back
> > # to inferior hack.
> > if test 2 -ne "$stderr_fileno_" && test ! -t "$stderr_fileno_"; then
> > eval "exec $stderr_fileno_>&2" # Or is `stderr_fileno_=2' enough?
> > fi
>
> Or is the whole point of this hack to init.sh to avoid the spurious
> warning about EBADF and to instead redirect errors to the
> (possibly-altered-by-make) fd 2 when fd 9 was lost, so that at least the
> warning messages appear in the logs even though we lost the chance to
> display them on the tty?
>
Yes, this was the idea. But it is moot now that we have a better solution
to be used in TESTS_ENVRIRONMENT.
> At any rate, in answer to your question:
>
> Solaris /bin/sh (the shell where failed redirections are silently
> ignored) does this:
>
> $ /bin/sh -c '(exec 3>&-; exec 4>&3) >/dev/null 2>&1; echo $?'
> 0
> $ cat k
> #!/bin/sh
> e=9; warn_ () { echo "$@" 1>&$e; }; warn_ x
> $ /bin/sh k
> x
> $ /bin/sh k >/dev/null
> $ /bin/sh k 9>&2
> x
> $ /bin/sh k 9>&2 >/dev/null
> x
>
> That is, if fd 9 is closed, then 1>&$e (aka >&9) is a no-op on that
> shell, and output goes to the original fd 1; but if fd 9 is open, then
> output goes to fd 9.
>
> Whereas ksh and HP-UX sh are both vocal on failed redirections, for
> reliable detection:
>
> $ ksh -c '(exec 3>&-; exec 4>&3) >/dev/null 2>&1; echo $?'
> 1
> $ ksh k
> k[2]: 9: cannot open [Bad file number]
> $ ksh k >/dev/null
> k[2]: 9: cannot open [Bad file number]
> $ ksh k 9>&2
> x
> $ ksh k 9>&2 >/dev/null
> x
>
This would have been a problem :-/
Luckily, no more need to worry about it now :-)
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Tue, 14 Jun 2011 07:49:01 GMT)
Full text and
rfc822 format available.
Message #119 received at 8846 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
[adding automake-patches on CC:]
Reference to original thread(s), mostly duplicated:
<http://lists.gnu.org/archive/html/bug-coreutils/2011-06/msg00051.html>
<http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
Reference to last relevant message there:
<http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00018.html>
<http://lists.gnu.org/archive/html/bug-coreutils/2011-06/msg00082.html>
On Tuesday 14 June 2011, Eric Blake wrote:
> On 06/13/2011 04:29 PM, Stefano Lattarini wrote:
> > If this work, then using a bare `>&2' *at the end of TESTS_ENVIRONMENT* and
>
> You meant a bare `9>&2',
>
Yes, sorry.
> but yes that does seem to be workable for what we want!
>
> > *without a following semicolon* might give a portable workaround, as if I'm
> > not mistaken POSIX mandates that redirections can be specified anywere on
> > the command line, and are to be evaluated from left to right.
>
> Yes, all shells support these as equivalent:
>
> 9>&2 sh k
> sh k 9>&2
>
> >
> > UPDATE: Yes, it seems to work. I'll add a testcase to the 'maint' branch in
> > case you and Jim decide to go with this solution (and you can confirm that it
> > really works).
>
> Cool! Definitely worth documenting in the automake manual, as owner of
> TESTS_ENVIRONMENT and as a client of init.sh functionality, as well as
> your proposed automake testcase addition to ensure we don't break it.
>
I'll then push the attached patch to automake master soonish (by tomorrow
or so) if there is no objection by then.
Regards,
Stefano
[0001-tests-check-portable-fd-redirection-in-TESTS_ENVIRON.patch (text/x-patch, inline)]
From 9f80020a24110e599b7dbdc5699dc8dbf5af0bd2 Mon Sep 17 00:00:00 2001
Message-Id: <9f80020a24110e599b7dbdc5699dc8dbf5af0bd2.1308037344.git.stefano.lattarini <at> gmail.com>
From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Date: Tue, 14 Jun 2011 09:41:14 +0200
Subject: [PATCH] tests: check portable fd redirection in TESTS_ENVIRONMENT
* tests/tests-environment-fd-redirect.test: New test.
* tests/Makefile.am (TESTS): Update.
Motivated by coreutils bug#8846:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
See also following CC:ed thread on bug-autoconf list:
<http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
---
ChangeLog | 10 ++++
tests/Makefile.am | 1 +
tests/Makefile.in | 1 +
tests/tests-environment-fd-redirect.test | 75 ++++++++++++++++++++++++++++++
4 files changed, 87 insertions(+), 0 deletions(-)
create mode 100755 tests/tests-environment-fd-redirect.test
diff --git a/ChangeLog b/ChangeLog
index aad41c7..e482cbd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,13 @@
+2011-06-14 Stefano Lattarini <stefano.lattarini <at> gmail.com>
+
+ tests: check portable fd redirection in TESTS_ENVIRONMENT
+ * tests/tests-environment-fd-redirect.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+ Motivated by coreutils bug#8846:
+ <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
+ See also following CC:ed thread on bug-autoconf list:
+ <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+
2011-06-08 Stefano Lattarini <stefano.lattarini <at> gmail.com>
test defs: new function 'fatal_', for hard errors
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e68f6d7..c0f39ce 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -949,6 +949,7 @@ tar2.test \
tar3.test \
target-cflags.test \
targetclash.test \
+tests-environment-fd-redirect.test \
txinfo.test \
txinfo2.test \
txinfo3.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 7e5fd09..4c223fc 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1216,6 +1216,7 @@ tar2.test \
tar3.test \
target-cflags.test \
targetclash.test \
+tests-environment-fd-redirect.test \
txinfo.test \
txinfo2.test \
txinfo3.test \
diff --git a/tests/tests-environment-fd-redirect.test b/tests/tests-environment-fd-redirect.test
new file mode 100755
index 0000000..2a0afa9
--- /dev/null
+++ b/tests/tests-environment-fd-redirect.test
@@ -0,0 +1,75 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test for a behaviour of `TESTS_ENVIRONMENT' and `AM_TESTS_ENVIRONMENT'
+# w.r.t. file descriptor redirections which, although undocumented,
+# is nonetheless required by Gnulib's 'tests/init.sh' and by coreutils'
+# testsuite.
+# The checked behaviour is that we can portably do file descriptor
+# redirections by placing them at the end of a {AM_,}TESTS_ENVIRONMENT
+# definition without a following semicolon. The need to support this
+# is detailedly motivated by coreutils bug#8846:
+# <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
+# and the following CC:ed thread on bug-autoconf list:
+# <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat >foo.test <<'END'
+#! /bin/sh
+echo " " $0: foofoofoo >&8
+echo " " $0: barbarbar >&9
+END
+chmod a+x foo.test
+
+$ACLOCAL
+$AUTOCONF
+
+# /bin/ksh seems more vulnerable to the issue highlighted in coreutils
+# bug#8846 than other shells are. In particular, the default Korn Shell
+# on Debian GNU/Linux is affected by the issue. So let's try to run our
+# test with /bin/ksh too, if that's available.
+if test "$SHELL" != /bin/ksh && test -f /bin/ksh; then
+ bin_ksh=/bin/ksh
+else
+ bin_ksh=:
+fi
+
+for sh in "$SHELL" "$bin_ksh"; do
+ test "$sh" = : && continue
+ for pfx in AM_ ''; do
+ unindent > Makefile.am <<END
+ TESTS = foo.test
+ ## No trailing semicolon here, *deliberately*.
+ ${pfx}TESTS_ENVIRONMENT = 8>&1 9>&8
+END
+ $AUTOMAKE -a
+ CONFIG_SHELL="$sh" $sh ./configure CONFIG_SHELL="$sh"
+ $MAKE check >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ grep '[ /]foo\.test: foofoofoo$' stdout
+ grep '[ /]foo\.test: barbarbar$' stdout
+ $EGREP '(foofoofoo|barbarbar)' foo.log && Exit 1
+ : # For shells with buggy 'set -e'.
+ done
+done
+
+:
--
1.7.2.3
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Tue, 14 Jun 2011 08:12:02 GMT)
Full text and
rfc822 format available.
Message #122 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Here's what I expect to do for coreutils,
along with an advice-update for gnulib's init.sh:
From e948173c1c461aac9f1c490061b257f55e42608d Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Tue, 14 Jun 2011 09:59:14 +0200
Subject: [PATCH] tests: accommodate HP-UX and Solaris shells
Running "make check" normally prints a diagnostic to the outermost
stderr (usually a tty) to explain why a test is skipped. It did this
by redirecting FD 9 to stderr (via "exec 9>&2") before invoking the
shell script. Shell scripts write skip-explanation to FD 9 via
init.sh's skip_ function. However, with Solaris 10's ksh and HP-UX,
the effects of "exec 9>&2" are canceled upon fork-and-exec, so we
would get a "Bad file number" diagnostic and no skip explanation on
those systems.
* tests/check.mk (TESTS_ENVIRONMENT): Redirect more portably, via
$(SHELL) "$$1" 9>&2, rather than the prior
exec 9>&2; $(SHELL) ...
Actually, we use "shell_or_perl_ 9>&2", to make this effective
also for the perl-based tests.
* tests/init.sh (stderr_fileno_): Update the advice in comments.
See http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488
for lots of discussion. Stefano Lattarini suggested the solution
of putting "9>&2" after the command. Reported by Bruno Haible.
---
tests/check.mk | 3 +--
tests/init.sh | 4 ++--
2 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/tests/check.mk b/tests/check.mk
index db7f067..9db96af 100644
--- a/tests/check.mk
+++ b/tests/check.mk
@@ -58,7 +58,6 @@ TESTS_ENVIRONMENT = \
test -d "$$tmp__" && test -w "$$tmp__" || tmp__=.; \
. $(srcdir)/envvar-check; \
TMPDIR=$$tmp__; export TMPDIR; \
- exec 9>&2; \
shell_or_perl_() { \
if grep '^\#!/usr/bin/perl' "$$1" > /dev/null; then \
if $(PERL) -e 'use warnings' > /dev/null 2>&1; then \
@@ -100,6 +99,6 @@ TESTS_ENVIRONMENT = \
REPLACE_GETCWD=$(REPLACE_GETCWD) \
; test -d /usr/xpg4/bin && PATH='/usr/xpg4/bin$(PATH_SEPARATOR)'"$$PATH"; \
PATH='$(abs_top_builddir)/src$(PATH_SEPARATOR)'"$$PATH" \
- ; shell_or_perl_
+ ; shell_or_perl_ 9>&2
VERBOSE = yes
diff --git a/tests/init.sh b/tests/init.sh
index 60d1bc1..a769d8b 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -68,8 +68,8 @@ Exit () { set +e; (exit $1); exit $1; }
# Print warnings (e.g., about skipped and failed tests) to this file number.
# Override by defining to say, 9, in init.cfg, and putting say,
-# "export ...ENVVAR_SETTINGS...; exec 9>&2; $(SHELL)" in the definition
-# of TESTS_ENVIRONMENT in your tests/Makefile.am file.
+# export ...ENVVAR_SETTINGS...; $(SHELL) "$$1" 9>&2
+# in the definition of TESTS_ENVIRONMENT in your tests/Makefile.am file.
# This is useful when using automake's parallel tests mode, to print
# the reason for skip/failure to console, rather than to the .log files.
: ${stderr_fileno_=2}
--
1.7.6.rc0.293.g40857
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Tue, 14 Jun 2011 08:17:02 GMT)
Full text and
rfc822 format available.
Message #125 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Jim Meyering wrote:
> Bruno Haible wrote:
>> On HP-UX 11.31, built with cc, coreutils-8.12 gives 3 test suite failures:
>>
>> FAIL: misc/printf-surprise (exit: 1)
>> FAIL: dd/nocache (exit: 1)
>> FAIL: du/inaccessible-cwd (exit: 1)
>>
>> Find attached the log file.
>>
> ...
>> FAIL: misc/printf-surprise (exit: 1)
>> ====================================
>>
>> ./init.sh: line 77: 1: Bad file number
>> printf (GNU coreutils) 8.12
>> Copyright (C) 2011 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.
> ...
>
> Hi Bruno
>
> Thank you for the testing and report.
> That "bad file number" error comes from this code in init.sh:
>
> : ${stderr_fileno_=2}
> warn_ () { echo "$@" 1>&$stderr_fileno_; }
>
> Because of that, the log contains less information than usual.
> If you can easily apply this patch and rerun the failing tests,
> I'd appreciate it. It should avoid the error and let us see
> more details about what is failing.
>
>
>>From 25e7bded3f2abff58540b0fcead2ba110d344bb0 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering <at> redhat.com>
> Date: Mon, 13 Jun 2011 12:07:14 +0200
> Subject: [PATCH] init.sh: accommodate shells for which 1>&$stderr_fileno_
> fails
>
> * tests/init.sh (warn_): Use eval to work around a bug in some shells,
> like those of Solaris 10 and HP-UX 11.11.
That change did not help, so I've reverted it.
That's the first patch.
The second one syncs from gnulib's init.sh.
From 83e424e7319e8ef2e3398aefbaade972770a6f97 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Mon, 13 Jun 2011 18:20:14 +0200
Subject: [PATCH 1/2] maint: revert previous commit
Revert "init.sh: accommodate shells for which 1>&$stderr_fileno_ fails"
This reverts commit 6fb9aeedd1b858a61d5cbf7f15782adf29ff733a.
That change did not solve the problem. For details, see
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846#74
---
tests/init.sh | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tests/init.sh b/tests/init.sh
index 25850af..14f2e26 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
# the reason for skip/failure to console, rather than to the .log files.
: ${stderr_fileno_=2}
-warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
+warn_ () { echo "$@" 1>&$stderr_fileno_; }
fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
--
1.7.6.rc0.293.g40857
From ef6e57b24b51f247119a83be6c81c6a8a30b87a4 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Mon, 13 Jun 2011 12:49:10 +0200
Subject: [PATCH 2/2] init.sh: sync from gnulib
* tests/init.sh: Sync recent changes from gnulib.
---
tests/init.sh | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/tests/init.sh b/tests/init.sh
index 14f2e26..60d1bc1 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -77,6 +77,7 @@ Exit () { set +e; (exit $1); exit $1; }
warn_ () { echo "$@" 1>&$stderr_fileno_; }
fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
+fatal_ () { warn_ "$ME_: hard error: $@"; Exit 99; }
framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
# Sanitize this shell to POSIX mode, if possible.
@@ -167,7 +168,10 @@ else
st_=$?
# $re_shell_ works just fine. Use it.
- test $st_ = 10 && break
+ if test $st_ = 10; then
+ gl_set_x_corrupts_stderr_=false
+ break
+ fi
# If this is our first marginally acceptable shell, remember it.
if test "$st_:$marginal_" = 9: ; then
@@ -400,7 +404,7 @@ mktempd_ ()
{
case $# in
2);;
- *) fail_ "Usage: $ME DIR TEMPLATE";;
+ *) fail_ "Usage: mktempd_ DIR TEMPLATE";;
esac
destdir_=$1
--
1.7.6.rc0.293.g40857
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Tue, 14 Jun 2011 08:34:02 GMT)
Full text and
rfc822 format available.
Message #128 received at 8846 <at> debbugs.gnu.org (full text, mbox):
On Tuesday 14 June 2011, Jim Meyering wrote:
> Here's what I expect to do for coreutils,
> along with an advice-update for gnulib's init.sh:
>
Thanks, I'll apply something similar to automake `tests/defs' soonish.
I have a doubt below, tough (see near the end).
> From e948173c1c461aac9f1c490061b257f55e42608d Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering <at> redhat.com>
> Date: Tue, 14 Jun 2011 09:59:14 +0200
> Subject: [PATCH] tests: accommodate HP-UX and Solaris shells
>
> Running "make check" normally prints a diagnostic to the outermost
> stderr (usually a tty) to explain why a test is skipped. It did this
> by redirecting FD 9 to stderr (via "exec 9>&2") before invoking the
> shell script. Shell scripts write skip-explanation to FD 9 via
> init.sh's skip_ function. However, with Solaris 10's ksh and HP-UX,
> the effects of "exec 9>&2" are canceled upon fork-and-exec,
>
BTW, this is true also for the default ksh on Debian GNU/Linux (but
note that this is just a "trivia fact", since on GNU/Linux basically
everybody uses either bash or dash as the non-interactive shell of
choice).
> so we would get a "Bad file number" diagnostic and no skip explanation
> on those systems.
> * tests/check.mk (TESTS_ENVIRONMENT): Redirect more portably, via
> $(SHELL) "$$1" 9>&2, rather than the prior
> exec 9>&2; $(SHELL) ...
> Actually, we use "shell_or_perl_ 9>&2", to make this effective
> also for the perl-based tests.
> * tests/init.sh (stderr_fileno_): Update the advice in comments.
> See http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488
> for lots of discussion. Stefano Lattarini suggested the solution
> of putting "9>&2" after the command. Reported by Bruno Haible.
> ---
> tests/check.mk | 3 +--
> tests/init.sh | 4 ++--
> 2 files changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/tests/check.mk b/tests/check.mk
> index db7f067..9db96af 100644
> --- a/tests/check.mk
> +++ b/tests/check.mk
> @@ -58,7 +58,6 @@ TESTS_ENVIRONMENT = \
> test -d "$$tmp__" && test -w "$$tmp__" || tmp__=.; \
> . $(srcdir)/envvar-check; \
> TMPDIR=$$tmp__; export TMPDIR; \
> - exec 9>&2; \
> shell_or_perl_() { \
> if grep '^\#!/usr/bin/perl' "$$1" > /dev/null; then \
> if $(PERL) -e 'use warnings' > /dev/null 2>&1; then \
> @@ -100,6 +99,6 @@ TESTS_ENVIRONMENT = \
> REPLACE_GETCWD=$(REPLACE_GETCWD) \
> ; test -d /usr/xpg4/bin && PATH='/usr/xpg4/bin$(PATH_SEPARATOR)'"$$PATH"; \
> PATH='$(abs_top_builddir)/src$(PATH_SEPARATOR)'"$$PATH" \
> - ; shell_or_perl_
> + ; shell_or_perl_ 9>&2
>
> VERBOSE = yes
> diff --git a/tests/init.sh b/tests/init.sh
> index 60d1bc1..a769d8b 100644
> --- a/tests/init.sh
> +++ b/tests/init.sh
> @@ -68,8 +68,8 @@ Exit () { set +e; (exit $1); exit $1; }
>
> # Print warnings (e.g., about skipped and failed tests) to this file number.
> # Override by defining to say, 9, in init.cfg, and putting say,
> -# "export ...ENVVAR_SETTINGS...; exec 9>&2; $(SHELL)" in the definition
> -# of TESTS_ENVIRONMENT in your tests/Makefile.am file.
> +# export ...ENVVAR_SETTINGS...; $(SHELL) "$$1" 9>&2
>
What is this `$$1' here for?
> +# in the definition of TESTS_ENVIRONMENT in your tests/Makefile.am file.
> # This is useful when using automake's parallel tests mode, to print
> # the reason for skip/failure to console, rather than to the .log files.
> : ${stderr_fileno_=2}
> --
> 1.7.6.rc0.293.g40857
>
Regards,
Stefano
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Tue, 14 Jun 2011 08:48:02 GMT)
Full text and
rfc822 format available.
Message #131 received at 8846 <at> debbugs.gnu.org (full text, mbox):
Stefano Lattarini wrote:
> On Tuesday 14 June 2011, Jim Meyering wrote:
>> Here's what I expect to do for coreutils,
>> along with an advice-update for gnulib's init.sh:
>>
> Thanks, I'll apply something similar to automake `tests/defs' soonish.
> I have a doubt below, tough (see near the end).
>
>> From e948173c1c461aac9f1c490061b257f55e42608d Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <meyering <at> redhat.com>
>> Date: Tue, 14 Jun 2011 09:59:14 +0200
>> Subject: [PATCH] tests: accommodate HP-UX and Solaris shells
>>
>> Running "make check" normally prints a diagnostic to the outermost
>> stderr (usually a tty) to explain why a test is skipped. It did this
>> by redirecting FD 9 to stderr (via "exec 9>&2") before invoking the
>> shell script. Shell scripts write skip-explanation to FD 9 via
>> init.sh's skip_ function. However, with Solaris 10's ksh and HP-UX,
>> the effects of "exec 9>&2" are canceled upon fork-and-exec,
>>
> BTW, this is true also for the default ksh on Debian GNU/Linux (but
> note that this is just a "trivia fact", since on GNU/Linux basically
> everybody uses either bash or dash as the non-interactive shell of
> choice).
Thanks.
>> so we would get a "Bad file number" diagnostic and no skip explanation
>> on those systems.
>> * tests/check.mk (TESTS_ENVIRONMENT): Redirect more portably, via
>> $(SHELL) "$$1" 9>&2, rather than the prior
>> exec 9>&2; $(SHELL) ...
>> Actually, we use "shell_or_perl_ 9>&2", to make this effective
>> also for the perl-based tests.
>> * tests/init.sh (stderr_fileno_): Update the advice in comments.
>> See http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488
>> for lots of discussion. Stefano Lattarini suggested the solution
>> of putting "9>&2" after the command. Reported by Bruno Haible.
>> ---
>> tests/check.mk | 3 +--
>> tests/init.sh | 4 ++--
>> 2 files changed, 3 insertions(+), 4 deletions(-)
>>
>> diff --git a/tests/check.mk b/tests/check.mk
>> index db7f067..9db96af 100644
>> --- a/tests/check.mk
>> +++ b/tests/check.mk
>> @@ -58,7 +58,6 @@ TESTS_ENVIRONMENT = \
>> test -d "$$tmp__" && test -w "$$tmp__" || tmp__=.; \
>> . $(srcdir)/envvar-check; \
>> TMPDIR=$$tmp__; export TMPDIR; \
>> - exec 9>&2; \
>> shell_or_perl_() { \
>> if grep '^\#!/usr/bin/perl' "$$1" > /dev/null; then \
>> if $(PERL) -e 'use warnings' > /dev/null 2>&1; then \
>> @@ -100,6 +99,6 @@ TESTS_ENVIRONMENT = \
>> REPLACE_GETCWD=$(REPLACE_GETCWD) \
>> ; test -d /usr/xpg4/bin && PATH='/usr/xpg4/bin$(PATH_SEPARATOR)'"$$PATH"; \
>> PATH='$(abs_top_builddir)/src$(PATH_SEPARATOR)'"$$PATH" \
>> - ; shell_or_perl_
>> + ; shell_or_perl_ 9>&2
>>
>> VERBOSE = yes
>> diff --git a/tests/init.sh b/tests/init.sh
>> index 60d1bc1..a769d8b 100644
>> --- a/tests/init.sh
>> +++ b/tests/init.sh
>> @@ -68,8 +68,8 @@ Exit () { set +e; (exit $1); exit $1; }
>>
>> # Print warnings (e.g., about skipped and failed tests) to this file number.
>> # Override by defining to say, 9, in init.cfg, and putting say,
>> -# "export ...ENVVAR_SETTINGS...; exec 9>&2; $(SHELL)" in the definition
>> -# of TESTS_ENVIRONMENT in your tests/Makefile.am file.
>> +# export ...ENVVAR_SETTINGS...; $(SHELL) "$$1" 9>&2
>>
> What is this `$$1' here for?
Good catch. It doesn't belong.
I fixed the log, too.
From 67811abfda04dafdde114a50556f0e1d2e9b6270 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Tue, 14 Jun 2011 09:59:14 +0200
Subject: [PATCH] tests: accommodate HP-UX and Solaris shells
Running "make check" normally prints a diagnostic to the outermost
stderr (usually a tty) to explain why a test is skipped. It did this
by redirecting FD 9 to stderr (via "exec 9>&2") before invoking the
shell script. Shell scripts write skip-explanation to FD 9 via
init.sh's skip_ function. However, with Solaris 10's ksh and HP-UX,
the effects of "exec 9>&2" are canceled upon fork-and-exec, so we
would get a "Bad file number" diagnostic and no skip explanation on
those systems.
* tests/check.mk (TESTS_ENVIRONMENT): Redirect more portably, via
"$(SHELL) 9>&2", rather than the prior "exec 9>&2; $(SHELL) ..."
Actually, we use "shell_or_perl_ 9>&2", to make this effective
also for the perl-based tests.
* tests/init.sh (stderr_fileno_): Update the advice in comments.
See http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488
for lots of discussion. Stefano Lattarini suggested the solution
of putting "9>&2" after the command. Reported by Bruno Haible.
---
tests/check.mk | 3 +--
tests/init.sh | 4 ++--
2 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/tests/check.mk b/tests/check.mk
index db7f067..9db96af 100644
--- a/tests/check.mk
+++ b/tests/check.mk
@@ -58,7 +58,6 @@ TESTS_ENVIRONMENT = \
test -d "$$tmp__" && test -w "$$tmp__" || tmp__=.; \
. $(srcdir)/envvar-check; \
TMPDIR=$$tmp__; export TMPDIR; \
- exec 9>&2; \
shell_or_perl_() { \
if grep '^\#!/usr/bin/perl' "$$1" > /dev/null; then \
if $(PERL) -e 'use warnings' > /dev/null 2>&1; then \
@@ -100,6 +99,6 @@ TESTS_ENVIRONMENT = \
REPLACE_GETCWD=$(REPLACE_GETCWD) \
; test -d /usr/xpg4/bin && PATH='/usr/xpg4/bin$(PATH_SEPARATOR)'"$$PATH"; \
PATH='$(abs_top_builddir)/src$(PATH_SEPARATOR)'"$$PATH" \
- ; shell_or_perl_
+ ; shell_or_perl_ 9>&2
VERBOSE = yes
diff --git a/tests/init.sh b/tests/init.sh
index 60d1bc1..4a52626 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -68,8 +68,8 @@ Exit () { set +e; (exit $1); exit $1; }
# Print warnings (e.g., about skipped and failed tests) to this file number.
# Override by defining to say, 9, in init.cfg, and putting say,
-# "export ...ENVVAR_SETTINGS...; exec 9>&2; $(SHELL)" in the definition
-# of TESTS_ENVIRONMENT in your tests/Makefile.am file.
+# export ...ENVVAR_SETTINGS...; $(SHELL) 9>&2
+# in the definition of TESTS_ENVIRONMENT in your tests/Makefile.am file.
# This is useful when using automake's parallel tests mode, to print
# the reason for skip/failure to console, rather than to the .log files.
: ${stderr_fileno_=2}
--
1.7.6.rc0.293.g40857
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Sun, 07 Aug 2011 09:05:01 GMT)
Full text and
rfc822 format available.
Message #134 received at submit <at> debbugs.gnu.org (full text, mbox):
tags 8846 + moreinfo
thanks
Bruno Haible wrote:
> Jim Meyering wrote:
>> --- a/tests/init.sh
>> +++ b/tests/init.sh
>> @@ -74,7 +74,7 @@ Exit () { set +e; (exit $1); exit $1; }
>> # the reason for skip/failure to console, rather than to the .log files.
>> : ${stderr_fileno_=2}
>>
>> -warn_ () { echo "$@" 1>&$stderr_fileno_; }
>> +warn_ () { eval 'echo "$@" 1>&'"$stderr_fileno_"; }
>> fail_ () { warn_ "$ME_: failed test: $@"; Exit 1; }
>> skip_ () { warn_ "$ME_: skipped test: $@"; Exit 77; }
>> framework_failure_ () { warn_ "$ME_: set-up failure: $@"; Exit 99; }
>
> After applying this patch, here's the test-suite.log that I now get.
Hi Bruno,
This turned into a long thread.
I think all issues raised here have been resolved.
Can you confirm?
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Sun, 07 Aug 2011 09:05:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Sun, 07 Aug 2011 12:41:02 GMT)
Full text and
rfc822 format available.
Message #140 received at submit <at> debbugs.gnu.org (full text, mbox):
Hi Jim,
> I think all issues raised here have been resolved.
> Can you confirm?
I think the du/inaccessible-cwd failure was not tackled. But anyway, feel
free to close it, and let's test again when you have a pre-release of
coreutils-8.13 that you want to get tested.
Bruno
--
In memoriam Szczepan Ścibior <http://pl.wikipedia.org/wiki/Szczepan_Ścibior> <http://home.clara.net/clinchy/neeb5.htm>
Information forwarded
to
owner <at> debbugs.gnu.org, bug-coreutils <at> gnu.org:
bug#8846; Package
coreutils.
(Sun, 07 Aug 2011 12:41:03 GMT)
Full text and
rfc822 format available.
Added tag(s) moreinfo.
Request was from
Jim Meyering <jim <at> meyering.net>
to
control <at> debbugs.gnu.org.
(Sun, 07 Aug 2011 16:52:01 GMT)
Full text and
rfc822 format available.
Reply sent
to
Jim Meyering <jim <at> meyering.net>:
You have taken responsibility.
(Sun, 07 Aug 2011 20:13:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Bruno Haible <bruno <at> clisp.org>:
bug acknowledged by developer.
(Sun, 07 Aug 2011 20:13:02 GMT)
Full text and
rfc822 format available.
Message #150 received at 8846-done <at> debbugs.gnu.org (full text, mbox):
Bruno Haible wrote:
> Hi Jim,
>
>> I think all issues raised here have been resolved.
>> Can you confirm?
>
> I think the du/inaccessible-cwd failure was not tackled. But anyway, feel
> free to close it, and let's test again when you have a pre-release of
> coreutils-8.13 that you want to get tested.
Thanks.
I think that failure is inevitable on any system with
such poor (save-cwd/restore-cwd--based) openat emulation.
I.e., since it has neither the *at functions nor the
/proc-based support to emulate them, the test must fail.
If only to forestall further bug reports, it'd be good to
arrange to skip that test on such systems.
From 24a047d93db642b8e73122a6e6553a475f216572 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Sun, 7 Aug 2011 22:09:42 +0200
Subject: [PATCH] tests: avoid lack-of-support du test failure on HP-UX 11.31
* tests/du/inaccessible-cwd: Skip this test on systems like HP-UX 11.31
that lack both the *at functions and the /proc/self/fd-based support
we might have used to emulate them. Reported by Bruno Haible in
http://debbugs.gnu.org/8846
---
tests/du/inaccessible-cwd | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/tests/du/inaccessible-cwd b/tests/du/inaccessible-cwd
index 220ac7b..3dfe601 100755
--- a/tests/du/inaccessible-cwd
+++ b/tests/du/inaccessible-cwd
@@ -22,6 +22,11 @@
. "${srcdir=.}/init.sh"; path_prepend_ ../src
print_ver_ du
+
+# Skip this test if your system has neither the openat-style functions
+# nor /proc/self/fd support with which to emulate them.
+require_openat_support_
+
skip_if_root_
cwd=`pwd`
--
1.7.4.4
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Mon, 05 Sep 2011 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 12 years and 237 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.