GNU bug report logs - #29266
gzip-1.8.41 test results: help-version

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29266 in the body.
You can then email your comments to 29266 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gzip <at> gnu.org
Subject: gzip-1.8.41 test results: help-version
Date: Sun, 12 Nov 2017 00:39:18 +0100

[Message part 1 (text/plain, inline)]

Test results of gzip git of today + gnulib git of today (only 32-bit platforms):

On
FreeBSD/i386:
Haiku/i386:
HP-UX/hppa:
HP-UX/ia64:

FAIL: help-version

Find attached the test-suite.log of each platform.

[test-suite.freebsd-i386 (text/plain, attachment)]

[test-suite.haiku-i386 (text/plain, attachment)]

[test-suite.hpux-ia64 (text/plain, attachment)]

[test-suite.hpux-hppa (text/plain, attachment)]

Message #8 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version
Date: Sun, 12 Nov 2017 09:05:37 -0800

On Sat, Nov 11, 2017 at 3:39 PM, Bruno Haible <bruno <at> clisp.org> wrote:
> Test results of gzip git of today + gnulib git of today (only 32-bit platforms):
>
> On
> FreeBSD/i386:
> Haiku/i386:
> HP-UX/hppa:
> HP-UX/ia64:
>
> FAIL: help-version

Thanks, Bruno.
The FreeBSD failure looks like it is due to "more" not detecting write
failure when its output is redirected to /dev/full (in the zmore
test).
The Haiku failure is due to gzexe failing due to that system's lack of
hard link support:

+ eval 'env $i  in-6369 < $tmp_in > $tmp_out'
++ env gzexe in-6369
in-6369:        -15.4%
ln: failed to create hard link 'in-6369~' => 'in-6369': Operation not supported
/boot/home/gzip-1.8.41-9d3bb-dirty/build/tests/../gzexe: cannot backup
in-6369 as
in-6369~
+ echo FAIL: gzexe
FAIL: gzexe
+ fail=1

On hppa, the test that runs `gunzip --help > /dev/full` fails
unexpectedly. It should run only on a system with writable "char"
device /dev/full, and it should fail like this:

$ gunzip --help > /dev/full
echo: write error: No space left on device
[Exit 1]

But on your system, it exits with status 2.

The other hppa failures may be similar:

$ grep '^\*' test-suite.hpux-hppa|sort -u
*** gunzip: bad exit status `2' (expected 1),
*** gzexe: bad exit status `2' (expected 1),
*** zcat: bad exit status `2' (expected 1),
*** zcmp: bad exit status `0' (expected 2),
*** zdiff: bad exit status `0' (expected 2),
*** zegrep: bad exit status `0' (expected 2),
*** zfgrep: bad exit status `0' (expected 2),
*** zforce: bad exit status `2' (expected 1),
*** zgrep: bad exit status `0' (expected 2),
*** zless: bad exit status `2' (expected 1),
*** zmore: bad exit status `0' (expected 1),
*** znew: bad exit status `2' (expected 1),

Message #11 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Jim Meyering <jim <at> meyering.net>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version on HP-UX
Date: Sun, 12 Nov 2017 18:40:24 +0100

Hi Jim,

> On hppa, the test that runs `gunzip --help > /dev/full` fails
> unexpectedly. It should run only on a system with writable "char"
> device /dev/full, and it should fail like this:
> 
> $ gunzip --help > /dev/full
> echo: write error: No space left on device
> [Exit 1]
> 
> But on your system, it exits with status 2.

Yes:

$ ./gunzip --help > /dev/full 
echo: No space left on device
$ echo $?
2

Bruno

Message #14 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Jim Meyering <jim <at> meyering.net>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Sun, 12 Nov 2017 18:53:41 +0100

[Message part 1 (text/plain, inline)]

Hi Jim,

> The Haiku failure is due to gzexe failing due to that system's lack of
> hard link support:
> 
> + eval 'env $i  in-6369 < $tmp_in > $tmp_out'
> ++ env gzexe in-6369
> in-6369:        -15.4%
> ln: failed to create hard link 'in-6369~' => 'in-6369': Operation not supported
> /boot/home/gzip-1.8.41-9d3bb-dirty/build/tests/../gzexe: cannot backup
> in-6369 as
> in-6369~
> + echo FAIL: gzexe
> FAIL: gzexe
> + fail=1

Why does gzexe expect that hard links work? Even on Linux, some file systems
do not support hard links.

$ dd if=/dev/zero of=vfat.img bs=1048576 count=100
$ mkfs.vfat vfat.img
$ sudo mount -o loop vfat.img /mnt
$ cd /mnt
$ sudo bash
# tar xfz ~/gzip-1.8.41-9d3bb-dirty.tar.gz
# cd gzip-1.8.41-9d3bb-dirty
# ./configure
# make
# make check
=>
FAIL: help-version

test-suite.log is attached.

Bruno

[test-suite.log (text/x-log, attachment)]

Message #17 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version on HP-UX
Date: Sun, 12 Nov 2017 13:42:15 -0800

On Sun, Nov 12, 2017 at 9:40 AM, Bruno Haible <bruno <at> clisp.org> wrote:
> Hi Jim,
>
>> On hppa, the test that runs `gunzip --help > /dev/full` fails
>> unexpectedly. It should run only on a system with writable "char"
>> device /dev/full, and it should fail like this:
>>
>> $ gunzip --help > /dev/full
>> echo: write error: No space left on device
>> [Exit 1]
>>
>> But on your system, it exits with status 2.
>
> Yes:
>
> $ ./gunzip --help > /dev/full
> echo: No space left on device
> $ echo $?
> 2

Ohh...
So it's the HP-UX shell's "echo" that is detecting the write failure
but setting errno to 2 rather than the 1 that gzip users would expect
from gunzip.

I'm considering testing only zero-vs-nonzero exit status in the
help-version tests of those wrappers on systems for which echo fails
that way. Otherwise, with all of gzip's wrappers depending on other
programs that do not fail with the gzip-package-expected status, we'd
see too many reports of these unrelated failures.

Message #20 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Jim Meyering <jim <at> meyering.net>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version on HP-UX
Date: Sun, 12 Nov 2017 23:47:54 +0100

Hi Jim,

> >> But on your system, it exits with status 2.
> >
> > Yes:
> >
> > $ ./gunzip --help > /dev/full
> > echo: No space left on device
> > $ echo $?
> > 2
> 
> Ohh...
> So it's the HP-UX shell's "echo" that is detecting the write failure
> but setting errno to 2 rather than the 1

'bash' on the same system behaves the same way:
  $ bash gunzip --help > /dev/full
  echo: No space left on device
  $ echo $?
  2

And that is despite of

  $ bash -c 'type echo'
  echo is a shell builtin

$ bash -c 'echo foo > /dev/full'
bash: line 0: echo: write error: No space left on device
$ echo $?
1
$ bash -c '/usr/bin/echo foo > /dev/full'
/usr/bin/echo: No space left on device
$ echo $?
2

It looks like the 'gunzip' script is never using the bash built-in
but always the 'echo' in $PATH. Probably because of 'exec echo ...'.

This patch:

*** gunzip.orig Sat Nov 11 20:05:54 2017
--- gunzip      Sun Nov 12 22:38:15 2017
***************
*** 50,57 ****
  Report bugs to <bug-gzip <at> gnu.org>."
  
  case $1 in
! --help)    exec echo "$usage";;
! --version) exec echo "$version";;
  esac
  
  exec gzip -d "$@"
--- 50,57 ----
  Report bugs to <bug-gzip <at> gnu.org>."
  
  case $1 in
! --help)    echo "$usage" || exit 1; exit 0;;
! --version) echo "$version" || exit 1; exit 0;;
  esac
  
  exec gzip -d "$@"

works fine with bash but not with /bin/sh:

$ bash gunzip --help > /dev/full
gunzip: line 53: echo: write error: No space left on device
$ echo $?
1
$ ./gunzip --help > /dev/full
$ echo $?
0

But this one works with both:

*** gunzip.orig Sat Nov 11 20:05:54 2017
--- gunzip      Sun Nov 12 22:46:39 2017
***************
*** 50,57 ****
  Report bugs to <bug-gzip <at> gnu.org>."
  
  case $1 in
! --help)    exec echo "$usage";;
! --version) exec echo "$version";;
  esac
  
  exec gzip -d "$@"
--- 50,62 ----
  Report bugs to <bug-gzip <at> gnu.org>."
  
  case $1 in
! # Produce output and exit with code 1 if there is a write error.
! # Use 'exec echo', not plain 'echo', because the 'echo' built-in in
! # HP-UX /bin/sh does not check for write errors.
! # Use '|| exit 1', because the 'echo' program on HP-UX exits with
! # code 2 in case of a write error, but we want code 1.
! --help)    (exec echo "$usage") || exit 1; exit 0;;
! --version) (exec echo "$version") || exit 1; exit 0;;
  esac
  
  exec gzip -d "$@"


$ bash gunzip --help > /dev/full
echo: No space left on device
$ echo $?
1
$ ./gunzip --help > /dev/full
echo: No space left on device
$ echo $?
1

Message #23 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bruno Haible <bruno <at> clisp.org>, Jim Meyering <jim <at> meyering.net>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version on HP-UX
Date: Sun, 12 Nov 2017 23:53:35 -0800

[Message part 1 (text/plain, inline)]

Bruno Haible wrote:
> ! # Produce output and exit with code 1 if there is a write error.
> ! # Use 'exec echo', not plain 'echo', because the 'echo' built-in in
> ! # HP-UX /bin/sh does not check for write errors.
> ! # Use '|| exit 1', because the 'echo' program on HP-UX exits with
> ! # code 2 in case of a write error, but we want code 1.
> ! --help)    (exec echo "$usage") || exit 1; exit 0;;
> ! --version) (exec echo "$version") || exit 1; exit 0;;

Thanks for the patch. I don't think we need worry about the first problem, since 
gzip assumes a working POSIX shell and the first problem is a failure to conform 
to POSIX. We can ask builders on HP-UX to work around the problem by configuring 
with SHELL=/bin/bash, or with some other POSIX-compatible shell.

The second problem is indeed a bug in gzip, since it shouldn't assume that echo 
exits with status 1 on failure (it could be some other positive status).

I notice that some gzip scripts already fix that bug, and some other scripts do 
not fix it. Also, while we're at it, scripts should use printf instead of echo 
if the strings might contain backslash (at least in theory; admittedly a 
backslash in a version number would be pretty weird). I looked for these 
problems in all the scripts and installed the attached to fix what I found.

[0001-maint-script-diagnostics-status-cleanup.txt (text/plain, attachment)]

Message #26 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 29266 <at> debbugs.gnu.org, Bruno Haible <bruno <at> clisp.org>
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version on HP-UX
Date: Mon, 13 Nov 2017 07:34:47 -0800

[Message part 1 (text/plain, inline)]

On Sun, Nov 12, 2017 at 11:53 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> Bruno Haible wrote:
>>
>> ! # Produce output and exit with code 1 if there is a write error.
>> ! # Use 'exec echo', not plain 'echo', because the 'echo' built-in in
>> ! # HP-UX /bin/sh does not check for write errors.
>> ! # Use '|| exit 1', because the 'echo' program on HP-UX exits with
>> ! # code 2 in case of a write error, but we want code 1.
>> ! --help)    (exec echo "$usage") || exit 1; exit 0;;
>> ! --version) (exec echo "$version") || exit 1; exit 0;;
>
>
> Thanks for the patch. I don't think we need worry about the first problem,
> since gzip assumes a working POSIX shell and the first problem is a failure
> to conform to POSIX. We can ask builders on HP-UX to work around the problem
> by configuring with SHELL=/bin/bash, or with some other POSIX-compatible
> shell.
>
> The second problem is indeed a bug in gzip, since it shouldn't assume that
> echo exits with status 1 on failure (it could be some other positive
> status).
>
> I notice that some gzip scripts already fix that bug, and some other scripts
> do not fix it. Also, while we're at it, scripts should use printf instead of
> echo if the strings might contain backslash (at least in theory; admittedly
> a backslash in a version number would be pretty weird). I looked for these
> problems in all the scripts and installed the attached to fix what I found.

Nice. Thanks, Paul.
I've pushed this additional fix:

[zless-znew-exit.diff (text/plain, attachment)]

Message #29 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Wed, 15 Nov 2017 21:32:26 -0800

[Message part 1 (text/plain, inline)]

On Sun, Nov 12, 2017 at 9:53 AM, Bruno Haible <bruno <at> clisp.org> wrote:
> Hi Jim,
>
>> The Haiku failure is due to gzexe failing due to that system's lack of
>> hard link support:
>>
>> + eval 'env $i  in-6369 < $tmp_in > $tmp_out'
>> ++ env gzexe in-6369
>> in-6369:        -15.4%
>> ln: failed to create hard link 'in-6369~' => 'in-6369': Operation not supported
>> /boot/home/gzip-1.8.41-9d3bb-dirty/build/tests/../gzexe: cannot backup
>> in-6369 as
>> in-6369~
>> + echo FAIL: gzexe
>> FAIL: gzexe
>> + fail=1
>
> Why does gzexe expect that hard links work? Even on Linux, some file systems
> do not support hard links.
>
> $ dd if=/dev/zero of=vfat.img bs=1048576 count=100
> $ mkfs.vfat vfat.img
> $ sudo mount -o loop vfat.img /mnt
> $ cd /mnt
> $ sudo bash
> # tar xfz ~/gzip-1.8.41-9d3bb-dirty.tar.gz
> # cd gzip-1.8.41-9d3bb-dirty
> # ./configure
> # make
> # make check

Thanks, Bruno.
Here is a proposed patch:

[gzexe-ln-vs-haiku.diff (text/plain, attachment)]

Message #32 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Jim Meyering <jim <at> meyering.net>, Bruno Haible <bruno <at> clisp.org>
Cc: 29266 <at> debbugs.gnu.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Wed, 15 Nov 2017 23:43:04 -0800

On 11/15/2017 09:32 PM, Jim Meyering wrote:
> +  ln -f "$file" "$file~" 2>/dev/null || cp -f "$file" "$file~" || {

This will be problematic if the destination already exists, as the 
resulting permissions etc. may not be what the user intend. How about if 
we fall back on "mv -f" instead? Althoug this has the disadvantage of 
having a small window where "$file" does not exist, I think that's 
preferable to the disadvantage of using "cp".

Message #35 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 29266 <at> debbugs.gnu.org, Jim Meyering <jim <at> meyering.net>
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Thu, 16 Nov 2017 11:37:31 +0100

Paul Eggert wrote:
> > +  ln -f "$file" "$file~" 2>/dev/null || cp -f "$file" "$file~" || {
> 
> This will be problematic if the destination already exists, as the 
> resulting permissions etc. may not be what the user intend. How about if 
> we fall back on "mv -f" instead? Although this has the disadvantage of 
> having a small window where "$file" does not exist, I think that's 
> preferable to the disadvantage of using "cp".

How about

  ln -f "$file" "$file~" 2>/dev/null || { rm -f "$file~" && cp "$file" "$file~"; } || {

then? It fixes the problem "if the destination already exists", and does
NOT leave a window where "$file" does not exist.

Bruno

Message #40 received at 29266-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 29266-done <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Thu, 16 Nov 2017 07:18:32 -0800

On Thu, Nov 16, 2017 at 2:37 AM, Bruno Haible <bruno <at> clisp.org> wrote:
> Paul Eggert wrote:
>> > +  ln -f "$file" "$file~" 2>/dev/null || cp -f "$file" "$file~" || {
>>
>> This will be problematic if the destination already exists, as the
>> resulting permissions etc. may not be what the user intend. How about if
>> we fall back on "mv -f" instead? Although this has the disadvantage of
>> having a small window where "$file" does not exist, I think that's
>> preferable to the disadvantage of using "cp".
>
> How about
>
>   ln -f "$file" "$file~" 2>/dev/null || { rm -f "$file~" && cp "$file" "$file~"; } || {
>
> then? It fixes the problem "if the destination already exists", and does
> NOT leave a window where "$file" does not exist.

Which is more important? (tempered with qualification that it probably
doesn't matter at all, since this is in the context of running a
program (gzexe) that is used very rarely, and even less frequently in
a context where ln fails to create a hard link):

 - preserving backup file metadata (that backup may end up being the
sole copy of the original)
 - ensuring that the specified name does not go missing briefly

Given that the backup file may be the only remaining copy of the
original, I prefer the risk of an ephemeral ENOENT, so that we can
guarantee the running of gzexe does not destroy any file metadata.

Message #43 received at 29266-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 29266-done <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Thu, 16 Nov 2017 07:29:14 -0800

[Message part 1 (text/plain, inline)]

On Thu, Nov 16, 2017 at 7:18 AM, Jim Meyering <jim <at> meyering.net> wrote:
> On Thu, Nov 16, 2017 at 2:37 AM, Bruno Haible <bruno <at> clisp.org> wrote:
>> Paul Eggert wrote:
>>> > +  ln -f "$file" "$file~" 2>/dev/null || cp -f "$file" "$file~" || {
>>>
>>> This will be problematic if the destination already exists, as the
>>> resulting permissions etc. may not be what the user intend. How about if
>>> we fall back on "mv -f" instead? Although this has the disadvantage of
>>> having a small window where "$file" does not exist, I think that's
>>> preferable to the disadvantage of using "cp".

P.S., thanks to both of you for the reviews and suggestions.

FYI, I've also pushed this:

[gzip-make-generated-unwritable.diff (text/plain, attachment)]

Message #46 received at 29266 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 29266 <at> debbugs.gnu.org, jim <at> meyering.net, bruno <at> clisp.org
Subject: Re: bug#29266: gzip-1.8.41 test results: help-version and hard links
Date: Thu, 16 Nov 2017 09:09:00 -0800

On 11/16/2017 07:18 AM, Jim Meyering wrote:
> Which is more important? (tempered with qualification that it probably
> doesn't matter at all, since this is in the context of running a
> program (gzexe) that is used very rarely, and even less frequently in
> a context where ln fails to create a hard link):

Yes, you're right, this is bikeshedding on wheels.

>
>   - preserving backup file metadata (that backup may end up being the
> sole copy of the original)
>   - ensuring that the specified name does not go missing briefly
>
> Given that the backup file may be the only remaining copy of the
> original, I prefer the risk of an ephemeral ENOENT, so that we can
> guarantee the running of gzexe does not destroy any file metadata.

That was basically the same analysis I used.

I tried to get fancier by using mv to rename the backup file to a 
"backup backup" file, but even that has problems. My guess is that if we 
don't have hard links we're going to have problems no matter what, so we 
might as well have the simplest problems we can for this rarely-used 
program, and "mv -f" ended up with the easiest-to-explain problem that I 
could think of.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.