GNU bug report logs - #19614
make dist exits succesfully even when tar exits with error

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: automake; Reported by: Dimitrios Apostolou <jimis@HIDDEN>; dated Fri, 16 Jan 2015 15:02:02 UTC; Maintainer for automake is bug-automake@HIDDEN.

Message received at 19614 <at> debbugs.gnu.org:


Received: (at 19614) by debbugs.gnu.org; 16 Jan 2015 17:15:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 16 12:15:21 2015
Received: from localhost ([127.0.0.1]:59501 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YCAUR-0001gI-Pm
	for submit <at> debbugs.gnu.org; Fri, 16 Jan 2015 12:15:20 -0500
Received: from mout.gmx.net ([212.227.17.22]:52461)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <jimis@HIDDEN>) id 1YCAUQ-0001Xm-7q
 for 19614 <at> debbugs.gnu.org; Fri, 16 Jan 2015 12:15:18 -0500
Received: from [10.25.0.194] ([195.159.234.190]) by mail.gmx.com (mrgmx102)
 with ESMTPSA (Nemesis) id 0M82zV-1Xpypi05eD-00ve9r; Fri, 16 Jan 2015 18:15:11
 +0100
Date: Fri, 16 Jan 2015 18:15:08 +0100 (CET)
From: Dimitrios Apostolou <jimis@HIDDEN>
To: Eric Blake <eblake@HIDDEN>
Subject: Re: bug#19614: make dist exits succesfully even when tar exits with
 error
In-Reply-To: <54B940B7.80002@HIDDEN>
Message-ID: <alpine.DEB.2.10.1501161807120.5189@localhost>
References: <alpine.DEB.2.10.1501161550450.5189@localhost>
 <54B940B7.80002@HIDDEN>
User-Agent: Alpine 2.10 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Provags-ID: V03:K0:u7uACp+p+2QRII+iUBiJTFs6pIRjyFPVepPBfn1LOpwIklW1Jow
 84NJqqK7u6SxQUiJlHUWLHsnmNCmyngczpPUv9qCYshXuJxvsFiUOTULt2XbGCEyneubJAt
 pEzqco2FYeC2jmd5SUSBC+9V27XzczvpZlDj6sCmEV0pkkep7Y+BxNIQBayzS0yT6pgjeVw
 37k0CBRQu+l2NfohzOHRQ==
X-UI-Out-Filterresults: notjunk:1;
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 19614
Cc: 19614 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

On Fri, 16 Jan 2015, Eric Blake wrote:
> On 01/16/2015 08:01 AM, Dimitrios Apostolou wrote:
>> Relevant output from "make V=1 dist":
>>
>> tardir=cfengine-3.7.0a1.5ffcc54 && tar --format=ustar -chf - "$tardir" |
>> GZIP=--best gzip -c >cfengine-3.7.0a1.5ffcc54.tar.gz
>
> It is portable to do something hairy like:
>
> { tar ...; echo $? > file; } | gzip ...
> inspect file
>
> It is even possible to avoid an intermediate file with even hairier exec
> operations to shuffle fds around.
>
> It may be simpler to break things into two steps, with an intermediate
> file instead of a pipeline, although I'm not sure it would be efficient.
>
> But you definitely make a point that we should fix things to detect tar
> failure.

What if we just grep stderr for *any* output from any command in the 
pipeline?

{ tar ... | gzip ... ; } 2>&1 1>/dev/null | grep .

Nobody guarantees that it can't fail silently or it can't succeed despite 
output in stderr, but it's pretty rare.


Dimitris





Information forwarded to bug-automake@HIDDEN:
bug#19614; Package automake. Full text available.

Message received at 19614 <at> debbugs.gnu.org:


Received: (at 19614) by debbugs.gnu.org; 16 Jan 2015 16:48:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 16 11:48:02 2015
Received: from localhost ([127.0.0.1]:59477 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YCA42-0000cm-8S
	for submit <at> debbugs.gnu.org; Fri, 16 Jan 2015 11:48:02 -0500
Received: from mx1.redhat.com ([209.132.183.28]:52524)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <eblake@HIDDEN>) id 1YCA3z-0000cF-Tf
 for 19614 <at> debbugs.gnu.org; Fri, 16 Jan 2015 11:48:00 -0500
Received: from int-mx13.intmail.prod.int.phx2.redhat.com
 (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
 by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t0GGlrPV008369
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL);
 Fri, 16 Jan 2015 11:47:53 -0500
Received: from [10.3.113.41] (ovpn-113-41.phx2.redhat.com [10.3.113.41])
 by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id
 t0GGlqpa014126; Fri, 16 Jan 2015 11:47:53 -0500
Message-ID: <54B940B7.80002@HIDDEN>
Date: Fri, 16 Jan 2015 09:47:51 -0700
From: Eric Blake <eblake@HIDDEN>
Organization: Red Hat, Inc.
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Thunderbird/31.3.0
MIME-Version: 1.0
To: Dimitrios Apostolou <jimis@HIDDEN>, 19614 <at> debbugs.gnu.org
Subject: Re: bug#19614: make dist exits succesfully even when tar exits with
 error
References: <alpine.DEB.2.10.1501161550450.5189@localhost>
In-Reply-To: <alpine.DEB.2.10.1501161550450.5189@localhost>
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="BtX7wL4Wl46EXoHEc2LnaqVjhB6bHHIj3"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 19614
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--BtX7wL4Wl46EXoHEc2LnaqVjhB6bHHIj3
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 01/16/2015 08:01 AM, Dimitrios Apostolou wrote:
> (Please keep me CC'd as I'm not subscribed.)

In general, GNU list policy is to reply-to-all, precisely so you don't
have to leave disclaimers like that :)

>=20
> Hello list,
>=20
> as the title says, I believe running "make dist" should fail when tar
> fails. The problem is that because there is a shell pipeline with gzip,=

> the exit code of the pipeline is 0. I'm not aware of a portable way to
> fix this (named pipes maybe?), but most convenient would be bash's "set=

> -o pipefail".

Alas, pipefail is not portable.

>=20
> Relevant output from "make V=3D1 dist":
>=20
> tardir=3Dcfengine-3.7.0a1.5ffcc54 && tar --format=3Dustar -chf - "$tard=
ir" |
> GZIP=3D--best gzip -c >cfengine-3.7.0a1.5ffcc54.tar.gz

It is portable to do something hairy like:

{ tar ...; echo $? > file; } | gzip ...
inspect file

It is even possible to avoid an intermediate file with even hairier exec
operations to shuffle fds around.

It may be simpler to break things into two steps, with an intermediate
file instead of a pipeline, although I'm not sure it would be efficient.

But you definitely make a point that we should fix things to detect tar
failure.

--=20
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--BtX7wL4Wl46EXoHEc2LnaqVjhB6bHHIj3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBCAAGBQJUuUC3AAoJEKeha0olJ0NqPTAH/1g0iC2Yb5fJ/omw3RBoeWms
qxiFUzeLSP6Kz4JVCV6e8MpJLcY9od9WnT8Z9fA2GAKnVtaXV3mXTHwO7yxvFHkD
aTTWMdDw+4cE3T88e8V5+0WnYmLODjeHxdwAuPV8Y+bIkG27fiu/u4bl+/Loi2uu
Uw+53X+tNAn7bVuwKORXHvlZBw8eljdoIMBQl/kmlUY2pHcJhaI6nBQtj9lsw7mR
Cify0OTtMuBonLVx1dmonVjfx/4PBV2kSVmXiMXuJI2O+hkB43rDSB0RTOdsN4EU
5EPAnk8yIyTxB6ZaR1V5WOxTdJIMnStCQXeNuy6P8aCtfbKtzvauDcrssmQesXk=
=Tv9E
-----END PGP SIGNATURE-----

--BtX7wL4Wl46EXoHEc2LnaqVjhB6bHHIj3--




Information forwarded to bug-automake@HIDDEN:
bug#19614; Package automake. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 16 Jan 2015 15:01:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 16 10:01:24 2015
Received: from localhost ([127.0.0.1]:59457 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YC8Oq-00065i-EX
	for submit <at> debbugs.gnu.org; Fri, 16 Jan 2015 10:01:24 -0500
Received: from eggs.gnu.org ([208.118.235.92]:41659)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <jimis@HIDDEN>) id 1YC8Oo-00065S-LQ
 for submit <at> debbugs.gnu.org; Fri, 16 Jan 2015 10:01:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <jimis@HIDDEN>) id 1YC8Of-0003PZ-9g
 for submit <at> debbugs.gnu.org; Fri, 16 Jan 2015 10:01:17 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,FREEMAIL_FROM
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:49151)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <jimis@HIDDEN>) id 1YC8Of-0003PV-5t
 for submit <at> debbugs.gnu.org; Fri, 16 Jan 2015 10:01:13 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:43376)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <jimis@HIDDEN>) id 1YC8Oe-0000Kt-0S
 for bug-automake@HIDDEN; Fri, 16 Jan 2015 10:01:13 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <jimis@HIDDEN>) id 1YC8OZ-0003Od-03
 for bug-automake@HIDDEN; Fri, 16 Jan 2015 10:01:11 -0500
Received: from mout.gmx.net ([212.227.15.18]:53831)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <jimis@HIDDEN>) id 1YC8OY-0003Nl-LQ
 for bug-automake@HIDDEN; Fri, 16 Jan 2015 10:01:06 -0500
Received: from [10.25.0.194] ([195.159.234.190]) by mail.gmx.com (mrgmx003)
 with ESMTPSA (Nemesis) id 0MhiTL-1YPgwH3cBf-00MrHD for
 <bug-automake@HIDDEN>; Fri, 16 Jan 2015 16:01:02 +0100
Date: Fri, 16 Jan 2015 16:01:01 +0100 (CET)
From: Dimitrios Apostolou <jimis@HIDDEN>
To: bug-automake@HIDDEN
Subject: make dist exits succesfully even when tar exits with error
Message-ID: <alpine.DEB.2.10.1501161550450.5189@localhost>
User-Agent: Alpine 2.10 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Provags-ID: V03:K0:euqsKeXGAaAL/zbJQNB7JKcccthNNivilFwJlBwNAHYbziB4oii
 PKGFpJ2/np4FS+bkn96IUf7bz2i/+LZ/2/OGvdM2boXY59+NVz5pmxt3IfMdxQp/PsFTLl7
 2uMsXob2jOQiHUjA/o+aoQI+LYYflUtEnbUEMqpeHfB1IZj6WaksM7x0S+yUqmvvUlhYT2C
 uTZ24/9ayUwBEQwGUMPJQ==
X-UI-Out-Filterresults: notjunk:1;
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic]
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.1 (----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.1 (----)

(Please keep me CC'd as I'm not subscribed.)

Hello list,

as the title says, I believe running "make dist" should fail when tar 
fails. The problem is that because there is a shell pipeline with gzip, 
the exit code of the pipeline is 0. I'm not aware of a portable way to fix 
this (named pipes maybe?), but most convenient would be bash's "set -o 
pipefail".

Relevant output from "make V=1 dist":

tardir=cfengine-3.7.0a1.5ffcc54 && tar --format=ustar -chf - "$tardir" | GZIP=--best gzip -c >cfengine-3.7.0a1.5ffcc54.tar.gz
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_password_hash_is_not_cached_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_unlock_with_password_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_modify_user_with_many_attributes_warn.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_with_hashed_password_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_lock_with_password_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_add_user_with_many_attributes_warn_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_with_password_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_newly_created_account_should_not_count_as_locked.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_with_many_attributes_hpux_trusted.cf: link name is too long; not dumped
tar: cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_add_user_with_hashed_password_hpux_trusted.cf: link name is too long; not dumped
tar: Exiting with failure status due to previous errors


(A separate bug report will come soon about the "link failure" despite tar 
"-h" option being used.)


Thanks,
Dimitris






Acknowledgement sent to Dimitrios Apostolou <jimis@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-automake@HIDDEN. Full text available.
Report forwarded to bug-automake@HIDDEN:
bug#19614; Package automake. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 16 Jan 2015 17:15:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.