GNU bug report logs - #22745
guix http downloads don't resume

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: wishlist; Reported by: Danny Milosavljevic <dannym@HIDDEN>; dated Sat, 20 Feb 2016 08:36:01 UTC; Maintainer for guix is bug-guix@HIDDEN.
Severity set to 'wishlist' from 'normal' Request was from ludo@HIDDEN (Ludovic Courtès) to control <at> debbugs.gnu.org. Full text available.

Message received at 22745 <at> debbugs.gnu.org:


Received: (at 22745) by debbugs.gnu.org; 25 Mar 2016 08:35:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 25 04:35:28 2016
Received: from localhost ([127.0.0.1]:36802 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ajNDL-0006Vf-PB
	for submit <at> debbugs.gnu.org; Fri, 25 Mar 2016 04:35:27 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55104)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1ajNDK-0006VT-DG
 for 22745 <at> debbugs.gnu.org; Fri, 25 Mar 2016 04:35:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1ajNDB-0000VG-Oy
 for 22745 <at> debbugs.gnu.org; Fri, 25 Mar 2016 04:35:20 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:40588)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1ajNDB-0000VC-MN; Fri, 25 Mar 2016 04:35:17 -0400
Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:60988 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1ajND9-0004ql-Qy; Fri, 25 Mar 2016 04:35:16 -0400
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Danny Milosavljevic <dannym@HIDDEN>
Subject: Re: bug#22745: guix http downloads don't resume
References: <20160220093445.45cd64d2@HIDDEN>
Date: Fri, 25 Mar 2016 09:35:12 +0100
In-Reply-To: <20160220093445.45cd64d2@HIDDEN> (Danny Milosavljevic's
 message of "Sat, 20 Feb 2016 09:34:45 +0100")
Message-ID: <87poujt0ov.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 22745
Cc: 22745 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Danny Milosavljevic <dannym@HIDDEN> skribis:

> By now, guix reconfigure has tried to download the same texlive binary fr=
om hydra at least 5 times to the same machine, unsuccessfully breaking afte=
r about 1 GB each, for a total of 5 GB, always starting from the beginning.
>
> Would it be possible to just resume?

There does not seem to be an easy way to achieve this.

> Additionally, it doesn't seem like it checks the Content-Length in order =
to find out whether the connection broke before the file was done. Why does=
n't it?

For archives (the /nar/foo URLs), there is currently no =E2=80=98Content-Le=
ngth=E2=80=99
header at all (this is because Hydra, the software, generates those
files on the fly; we should tweak nginx to add a =E2=80=98Content-Length=E2=
=80=99
header, but this seems to require an external nginx plugin.)

However, the {mirror.,}hydra.gnu.org use chunked encoding, indeed, which
allows the HTTP client to detect truncated transfers.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22745; Package guix. Full text available.

Message received at 22745 <at> debbugs.gnu.org:


Received: (at 22745) by debbugs.gnu.org; 20 Feb 2016 08:44:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 20 03:44:50 2016
Received: from localhost ([127.0.0.1]:34562 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aX39m-0005td-DB
	for submit <at> debbugs.gnu.org; Sat, 20 Feb 2016 03:44:50 -0500
Received: from dd1012.kasserver.com ([85.13.128.8]:51665)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <dannym@HIDDEN>) id 1aX39k-0005tU-BL
 for 22745 <at> debbugs.gnu.org; Sat, 20 Feb 2016 03:44:48 -0500
Received: from localhost (91.141.3.82.wireless.dyn.drei.com [91.141.3.82])
 by dd1012.kasserver.com (Postfix) with ESMTPSA id 86FFC1CA08B0
 for <22745 <at> debbugs.gnu.org>; Sat, 20 Feb 2016 09:44:46 +0100 (CET)
Date: Sat, 20 Feb 2016 09:44:42 +0100
From: Danny Milosavljevic <dannym@HIDDEN>
To: 22745 <at> debbugs.gnu.org
Subject: error messages
Message-ID: <20160220094442.6033b960@HIDDEN>
X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 22745
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

----
Found valid signature for /gnu/store/36wqhbch45x055wj6gfsng00zkwfqg6n-texli=
ve-20150523-texmf.tar.xz
=46rom http://hydra.gnu.org/nar/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-2015=
0523-texmf.tar.xz
Downloading 36wqhb...-texlive-20150523-texmf.tar.xz (1.76GiB installed)...
 http://hydra.gnu.org/nar/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523=
-texmf.tar.xz 1.1MiB/s 13:24 | 893.9MiB transferred
bzip2: Data integrity error when decompressing.
 http://Input file =3D (stdin), output file =3D (stdout)wfqg6n-texlive-2015=
0523-texmf.tar.xz 1.1MiB/s 13:24 | 893.9MiB transferred

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

guix substitute: error: corrupt input while restoring '/gnu/store/36wqhbch4=
5x055wj6gfsng00zkwfqg6n-texlive-20150523-texmf.tar.xz' from #{read pipe}#
killing process 3867g/nar/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523=
-texmf.tar.xz 1.1MiB/s 13:24 | 893.9MiB transferred
guix system: error: build failed: some substitutes for the outputs of deriv=
ation `/gnu/store/83nkdyp9wl6zwflcm416xf1imppp7v9f-texlive-20150523-texmf.t=
ar.xz.drv' failed (usually happens due to networking issues); try `--fallba=
ck' to build derivation from source=20
---

Also, downloading it by wget right afterwards, it works just fine, all 1.77=
 GB of it. Wtf?




Information forwarded to bug-guix@HIDDEN:
bug#22745; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 20 Feb 2016 08:35:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 20 03:35:07 2016
Received: from localhost ([127.0.0.1]:34557 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aX30N-0005g2-Ef
	for submit <at> debbugs.gnu.org; Sat, 20 Feb 2016 03:35:07 -0500
Received: from eggs.gnu.org ([208.118.235.92]:45021)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <dannym@HIDDEN>) id 1aX30K-0005fN-Gb
 for submit <at> debbugs.gnu.org; Sat, 20 Feb 2016 03:35:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <dannym@HIDDEN>) id 1aX30E-000659-KJ
 for submit <at> debbugs.gnu.org; Sat, 20 Feb 2016 03:34:59 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:36487)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <dannym@HIDDEN>) id 1aX30E-00064w-H4
 for submit <at> debbugs.gnu.org; Sat, 20 Feb 2016 03:34:58 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35986)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <dannym@HIDDEN>) id 1aX30D-0005YE-JH
 for bug-guix@HIDDEN; Sat, 20 Feb 2016 03:34:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <dannym@HIDDEN>) id 1aX308-00061c-Ia
 for bug-guix@HIDDEN; Sat, 20 Feb 2016 03:34:57 -0500
Received: from dd1012.kasserver.com ([85.13.128.8]:56147)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <dannym@HIDDEN>) id 1aX308-00061L-CL
 for bug-guix@HIDDEN; Sat, 20 Feb 2016 03:34:52 -0500
Received: from localhost (91.141.3.82.wireless.dyn.drei.com [91.141.3.82])
 by dd1012.kasserver.com (Postfix) with ESMTPSA id BC0FF1CA08B0
 for <bug-guix@HIDDEN>; Sat, 20 Feb 2016 09:34:47 +0100 (CET)
Date: Sat, 20 Feb 2016 09:34:45 +0100
From: Danny Milosavljevic <dannym@HIDDEN>
To: bug-guix@HIDDEN
Subject: guix http downloads don't resume
Message-ID: <20160220093445.45cd64d2@HIDDEN>
X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.28; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Package: guix

By now, guix reconfigure has tried to download the same texlive binary from hydra at least 5 times to the same machine, unsuccessfully breaking after about 1 GB each, for a total of 5 GB, always starting from the beginning.

Would it be possible to just resume?

Additionally, it doesn't seem like it checks the Content-Length in order to find out whether the connection broke before the file was done. Why doesn't it?

The http-client seems to handle chunked encoding - so not sure whether it's a problem with hydra or with guix.




Acknowledgement sent to Danny Milosavljevic <dannym@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#22745; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 26 Apr 2016 10:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.