GNU bug report logs - #61052
[PATCH] download: Add url-fetch/xz-file.

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: Hilton Chain <hako@HIDDEN>; Keywords: patch moreinfo; dated Wed, 25 Jan 2023 09:09:01 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.
Added tag(s) moreinfo. Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 25 Feb 2023 19:25:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 25 14:25:44 2023
Received: from localhost ([127.0.0.1]:41564 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pW0Ax-0002fi-U1
	for submit <at> debbugs.gnu.org; Sat, 25 Feb 2023 14:25:44 -0500
Received: from lists.gnu.org ([209.51.188.17]:57704)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <me@HIDDEN>) id 1pW0Aw-0002fZ-7C
 for submit <at> debbugs.gnu.org; Sat, 25 Feb 2023 14:25:42 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <me@HIDDEN>)
 id 1pW0As-0008DI-D5; Sat, 25 Feb 2023 14:25:39 -0500
Received: from tobias.gr ([2a02:c205:2020:6054::1])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <me@HIDDEN>)
 id 1pW0Al-0002j7-K2; Sat, 25 Feb 2023 14:25:37 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=2018; bh=56ZWVc5z9JtMM
 kcVvITyxSkkkmYkr121MSA7Zl1ApXM=;
 h=references:in-reply-to:subject:cc:
 to:from:date; d=tobias.gr; b=N06CKa7KJ7KOyML/VqcHv3tsGz3QY0u8/W+81m6zf
 Q0RirRPKncJhsJZ++o8x+/nr9Bp0TMGneVC+HuEm5SE3uJJHXyTjC7N6Br2wbhTbbWQBIm
 9AXldX7KlQKCDYMP/2LTLugI3BR6jCLWnD1xXQbDwGJc084tdNrBlygihM8McjEShD+91r
 KQsABD0sNr5gE0us/u6nlBIH528t+daF/gX1pM7KyZBykTjrP9A2YfON//Y4Z5nCZATpYL
 O6Aw6fR7N/bBws/lZQZTGEZGPcxAYR4Amqne1aPXttOWLKWhsJyenaTcKubBlP75p2KcG4
 Zwvjh+Z0s6WFDc2oIgFzA==
Received: by submission.tobias.gr (OpenSMTPD) with ESMTP id 0a07d9f6;
 Sat, 25 Feb 2023 19:25:24 +0000 (UTC)
MIME-Version: 1.0
Date: Sat, 25 Feb 2023 20:25:24 +0100
From: Tobias Geerinckx-Rice <me@HIDDEN>
To: Hilton Chain <hako@HIDDEN>
Subject: Re: [PATCH] download: Add url-fetch/xz-file.
In-Reply-To: <87sffz2d9a.wl-hako@HIDDEN>
References: <87sffz2d9a.wl-hako@HIDDEN>
Message-ID: <16189d896f675c30ba117222b7dd36da@HIDDEN>
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=2a02:c205:2020:6054::1; envelope-from=me@HIDDEN;
 helo=tobias.gr
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-Debbugs-Envelope-To: submit
Cc: guix-devel@HIDDEN, guix-patches@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

Hi Hilton,

I agree with Ludo' and also wonder if a generic 
‘url-fetch/compressed-file’ wouldn't be better.  There are closure 
arguments to be made for this xz-only approach.  I don't know if they're 
convincing.  Cluebats welcome.

(I was going to bring up ‘url-fetch/tarbomb’ as an example, but it 
doesn't actually handle anything besides gzip!  Madness.)

On 2023-01-25 10:07, Hilton Chain wrote:
> +                            (setenv "XZ_OPT"
> +                                    (string-join (%xz-parallel-args)))

Why set this kluge…

> +                            (invoke (string-append #+xz "/bin/unxz")
> +                                    #$file-name)

…when we have full control over xz's arguments?

Kind regards,

T G-R

Sent from a Web browser.  Excuse or enjoy my brevity.




Information forwarded to guix-patches@HIDDEN:
bug#61052; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 25 Feb 2023 17:48:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 25 12:48:22 2023
Received: from localhost ([127.0.0.1]:41371 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pVyej-0005nL-QV
	for submit <at> debbugs.gnu.org; Sat, 25 Feb 2023 12:48:22 -0500
Received: from lists.gnu.org ([209.51.188.17]:48730)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1pVyeh-0005n4-K5
 for submit <at> debbugs.gnu.org; Sat, 25 Feb 2023 12:48:19 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1pVyeh-0004cL-7L; Sat, 25 Feb 2023 12:48:19 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1pVyeg-0002Zc-N7; Sat, 25 Feb 2023 12:48:18 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=8XDLBScZeQcCEz1ibxO1FuZ1mKj3Hov6RdYcguiRFPo=; b=anJ0Sgmc1KPFKeYEzmJM
 /dawlo4D4/RPYXjzpvVuQ4iT4ukwGHdqIVFHQ/KLLYgER6+NAzhb7WW067RVp6/P4Fe04fk3I0oPO
 k6Ahq36ymddaNEwJcmvxFj1nL5L9gBQkBFmBgz4CqA1P94GSmPBmsEXZiHOsMWLAXQNY/CxKq5Fqn
 y9sj25HGeu5BnvX6tReMCBK+ywNfAvOWftPsCK3zYPuxT7u87DYFebAW13Ln5mK0MtvKqVPLViZqz
 Ic8FHhsQZ8PIUyHnjv3xB6FqwH1lC4VLQzOQcpt82o4Yd4OWCbxMip0qNt4MM/LVDToIrWpryd9hW
 4aiXceL0SP4uZg==;
Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1pVyeg-0003Gy-13; Sat, 25 Feb 2023 12:48:18 -0500
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Hilton Chain <hako@HIDDEN>
Subject: Re: [PATCH] download: Add url-fetch/xz-file.
References: <87sffz2d9a.wl-hako@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Septidi 7 =?utf-8?Q?Vent=C3=B4se?= an 231 de la
 =?utf-8?Q?R=C3=A9volution=2C?= jour de l'Alaterne
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Sat, 25 Feb 2023 18:48:15 +0100
In-Reply-To: <87sffz2d9a.wl-hako@HIDDEN> (Hilton Chain's message of
 "Wed, 25 Jan 2023 17:07:45 +0800")
Message-ID: <87lekly6u8.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
Cc: guix-devel@HIDDEN, guix-patches@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Hilton Chain <hako@HIDDEN> skribis:

> * guix/download.scm (url-fetch/xz-file): New variable.

This LGTM, but do you know of a package that would use it?

I think we should add it if and only if it=E2=80=99s going to be used in Gu=
ix
proper.

Thanks,
Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#61052; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 25 Jan 2023 09:08:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 25 04:08:25 2023
Received: from localhost ([127.0.0.1]:58261 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pKblY-0001dN-Os
	for submit <at> debbugs.gnu.org; Wed, 25 Jan 2023 04:08:25 -0500
Received: from lists.gnu.org ([209.51.188.17]:60998)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hako@HIDDEN>) id 1pKblX-0001dG-6V
 for submit <at> debbugs.gnu.org; Wed, 25 Jan 2023 04:08:23 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <hako@HIDDEN>)
 id 1pKblR-0003WI-Fs; Wed, 25 Jan 2023 04:08:19 -0500
Received: from mail.boiledscript.com ([144.168.59.46])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <hako@HIDDEN>)
 id 1pKblO-00032p-3M; Wed, 25 Jan 2023 04:08:17 -0500
Date: Wed, 25 Jan 2023 17:07:45 +0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ultrarare.space;
 s=dkim; t=1674637686;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type;
 bh=EJvNdeN4Vy+EQYuAf55Ty8L8DYWCL5GMEfN88ZYOHt4=;
 b=GRT4VzGl0fbVPTlIgLn5xlN6O5/Y3hGt76Hg+r6lbPuHmM0O87G8RXsfFUsVAgGI+dpRBT
 4IHkb2IGTHX7FFKM0IAZwyHWtd6Vnkbc0Wej3Wp5zk//DbQ4WafzR9DTdHAQ9SaGgKM232
 va3CwZ7sSomBubk17XuKO8gTRCVPX/6hXHHwWRiDe4wY9LCydxX3u7w8YVUovID4TBre4S
 4EQ/G6iKH7V3qhmwBxnpb5NHn8gEqZykz4/MENCxQo8AHTAKsvq/BjDIMEKBxpkwSHRriZ
 zPXQ+hl45nE2noafgwe7KAJuxh39OQX1sAmP69iw9qk7/Z/328CA/RHV94v0ig==
Message-ID: <87sffz2d9a.wl-hako@HIDDEN>
From: Hilton Chain <hako@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] download: Add url-fetch/xz-file.
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Authentication-Results: mail.boiledscript.com;
 auth=pass smtp.mailfrom=hako@HIDDEN
X-Spamd-Bar: /
Received-SPF: pass client-ip=144.168.59.46; envelope-from=hako@HIDDEN;
 helo=mail.boiledscript.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
Cc: guix-devel@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)


* guix/download.scm (url-fetch/xz-file): New variable.
---
 guix/download.scm | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/guix/download.scm b/guix/download.scm
index 2e9ecb43fc..cce62c4185 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -41,6 +41,7 @@ (define-module (guix download)
             (url-fetch* . url-fetch)
             url-fetch/executable
             url-fetch/tarbomb
+            url-fetch/xz-file
             url-fetch/zipbomb
             download-to-store))
 
@@ -602,6 +603,48 @@ (define tar
                       #:graft? #f
                       #:local-build? #t)))
 
+(define* (url-fetch/xz-file url hash-algo hash
+                            #:optional name
+                            #:key (system (%current-system))
+                            (guile (default-guile)))
+  "Similar to 'url-fetch' but decompress the xz file at URL as the result.
+This is mainly used for adding xz-compressed patches to a origin definition."
+  (define file-name
+    (match url
+      ((head _ ...)
+       (basename head))
+      (_
+       (basename url))))
+  (define xz
+    (module-ref (resolve-interface '(gnu packages compression)) 'xz))
+
+  (mlet %store-monad ((drv (url-fetch* url hash-algo hash
+                                       (or name (basename file-name ".xz"))
+                                       #:system system
+                                       #:guile guile))
+                      (guile (package->derivation guile system)))
+    ;; Take the xz file, and simply decompress it.
+    ;; Use ungrafted xz so that the resulting file doesn't depend on whether
+    ;; grafts are enabled.
+    (gexp->derivation (or name file-name)
+                      (with-imported-modules '((guix build utils))
+                        #~(begin
+                            (use-modules (guix build utils))
+                            (setenv "XZ_OPT"
+                                    (string-join (%xz-parallel-args)))
+
+                            (copy-file #$drv #$file-name)
+                            (make-file-writable #$file-name)
+                            (invoke (string-append #+xz "/bin/unxz")
+                                    #$file-name)
+
+                            (copy-file (basename #$file-name ".xz")
+                                       #$output)))
+                      #:system system
+                      #:guile-for-build guile
+                      #:graft? #f
+                      #:local-build? #t)))
+
 (define* (url-fetch/zipbomb url hash-algo hash
                             #:optional name
                             #:key (system (%current-system))

base-commit: 718223c58c20fa066527fb30da2b5dccca82913f
-- 
2.39.1





Acknowledgement sent to Hilton Chain <hako@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#61052; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 16 Mar 2023 11:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.