Christopher Baines <mail@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 3 Jun 2023 11:44:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 03 07:44:11 2023
Received: from localhost ([127.0.0.1]:41688 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1q5Pg3-0005Jg-46
for submit <at> debbugs.gnu.org; Sat, 03 Jun 2023 07:44:11 -0400
Received: from mira.cbaines.net ([212.71.252.8]:42596)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1q5Pg0-0005JX-Bh
for 61363 <at> debbugs.gnu.org; Sat, 03 Jun 2023 07:44:09 -0400
Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699])
by mira.cbaines.net (Postfix) with ESMTPSA id DB06C27BBE2;
Sat, 3 Jun 2023 12:44:07 +0100 (BST)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id e36f618a;
Sat, 3 Jun 2023 11:44:07 +0000 (UTC)
References: <871qn08uo7.fsf@HIDDEN>
<20230417145928.16693-1-mail@HIDDEN> <86353w76i6.fsf@HIDDEN>
User-agent: mu4e 1.10.2; emacs 28.2
From: Christopher Baines <mail@HIDDEN>
To: Simon Tournier <zimon.toutoune@HIDDEN>
Subject: Re: [bug#61363] [PATCH v3] self: Apply grafts to the outputs of the
guix derivation.
Date: Sat, 03 Jun 2023 12:41:57 +0100
In-reply-to: <86353w76i6.fsf@HIDDEN>
Message-ID: <87bkhw6a9k.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
Cc: 61363 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain
Simon Tournier <zimon.toutoune@HIDDEN> writes:
> Hi Chris,
>
> I am late to the party and probably do not well understand all that
> part. Just a quick comment in the same direction as Ludo.
>
> On Mon, 17 Apr 2023 at 15:59, Christopher Baines <mail@HIDDEN> wrote:
>
>> diff --git a/guix/self.scm b/guix/self.scm
>> index 74c953bd50..bbc0beaca8 100644
>> --- a/guix/self.scm
>> +++ b/guix/self.scm
>
> [...]
>
>> + (if graft?
>> + (explicit-grafting obj
>> + (map (compose force cdr) %packages))
>> + obj)))
>
> [...]
>
>> + (if graft?
>> + (explicit-grafting obj
>> + (map (compose force cdr) %packages))
>> + obj)))
>
> It means that the grafts are only applied to %packages, right?
>
> Other said, defined by:
>
> (define %packages
> (let ((ref (lambda (module variable)
> (delay
> (module-ref (resolve-interface
> `(gnu packages ,module))
> variable)))))
> `(("guile" . ,(ref 'guile 'guile-3.0-latest))
> ("guile-avahi" . ,(ref 'guile-xyz 'guile-avahi))
> ("guile-json" . ,(ref 'guile 'guile-json-4))
> ("guile-ssh" . ,(ref 'ssh 'guile-ssh))
> ("guile-git" . ,(ref 'guile 'guile-git))
> ("guile-semver" . ,(ref 'guile-xyz 'guile-semver))
> ("guile-lib" . ,(ref 'guile-xyz 'guile-lib))
> ("guile-sqlite3" . ,(ref 'guile 'guile-sqlite3))
> ("guile-zlib" . ,(ref 'guile 'guile-zlib))
> ("guile-lzlib" . ,(ref 'guile 'guile-lzlib))
> ("guile-zstd" . ,(ref 'guile 'guile-zstd))
> ("guile-gcrypt" . ,(ref 'gnupg 'guile-gcrypt))
> ("guile-gnutls" . ,(ref 'tls 'guile-gnutls))
> ("guix-daemon" . ,(ref 'package-management 'guix-daemon))
> ("disarchive" . ,(ref 'backup 'disarchive))
> ("guile-lzma" . ,(ref 'guile 'guile-lzma))
> ("gzip" . ,(ref 'compression 'gzip))
> ("bzip2" . ,(ref 'compression 'bzip2))
> ("xz" . ,(ref 'compression 'xz))
> ("po4a" . ,(ref 'gettext 'po4a))
> ("gettext-minimal" . ,(ref 'gettext 'gettext-minimal))
> ("gcc-toolchain" . ,(ref 'commencement 'gcc-toolchain))
> ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
> ("graphviz" . ,(ref 'graphviz 'graphviz))
> ("texinfo" . ,(ref 'texinfo 'texinfo)))))
>
> tweaked by e5c33837cbee98d460d9ae09b463501de6f15d97. And there is a
> slippery slope: the manual addition. These had been added with
> e5c33837cbee98d460d9ae09b463501de6f15d97:
>
> + ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
> + ("graphviz" . ,(ref 'graphviz 'graphviz))
> + ("guix-daemon" . ,(ref 'package-management 'guix-daemon))
> + ("texinfo" . ,(ref 'texinfo 'texinfo)))))
>
> Other said, what does it happen if we forget to manually update this
> list?
Well, specification->package in (guix self) won't work for the missing
packages.
It's possible to use packages outside of this list, but that doesn't
happen currently.
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmR7J4dfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XdA9Q//dkxQkEFKMrfuSmaL04m90ysfEDI1pa8e
ZPAXcD4UaEwHOPq/uKjmyAmUdsFnzRdsMIZjWJZkGh1UVJiGmA+g55h46F6Zf5Pb
jTHHvZeonmMSoi/YP5cLif4W0mvRpRoGRgtjdhBbHMEbfR3ww3AYoU8CrYs0AhHD
8C+pgnw6p+muWXQ4qVtaoXvN6EsyMzh++C4LrhMH7uTJp+C4Ki87LmAESTa8f7vl
4ti2k/Geawu/ENIouA8jG9SFOBOAGnGMacxd36XgpHBJs70e/oug5tkA/u9MwA20
pEXBXCDaZJiu1WkX3iLdiaQuRvE78pG8TJ2afl1iGIrG7HZ1E33kTXJqTYyYMcaH
8mBtar22hmRvvmQWLHUDchgA+wBeWWcuJZMLGQHrwV0zRyBPqD5HfQlKWRU4NFg+
PD+svkV9O1VkCVCqJ4Azm8CNE6FIyp+AuY+gdETZtWk+srkIC3yUD/30n3rssrKI
PqhfftkVQQNi4Ehdzt5ck7h+lIYBXXHv+nqitD1QtRVy1DCc+U9wAHRdMhbjBxYN
5Bp3xXOH4v/0+kLzx3GqfupU/8E7ik4xYAly6QlBUrIS7yiuOl7ktM/6uZkrJ3iM
UWIQCexXpQTXkrT7y1FIHPtTcPExgdLsotKj7LwCZlhPP6BlqvMeFIIoQDV8cKU2
xlwMQWIfyqw=
=0LsZ
-----END PGP SIGNATURE-----
--=-=-=--
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 16 May 2023 15:06:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 16 11:06:41 2023
Received: from localhost ([127.0.0.1]:46921 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pywG8-0001sQ-V8
for submit <at> debbugs.gnu.org; Tue, 16 May 2023 11:06:41 -0400
Received: from mail-wm1-f49.google.com ([209.85.128.49]:41280)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <zimon.toutoune@HIDDEN>) id 1pywFy-0001ql-Li
for 61363 <at> debbugs.gnu.org; Tue, 16 May 2023 11:06:31 -0400
Received: by mail-wm1-f49.google.com with SMTP id
5b1f17b1804b1-3f475364065so14833815e9.1
for <61363 <at> debbugs.gnu.org>; Tue, 16 May 2023 08:06:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20221208; t=1684249585; x=1686841585;
h=mime-version:message-id:date:references:in-reply-to:subject:cc:to
:from:from:to:cc:subject:date:message-id:reply-to;
bh=eq0vtNrtSRniouaCKk5q6iDiEAw2l1f1DG9TyiHhkUY=;
b=k2jACkb3v5zUalvMVdKj0FAWx20k+TXngMPMt9zndBABz3/63bNRpBrmDj5rf8Qqvh
1DZeIfAFIBsRUaYmdWBAo42cVg5wEmFKt1gmtu00XBlx85GkwqcpSR7pOng0B2tdeu4D
g/5PRTzjMm2Sf34BDHFfAn96oxfnoLiizNOZGeRONJa9bc8y4N438dLIU7TXoA7klPSk
D3nbZsxOvwlXUkRrTec/+XGj7/iJUP3cai3pM7oQcU0Gy+fORcNKVGEHa497vZNpQ08z
aCihWv11eJpdvTwQEPh8Ko8LWnc+OO8bCWFryU+UtwCDT87R043HRG6sRFjZqhsTTlss
iEBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20221208; t=1684249585; x=1686841585;
h=mime-version:message-id:date:references:in-reply-to:subject:cc:to
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=eq0vtNrtSRniouaCKk5q6iDiEAw2l1f1DG9TyiHhkUY=;
b=kfxsHsn6AQBtZKPW9rHQ7cG0S4xt7rLxprVttehLwT8ShsvUJ8CFOc7uROxE9Q6pgn
X6hR3RXj/wQqqUGEuGDQt3mDLZh1Uv0qVOnXO7VAuQGnKkyEWxs1Q+ZzRlxxpJdPY1jn
AD5OcZXzzIL5tTuxYGiZurto77sS8y4vS4UBo8snqWGFFvyT8n6562VE/XPcoScEsrmi
y1Hw88XgGWD5VcEM89nNu0rjXRfHYDi3a6ZlwbGJjEvVChO93ABb1/Sh1TRYSqG+gMnK
Xomqwohx9Mij43tqZfTZPRY9+fQi8Y4B8b+KsIIHTcINMRbC6XLFNJO9kxHiU27CrcI+
PW8Q==
X-Gm-Message-State: AC+VfDwQFvEXHyc+c4oKpgs5SuctOR8nHYTPW+X8iSPp8PGspg5oVrOE
knm6K2+0dJVHMf9XGBYopDI=
X-Google-Smtp-Source: ACHHUZ5HebFanQhk6sZ3CLDtK4FFS0ICrnsWgRkaFBK3G/aYfSLXtz5wz2qYdpPEC3cNkaWklJKs7w==
X-Received: by 2002:a05:600c:3514:b0:3f4:e426:e0b7 with SMTP id
h20-20020a05600c351400b003f4e426e0b7mr2764647wmq.3.1684249584283;
Tue, 16 May 2023 08:06:24 -0700 (PDT)
Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e])
by smtp.gmail.com with ESMTPSA id
m7-20020a7bcb87000000b003f4290720d0sm2562061wmi.47.2023.05.16.08.06.23
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Tue, 16 May 2023 08:06:24 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Christopher Baines <mail@HIDDEN>, 61363 <at> debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH v3] self: Apply grafts to the outputs of the
guix derivation.
In-Reply-To: <20230417145928.16693-1-mail@HIDDEN>
References: <871qn08uo7.fsf@HIDDEN>
<20230417145928.16693-1-mail@HIDDEN>
Date: Tue, 16 May 2023 15:25:37 +0200
Message-ID: <86353w76i6.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61363
Cc: Christopher Baines <mail@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi Chris,
I am late to the party and probably do not well understand all that
part. Just a quick comment in the same direction as Ludo.
On Mon, 17 Apr 2023 at 15:59, Christopher Baines <mail@HIDDEN> wrote:
> diff --git a/guix/self.scm b/guix/self.scm
> index 74c953bd50..bbc0beaca8 100644
> --- a/guix/self.scm
> +++ b/guix/self.scm
[...]
> + (if graft?
> + (explicit-grafting obj
> + (map (compose force cdr) %packages))
> + obj)))
[...]
> + (if graft?
> + (explicit-grafting obj
> + (map (compose force cdr) %packages))
> + obj)))
It means that the grafts are only applied to %packages, right?
Other said, defined by:
--8<---------------cut here---------------start------------->8---
(define %packages
(let ((ref (lambda (module variable)
(delay
(module-ref (resolve-interface
`(gnu packages ,module))
variable)))))
`(("guile" . ,(ref 'guile 'guile-3.0-latest))
("guile-avahi" . ,(ref 'guile-xyz 'guile-avahi))
("guile-json" . ,(ref 'guile 'guile-json-4))
("guile-ssh" . ,(ref 'ssh 'guile-ssh))
("guile-git" . ,(ref 'guile 'guile-git))
("guile-semver" . ,(ref 'guile-xyz 'guile-semver))
("guile-lib" . ,(ref 'guile-xyz 'guile-lib))
("guile-sqlite3" . ,(ref 'guile 'guile-sqlite3))
("guile-zlib" . ,(ref 'guile 'guile-zlib))
("guile-lzlib" . ,(ref 'guile 'guile-lzlib))
("guile-zstd" . ,(ref 'guile 'guile-zstd))
("guile-gcrypt" . ,(ref 'gnupg 'guile-gcrypt))
("guile-gnutls" . ,(ref 'tls 'guile-gnutls))
("guix-daemon" . ,(ref 'package-management 'guix-daemon))
("disarchive" . ,(ref 'backup 'disarchive))
("guile-lzma" . ,(ref 'guile 'guile-lzma))
("gzip" . ,(ref 'compression 'gzip))
("bzip2" . ,(ref 'compression 'bzip2))
("xz" . ,(ref 'compression 'xz))
("po4a" . ,(ref 'gettext 'po4a))
("gettext-minimal" . ,(ref 'gettext 'gettext-minimal))
("gcc-toolchain" . ,(ref 'commencement 'gcc-toolchain))
("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
("graphviz" . ,(ref 'graphviz 'graphviz))
("texinfo" . ,(ref 'texinfo 'texinfo)))))
--8<---------------cut here---------------end--------------->8---
tweaked by e5c33837cbee98d460d9ae09b463501de6f15d97. And there is a
slippery slope: the manual addition. These had been added with
e5c33837cbee98d460d9ae09b463501de6f15d97:
+ ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
+ ("graphviz" . ,(ref 'graphviz 'graphviz))
+ ("guix-daemon" . ,(ref 'package-management 'guix-daemon))
+ ("texinfo" . ,(ref 'texinfo 'texinfo)))))
Other said, what does it happen if we forget to manually update this
list?
Cheers,
simon
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.Received: (at 61363) by debbugs.gnu.org; 17 Apr 2023 15:07:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Apr 17 11:07:35 2023 Received: from localhost ([127.0.0.1]:55784 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1poQS7-0000gI-1O for submit <at> debbugs.gnu.org; Mon, 17 Apr 2023 11:07:35 -0400 Received: from mira.cbaines.net ([212.71.252.8]:42480) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1poQS5-0000gA-72 for 61363 <at> debbugs.gnu.org; Mon, 17 Apr 2023 11:07:33 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699]) by mira.cbaines.net (Postfix) with ESMTPSA id 2CFFC17923 for <61363 <at> debbugs.gnu.org>; Mon, 17 Apr 2023 16:07:32 +0100 (BST) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id bf600664 for <61363 <at> debbugs.gnu.org>; Mon, 17 Apr 2023 15:07:31 +0000 (UTC) References: <20230208075403.11788-1-mail@HIDDEN> <20230208075403.11788-2-mail@HIDDEN> <87sfey9i1t.fsf@HIDDEN> <878rgpeo28.fsf@HIDDEN> <871qm9aiw7.fsf@HIDDEN> User-agent: mu4e 1.8.13; emacs 28.2 From: Christopher Baines <mail@HIDDEN> To: 61363 <at> debbugs.gnu.org Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation. Date: Mon, 17 Apr 2023 16:06:16 +0100 In-reply-to: <871qm9aiw7.fsf@HIDDEN> Message-ID: <87bkjmfsy5.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 61363 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Christopher Baines <mail@HIDDEN> writes: > [[PGP Signed Part:Undecided]] > > Christopher Baines <mail@HIDDEN> writes: > >> Ludovic Court=C3=A8s <ludo@HIDDEN> writes: >> >>> 2. More importantly, manually listing packages that might require >>> grafting looks like a slippery slope (=E2=80=9Coops! we=E2=80=99re= not getting the >>> GnuTLS graft for that CVE, too bad=E2=80=9D). >>> >>> I designed and implemented several variants to try and delay grafting. >>> One of them consisted in carrying graft information in gexps: >>> >>> https://git.savannah.gnu.org/cgit/guix.git/log?h=3Dwip-gexp-grafts >>> >>> It=E2=80=99s kinda similar to what you=E2=80=99re proposing in that gra= ft information is >>> carried as far as possible. The main difference is that it=E2=80=99s a= utomated. >> >> That's interesting, I think that making grafting not specific to >> packages, and something where the replacement is handled at a lower >> level (e.g. gexps) would be an alternative way to handle this. >> >> Given that this approach works though, maybe the explicit-grafting >> functionality could just sit and be used inside of (guix self). Given >> that module is very explicit about what packages are used, it should be >> possible to arrange the code so it's very hard to miss a package out, >> which should address your concern about manually listing packages (maybe >> specification->package can be tweaked so that it's possible to get all >> the packages, and that can be the list considered for grafting). >> >> I don't know of any other places where this approach would be useful, so >> while it would be nice to have a more general grafting mechanism >> eventually, I'd also like to be able to make these changes to channel >> instance grafts sooner rather than later. > > I've sent a v2 series which changes along the above lines. The explicit > grafting stuff just sits in (guix self), and (guix self) more > rigeriously uses it's own definition of specification->package, which > should provide some protection against missing packages out. Obviously > it's not quite as rigerous as moving the grafting functionality in to > gexps, but hopefully it's rigerous enough for now. This has stalled a bit, but it would be good to try and get things merged. I've gone ahead and pushed the first two patches in the series I last sent, these just make minor changes to prepare for the functional change here. I've also resent that patch as as v3. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmQ9YLJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xcl8BAAm4rgvPrHIUH95Ip/HYbEafd0LmxShmR2 Wo7T1NXRkriwnp7S4PZ7xMALF8ztK5ggzZBvWrHdRo52GiEWen9nxyj7a6/PMZ5y IdWgjqJGLn4KNXo5E4FPEJ2ahgLtXwWcfF97sI8n4JSrrRB3B7cGO/0++P2I6ZsB kOd1/cKQTKvn+I35umZijhGUNbgewv0T5m5Cj0CW9wQPUyYSj1LKsp0SmMbz36Uk kVWtRE1rHbjqqMDqckbsqt8T2C4hagzkZXoRlKLvT2PJoQg8pjdoRMiEdzmz7WOU BNUVN8ATuLUTQaYqu349YeYrcmJWBgLNnw6FKaFhbqrhy3TfUcuZ6a8KavyF6riN 4LXpfFZOz05J9l+VExd6d+jWzfn8YX5Hnvj1WNOOuNbZngLstpM2prMXbJr3Lfv7 pdrpOULYcE8sBIAQfUMH7Gs9Ca7ktyW/GSsaq2wdYdNKkT3uWqIkpnuae+u3tAZL sOlRQ5qAsgc78AAcjscWxuhbvDjQ/ghtbLT4coIJ3S9Yk48bWkNzrZ6pTXZd8Fen cHlY2pzQgObFbBamZdsV+xLFlMOHbt87IXP/WqwsGLKKCjQ8P4I7FPvuegZy2l5j LNNmHsJKck4++Mr5HNH626jUKdgo0H9d5JjaF2mBahFlFUHj266SBuzVwnSsgtWI YzNQZs7oCI0= =1IL7 -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 17 Apr 2023 14:59:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Apr 17 10:59:33 2023
Received: from localhost ([127.0.0.1]:55779 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1poQKK-0000Sa-Qf
for submit <at> debbugs.gnu.org; Mon, 17 Apr 2023 10:59:33 -0400
Received: from mira.cbaines.net ([212.71.252.8]:42478)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1poQKI-0000SM-0x
for 61363 <at> debbugs.gnu.org; Mon, 17 Apr 2023 10:59:31 -0400
Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699])
by mira.cbaines.net (Postfix) with ESMTPSA id DE37317923;
Mon, 17 Apr 2023 15:59:28 +0100 (BST)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id 92f05399;
Mon, 17 Apr 2023 14:59:28 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 61363 <at> debbugs.gnu.org
Subject: [PATCH v3] self: Apply grafts to the outputs of the guix derivation.
Date: Mon, 17 Apr 2023 15:59:28 +0100
Message-Id: <20230417145928.16693-1-mail@HIDDEN>
X-Mailer: git-send-email 2.39.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
Cc: Christopher Baines <mail@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.
I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.
* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
#:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
Signed-off-by: Christopher Baines <mail@HIDDEN>
---
build-aux/build-self.scm | 4 +-
guix/self.scm | 101 +++++++++++++++++++++++++++++++--------
2 files changed, 84 insertions(+), 21 deletions(-)
diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
#:channel-metadata
'#$channel-metadata
#:pull-version
- #$pull-version)
+ #$pull-version
+ #:graft?
+ #$(%graft?))
#:system system))
derivation-file-name))))))
#:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index 74c953bd50..bbc0beaca8 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -21,6 +21,7 @@ (define-module (guix self)
#:use-module (guix config)
#:use-module (guix modules)
#:use-module (guix gexp)
+ #:use-module (guix grafts)
#:use-module (guix store)
#:use-module (guix monads)
#:use-module (guix discovery)
@@ -31,6 +32,7 @@ (define-module (guix self)
#:use-module ((guix build utils) #:select (find-files))
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-9)
+ #:use-module (srfi srfi-9 gnu)
#:use-module (srfi srfi-35)
#:use-module (ice-9 match)
#:export (make-config.scm
@@ -243,6 +245,50 @@ (define* (file-append* item file #:key (recursive? #t))
;; which isn't great.
(file-append item "/" file))))
+(define graft-derivation*
+ (store-lift graft-derivation))
+
+(define package-grafts*
+ (store-lift package-grafts))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+ (%explicit-grafting obj packages)
+ explicit-grafting?
+ (obj explicit-grafting-obj) ;obj
+ (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+ (match rec
+ (($ <explicit-grafting> obj packages)
+ (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+ (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+ system target)
+ (match explicit-grafting
+ (($ <explicit-grafting> obj packages)
+ (mlet* %store-monad ((drv (without-grafting
+ (lower-object obj system #:target target)))
+ (grafts
+ (mapm %store-monad
+ (lambda (pkg)
+ (package-grafts* pkg system #:target target))
+ packages)))
+ (match (delete-duplicates
+ (concatenate grafts))
+ (()
+ (return drv))
+ (grafts
+ (mlet %store-monad ((guile (package->derivation
+ (guile-for-grafts)
+ system #:graft? #f)))
+ (graft-derivation* drv grafts
+ #:system system
+ #:guile guile))))))))
+
(define* (locale-data source domain
#:optional (directory domain))
"Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
@@ -753,7 +799,8 @@ (define* (compiled-guix source #:key
(gzip (specification->package "gzip"))
(bzip2 (specification->package "bzip2"))
(xz (specification->package "xz"))
- (guix (specification->package "guix")))
+ (guix (specification->package "guix"))
+ (graft? #t))
"Return a file-like object that contains a compiled Guix."
(define guile-avahi
(specification->package "guile-avahi"))
@@ -1023,25 +1070,34 @@ (define (built-modules node-subset)
guile-lzma
dependencies)
#:guile guile-for-build
- #:guile-version guile-version)))
- (whole-package name modules dependencies
- #:command command
- #:guile guile-for-build
-
- ;; Include 'guix-daemon'. XXX: Here we inject an
- ;; older snapshot of guix-daemon, but that's a good
- ;; enough approximation for now.
- #:daemon (specification->package "guix-daemon")
-
- #:info (info-manual source)
- #:miscellany (miscellaneous-files source)
- #:guile-version guile-version)))
+ #:guile-version guile-version))
+ (obj
+ (whole-package name modules dependencies
+ #:command command
+ #:guile guile-for-build
+
+ ;; Include 'guix-daemon'. XXX: Here we inject
+ ;; an older snapshot of guix-daemon, but
+ ;; that's a good enough approximation for now.
+ #:daemon (specification->package "guix-daemon")
+
+ #:info (info-manual source)
+ #:miscellany (miscellaneous-files source)
+ #:guile-version guile-version)))
+ (if graft?
+ (explicit-grafting obj
+ (map (compose force cdr) %packages))
+ obj)))
((= 0 pull-version)
;; Legacy 'guix pull': return the .scm and .go files as one
;; directory.
- (built-modules (lambda (node)
- (list (node-source node)
- (node-compiled node)))))
+ (let ((obj (built-modules (lambda (node)
+ (list (node-source node)
+ (node-compiled node))))))
+ (if graft?
+ (explicit-grafting obj
+ (map (compose force cdr) %packages))
+ obj)))
(else
;; Unsupported 'guix pull' version.
#f)))
@@ -1271,7 +1327,8 @@ (define (process-directory directory files output)
(define* (guix-derivation source version
#:optional (guile-version (effective-version))
#:key (pull-version 0)
- channel-metadata)
+ channel-metadata
+ (graft? #t))
"Return, as a monadic value, the derivation to build the Guix from SOURCE
for GUILE-VERSION. Use VERSION as the version string. Use CHANNEL-METADATA
as the channel metadata sexp to include in (guix config).
@@ -1308,7 +1365,11 @@ (define guile
#:pull-version pull-version
#:guile-version (if (>= pull-version 1)
"3.0" guile-version)
- #:guile-for-build guile)))
+ #:guile-for-build guile
+ #:graft? graft?)))
(if guix
- (lower-object guix)
+ (if graft?
+ (lower-object guix)
+ (without-grafting
+ (lower-object guix)))
(return #f)))))
--
2.39.1
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.Received: (at 61363) by debbugs.gnu.org; 28 Feb 2023 15:50:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 28 10:50:55 2023 Received: from localhost ([127.0.0.1]:51806 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pX2Fi-0003yW-JD for submit <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:50:54 -0500 Received: from mira.cbaines.net ([212.71.252.8]:42294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1pX2Fg-0003yN-05 for 61363 <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:50:52 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699]) by mira.cbaines.net (Postfix) with ESMTPSA id 743DF16BBA; Tue, 28 Feb 2023 15:50:51 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 3b7f82bd; Tue, 28 Feb 2023 15:50:51 +0000 (UTC) References: <20230208075403.11788-1-mail@HIDDEN> <20230208075403.11788-2-mail@HIDDEN> <87sfey9i1t.fsf@HIDDEN> <878rgpeo28.fsf@HIDDEN> User-agent: mu4e 1.8.13; emacs 28.2 From: Christopher Baines <mail@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation. Date: Tue, 28 Feb 2023 15:47:11 +0000 In-reply-to: <878rgpeo28.fsf@HIDDEN> Message-ID: <871qm9aiw7.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 61363 Cc: 61363 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Christopher Baines <mail@HIDDEN> writes: > Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > >> 2. More importantly, manually listing packages that might require >> grafting looks like a slippery slope (=E2=80=9Coops! we=E2=80=99re = not getting the >> GnuTLS graft for that CVE, too bad=E2=80=9D). >> >> I designed and implemented several variants to try and delay grafting. >> One of them consisted in carrying graft information in gexps: >> >> https://git.savannah.gnu.org/cgit/guix.git/log?h=3Dwip-gexp-grafts >> >> It=E2=80=99s kinda similar to what you=E2=80=99re proposing in that graf= t information is >> carried as far as possible. The main difference is that it=E2=80=99s au= tomated. > > That's interesting, I think that making grafting not specific to > packages, and something where the replacement is handled at a lower > level (e.g. gexps) would be an alternative way to handle this. > > Given that this approach works though, maybe the explicit-grafting > functionality could just sit and be used inside of (guix self). Given > that module is very explicit about what packages are used, it should be > possible to arrange the code so it's very hard to miss a package out, > which should address your concern about manually listing packages (maybe > specification->package can be tweaked so that it's possible to get all > the packages, and that can be the list considered for grafting). > > I don't know of any other places where this approach would be useful, so > while it would be nice to have a more general grafting mechanism > eventually, I'd also like to be able to make these changes to channel > instance grafts sooner rather than later. I've sent a v2 series which changes along the above lines. The explicit grafting stuff just sits in (guix self), and (guix self) more rigeriously uses it's own definition of specification->package, which should provide some protection against missing packages out. Obviously it's not quite as rigerous as moving the grafting functionality in to gexps, but hopefully it's rigerous enough for now. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmP+IthfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xf2iw/8DZikf+lixwrP883zXiQT2XSlzldmnxnT Hv0mZyZl1pPIo78jVTcgumLasEsPa9CL7pVKjwvMYSmGLIhtdsAnV0heII2G15ch GVigOj2u74rexLjjbP1FabA244qDLFN+3BZAtwl9beqC+1B5FwLZzoHWPE/CW2/Z WluT3F2gBr/GX+61ksRHETzm+OHnwMCawHEOkhcmTtF9yQiSW2K0NkOvI8k1n5v7 rsQIbJ5BQdzZvbwKl5Q2oQd6y7nyliZctOl+rFjZ9a9rkrTNT4LdWC+5bPhsai8m uV8d+G0fOPgfaUvomvsxRqo9Xhi92KSFaoBwdgxMzoByuBF0U6k2QUQEyS1CwZVO j8jTGb+qnDfRlGbitdqjERGf4Yx4XAFfpuXYqBUJJLqG9dnES5JyOT7WLhrsVdFx wAI35UkCoopEirexXJ8gdmKFWKo3YEeRFXKme8qP+8wvFZ0jv4p7quEwBibE+8Tw JHh7my12WxJ/YURo4DoM6h2U25kq5QUtzHVyTSdrYFIeM1kCcy0c2DllWG7nUYzW 23Kz4veoXfY7hyfuU4Bi+VWFdVxlDcZS06wItO4VUbibJ3PPcOZ7yt1cA2DLLMDs zJgxebMvTusv/l2QLXvRJTv9/Y1h66Brs8sE4N4UQOb7Fmtom07N+u5C+MtXzcq7 re6XFrk5shU= =afSK -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 28 Feb 2023 15:47:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 28 10:47:14 2023
Received: from localhost ([127.0.0.1]:51796 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pX2CA-0003oC-Cx
for submit <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:47:14 -0500
Received: from mira.cbaines.net ([212.71.252.8]:42292)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1pX2C1-0003nR-PG
for 61363 <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:47:06 -0500
Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699])
by mira.cbaines.net (Postfix) with ESMTPSA id 6EAB916C22
for <61363 <at> debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:04 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id 11115824
for <61363 <at> debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:03 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 61363 <at> debbugs.gnu.org
Subject: [PATCH v2 3/3] self: Apply grafts to the outputs of the guix
derivation.
Date: Tue, 28 Feb 2023 15:47:03 +0000
Message-Id: <20230228154703.3952-3-mail@HIDDEN>
X-Mailer: git-send-email 2.39.1
In-Reply-To: <20230228154703.3952-1-mail@HIDDEN>
References: <20230228154703.3952-1-mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.
I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.
* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
#:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
---
build-aux/build-self.scm | 4 +-
guix/self.scm | 101 +++++++++++++++++++++++++++++++--------
2 files changed, 84 insertions(+), 21 deletions(-)
diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
#:channel-metadata
'#$channel-metadata
#:pull-version
- #$pull-version)
+ #$pull-version
+ #:graft?
+ #$(%graft?))
#:system system))
derivation-file-name))))))
#:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index c5de3ab8fc..8842275ff8 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -22,6 +22,7 @@ (define-module (guix self)
#:use-module (guix i18n)
#:use-module (guix modules)
#:use-module (guix gexp)
+ #:use-module (guix grafts)
#:use-module (guix store)
#:use-module (guix monads)
#:use-module (guix discovery)
@@ -32,6 +33,7 @@ (define-module (guix self)
#:use-module ((guix build utils) #:select (find-files))
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-9)
+ #:use-module (srfi srfi-9 gnu)
#:use-module (srfi srfi-35)
#:use-module (ice-9 match)
#:export (make-config.scm
@@ -244,6 +246,50 @@ (define* (file-append* item file #:key (recursive? #t))
;; which isn't great.
(file-append item "/" file))))
+(define graft-derivation*
+ (store-lift graft-derivation))
+
+(define package-grafts*
+ (store-lift package-grafts))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+ (%explicit-grafting obj packages)
+ explicit-grafting?
+ (obj explicit-grafting-obj) ;obj
+ (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+ (match rec
+ (($ <explicit-grafting> obj packages)
+ (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+ (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+ system target)
+ (match explicit-grafting
+ (($ <explicit-grafting> obj packages)
+ (mlet* %store-monad ((drv (without-grafting
+ (lower-object obj system #:target target)))
+ (grafts
+ (mapm %store-monad
+ (lambda (pkg)
+ (package-grafts* pkg system #:target target))
+ packages)))
+ (match (delete-duplicates
+ (concatenate grafts))
+ (()
+ (return drv))
+ (grafts
+ (mlet %store-monad ((guile (package->derivation
+ (guile-for-grafts)
+ system #:graft? #f)))
+ (graft-derivation* drv grafts
+ #:system system
+ #:guile guile))))))))
+
(define* (locale-data source domain
#:optional (directory domain))
"Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
@@ -754,7 +800,8 @@ (define* (compiled-guix source #:key
(gzip (specification->package "gzip"))
(bzip2 (specification->package "bzip2"))
(xz (specification->package "xz"))
- (guix (specification->package "guix")))
+ (guix (specification->package "guix"))
+ (graft? #t))
"Return a file-like object that contains a compiled Guix."
(define guile-avahi
(specification->package "guile-avahi"))
@@ -1024,25 +1071,34 @@ (define (built-modules node-subset)
guile-lzma
dependencies)
#:guile guile-for-build
- #:guile-version guile-version)))
- (whole-package name modules dependencies
- #:command command
- #:guile guile-for-build
-
- ;; Include 'guix-daemon'. XXX: Here we inject an
- ;; older snapshot of guix-daemon, but that's a good
- ;; enough approximation for now.
- #:daemon (specification->package "guix-daemon")
-
- #:info (info-manual source)
- #:miscellany (miscellaneous-files source)
- #:guile-version guile-version)))
+ #:guile-version guile-version))
+ (obj
+ (whole-package name modules dependencies
+ #:command command
+ #:guile guile-for-build
+
+ ;; Include 'guix-daemon'. XXX: Here we inject
+ ;; an older snapshot of guix-daemon, but
+ ;; that's a good enough approximation for now.
+ #:daemon (specification->package "guix-daemon")
+
+ #:info (info-manual source)
+ #:miscellany (miscellaneous-files source)
+ #:guile-version guile-version)))
+ (if graft?
+ (explicit-grafting obj
+ (map (compose force cdr) %packages))
+ obj)))
((= 0 pull-version)
;; Legacy 'guix pull': return the .scm and .go files as one
;; directory.
- (built-modules (lambda (node)
- (list (node-source node)
- (node-compiled node)))))
+ (let ((obj (built-modules (lambda (node)
+ (list (node-source node)
+ (node-compiled node))))))
+ (if graft?
+ (explicit-grafting obj
+ (map (compose force cdr) %packages))
+ obj)))
(else
;; Unsupported 'guix pull' version.
#f)))
@@ -1272,7 +1328,8 @@ (define (process-directory directory files output)
(define* (guix-derivation source version
#:optional (guile-version (effective-version))
#:key (pull-version 0)
- channel-metadata)
+ channel-metadata
+ (graft? #t))
"Return, as a monadic value, the derivation to build the Guix from SOURCE
for GUILE-VERSION. Use VERSION as the version string. Use CHANNEL-METADATA
as the channel metadata sexp to include in (guix config).
@@ -1309,7 +1366,11 @@ (define guile
#:pull-version pull-version
#:guile-version (if (>= pull-version 1)
"3.0" guile-version)
- #:guile-for-build guile)))
+ #:guile-for-build guile
+ #:graft? graft?)))
(if guix
- (lower-object guix)
+ (if graft?
+ (lower-object guix)
+ (without-grafting
+ (lower-object guix)))
(return #f)))))
--
2.39.1
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 28 Feb 2023 15:47:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 28 10:47:08 2023
Received: from localhost ([127.0.0.1]:51794 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pX2C3-0003nq-PM
for submit <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:47:08 -0500
Received: from mira.cbaines.net ([212.71.252.8]:42288)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1pX2C1-0003nQ-Oj
for 61363 <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:47:06 -0500
Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699])
by mira.cbaines.net (Postfix) with ESMTPSA id 645EC16BBF
for <61363 <at> debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:04 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id 667d7896
for <61363 <at> debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:03 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 61363 <at> debbugs.gnu.org
Subject: [PATCH v2 2/3] self: Restructure accessing packages.
Date: Tue, 28 Feb 2023 15:47:02 +0000
Message-Id: <20230228154703.3952-2-mail@HIDDEN>
X-Mailer: git-send-email 2.39.1
In-Reply-To: <20230228154703.3952-1-mail@HIDDEN>
References: <20230228154703.3952-1-mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Both for consistency (always use specification->package as defined in this
module) and so that all the packages that are used can be accessed (which
comes in useful when applying grafts).
* guix/self.scm (%packages): New variable.
(specification->package): Use %packages.
(locale-data, translate-texi-manuals, info-manual, guix-command,
compiled-guix): Use specification->package.
---
guix/self.scm | 97 +++++++++++++++++++++++++--------------------------
1 file changed, 48 insertions(+), 49 deletions(-)
diff --git a/guix/self.scm b/guix/self.scm
index 93019e1c64..c5de3ab8fc 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -44,34 +44,42 @@ (define-module (guix self)
;;; Dependency handling.
;;;
-(define specification->package
+(define %packages
+ (let ((ref (lambda (module variable)
+ (delay
+ (module-ref (resolve-interface
+ `(gnu packages ,module))
+ variable)))))
+ `(("guile" . ,(ref 'guile 'guile-3.0-latest))
+ ("guile-avahi" . ,(ref 'guile-xyz 'guile-avahi))
+ ("guile-json" . ,(ref 'guile 'guile-json-4))
+ ("guile-ssh" . ,(ref 'ssh 'guile-ssh))
+ ("guile-git" . ,(ref 'guile 'guile-git))
+ ("guile-semver" . ,(ref 'guile-xyz 'guile-semver))
+ ("guile-lib" . ,(ref 'guile-xyz 'guile-lib))
+ ("guile-sqlite3" . ,(ref 'guile 'guile-sqlite3))
+ ("guile-zlib" . ,(ref 'guile 'guile-zlib))
+ ("guile-lzlib" . ,(ref 'guile 'guile-lzlib))
+ ("guile-zstd" . ,(ref 'guile 'guile-zstd))
+ ("guile-gcrypt" . ,(ref 'gnupg 'guile-gcrypt))
+ ("guile-gnutls" . ,(ref 'tls 'guile-gnutls))
+ ("guix-daemon" . ,(ref 'package-management 'guix-daemon))
+ ("disarchive" . ,(ref 'backup 'disarchive))
+ ("guile-lzma" . ,(ref 'guile 'guile-lzma))
+ ("gzip" . ,(ref 'compression 'gzip))
+ ("bzip2" . ,(ref 'compression 'bzip2))
+ ("xz" . ,(ref 'compression 'xz))
+ ("po4a" . ,(ref 'gettext 'po4a))
+ ("gettext-minimal" . ,(ref 'gettext 'gettext-minimal))
+ ("gcc-toolchain" . ,(ref 'commencement 'gcc-toolchain))
+ ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
+ ("graphviz" . ,(ref 'graphviz 'graphviz))
+ ("texinfo" . ,(ref 'texinfo 'texinfo)))))
+
+(define (specification->package name)
;; Use our own variant of that procedure because that of (gnu packages)
;; would traverse all the .scm files, which is wasteful.
- (let ((ref (lambda (module variable)
- (module-ref (resolve-interface module) variable))))
- (match-lambda
- ("guile" (ref '(gnu packages guile) 'guile-3.0-latest))
- ("guile-avahi" (ref '(gnu packages guile-xyz) 'guile-avahi))
- ("guile-json" (ref '(gnu packages guile) 'guile-json-4))
- ("guile-ssh" (ref '(gnu packages ssh) 'guile-ssh))
- ("guile-git" (ref '(gnu packages guile) 'guile-git))
- ("guile-semver" (ref '(gnu packages guile-xyz) 'guile-semver))
- ("guile-lib" (ref '(gnu packages guile-xyz) 'guile-lib))
- ("guile-sqlite3" (ref '(gnu packages guile) 'guile-sqlite3))
- ("guile-zlib" (ref '(gnu packages guile) 'guile-zlib))
- ("guile-lzlib" (ref '(gnu packages guile) 'guile-lzlib))
- ("guile-zstd" (ref '(gnu packages guile) 'guile-zstd))
- ("guile-gcrypt" (ref '(gnu packages gnupg) 'guile-gcrypt))
- ("guile-gnutls" (ref '(gnu packages tls) 'guile-gnutls))
- ("disarchive" (ref '(gnu packages backup) 'disarchive))
- ("guile-lzma" (ref '(gnu packages guile) 'guile-lzma))
- ("gzip" (ref '(gnu packages compression) 'gzip))
- ("bzip2" (ref '(gnu packages compression) 'bzip2))
- ("xz" (ref '(gnu packages compression) 'xz))
- ("po4a" (ref '(gnu packages gettext) 'po4a))
- ("gettext" (ref '(gnu packages gettext) 'gettext-minimal))
- ("gcc-toolchain" (ref '(gnu packages commencement) 'gcc-toolchain))
- (_ #f)))) ;no such package
+ (and=> (assoc-ref %packages name) force))
;;;
@@ -240,9 +248,8 @@ (define* (locale-data source domain
#:optional (directory domain))
"Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
DOMAIN, a gettext domain."
- (define gettext
- (module-ref (resolve-interface '(gnu packages gettext))
- 'gettext-minimal))
+ (define gettext-minimal
+ (specification->package "gettext-minimal"))
(define build
(with-imported-modules '((guix build utils))
@@ -258,7 +265,7 @@ (define (compile language)
(let ((gmo (string-append #$output "/" language "/LC_MESSAGES/"
#$domain ".mo")))
(mkdir-p (dirname gmo))
- (invoke #+(file-append gettext "/bin/msgfmt")
+ (invoke #+(file-append gettext-minimal "/bin/msgfmt")
"-c" "--statistics" "--verbose"
"-o" gmo
(string-append po-directory "/" language ".po"))))
@@ -280,20 +287,19 @@ (define (translate-texi-manuals source)
"Return the translated texinfo manuals built from SOURCE."
(define po4a
(specification->package "po4a"))
-
- (define gettext
- (specification->package "gettext"))
+
+ (define gettext-minimal
+ (specification->package "gettext-minimal"))
(define glibc-utf8-locales
- (module-ref (resolve-interface '(gnu packages base))
- 'glibc-utf8-locales))
+ (specification->package "glibc-utf8-locales"))
(define documentation
(file-append* source "doc"))
(define documentation-po
(file-append* source "po/doc"))
-
+
(define build
(with-imported-modules '((guix build utils) (guix build po))
#~(begin
@@ -365,7 +371,7 @@ (define parallel-jobs
(setenv "GUIX_LOCPATH"
#+(file-append glibc-utf8-locales "/lib/locale"))
- (setenv "PATH" #+(file-append gettext "/bin"))
+ (setenv "PATH" #+(file-append gettext-minimal "/bin"))
(setenv "LC_ALL" "en_US.UTF-8")
(setlocale LC_ALL "en_US.UTF-8")
@@ -394,16 +400,13 @@ (define parallel-jobs
(define (info-manual source)
"Return the Info manual built from SOURCE."
(define texinfo
- (module-ref (resolve-interface '(gnu packages texinfo))
- 'texinfo))
+ (specification->package "texinfo"))
(define graphviz
- (module-ref (resolve-interface '(gnu packages graphviz))
- 'graphviz))
+ (specification->package "graphviz"))
(define glibc-utf8-locales
- (module-ref (resolve-interface '(gnu packages base))
- 'glibc-utf8-locales))
+ (specification->package "glibc-utf8-locales"))
(define documentation
(file-append* source "doc"))
@@ -586,8 +589,7 @@ (define* (guix-command modules
"Return the 'guix' command such that it adds MODULES and DEPENDENCIES in its
load path."
(define glibc-utf8-locales
- (module-ref (resolve-interface '(gnu packages base))
- 'glibc-utf8-locales))
+ (specification->package "glibc-utf8-locales"))
(define module-directory
;; To minimize the number of 'stat' calls needed to locate a module,
@@ -1030,10 +1032,7 @@ (define (built-modules node-subset)
;; Include 'guix-daemon'. XXX: Here we inject an
;; older snapshot of guix-daemon, but that's a good
;; enough approximation for now.
- #:daemon (module-ref (resolve-interface
- '(gnu packages
- package-management))
- 'guix-daemon)
+ #:daemon (specification->package "guix-daemon")
#:info (info-manual source)
#:miscellany (miscellaneous-files source)
--
2.39.1
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 28 Feb 2023 15:47:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 28 10:47:07 2023
Received: from localhost ([127.0.0.1]:51792 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pX2C3-0003no-HP
for submit <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:47:07 -0500
Received: from mira.cbaines.net ([212.71.252.8]:42290)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1pX2C1-0003nS-O2
for 61363 <at> debbugs.gnu.org; Tue, 28 Feb 2023 10:47:06 -0500
Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699])
by mira.cbaines.net (Postfix) with ESMTPSA id 4F77E16BBA
for <61363 <at> debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:04 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id cb7815f2
for <61363 <at> debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:03 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 61363 <at> debbugs.gnu.org
Subject: [PATCH v2 1/3] packages: Export guile-for-grafts.
Date: Tue, 28 Feb 2023 15:47:01 +0000
Message-Id: <20230228154703.3952-1-mail@HIDDEN>
X-Mailer: git-send-email 2.39.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
So this can be used in (guix self).
* guix/packages.scm (guile-for-grafts): Export.
---
guix/packages.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..2f81ad0284 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -183,6 +183,7 @@ (define-module (guix packages)
package-closure
default-guile
+ guile-for-grafts
default-guile-derivation
set-guile-for-build
package-file
--
2.39.1
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 22 Feb 2023 15:07:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 22 10:07:38 2023
Received: from localhost ([127.0.0.1]:60064 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pUqiX-0005Wl-I7
for submit <at> debbugs.gnu.org; Wed, 22 Feb 2023 10:07:38 -0500
Received: from mira.cbaines.net ([212.71.252.8]:42286)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1pUqiT-0005Wa-8Q
for 61363 <at> debbugs.gnu.org; Wed, 22 Feb 2023 10:07:35 -0500
Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699])
by mira.cbaines.net (Postfix) with ESMTPSA id 2118416B9E;
Wed, 22 Feb 2023 15:07:31 +0000 (GMT)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id 32853173;
Wed, 22 Feb 2023 15:07:29 +0000 (UTC)
References: <20230208075403.11788-1-mail@HIDDEN>
<20230208075403.11788-2-mail@HIDDEN> <87sfey9i1t.fsf@HIDDEN>
User-agent: mu4e 1.8.13; emacs 28.2
From: Christopher Baines <mail@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of
the guix derivation.
Date: Wed, 22 Feb 2023 11:17:48 +0000
In-reply-to: <87sfey9i1t.fsf@HIDDEN>
Message-ID: <878rgpeo28.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: 1.1 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: Ludovic Courtès <ludo@HIDDEN> writes: >> I'm looking at
this as it'll allow the Guix Data Service to compute the >> derivations without
grafts, and for these to be useful for substitutes >> regardless of whether
users are using grafts. > [...]
Content analysis details: (1.1 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 SPF_PASS SPF: sender matches SPF record
1.1 DATE_IN_PAST_03_06 Date: is 3 to 6 hours before Received: date
X-Debbugs-Envelope-To: 61363
Cc: 61363 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.1 (/)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>> I'm looking at this as it'll allow the Guix Data Service to compute the
>> derivations without grafts, and for these to be useful for substitutes
>> regardless of whether users are using grafts.
>
> How does it help exactly? By disabling grafts in that context?
So the Guix Data Service is somewhat built on the assumption that it's
cheap to compute derivations, at least with grafts disabled. That's
always been the case for packages, but for channel instance derivations
it's not reliably the case, since currently disabling grafts doesn't
apply to the whole process, and even if it did, the derivations you'd
get out wouldn't be that useful (since you can't transform the outputs
from those derivations to the outputs you'd get if using grafts).
With these changes, it's always relatively cheap to compute the channel
instance derivations, and it's always possible to compute the
derivations for any system without needing to be able to perform builds
for that system.
You can see this in how the data service has processed Guix before and
after these patches.
This is the channel instances before:
https://data.qa.guix.gnu.org/revision/a582d863465990642d331bc05bf073f47fb=
80908/channel-instances
and this is after:
https://data.qa.guix.gnu.org/revision/9cfbb22b556d28a0af345824ae5b3e00eb3=
f4a15/channel-instances
Given data.qa.guix.gnu.org is running on an x86_64-linux system, that
and i686-linux isn't generally a problem, but I'm guessing it only
managed to compute the powerpc64le-linux and aarch64-linux derivations
because it was able to substitute the necessary store items. For other
system computing the derivations would have failed.
I believe this change will also mean that the build farms will go from
performing the grafting for these builds, to being able to not do so, in
line with how builds for packages are handled. This isn't a big thing,
but I think it makes sense.
>> +++ b/guix/self.scm
>> @@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
>> (gzip (specification->package "gzip"))
>> (bzip2 (specification->package "bzip2"))
>> (xz (specification->package "xz"))
>> - (guix (specification->package "guix")))
>> + (guix (specification->package "guix"))
>> + (graft? #t))
>> "Return a file-like object that contains a compiled Guix."
>> (define guile-avahi
>> (specification->package "guile-avahi"))
>> @@ -802,6 +803,12 @@ (define dependencies
>> guile-json guile-semver guile-ssh guile-sqlite3
>> guile-lib guile-zlib guile-lzlib guile-zstd)))
>>=20=20
>> + (define packages
>> + (cons* gzip
>> + bzip2
>> + xz
>> + dependencies))
>> +
>
> [...]
>
>> + (let ((obj (built-modules (lambda (node)
>> + (list (node-source node)
>> + (node-compiled node))))))
>> + (if graft?
>> + (explicit-grafting obj packages)
>> + obj)))
>
> There are two things I=E2=80=99m not comfortable with:
>
> 1. Having <explicit-grafting> in (guix packages); it looks misplaced.
I didn't put it there at first, but I think it makes sense since
grafting is currently specific to packages, as is this additional code.
> 2. More importantly, manually listing packages that might require
> grafting looks like a slippery slope (=E2=80=9Coops! we=E2=80=99re n=
ot getting the
> GnuTLS graft for that CVE, too bad=E2=80=9D).
>
> I designed and implemented several variants to try and delay grafting.
> One of them consisted in carrying graft information in gexps:
>
> https://git.savannah.gnu.org/cgit/guix.git/log?h=3Dwip-gexp-grafts
>
> It=E2=80=99s kinda similar to what you=E2=80=99re proposing in that graft=
information is
> carried as far as possible. The main difference is that it=E2=80=99s aut=
omated.
That's interesting, I think that making grafting not specific to
packages, and something where the replacement is handled at a lower
level (e.g. gexps) would be an alternative way to handle this.
Given that this approach works though, maybe the explicit-grafting
functionality could just sit and be used inside of (guix self). Given
that module is very explicit about what packages are used, it should be
possible to arrange the code so it's very hard to miss a package out,
which should address your concern about manually listing packages (maybe
specification->package can be tweaked so that it's possible to get all
the packages, and that can be the list considered for grafting).
I don't know of any other places where this approach would be useful, so
while it would be nice to have a more general grafting mechanism
eventually, I'd also like to be able to make these changes to channel
instance grafts sooner rather than later.
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmP2L69fFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xf3pw/9EtEo0UOool4+8PfEFZ/Df9MYigLmm/y9
MWN0gQrfhosBdRVVGGZlCDlz9Hgh1ix07pJKSUELHxaoLGwhyKgPvxghz7A9BbBi
DN4D+AJL/8Xe8aPblQvoXWwMnGViHdK1HW9uxey8UJwCl2Np/0jrYq3H30WzXPCU
XWCSEz4hcFV4DTb7VDaLEINtkthv0lEPkKnaMP0ojGNoSztPxaUtCzNVc78zMxCV
6wMxa7HK7ew5131RYnGpn13p1hHFhhfnnY8EbmhkxfnZ9huZOCCNez4ifkO5jjjP
CpztgnQPBe+/EgApp71Vwmx5s2ZrQMIhNykLg6e1wtRhDygtEYmI7RlzXMWVrYTn
XY0XZ9d5xlc9K4hvFbvl3BA8yzjRXNSeSJMDKlvEGqy8qpl3BxH1MxXdGHkN0yGc
9M7cv9hMqwFcE7rVUKrEKfHwASDGwMwe9yTnE7eSlLhmO/xv2h+aHsmQQfrGe3y2
g5KnoOlBO26bbfXIf5rGOOSOCAmQUHMb25KFMMhdfTgP0Z6BLqNtLMbIKptEr6cM
dGhaxv47kEXEEP8TcuCezdAd3N3oTwPneUM/BKmUVEKmktPlRBkR2beKxp35tFb8
ngxgSLlUj8xhf7wTf22OzaUeuXmGNmQkaycpxwQhtuEfCtZXF6G6XUU5xouXO7RE
tUzp7aAE1oU=
=1/IL
-----END PGP SIGNATURE-----
--=-=-=--
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 22 Feb 2023 09:16:30 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 22 04:16:30 2023
Received: from localhost ([127.0.0.1]:57955 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pUlEk-0007k9-5C
for submit <at> debbugs.gnu.org; Wed, 22 Feb 2023 04:16:30 -0500
Received: from eggs.gnu.org ([209.51.188.92]:50982)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1pUlEh-0007jv-3s
for 61363 <at> debbugs.gnu.org; Wed, 22 Feb 2023 04:16:29 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1pUlEb-0004rr-H0; Wed, 22 Feb 2023 04:16:21 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
From; bh=tQ0D4ojXhWGmIlecBqN6ZOKnGeejzQ+ekk7iCRmDpRs=; b=ppJADP/UqUfMKCvtJY/N
ofBM9eC+F7jgefGzu77zJ1K864V3ymxPaHN9Sh2+RZqtHH7ULOnYJijQNhmL5bAkCzkKNNWRhNHwA
mpeIeVOCtlfTtoSPHmLBS9z7S+aUMpj+Ni3OlMObunnKvUpgAFZ22UuyYmhc7bMAvIq8ORi3K51Kg
vkp65luxa+Qk9Oir95+166nO+qClMZgOV4R8TkC9/ui3ZHJ1Lum0dNrpQ8XeFKkHQMJiYhoB53VF4
8vqeHdRBopwsTWHAuvzHh6ufJRyDDDnAyQ8otc4EVHgIJtiYqoAxWhaWzXSKv8YIaKeD5VYhULXFC
b+Ps9k/+DbE2IA==;
Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1pUlEW-0006gx-AX; Wed, 22 Feb 2023 04:16:21 -0500
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Christopher Baines <mail@HIDDEN>
Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of
the guix derivation.
References: <20230208075403.11788-1-mail@HIDDEN>
<20230208075403.11788-2-mail@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Quartidi 4 =?utf-8?Q?Vent=C3=B4se?= an 231 de la
=?utf-8?Q?R=C3=A9volution=2C?= jour du
=?utf-8?Q?Tro=C3=A8ne?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Wed, 22 Feb 2023 10:16:14 +0100
In-Reply-To: <20230208075403.11788-2-mail@HIDDEN> (Christopher Baines's
message of "Wed, 8 Feb 2023 08:54:03 +0100")
Message-ID: <87sfey9i1t.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 61363
Cc: 61363 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi,
Christopher Baines <mail@HIDDEN> skribis:
> Rather than having grafts apply to the derivation itself. This moves graf=
ting
> here to work like grafting for packages, where you can think of the graft=
ed
> outputs as a transformed variant of the ungrafted outputs.
Hmm.
> I'm looking at this as it'll allow the Guix Data Service to compute the
> derivations without grafts, and for these to be useful for substitutes
> regardless of whether users are using grafts.
How does it help exactly? By disabling grafts in that context?
> +++ b/guix/self.scm
> @@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
> (gzip (specification->package "gzip"))
> (bzip2 (specification->package "bzip2"))
> (xz (specification->package "xz"))
> - (guix (specification->package "guix")))
> + (guix (specification->package "guix"))
> + (graft? #t))
> "Return a file-like object that contains a compiled Guix."
> (define guile-avahi
> (specification->package "guile-avahi"))
> @@ -802,6 +803,12 @@ (define dependencies
> guile-json guile-semver guile-ssh guile-sqlite3
> guile-lib guile-zlib guile-lzlib guile-zstd)))
>=20=20
> + (define packages
> + (cons* gzip
> + bzip2
> + xz
> + dependencies))
> +
[...]
> + (let ((obj (built-modules (lambda (node)
> + (list (node-source node)
> + (node-compiled node))))))
> + (if graft?
> + (explicit-grafting obj packages)
> + obj)))
There are two things I=E2=80=99m not comfortable with:
1. Having <explicit-grafting> in (guix packages); it looks misplaced.
2. More importantly, manually listing packages that might require
grafting looks like a slippery slope (=E2=80=9Coops! we=E2=80=99re not=
getting the
GnuTLS graft for that CVE, too bad=E2=80=9D).
I designed and implemented several variants to try and delay grafting.
One of them consisted in carrying graft information in gexps:
https://git.savannah.gnu.org/cgit/guix.git/log?h=3Dwip-gexp-grafts
It=E2=80=99s kinda similar to what you=E2=80=99re proposing in that graft i=
nformation is
carried as far as possible. The main difference is that it=E2=80=99s autom=
ated.
Hmm needs more thought.
Ludo=E2=80=99.
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.Received: (at 61363) by debbugs.gnu.org; 10 Feb 2023 09:19:33 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 04:19:33 2023 Received: from localhost ([127.0.0.1]:34334 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQPZ7-0002l6-HH for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 04:19:33 -0500 Received: from mira.cbaines.net ([212.71.252.8]:42190) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1pQPZ5-0002kx-8r for 61363 <at> debbugs.gnu.org; Fri, 10 Feb 2023 04:19:32 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699]) by mira.cbaines.net (Postfix) with ESMTPSA id 1A2F7165A5; Fri, 10 Feb 2023 09:19:30 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 4e1784b4; Fri, 10 Feb 2023 09:19:28 +0000 (UTC) References: <871qn08uo7.fsf@HIDDEN> User-agent: mu4e 1.8.11; emacs 28.2 From: Christopher Baines <mail@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#61363] [PATCH 0/2] self: Apply grafts to the outputs of the guix derivation. Date: Fri, 10 Feb 2023 09:16:12 +0000 In-reply-to: <871qn08uo7.fsf@HIDDEN> Message-ID: <87h6vt6foi.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 61363 Cc: 61363 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain The data service comparison is now available for this, and while there are no differences in the packages, you can see some information. This is the channel instances before: https://data.qa.guix.gnu.org/revision/a582d863465990642d331bc05bf073f47fb80908/channel-instances and this is after: https://data.qa.guix.gnu.org/revision/9cfbb22b556d28a0af345824ae5b3e00eb3f4a15/channel-instances --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmPmDB1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XfyxA//WSnQZUWm2/qLSUknAYqL/Mr+YiaqtIw+ oCTLm96qrlZSl1VU7XSPsSCn7TbS6n6f7gSUBh7MSmaWHBbVRr/HsoLK2owTRqRv w34wkn5m0fZUKLqtt8fTum5Zkc8tQoz+ZgABRYLQQwRymssVistnG25A7vgrq6bD sfuEwQjk02Aejc9Mmshh6b14Nd1bq/NulgNbncqNkkMiHoiaGGHPvbBkfoCGAVrM TwhO6VY4cfge9c5cGrtrAKZo5YsB7Ergy2Ixx9KkTOUN6O1FWVtrXMijP6Nf+Gky asCHWfDR+YXl6EVOKcL8JZ/4JNKKkPP9qHT69JTvL+kgNdBaE18ucWvlBJMagQyk FqwotXQVqTON6ZHYR5IE6FV/RK+KgqXvJcBzIUdgVUKfQeSqRr/J2atXhJZ80TDr rHlS/RUnDG/Qcp//vTxdPX/ZkUulumVDmtKSuoWZ/xtaEeIy0EOsJCK0oZ0xSJZB P6ZwmYTJUVDSpGlstgaPpTKo/5zwOdKmnZlcAv3zT5G7v2pediK49g9FVqMBQA0h edsbnMCjVGHXmJgvvyBJdAyRxU28MiYM4gm0wbAe9AqnAnStKxV2KMMK03joLK08 S6UeAfr4HSeUkT6UNUH/0ET9P7+Th8kFGlr78yn0HomTgiXaj9NozbHZ3V4m5GqE KeIOjfiahbg= =4/2T -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 8 Feb 2023 07:54:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 08 02:54:08 2023
Received: from localhost ([127.0.0.1]:54685 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pPfHL-00009H-Jt
for submit <at> debbugs.gnu.org; Wed, 08 Feb 2023 02:54:08 -0500
Received: from mira.cbaines.net ([212.71.252.8]:42178)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1pPfHJ-000092-JQ
for 61363 <at> debbugs.gnu.org; Wed, 08 Feb 2023 02:54:06 -0500
Received: from localhost (unknown [IPv6:2a02:1808:280:f63d:f055:17e:cfb4:3])
by mira.cbaines.net (Postfix) with ESMTPSA id D08DF16D3C
for <61363 <at> debbugs.gnu.org>; Wed, 8 Feb 2023 07:54:04 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id d7e7ca6f
for <61363 <at> debbugs.gnu.org>; Wed, 8 Feb 2023 07:54:03 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 61363 <at> debbugs.gnu.org
Subject: [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.
Date: Wed, 8 Feb 2023 08:54:03 +0100
Message-Id: <20230208075403.11788-2-mail@HIDDEN>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <20230208075403.11788-1-mail@HIDDEN>
References: <20230208075403.11788-1-mail@HIDDEN>
MIME-Version: 1.0
X-Debbugs-Cc: zimon.toutoune@HIDDEN
X-Debbugs-Cc: dev@HIDDEN
X-Debbugs-Cc: rekado@HIDDEN
X-Debbugs-Cc: othacehe@HIDDEN
X-Debbugs-Cc: ludo@HIDDEN
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.
I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.
* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
#:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
---
build-aux/build-self.scm | 4 ++-
guix/self.scm | 65 ++++++++++++++++++++++++++--------------
2 files changed, 45 insertions(+), 24 deletions(-)
diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
#:channel-metadata
'#$channel-metadata
#:pull-version
- #$pull-version)
+ #$pull-version
+ #:graft?
+ #$(%graft?))
#:system system))
derivation-file-name))))))
#:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index 93019e1c64..c944dbe9ce 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
(gzip (specification->package "gzip"))
(bzip2 (specification->package "bzip2"))
(xz (specification->package "xz"))
- (guix (specification->package "guix")))
+ (guix (specification->package "guix"))
+ (graft? #t))
"Return a file-like object that contains a compiled Guix."
(define guile-avahi
(specification->package "guile-avahi"))
@@ -802,6 +803,12 @@ (define dependencies
guile-json guile-semver guile-ssh guile-sqlite3
guile-lib guile-zlib guile-lzlib guile-zstd)))
+ (define packages
+ (cons* gzip
+ bzip2
+ xz
+ dependencies))
+
(define *core-modules*
(scheme-node "guix-core"
'((guix)
@@ -1022,28 +1029,35 @@ (define (built-modules node-subset)
guile-lzma
dependencies)
#:guile guile-for-build
- #:guile-version guile-version)))
- (whole-package name modules dependencies
- #:command command
- #:guile guile-for-build
-
- ;; Include 'guix-daemon'. XXX: Here we inject an
- ;; older snapshot of guix-daemon, but that's a good
- ;; enough approximation for now.
- #:daemon (module-ref (resolve-interface
- '(gnu packages
- package-management))
- 'guix-daemon)
-
- #:info (info-manual source)
- #:miscellany (miscellaneous-files source)
- #:guile-version guile-version)))
+ #:guile-version guile-version))
+ (obj
+ (whole-package name modules dependencies
+ #:command command
+ #:guile guile-for-build
+
+ ;; Include 'guix-daemon'. XXX: Here we inject
+ ;; an older snapshot of guix-daemon, but
+ ;; that's a good enough approximation for now.
+ #:daemon (module-ref (resolve-interface
+ '(gnu packages
+ package-management))
+ 'guix-daemon)
+
+ #:info (info-manual source)
+ #:miscellany (miscellaneous-files source)
+ #:guile-version guile-version)))
+ (if graft?
+ (explicit-grafting obj packages)
+ obj)))
((= 0 pull-version)
;; Legacy 'guix pull': return the .scm and .go files as one
;; directory.
- (built-modules (lambda (node)
- (list (node-source node)
- (node-compiled node)))))
+ (let ((obj (built-modules (lambda (node)
+ (list (node-source node)
+ (node-compiled node))))))
+ (if graft?
+ (explicit-grafting obj packages)
+ obj)))
(else
;; Unsupported 'guix pull' version.
#f)))
@@ -1273,7 +1287,8 @@ (define (process-directory directory files output)
(define* (guix-derivation source version
#:optional (guile-version (effective-version))
#:key (pull-version 0)
- channel-metadata)
+ channel-metadata
+ (graft? #t))
"Return, as a monadic value, the derivation to build the Guix from SOURCE
for GUILE-VERSION. Use VERSION as the version string. Use CHANNEL-METADATA
as the channel metadata sexp to include in (guix config).
@@ -1310,7 +1325,11 @@ (define guile
#:pull-version pull-version
#:guile-version (if (>= pull-version 1)
"3.0" guile-version)
- #:guile-for-build guile)))
+ #:guile-for-build guile
+ #:graft? graft?)))
(if guix
- (lower-object guix)
+ (if graft?
+ (lower-object guix)
+ (without-grafting
+ (lower-object guix)))
(return #f)))))
--
2.38.1
ludo@HIDDEN, guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
Received: (at 61363) by debbugs.gnu.org; 8 Feb 2023 07:54:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 08 02:54:07 2023
Received: from localhost ([127.0.0.1]:54683 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pPfHL-00009F-8S
for submit <at> debbugs.gnu.org; Wed, 08 Feb 2023 02:54:07 -0500
Received: from mira.cbaines.net ([212.71.252.8]:42176)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1pPfHJ-000091-K3
for 61363 <at> debbugs.gnu.org; Wed, 08 Feb 2023 02:54:06 -0500
Received: from localhost (unknown [IPv6:2a02:1808:280:f63d:f055:17e:cfb4:3])
by mira.cbaines.net (Postfix) with ESMTPSA id 9081C16D28
for <61363 <at> debbugs.gnu.org>; Wed, 8 Feb 2023 07:54:04 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id eac9e016
for <61363 <at> debbugs.gnu.org>; Wed, 8 Feb 2023 07:54:03 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 61363 <at> debbugs.gnu.org
Subject: [PATCH 1/2] packages: Add explicit-grafting record type to assist
with grafts.
Date: Wed, 8 Feb 2023 08:54:02 +0100
Message-Id: <20230208075403.11788-1-mail@HIDDEN>
X-Mailer: git-send-email 2.38.1
MIME-Version: 1.0
X-Debbugs-Cc: zimon.toutoune@HIDDEN
X-Debbugs-Cc: dev@HIDDEN
X-Debbugs-Cc: rekado@HIDDEN
X-Debbugs-Cc: othacehe@HIDDEN
X-Debbugs-Cc: ludo@HIDDEN
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 61363
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Normally the grafting takes place when lowering packages, but this record
assists with applying the same transformation to arbitrary objects/store
items.
I'm adding this to allow grafting the channel instance derivation outputs.
* guix/packages.scm (explicit-grafting, explicit-grafting?,
explicit-grafting-obj, explicit-grafting-grafts): New procedures.
---
guix/packages.scm | 45 ++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 44 insertions(+), 1 deletion(-)
diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..877bf89522 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -188,7 +188,12 @@ (define-module (guix packages)
package-file
package->derivation
package->cross-derivation
- origin->derivation))
+ origin->derivation
+
+ explicit-grafting
+ explicit-grafting?
+ explicit-grafting-obj
+ explicit-grafting-grafts))
;; The 'source-module-closure' procedure ca. 1.2.0 did not recognize
;; #:re-export-and-replace: <https://issues.guix.gnu.org/52694>.
@@ -2093,3 +2098,41 @@ (define package-source-derivation ;somewhat deprecated
(add-to-store store (basename file) #t "sha256" file))
(_
(lower store source system))))))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+ (%explicit-grafting obj packages)
+ explicit-grafting?
+ (obj explicit-grafting-obj) ;obj
+ (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+ (match rec
+ (($ <explicit-grafting> obj packages)
+ (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+ (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+ system target)
+ (match explicit-grafting
+ (($ <explicit-grafting> obj packages)
+ (mlet* %store-monad ((drv (without-grafting
+ (lower-object obj system #:target target)))
+ (grafts
+ (mapm %store-monad
+ (lambda (pkg)
+ (package-grafts* pkg system #:target target))
+ packages)))
+ (match (delete-duplicates
+ (concatenate grafts))
+ (()
+ (return drv))
+ (grafts
+ (mlet %store-monad ((guile (package->derivation
+ (guile-for-grafts)
+ system #:graft? #f)))
+ (graft-derivation* drv grafts
+ #:system system
+ #:guile guile))))))))
--
2.38.1
ludo@HIDDEN, guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.Received: (at submit) by debbugs.gnu.org; 8 Feb 2023 07:48:17 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 08 02:48:17 2023 Received: from localhost ([127.0.0.1]:54677 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pPfBh-0008Qj-HS for submit <at> debbugs.gnu.org; Wed, 08 Feb 2023 02:48:17 -0500 Received: from lists.gnu.org ([209.51.188.17]:39608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1pPfBg-0008QW-5p for submit <at> debbugs.gnu.org; Wed, 08 Feb 2023 02:48:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1pPfBf-0007dW-3N for guix-patches@HIDDEN; Wed, 08 Feb 2023 02:48:15 -0500 Received: from mira.cbaines.net ([212.71.252.8]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1pPfBd-0001UM-Dj for guix-patches@HIDDEN; Wed, 08 Feb 2023 02:48:14 -0500 Received: from localhost (unknown [IPv6:2a02:1808:280:f63d:f055:17e:cfb4:3]) by mira.cbaines.net (Postfix) with ESMTPSA id 05A9A16D09 for <guix-patches@HIDDEN>; Wed, 8 Feb 2023 07:48:11 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id b1670436 for <guix-patches@HIDDEN>; Wed, 8 Feb 2023 07:48:10 +0000 (UTC) User-agent: mu4e 1.8.11; emacs 28.2 From: Christopher Baines <mail@HIDDEN> To: guix-patches@HIDDEN Subject: [PATCH 0/2] self: Apply grafts to the outputs of the guix derivation. Date: Wed, 08 Feb 2023 08:46:17 +0100 Message-ID: <871qn08uo7.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@HIDDEN; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable These patches mean that grafts apply to the outputs of the guix derivation, rather than having grafts apply to the derivation itself. This moves grafting here to work like grafting for packages, where you can think of the grafted outputs as a transformed variant of the ungrafted outputs. I'm looking at this as it'll allow the Guix Data Service to compute the derivations without grafts, and for these to be useful for substitutes regardless of whether users are using grafts. Christopher Baines (2): packages: Add explicit-grafting record type to assist with grafts. self: Apply grafts to the outputs of the guix derivation. build-aux/build-self.scm | 4 ++- guix/packages.scm | 45 +++++++++++++++++++++++++++- guix/self.scm | 65 ++++++++++++++++++++++++++-------------- 3 files changed, 89 insertions(+), 25 deletions(-) =2D-=20 2.38.1 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmPjU7hfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcgJw//bYzgsMxFE0DJYm1QniQG6g+NI11p6Yes JYdFHEAzYUWGXXyjosqG9o83956yfrNLIpXykHOejthOd8u64pv4KpetZq60aak4 QVu40WXHgeXeYySflnPzQoc7L27fQFE7uaDPXTei3Re/B73A+f6DYyCuiEfkc62f iqxfk9ImY/5sXQWoIU+ebk+ZyVByGBr4HhaSb9IGM79yWKjht+FS+Vl7+wmysgLg GvAz9MmRvGubwv3I6uFks7jHR0x+z6hcOiZvMneQSTq0uVhmq3zYcLNGW/oc/e/6 VfqDkNF0NZs/HV9iPgQgYIX0FO/9PAWeXsroHrv8bBqP0n/RCrJUbUjv569RF94n e4tPEbWluGtGYxq8fc8KGkgQvJ29WqF+mCweYdPlGSOtVQvP+Cuq38KDnQ+Z1IwO kDgTryb/OmC6+y78k0TUEizEA3IBQFdvxC/yVT4Sv5tjRVtOJreDhIp0LtV1Ajbk 9gv/N1m8RviWRfh6neNvrLAk9SytXlJn7JpbUTbOnkbKlOFDFFlOUGmPVf2wyWF2 7bl616qm/jatiBXjChYkRCxuTeM5HcF35onHh92tn93LygdkIGWsHb2E6Z/zYJcs /j2BsG61oNEKBntptu7+/zQhyjIfhuzwaucFpTu/kEIhrrVEDlIYbrXvp7a1DZ7I eRetV7hjFpY= =EhSi -----END PGP SIGNATURE----- --=-=-=--
Christopher Baines <mail@HIDDEN>:guix-patches@HIDDEN.
Full text available.guix-patches@HIDDEN:bug#61363; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.