Received: (at 71038) by debbugs.gnu.org; 18 May 2024 13:19:48 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat May 18 09:19:48 2024 Received: from localhost ([127.0.0.1]:33449 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1s8JyV-00069y-Um for submit <at> debbugs.gnu.org; Sat, 18 May 2024 09:19:48 -0400 Received: from mira.cbaines.net ([212.71.252.8]:43558) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1s8JyQ-00069g-OB for 71038 <at> debbugs.gnu.org; Sat, 18 May 2024 09:19:43 -0400 Received: from localhost (unknown [89.207.171.92]) by mira.cbaines.net (Postfix) with ESMTPSA id 7257B27BBE9 for <71038 <at> debbugs.gnu.org>; Sat, 18 May 2024 14:19:38 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id ba9ae70a for <71038 <at> debbugs.gnu.org>; Sat, 18 May 2024 13:19:36 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 71038 <at> debbugs.gnu.org Subject: [PATCH 2/2] guix: channels: Enable specifiying available builtin builders. Date: Sat, 18 May 2024 14:19:35 +0100 Message-ID: <3724435f5b2e86be53ffba5bffcbfdf493b48e5c.1716038375.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <6b9c634c024b6fd7a50d3b82edc244676a8ca8e8.1716038375.git.mail@HIDDEN> References: <6b9c634c024b6fd7a50d3b82edc244676a8ca8e8.1716038375.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 71038 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) When computing channel instance derivations. This is useful when you want to generate compatible derivations that can be run with a daemon that potentially doesn't support builtin builders that the daemon you're using to generate the derivations has. I'm looking at this in particular because I want to use this in the data service, since it provides substitutes for derivations, and since these can be built on other machines, it's useful to control which builtin builders they depend on. Fixes: <https://issues.guix.gnu.org/67250>. * build-aux/build-self.scm (build-program): Accept #:assume-available-builtin-builders and pass along to port->connection or open-connection as approriate. (build): Accept and pass on #:assume-available-builtin-builders. * guix/channels.scm (build-from-source, build-channel-instance, channel-instance-derivations, channel-instances->manifest, channel-instances->derivation): Accept and pass on #:assume-available-builtin-builders. Change-Id: I315c990de66c6f7dca25a859165a5568abe385ea --- build-aux/build-self.scm | 28 ++++++++++++++++++-------- guix/channels.scm | 43 +++++++++++++++++++++++++++++----------- 2 files changed, 51 insertions(+), 20 deletions(-) diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm index 02822a2ee8..7afb13c1e4 100644 --- a/build-aux/build-self.scm +++ b/build-aux/build-self.scm @@ -241,7 +241,8 @@ (define guile-gcrypt (define* (build-program source version #:optional (guile-version (effective-version)) - #:key (pull-version 0) (channel-metadata #f)) + #:key (pull-version 0) (channel-metadata #f) + assume-available-builtin-builders) "Return a program that computes the derivation to build Guix from SOURCE." (define select? ;; Select every module but (guix config) and non-Guix modules. @@ -331,11 +332,20 @@ (define* (build-program source version ;; case, attempt to open a new connection. (let* ((proto (string->number protocol-version)) (store (if (integer? proto) - (port->connection (duplicate-port - (current-input-port) - "w+0") - #:version proto) - (open-connection))) + (port->connection + (duplicate-port + (current-input-port) + "w+0") + #:version proto + #$@(if assume-available-builtin-builders + #~(#:assume-available-builtin-builders + '(#$@assume-available-builtin-builders)) + '())) + (open-connection + #$@(if assume-available-builtin-builders + #~(#:assume-available-builtin-builders + '(#$@assume-available-builtin-builders)) + '())))) (sock (socket AF_UNIX SOCK_STREAM 0))) ;; Connect to BUILD-OUTPUT and send it the raw ;; build output. @@ -406,7 +416,7 @@ (define* (build source (guile-version (if (> pull-version 0) "3.0" (effective-version))) - + assume-available-builtin-builders #:allow-other-keys #:rest rest) "Return a derivation that unpacks SOURCE into STORE and compiles Scheme @@ -415,7 +425,9 @@ (define* (build source ;; SOURCE. (mlet %store-monad ((build (build-program source version guile-version #:channel-metadata channel-metadata - #:pull-version pull-version)) + #:pull-version pull-version + #:assume-available-builtin-builders + assume-available-builtin-builders)) (system (if system (return system) (current-system))) (home -> (getenv "HOME")) diff --git a/guix/channels.scm b/guix/channels.scm index 0d7bc541cc..6469d08bb9 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -704,7 +704,8 @@ (define (with-trivial-build-handler mvalue) store)))) (define* (build-from-source instance - #:key core verbose? (dependencies '()) system) + #:key core verbose? (dependencies '()) system + assume-available-builtin-builders) "Return a derivation to build Guix from INSTANCE, using the self-build script contained therein. When CORE is true, build package modules under SOURCE using CORE, an instance of Guix. By default, build for the current @@ -750,20 +751,25 @@ (define* (build-from-source instance #:verbose? verbose? #:version commit #:system system #:channel-metadata (channel-instance->sexp instance) - #:pull-version %pull-version)))) + #:pull-version %pull-version + #:assume-available-builtin-builders + assume-available-builtin-builders)))) ;; Build a set of modules that extend Guix using the standard method. (standard-module-derivation name source core dependencies))) (define* (build-channel-instance instance system - #:optional core (dependencies '())) + #:optional core (dependencies '()) + #:key assume-available-builtin-builders) "Return, as a monadic value, the derivation for INSTANCE, a channel instance, for SYSTEM. DEPENDENCIES is a list of extensions providing Guile modules that INSTANCE depends on." (build-from-source instance #:core core #:dependencies dependencies - #:system system)) + #:system system + #:assume-available-builtin-builders + assume-available-builtin-builders)) (define (resolve-dependencies instances) "Return a procedure that, given one of the elements of INSTANCES, returns @@ -793,7 +799,8 @@ (define (resolve-dependencies instances) (lambda (instance) (vhash-foldq* cons '() instance edges))) -(define* (channel-instance-derivations instances #:key system) +(define* (channel-instance-derivations instances #:key system + assume-available-builtin-builders) "Return the list of derivations to build INSTANCES, in the same order as INSTANCES. Build for the current system by default, or SYSTEM if specified." (define core-instance @@ -809,11 +816,15 @@ (define* (channel-instance-derivations instances #:key system) (define (instance->derivation instance) (mlet %store-monad ((system (if system (return system) (current-system)))) (mcached (if (eq? instance core-instance) - (build-channel-instance instance system) + (build-channel-instance instance system + #:assume-available-builtin-builders + assume-available-builtin-builders) (mlet %store-monad ((core (instance->derivation core-instance)) (deps (mapm %store-monad instance->derivation (edges instance)))) - (build-channel-instance instance system core deps))) + (build-channel-instance instance system core deps + #:assume-available-builtin-builders + assume-available-builtin-builders))) instance system))) @@ -915,7 +926,8 @@ (define (channel-instance->sexp instance) intro)))))) '())))) -(define* (channel-instances->manifest instances #:key system) +(define* (channel-instances->manifest instances #:key system + assume-available-builtin-builders) "Return a profile manifest with entries for all of INSTANCES, a list of channel instances. By default, build for the current system, or SYSTEM if specified." @@ -934,8 +946,11 @@ (define* (channel-instances->manifest instances #:key system) (properties `((source ,(channel-instance->sexp instance))))))) - (mlet* %store-monad ((derivations (channel-instance-derivations instances - #:system system)) + (mlet* %store-monad ((derivations (channel-instance-derivations + instances + #:system system + #:assume-available-builtin-builders + assume-available-builtin-builders)) (entries -> (map instance->entry instances derivations))) (return (manifest entries)))) @@ -990,10 +1005,14 @@ (define %channel-profile-hooks ;; The default channel profile hooks. (cons package-cache-file %default-profile-hooks)) -(define (channel-instances->derivation instances) +(define* (channel-instances->derivation instances + #:key assume-available-builtin-builders) "Return the derivation of the profile containing INSTANCES, a list of channel instances." - (mlet %store-monad ((manifest (channel-instances->manifest instances))) + (mlet %store-monad ((manifest (channel-instances->manifest + instances + #:assume-available-builtin-builders + assume-available-builtin-builders))) ;; Emit a profile in format version so that, if INSTANCES denotes an old ;; Guix, it can still read that profile, for instance for the purposes of ;; 'guix describe'. -- 2.41.0
guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN
:bug#71038
; Package guix-patches
.
Full text available.Received: (at 71038) by debbugs.gnu.org; 18 May 2024 13:19:44 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat May 18 09:19:43 2024 Received: from localhost ([127.0.0.1]:33447 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1s8JyR-00069r-IK for submit <at> debbugs.gnu.org; Sat, 18 May 2024 09:19:43 -0400 Received: from mira.cbaines.net ([212.71.252.8]:43556) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1s8JyQ-00069f-6r for 71038 <at> debbugs.gnu.org; Sat, 18 May 2024 09:19:42 -0400 Received: from localhost (unknown [89.207.171.92]) by mira.cbaines.net (Postfix) with ESMTPSA id 490D227BBE2 for <71038 <at> debbugs.gnu.org>; Sat, 18 May 2024 14:19:36 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id c02c102f for <71038 <at> debbugs.gnu.org>; Sat, 18 May 2024 13:19:35 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 71038 <at> debbugs.gnu.org Subject: [PATCH 1/2] guix: store: Enable specifying the available builtin builders. Date: Sat, 18 May 2024 14:19:34 +0100 Message-ID: <6b9c634c024b6fd7a50d3b82edc244676a8ca8e8.1716038375.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 71038 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To open-connection and port->connection. This overrides the discovered builtin builders that the daemon says it provides. This is useful when you want to generate compatible derivations that can be run with a daemon that potentially doesn't support builtin builders that the daemon you're using to generate the derivations has. I'm looking at this in particular because I want to use this in the data service, since it provides substitutes for derivations, and since these can be built on other machines, it's useful to control which builtin builders they depend on. * guix/store.scm (open-connection, port->connection): Accept #:assume-available-builtin-builders and use this instead of %built-in-builders. Fixes: <https://issues.guix.gnu.org/67250>. Change-Id: I45d58ab93b6d276d280552858fc81ebc2b58828a --- guix/store.scm | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/guix/store.scm b/guix/store.scm index 58ddaa8d15..0c734cdca7 100644 --- a/guix/store.scm +++ b/guix/store.scm @@ -571,7 +571,7 @@ (define* (connect-to-daemon uri #:key non-blocking?) (define* (open-connection #:optional (uri (%daemon-socket-uri)) #:key port (reserve-space? #t) cpu-affinity - non-blocking?) + non-blocking? assume-available-builtin-builders) "Connect to the daemon at URI (a string), or, if PORT is not #f, use it as the I/O port over which to communicate to a build daemon. @@ -616,7 +616,9 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri)) (when (>= (protocol-minor v) 11) (write-int (if reserve-space? 1 0) port)) (letrec* ((built-in-builders - (delay (%built-in-builders conn))) + (if assume-available-builtin-builders + (delay assume-available-builtin-builders) + (delay (%built-in-builders conn)))) (caches (make-vector (atomic-box-ref %store-connection-caches) @@ -635,7 +637,8 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri)) conn)))))) (define* (port->connection port - #:key (version %protocol-version)) + #:key (version %protocol-version) + assume-available-builtin-builders) "Assimilate PORT, an input/output port, and return a connection to the daemon, assuming the given protocol VERSION. @@ -654,7 +657,9 @@ (define* (port->connection port (make-vector (atomic-box-ref %store-connection-caches) vlist-null) - (delay (%built-in-builders connection)))) + (if assume-available-builtin-builders + (delay assume-available-builtin-builders) + (delay (%built-in-builders connection))))) connection)) base-commit: 0846eaecd45783bf40e8dc67b0c16f71068524b7 -- 2.41.0
guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN
:bug#71038
; Package guix-patches
.
Full text available.Received: (at submit) by debbugs.gnu.org; 18 May 2024 13:12:14 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat May 18 09:12:14 2024 Received: from localhost ([127.0.0.1]:33409 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1s8JrC-0003IL-0g for submit <at> debbugs.gnu.org; Sat, 18 May 2024 09:12:14 -0400 Received: from lists.gnu.org ([209.51.188.17]:53410) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1s8JrA-0003IF-Ls for submit <at> debbugs.gnu.org; Sat, 18 May 2024 09:12:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1s8Jr6-0007iC-Qg for guix-patches@HIDDEN; Sat, 18 May 2024 09:12:08 -0400 Received: from mira.cbaines.net ([212.71.252.8]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1s8Jr0-0003mb-UU for guix-patches@HIDDEN; Sat, 18 May 2024 09:12:08 -0400 Received: from localhost (unknown [89.207.171.92]) by mira.cbaines.net (Postfix) with ESMTPSA id 750FE27BBE2 for <guix-patches@HIDDEN>; Sat, 18 May 2024 14:12:00 +0100 (BST) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 4c30137f for <guix-patches@HIDDEN>; Sat, 18 May 2024 13:11:56 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: guix-patches@HIDDEN Subject: [PATCH 0/2] Enable specifying the available builtin builders. User-Agent: mu4e 1.12.2; emacs 29.3 Date: Sat, 18 May 2024 14:11:56 +0100 Message-ID: <87y187b6xv.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@HIDDEN; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable This will enable addressing [11 by having the data service specify '("download") for the builtin builders. 1: https://issues.guix.gnu.org/67250 Christopher Baines (2): guix: store: Enable specifying the available builtin builders. guix: channels: Enable specifiying available builtin builders. build-aux/build-self.scm | 28 ++++++++++++++++++-------- guix/channels.scm | 43 +++++++++++++++++++++++++++++----------- guix/store.scm | 13 ++++++++---- 3 files changed, 60 insertions(+), 24 deletions(-) base-commit: 0846eaecd45783bf40e8dc67b0c16f71068524b7 =2D-=20 2.41.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmZIqRxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XdkqhAAiSoK89xGVo8caiP6YdEi+YmUOH08gEmi C6NajcBASwFCeOI6fHxUpeaK4kCJ4DE7AhTBFcOwEgcfo6xELL/LiIdhEwvWi9hy P4cGCBP/VyN/Vbthaa2LOsncr81dBqrtyrjBugTi6nmy4Ri2TckzTekN9gZ+XO8E 4vHgDghaKtiBHYR6insGzie9JbGSoziiUCq/2qe49DJy4cZEqzuaZe/5+n5uAlRc UE062bgBwWjE+SdAJu7UDtu0peiGhxeuoBdqfL92vBQYpP75UtuyRymRCdbX1FBB r5Quq9lfXjLHNd2nAfkSU7ghrjNfGUSwR+1vr4vPni43nOjoJlm5OEihPU8ZhwIF dq6fZAzG0RhYbOmWIAjt3KLqg+EgNptIkLerram9fBoPg4+C2BOC/0E0Dt2qheWa B1fFkdLK/M3Q86b2UsevyXq/Wrsbj0g8KlzNDz86Zu9adIzKplu5cOUUEeG+WPau Dh9K7+6BZ9HoOov4ckSU7F1P+OkIY/B4MNJpLw7Uy9gLFfL5vAPHnTbuNOlXz0PJ vvXQfUiTjVs38wg6Fph2UU8+WHGjBONFfsWyQMF9un8xHoRWo1xbLoMqllAguAjZ Nahn+jWuW25D3o6cXbaeoSoarY6lFs8EKgUgSqMVve0yaXud45YlEXGa03vY6gAS d5EaryoyYoQ= =OOIg -----END PGP SIGNATURE----- --=-=-=--
Christopher Baines <mail@HIDDEN>
:guix-patches@HIDDEN
.
Full text available.guix-patches@HIDDEN
:bug#71038
; Package guix-patches
.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.