X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 0/7] Memoize packages associated with cross building.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:38:02 +0000
Resent-Message-ID: <handler.68266.B.170447263014493 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
X-Debbugs-Original-To: guix-patches@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.170447263014493
(code B ref -1); Fri, 05 Jan 2024 16:38:02 +0000
Received: (at submit) by debbugs.gnu.org; 5 Jan 2024 16:37:10 +0000
Received: from localhost ([127.0.0.1]:57711 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnC2-0003lg-El
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:37:10 -0500
Received: from lists.gnu.org ([2001:470:142::17]:32926)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnC0-0003lG-WC
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:37:09 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1rLnBr-0004tJ-BC
for guix-patches@HIDDEN; Fri, 05 Jan 2024 11:36:59 -0500
Received: from mira.cbaines.net ([212.71.252.8])
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <mail@HIDDEN>) id 1rLnBp-0007mr-SH
for guix-patches@HIDDEN; Fri, 05 Jan 2024 11:36:59 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 10CD727BBE2
for <guix-patches@HIDDEN>; Fri, 5 Jan 2024 16:36:55 +0000 (GMT)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id bd3839a5
for <guix-patches@HIDDEN>; Fri, 5 Jan 2024 16:36:54 +0000 (UTC)
User-agent: mu4e 1.10.7; emacs 29.1
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 05 Jan 2024 16:35:25 +0000
Message-ID: <87plyfrb2x.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@HIDDEN;
helo=mira.cbaines.net
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001,
SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.9 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.1 (/)
--=-=-=
Content-Type: text/plain
This comes from dumping the contents of the store connection object
cache after computing derivations targeting i586-pc-gnu, and looking at
what packages appear multiple times as cache keys.
Christopher Baines (7):
gnu: Memozise make-ld-wrapper results.
gnu: Memozise cross-binutils results.
gnu: Memozise cross-gcc results.
gnu: Memozise cross-kernel-headers results.
gnu: Memozise cross-mig results.
gnu: Memozise cross-libc results.
packages: rust: Memoize make-rust-sysroot results.
gnu/packages/base.scm | 126 +++---
gnu/packages/cross-base.scm | 804 +++++++++++++++++++-----------------
gnu/packages/rust.scm | 6 +-
3 files changed, 486 insertions(+), 450 deletions(-)
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWYMCZfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XewgBAAj9aTBQ3J1mNMlCZ0aFsIWTxOEQFusvol
V3IOeUboC8w1Oe2wxGZtgtQBsArcZivfo8Wt9gSsL1z0o6d3i/m88/6NpIBwaAMq
hhP3f2b8zjngI+YYY3Tz6oEzG5GUGPLSQSb31bnPnmGEfkFTracvdb1r0/gshm9X
sLy+RaSy5z7UMLaSELEBH5G/a2GfRv++2WGYasftCKMludVFw99db9XkhsA8SO6b
TIH3bMNHapr1B43gRqc3AstxuAXjgXyPCMrJq/8gFezIoqjTdi+BOLLn0xk3G0nZ
WZoqPRNJdWfp5NPmYinLpU+ylUYCicXhxgWRRbfMjP6h+F1zmGZezI9E+h74m26G
/gQotLAQi/Y74jezsWcaPWEByB9Omouv+LncTClF26J5EB8ZaIf/sG19J/pggOWJ
5AY+hHDzgzZsME8UhCU5TvMXz+0brV39cHvOSJYQPKIP7ml9RtWe6GumhYTZdUrw
ClQGm5DycXXuFY10NUX+Ms7oEEOfYzWoWMIV/Q+T2MSTaeii9TYFutjG4OJDVzkx
Pn76Ik18eRWhgaNY4jJ9TsBTaZeU+3zbxoFYnRSLgsS7Xal6VAbZGmZ9qHciBwN2
9ktBORdciRnjP2b2venuJ27PqGu2K8nxrZXLvYcwye1HLk1FF5xf6Kk623nKTOqT
6yzwOXCRZd0=
=ElRU
-----END PGP SIGNATURE-----
--=-=-=--
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Christopher Baines <mail@HIDDEN> Subject: bug#68266: Acknowledgement ([PATCH 0/7] Memoize packages associated with cross building.) Message-ID: <handler.68266.B.170447263014493.ack <at> debbugs.gnu.org> References: <87plyfrb2x.fsf@HIDDEN> X-Gnu-PR-Message: ack 68266 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 68266 <at> debbugs.gnu.org Date: Fri, 05 Jan 2024 16:38:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): guix-patches@HIDDEN If you wish to submit further information on this problem, please send it to 68266 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 68266: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D68266 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results.
References: <87plyfrb2x.fsf@HIDDEN>
In-Reply-To: <87plyfrb2x.fsf@HIDDEN>
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: ludo@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:41:02 +0000
Resent-Message-ID: <handler.68266.B68266.170447286014927 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286014927
(code B ref 68266); Fri, 05 Jan 2024 16:41:02 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:00 +0000
Received: from localhost ([127.0.0.1]:57723 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFj-0003sO-0f
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:00 -0500
Received: from mira.cbaines.net
([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:50159)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFg-0003rw-EA
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:57 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 2BEF927BBE2
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id 019e4bde
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:50 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:43 +0000
Message-ID: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/base.scm (make-ld-wrapper): Move code to
make-ld-wrapper/implementation and call it.
(make-ld-wrapper/implementation) New procedure.
Change-Id: Id6fc805a4a7ffbc5ff0a5174eafcdf2c7c46854d
---
gnu/packages/base.scm | 126 ++++++++++++++++++++++--------------------
1 file changed, 66 insertions(+), 60 deletions(-)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 8b25af6a5e..929bf9f422 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -66,6 +66,7 @@ (define-module (gnu packages base)
#:use-module (guix gexp)
#:use-module (guix packages)
#:use-module (guix download)
+ #:use-module (guix memoization)
#:use-module (guix git-download)
#:use-module (guix build-system gnu)
#:use-module (guix build-system trivial)
@@ -715,68 +716,73 @@ (define* (make-ld-wrapper name #:key
wrapper for the cross-linker for that target, called 'TARGET-ld'. To use a
different linker than the default \"ld\", such as \"ld.gold\" the linker name
can be provided via the LINKER argument."
- ;; Note: #:system->target-triplet is a procedure so that the evaluation of
- ;; its result can be delayed until the 'arguments' field is evaluated, thus
- ;; in a context where '%current-system' is accurate.
- (package
- (name name)
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (inputs `(("binutils" ,binutils)
- ("guile" ,guile)
- ("bash" ,bash)
- ("wrapper" ,(search-path %load-path
- "gnu/packages/ld-wrapper.in"))))
- (arguments
- (let ((target (target (%current-system))))
- `(#:guile ,guile-for-build
- #:modules ((guix build utils))
- #:builder (begin
- (use-modules (guix build utils)
- (system base compile))
-
- (let* ((out (assoc-ref %outputs "out"))
- (bin (string-append out "/bin"))
- (ld ,(if target
- `(string-append bin "/" ,target "-"
- ,linker)
- `(string-append bin "/" ,linker)))
- (go (string-append ld ".go")))
-
- (setvbuf (current-output-port)
- (cond-expand (guile-2.0 _IOLBF)
- (else 'line)))
- (format #t "building ~s/bin/ld wrapper in ~s~%"
- (assoc-ref %build-inputs "binutils")
- out)
-
- (mkdir-p bin)
- (copy-file (assoc-ref %build-inputs "wrapper") ld)
- (substitute* ld
- (("@SELF@")
- ld)
- (("@GUILE@")
- (string-append (assoc-ref %build-inputs "guile")
- "/bin/guile"))
- (("@BASH@")
- (string-append (assoc-ref %build-inputs "bash")
- "/bin/bash"))
- (("@LD@")
- (string-append (assoc-ref %build-inputs "binutils")
- ,(if target
- (string-append "/bin/"
- target "-" linker)
- (string-append "/bin/" linker)))))
- (chmod ld #o555)
- (compile-file ld #:output-file go))))))
- (synopsis "The linker wrapper")
- (description
- "The linker wrapper (or @code{ld-wrapper}) wraps the linker to add any
+ (make-ld-wrapper/implementation name target binutils linker
+ guile bash guile-for-build))
+
+(define make-ld-wrapper/implementation
+ (mlambda (name target binutils linker guile bash guile-for-build)
+ ;; Note: #:system->target-triplet is a procedure so that the evaluation of
+ ;; its result can be delayed until the 'arguments' field is evaluated,
+ ;; thus in a context where '%current-system' is accurate.
+ (package
+ (name name)
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (inputs `(("binutils" ,binutils)
+ ("guile" ,guile)
+ ("bash" ,bash)
+ ("wrapper" ,(search-path %load-path
+ "gnu/packages/ld-wrapper.in"))))
+ (arguments
+ (let ((target (target (%current-system))))
+ `(#:guile ,guile-for-build
+ #:modules ((guix build utils))
+ #:builder (begin
+ (use-modules (guix build utils)
+ (system base compile))
+
+ (let* ((out (assoc-ref %outputs "out"))
+ (bin (string-append out "/bin"))
+ (ld ,(if target
+ `(string-append bin "/" ,target "-"
+ ,linker)
+ `(string-append bin "/" ,linker)))
+ (go (string-append ld ".go")))
+
+ (setvbuf (current-output-port)
+ (cond-expand (guile-2.0 _IOLBF)
+ (else 'line)))
+ (format #t "building ~s/bin/ld wrapper in ~s~%"
+ (assoc-ref %build-inputs "binutils")
+ out)
+
+ (mkdir-p bin)
+ (copy-file (assoc-ref %build-inputs "wrapper") ld)
+ (substitute* ld
+ (("@SELF@")
+ ld)
+ (("@GUILE@")
+ (string-append (assoc-ref %build-inputs "guile")
+ "/bin/guile"))
+ (("@BASH@")
+ (string-append (assoc-ref %build-inputs "bash")
+ "/bin/bash"))
+ (("@LD@")
+ (string-append (assoc-ref %build-inputs "binutils")
+ ,(if target
+ (string-append "/bin/"
+ target "-" linker)
+ (string-append "/bin/" linker)))))
+ (chmod ld #o555)
+ (compile-file ld #:output-file go))))))
+ (synopsis "The linker wrapper")
+ (description
+ "The linker wrapper (or @code{ld-wrapper}) wraps the linker to add any
missing @code{-rpath} flags, and to detect any misuse of libraries outside of
the store.")
- (home-page "https://www.gnu.org/software/guix//")
- (license gpl3+)))
+ (home-page "https://www.gnu.org/software/guix//")
+ (license gpl3+))))
(define-public %glibc/hurd-configure-flags
;; 'configure' in glibc 2.35 omits to pass '-ffreestanding' when detecting
base-commit: 5279bd453f354cbbaafff44e46c6fa03a39bc10a
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 2/7] gnu: Memozise cross-binutils results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: ludo@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:41:02 +0000
Resent-Message-ID: <handler.68266.B68266.170447286114940 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286114940
(code B ref 68266); Fri, 05 Jan 2024 16:41:02 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:01 +0000
Received: from localhost ([127.0.0.1]:57725 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFk-0003si-64
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:00 -0500
Received: from mira.cbaines.net
([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:57531)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFg-0003rx-EC
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:58 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 37E7C27BBE9
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id 8ed7c751
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:44 +0000
Message-ID: <cc0b70d8f909618989aa669c8e9689c466210932.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/cross-base.scm (cross-binutils*): Move code to
cross-binutils/implementation and call it.
(cross-binutils/implementation) New procedure.
(cross-binutils/deprecated): Call cross-binutils/implementation.
Change-Id: Ic7a493177026c7a699108ab6d75482ff3c189340
---
gnu/packages/cross-base.scm | 92 +++++++++++++++++++------------------
1 file changed, 48 insertions(+), 44 deletions(-)
diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
index 6ee7b315d8..a04e4f9c9e 100644
--- a/gnu/packages/cross-base.scm
+++ b/gnu/packages/cross-base.scm
@@ -89,56 +89,60 @@ (define (contains-keyword? args)
(find keyword? args))
(define* (cross-binutils . args)
+ "Return a cross-Binutils for TARGET using BINUTILS."
(if (or (= (length args) 1) (contains-keyword? args))
(apply cross-binutils* args)
(apply cross-binutils/deprecated args)))
+(define* (cross-binutils* target #:key (binutils binutils))
+ (cross-binutils/implementation target binutils))
+
(define* (cross-binutils/deprecated target #:optional (binutils binutils))
(warning (G_ "'cross-binutils' must be used with keyword arguments~%"))
- (cross-binutils* target #:binutils binutils))
-
-(define* (cross-binutils* target #:key (binutils binutils))
- "Return a cross-Binutils for TARGET using BINUTILS."
- (let ((binutils (package
- (inherit binutils)
- (arguments
- (substitute-keyword-arguments (package-arguments
- binutils)
- ((#:configure-flags flags)
- ;; Build with `--with-sysroot' so that ld honors
- ;; DT_RUNPATH entries when searching for a needed
- ;; library. This works because as a side effect
- ;; `genscripts.sh' sets `USE_LIBPATH=yes', which tells
- ;; elf32.em to use DT_RUNPATH in its search list.
- ;; See <http://sourceware.org/ml/binutils/2013-05/msg00312.html>.
- ;;
- ;; In theory choosing / as the sysroot could lead ld
- ;; to pick up native libs instead of target ones. In
- ;; practice the RUNPATH of target libs only refers to
- ;; target libs, not native libs, so this is safe.
- `(cons "--with-sysroot=/" ,flags)))))))
-
- ;; For Xtensa, apply Qualcomm's patch.
- (cross (cond ((string-prefix? "xtensa-" target)
- (package-with-patches binutils
- (search-patches
- "ath9k-htc-firmware-binutils.patch")))
- ((target-mingw? target)
- (package-with-extra-patches
- (package-with-extra-configure-variable
- ;; mingw binutils does not work correctly when configured
- ;; with `--enable-compressed-debug-sections`. An error
- ;; like the following will occur whenever you try to link:
- ;;
- ;; x86_64-w64-mingw32-ld: final link failed: bad value
- ;;
- ;; TODO: This seems like a deeper problem that warrants
- ;; deeper investigation.
- binutils "--enable-compressed-debug-sections" "no")
- (search-patches "binutils-mingw-w64-timestamp.patch"
- "binutils-mingw-w64-deterministic.patch")))
- (else binutils))
- target)))
+ (cross-binutils/implementation target binutils))
+
+(define cross-binutils/implementation
+ (mlambda (target binutils)
+ (let ((binutils (package
+ (inherit binutils)
+ (arguments
+ (substitute-keyword-arguments (package-arguments
+ binutils)
+ ((#:configure-flags flags)
+ ;; Build with `--with-sysroot' so that ld honors
+ ;; DT_RUNPATH entries when searching for a needed
+ ;; library. This works because as a side effect
+ ;; `genscripts.sh' sets `USE_LIBPATH=yes', which tells
+ ;; elf32.em to use DT_RUNPATH in its search list.
+ ;; See <http://sourceware.org/ml/binutils/2013-05/msg00312.html>.
+ ;;
+ ;; In theory choosing / as the sysroot could lead ld
+ ;; to pick up native libs instead of target ones. In
+ ;; practice the RUNPATH of target libs only refers to
+ ;; target libs, not native libs, so this is safe.
+ `(cons "--with-sysroot=/" ,flags)))))))
+
+ ;; For Xtensa, apply Qualcomm's patch.
+ (cross (cond ((string-prefix? "xtensa-" target)
+ (package-with-patches binutils
+ (search-patches
+ "ath9k-htc-firmware-binutils.patch")))
+ ((target-mingw? target)
+ (package-with-extra-patches
+ (package-with-extra-configure-variable
+ ;; mingw binutils does not work correctly when configured
+ ;; with `--enable-compressed-debug-sections`. An error
+ ;; like the following will occur whenever you try to link:
+ ;;
+ ;; x86_64-w64-mingw32-ld: final link failed: bad value
+ ;;
+ ;; TODO: This seems like a deeper problem that warrants
+ ;; deeper investigation.
+ binutils "--enable-compressed-debug-sections" "no")
+ (search-patches "binutils-mingw-w64-timestamp.patch"
+ "binutils-mingw-w64-deterministic.patch")))
+ (else binutils))
+ target))))
(define (cross-gcc-arguments target xgcc libc)
"Return build system arguments for a cross-gcc for TARGET, using XGCC as the
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 3/7] gnu: Memozise cross-gcc results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: ludo@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:41:02 +0000
Resent-Message-ID: <handler.68266.B68266.170447286114953 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286114953
(code B ref 68266); Fri, 05 Jan 2024 16:41:02 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:01 +0000
Received: from localhost ([127.0.0.1]:57727 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFk-0003sv-Tz
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:01 -0500
Received: from mira.cbaines.net ([212.71.252.8]:43064)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFg-0003ry-Fo
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:58 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 4344327BBEA
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id e46cabc2
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:45 +0000
Message-ID: <f65514f80f2a29cd40d6fb172101f31d19b65c04.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/cross-base.scm (cross-gcc): Move code to
cross-gcc/implementation and call it.
(cross-gcc/implementation) New procedure.
Change-Id: Ibeafaa4d652fc1d6fd27870b82a309c177b66a05
---
gnu/packages/cross-base.scm | 183 +++++++++++++++++++-----------------
1 file changed, 95 insertions(+), 88 deletions(-)
diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
index a04e4f9c9e..a4e361b476 100644
--- a/gnu/packages/cross-base.scm
+++ b/gnu/packages/cross-base.scm
@@ -301,6 +301,97 @@ (define (cross-gcc-search-paths target)
,(string-append target "/include")))))))
%gcc-cross-include-paths)))
+(define cross-gcc/implementation
+ (mlambda (target xgcc xbinutils libc)
+ (package
+ (inherit xgcc)
+ (name (string-append "gcc-cross-"
+ (if libc "" "sans-libc-")
+ target))
+ (source
+ (origin
+ (inherit
+ (package-source xgcc))
+ (patches
+ (append
+ (origin-patches (package-source xgcc))
+ (append (cond
+ ((version>=? (package-version xgcc) "12.0")
+ (search-patches "gcc-12-cross-environment-variables.patch"
+ "gcc-cross-gxx-include-dir.patch"))
+ ((version>=? (package-version xgcc) "10.0")
+ (search-patches "gcc-10-cross-environment-variables.patch"
+ "gcc-cross-gxx-include-dir.patch"))
+ ((version>=? (package-version xgcc) "8.0")
+ (search-patches "gcc-8-cross-environment-variables.patch"))
+ ((version>=? (package-version xgcc) "6.0")
+ (search-patches "gcc-7-cross-toolexeclibdir.patch"
+ "gcc-6-cross-environment-variables.patch"))
+ (else
+ (search-patches "gcc-cross-environment-variables.patch")))
+ (cross-gcc-patches xgcc target))))
+ (modules '((guix build utils)))
+ (snippet
+ (cross-gcc-snippet target))))
+
+ (outputs '("out" "lib"))
+
+ (arguments
+ `(#:implicit-inputs? #f
+ #:imported-modules ((gnu build cross-toolchain)
+ ,@%gnu-build-system-modules)
+ #:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (gnu build cross-toolchain)
+ (srfi srfi-1)
+ (srfi srfi-26)
+ (ice-9 regex))
+
+ ,@(cross-gcc-arguments target xgcc libc)))
+
+ (native-inputs
+ `(("ld-wrapper-cross" ,(make-ld-wrapper
+ (string-append "ld-wrapper-" target)
+ #:target (const target)
+ #:binutils xbinutils))
+ ("binutils-cross" ,xbinutils)
+
+ ,@(let ((inputs (append (package-inputs xgcc)
+ (fold alist-delete (%final-inputs)
+ '("libc" "libc:static"))
+
+ ;; Call it differently so that the builder can
+ ;; check whether the "libc" input is #f.
+ `(("libc-native"
+ ,@(assoc-ref (%final-inputs) "libc"))
+ ("libc-native:static"
+ ,@(assoc-ref (%final-inputs)
+ "libc:static"))))))
+ (cond
+ ((target-mingw? target)
+ (if libc
+ `(,@inputs
+ ("libc" ,libc))
+ `(,@inputs
+ ("mingw-source" ,(package-source mingw-w64)))))
+ ((and libc (target-avr? target))
+ `(,@inputs
+ ("libc" ,libc)))
+ (libc
+ `(,@inputs
+ ("libc" ,libc)
+ ("libc:static" ,libc "static")
+ ("xkernel-headers" ;the target headers
+ ,@(assoc-ref (package-propagated-inputs libc)
+ "kernel-headers"))))
+ (else inputs)))))
+
+ (inputs '())
+
+ ;; Only search target inputs, not host inputs.
+ (search-paths (cross-gcc-search-paths target))
+ (native-search-paths '()))))
+
(define* (cross-gcc target
#:key
(xgcc %xgcc)
@@ -310,94 +401,10 @@ (define* (cross-gcc target
XGCC as the base compiler. Use XBINUTILS as the associated cross-Binutils.
If LIBC is false, then build a GCC that does not target a libc; otherwise,
target that libc."
- (package
- (inherit xgcc)
- (name (string-append "gcc-cross-"
- (if libc "" "sans-libc-")
- target))
- (source
- (origin
- (inherit
- (package-source xgcc))
- (patches
- (append
- (origin-patches (package-source xgcc))
- (append (cond
- ((version>=? (package-version xgcc) "12.0")
- (search-patches "gcc-12-cross-environment-variables.patch"
- "gcc-cross-gxx-include-dir.patch"))
- ((version>=? (package-version xgcc) "10.0")
- (search-patches "gcc-10-cross-environment-variables.patch"
- "gcc-cross-gxx-include-dir.patch"))
- ((version>=? (package-version xgcc) "8.0")
- (search-patches "gcc-8-cross-environment-variables.patch"))
- ((version>=? (package-version xgcc) "6.0")
- (search-patches "gcc-7-cross-toolexeclibdir.patch"
- "gcc-6-cross-environment-variables.patch"))
- (else
- (search-patches "gcc-cross-environment-variables.patch")))
- (cross-gcc-patches xgcc target))))
- (modules '((guix build utils)))
- (snippet
- (cross-gcc-snippet target))))
-
- (outputs '("out" "lib"))
-
- (arguments
- `(#:implicit-inputs? #f
- #:imported-modules ((gnu build cross-toolchain)
- ,@%gnu-build-system-modules)
- #:modules ((guix build gnu-build-system)
- (guix build utils)
- (gnu build cross-toolchain)
- (srfi srfi-1)
- (srfi srfi-26)
- (ice-9 regex))
-
- ,@(cross-gcc-arguments target xgcc libc)))
-
- (native-inputs
- `(("ld-wrapper-cross" ,(make-ld-wrapper
- (string-append "ld-wrapper-" target)
- #:target (const target)
- #:binutils xbinutils))
- ("binutils-cross" ,xbinutils)
-
- ,@(let ((inputs (append (package-inputs xgcc)
- (fold alist-delete (%final-inputs)
- '("libc" "libc:static"))
-
- ;; Call it differently so that the builder can
- ;; check whether the "libc" input is #f.
- `(("libc-native"
- ,@(assoc-ref (%final-inputs) "libc"))
- ("libc-native:static"
- ,@(assoc-ref (%final-inputs)
- "libc:static"))))))
- (cond
- ((target-mingw? target)
- (if libc
- `(,@inputs
- ("libc" ,libc))
- `(,@inputs
- ("mingw-source" ,(package-source mingw-w64)))))
- ((and libc (target-avr? target))
- `(,@inputs
- ("libc" ,libc)))
- (libc
- `(,@inputs
- ("libc" ,libc)
- ("libc:static" ,libc "static")
- ("xkernel-headers" ;the target headers
- ,@(assoc-ref (package-propagated-inputs libc)
- "kernel-headers"))))
- (else inputs)))))
-
- (inputs '())
-
- ;; Only search target inputs, not host inputs.
- (search-paths (cross-gcc-search-paths target))
- (native-search-paths '())))
+ (cross-gcc/implementation target
+ xgcc
+ xbinutils
+ libc))
(define* (cross-kernel-headers . args)
(if (or (= (length args) 1) (contains-keyword? args))
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 6/7] gnu: Memozise cross-libc results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: ludo@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:42:02 +0000
Resent-Message-ID: <handler.68266.B68266.170447286314991 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286314991
(code B ref 68266); Fri, 05 Jan 2024 16:42:02 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:03 +0000
Received: from localhost ([127.0.0.1]:57730 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFm-0003tU-FI
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:03 -0500
Received: from mira.cbaines.net ([212.71.252.8]:43066)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFh-0003sA-Tp
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:59 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 7241827BBED
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id d5431592
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:48 +0000
Message-ID: <64ae83ce361b2883e9bdf4bcdaacb03a4fbd9611.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: 1.7 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: To ensure that it just returns a single package record for
some given arguments, as this helps to avoid poor performance of the store
connection object cache. * gnu/packages/cross-base.scm (cross-libc*): Move
code to cross-libc/implementation and call it. (cross-libc/implementation)
New procedure.
Content analysis details: (1.7 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: libc.so]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_SCC_BODY_TEXT_LINE No description available.
0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay
lines
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.7 (/)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/cross-base.scm (cross-libc*): Move code to
cross-libc/implementation and call it.
(cross-libc/implementation) New procedure.
Change-Id: I72f430136860e5d1fd9edeb3274678186b896bd4
---
gnu/packages/cross-base.scm | 165 +++++++++++++++++++-----------------
1 file changed, 85 insertions(+), 80 deletions(-)
diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
index 6c6c6e7636..8a40211456 100644
--- a/gnu/packages/cross-base.scm
+++ b/gnu/packages/cross-base.scm
@@ -651,10 +651,22 @@ (define cross-kernel-headers/implementation
(_ xlinux-headers))))
(define* (cross-libc . args)
+ "Return LIBC cross-built for TARGET, a GNU triplet. Use XGCC and XBINUTILS
+and the cross tool chain. If TARGET doesn't have a standard C library #f is
+returned."
(if (or (= (length args) 1) (contains-keyword? args))
(apply cross-libc* args)
(apply cross-libc/deprecated args)))
+(define* (cross-libc* target
+ #:key
+ (libc (libc-for-target target))
+ (xgcc (cross-gcc target))
+ (xbinutils (cross-binutils target))
+ (xheaders (cross-kernel-headers target)))
+ (cross-libc/implementation target libc
+ xgcc xbinutils xheaders))
+
(define* (cross-libc/deprecated target
#:optional
(libc (libc-for-target target))
@@ -668,88 +680,81 @@ (define* (cross-libc/deprecated target
#:xbinutils xbinutils
#:xheaders xheaders))
-(define* (cross-libc* target
- #:key
- (libc (libc-for-target target))
- (xgcc (cross-gcc target))
- (xbinutils (cross-binutils target))
- (xheaders (cross-kernel-headers target)))
- "Return LIBC cross-built for TARGET, a GNU triplet. Use XGCC and XBINUTILS
-and the cross tool chain. If TARGET doesn't have a standard C library #f is
-returned."
- (match target
- ((? target-mingw?)
- (let ((machine (substring target 0 (string-index target #\-))))
- (make-mingw-w64 machine
- #:xgcc xgcc
- #:xbinutils xbinutils)))
- ((or (? target-linux?) (? target-hurd?))
- (package
- (inherit libc)
- (name (string-append "glibc-cross-" target))
- (arguments
- (substitute-keyword-arguments
- `(;; Disable stripping (see above.)
- #:strip-binaries? #f
-
- ;; This package is used as a target input, but it should not have
- ;; the usual cross-compilation inputs since that would include
- ;; itself.
- #:implicit-cross-inputs? #f
-
- ;; We need SRFI 26.
- #:modules ((guix build gnu-build-system)
- (guix build utils)
- (srfi srfi-26))
-
- ,@(package-arguments libc))
- ((#:configure-flags flags)
- `(cons ,(string-append "--host=" target)
- ,(if (target-hurd? target)
- `(append (list "--disable-werror"
- ,@%glibc/hurd-configure-flags)
- ,flags)
- flags)))
- ((#:phases phases)
- `(modify-phases ,phases
- (add-before 'configure 'set-cross-kernel-headers-path
- (lambda* (#:key inputs #:allow-other-keys)
- (let* ((kernel (assoc-ref inputs "kernel-headers"))
- (cpath (string-append kernel "/include")))
- (for-each (cut setenv <> cpath)
- ',%gcc-cross-include-paths)
- (setenv "CROSS_LIBRARY_PATH"
+(define cross-libc/implementation
+ (mlambda (target libc xgcc xbinutils xheaders)
+ (match target
+ ((? target-mingw?)
+ (let ((machine (substring target 0 (string-index target #\-))))
+ (make-mingw-w64 machine
+ #:xgcc xgcc
+ #:xbinutils xbinutils)))
+ ((or (? target-linux?) (? target-hurd?))
+ (package
+ (inherit libc)
+ (name (string-append "glibc-cross-" target))
+ (arguments
+ (substitute-keyword-arguments
+ `(;; Disable stripping (see above.)
+ #:strip-binaries? #f
+
+ ;; This package is used as a target input, but it should not have
+ ;; the usual cross-compilation inputs since that would include
+ ;; itself.
+ #:implicit-cross-inputs? #f
+
+ ;; We need SRFI 26.
+ #:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (srfi srfi-26))
+
+ ,@(package-arguments libc))
+ ((#:configure-flags flags)
+ `(cons ,(string-append "--host=" target)
+ ,(if (target-hurd? target)
+ `(append (list "--disable-werror"
+ ,@%glibc/hurd-configure-flags)
+ ,flags)
+ flags)))
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (add-before 'configure 'set-cross-kernel-headers-path
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((kernel (assoc-ref inputs "kernel-headers"))
+ (cpath (string-append kernel "/include")))
+ (for-each (cut setenv <> cpath)
+ ',%gcc-cross-include-paths)
+ (setenv "CROSS_LIBRARY_PATH"
(string-append kernel "/lib")) ; for Hurd's libihash
#t)))
- ,@(if (target-hurd? target)
- '((add-after 'install 'augment-libc.so
- (lambda* (#:key outputs #:allow-other-keys)
- (let* ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/libc.so")
- (("/[^ ]+/lib/libc.so.0.3")
- (string-append out "/lib/libc.so.0.3"
- " libmachuser.so libhurduser.so"))))
- #t)))
- '())))))
-
- ;; Shadow the native "kernel-headers" because glibc's recipe expects the
- ;; "kernel-headers" input to point to the right thing.
- (propagated-inputs `(("kernel-headers" ,xheaders)))
-
- (native-inputs `(("cross-gcc" ,xgcc)
- ("cross-binutils" ,xbinutils)
- ,@(if (target-hurd? target)
- `(("cross-mig"
- ,(cross-mig target
- #:xgcc xgcc
- #:xbinutils xbinutils)))
- '())
- ,@(package-inputs libc) ;FIXME: static-bash
- ,@(package-native-inputs libc)))))
- ((? target-avr?)
- (make-avr-libc #:xbinutils xbinutils
- #:xgcc xgcc))
- (else #f)))
+ ,@(if (target-hurd? target)
+ '((add-after 'install 'augment-libc.so
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out")))
+ (substitute* (string-append out "/lib/libc.so")
+ (("/[^ ]+/lib/libc.so.0.3")
+ (string-append out "/lib/libc.so.0.3"
+ " libmachuser.so libhurduser.so"))))
+ #t)))
+ '())))))
+
+ ;; Shadow the native "kernel-headers" because glibc's recipe expects the
+ ;; "kernel-headers" input to point to the right thing.
+ (propagated-inputs `(("kernel-headers" ,xheaders)))
+
+ (native-inputs `(("cross-gcc" ,xgcc)
+ ("cross-binutils" ,xbinutils)
+ ,@(if (target-hurd? target)
+ `(("cross-mig"
+ ,(cross-mig target
+ #:xgcc xgcc
+ #:xbinutils xbinutils)))
+ '())
+ ,@(package-inputs libc) ;FIXME: static-bash
+ ,@(package-native-inputs libc)))))
+ ((? target-avr?)
+ (make-avr-libc #:xbinutils xbinutils
+ #:xgcc xgcc))
+ (else #f))))
(define* (cross-gcc-toolchain/implementation target
#:key
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: efraim@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:42:02 +0000
Resent-Message-ID: <handler.68266.B68266.170447286315008 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim@HIDDEN>
X-Debbugs-Original-Xcc: Efraim Flashner <efraim@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286315008
(code B ref 68266); Fri, 05 Jan 2024 16:42:02 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:03 +0000
Received: from localhost ([127.0.0.1]:57734 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFn-0003tk-3H
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:03 -0500
Received: from mira.cbaines.net
([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:55299)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFh-0003s9-Rb
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:59 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 7D6CA27BBEE
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id d075a7ef
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:49 +0000
Message-ID: <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/rust.scm (make-rust-sysroot): Move code to
make-rust-sysroot/implementation.
(make-rust-sysroot/implementation): New variable.
Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167
---
gnu/packages/rust.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index ea95d27476..64cde7ea14 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -60,6 +60,7 @@ (define-module (gnu packages rust)
#:use-module (guix build-system gnu)
#:use-module (guix build-system trivial)
#:use-module (guix download)
+ #:use-module (guix memoization)
#:use-module (guix git-download)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix packages)
@@ -1057,7 +1058,10 @@ (define-public rust
(package-native-inputs base-rust))))))
(define*-public (make-rust-sysroot target)
- (let ((base-rust rust))
+ (make-rust-sysroot/implementation target rust))
+
+(define make-rust-sysroot/implementation
+ (mlambda (target base-rust)
(package
(inherit base-rust)
(name (string-append "rust-sysroot-for-" target))
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 5/7] gnu: Memozise cross-mig results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: ludo@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:42:02 +0000
Resent-Message-ID: <handler.68266.B68266.170447286415017 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286415017
(code B ref 68266); Fri, 05 Jan 2024 16:42:02 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:04 +0000
Received: from localhost ([127.0.0.1]:57736 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFn-0003tz-Iw
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:04 -0500
Received: from mira.cbaines.net
([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:33519)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFh-0003s1-8J
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:59 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 5C84D27BBEC
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id d12c39fb
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:47 +0000
Message-ID: <e1640e9c30677bb61093dd3b8e409dc0033ebe3a.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/cross-base.scm (cross-mig): Move code to
cross-mig/implementation and call it.
(cross-mig/implementation) New procedure.
Change-Id: Iaf2a69c48664d2f0766b9d2f6e981653e0e3c44c
---
gnu/packages/cross-base.scm | 56 +++++++++++++++++++------------------
1 file changed, 29 insertions(+), 27 deletions(-)
diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
index f966e2f5ac..6c6c6e7636 100644
--- a/gnu/packages/cross-base.scm
+++ b/gnu/packages/cross-base.scm
@@ -467,37 +467,39 @@ (define* (cross-mig target
(xbinutils (cross-binutils target)))
"Return a cross-mig for TARGET, where TARGET is a GNU triplet. Use XGCC as
the base compiler. Use XBINUTILS as the associated cross-Binutils."
- (define xgnumach-headers
- (cross-gnumach-headers target
- #:xgcc xgcc
- #:xbinutils xbinutils))
- (package
- (inherit mig)
- (name (string-append "mig-cross"))
- (arguments
- (substitute-keyword-arguments (package-arguments mig)
- ((#:configure-flags flags #~'())
- #~(list #$(string-append "--target=" target)))
- ((#:tests? _ #f)
- #f)
- ((#:phases phases #~%standard-phases)
- #~(modify-phases #$phases
- (add-before 'configure 'set-cross-headers-path
- (lambda* (#:key inputs #:allow-other-keys)
- (let* ((mach #+xgnumach-headers)
- (cpath (string-append mach "/include")))
- (for-each (lambda (variable)
- (setenv variable cpath))
- '#$%gcc-cross-include-paths))))))))
- (propagated-inputs
- (list xgnumach-headers))
- (native-inputs
- (modify-inputs (package-native-inputs mig)
- (prepend xgcc xbinutils)))))
+ (cross-mig/implementation target xgcc xbinutils))
+(define cross-mig/implementation
+ (mlambda (target xgcc xbinutils)
+ "Return a cross-mig for TARGET, where TARGET is a GNU triplet. Use XGCC as
+the base compiler. Use XBINUTILS as the associated cross-Binutils."
+ (define xgnumach-headers
+ (cross-gnumach-headers target
+ #:xgcc xgcc
+ #:xbinutils xbinutils))
(package
+ (inherit mig)
+ (name (string-append "mig-cross"))
(arguments
+ (substitute-keyword-arguments (package-arguments mig)
+ ((#:configure-flags flags #~'())
+ #~(list #$(string-append "--target=" target)))
+ ((#:tests? _ #f)
+ #f)
+ ((#:phases phases #~%standard-phases)
#~(modify-phases #$phases
+ (add-before 'configure 'set-cross-headers-path
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((mach #+xgnumach-headers)
+ (cpath (string-append mach "/include")))
+ (for-each (lambda (variable)
+ (setenv variable cpath))
+ '#$%gcc-cross-include-paths))))))))
+ (propagated-inputs
+ (list xgnumach-headers))
+ (native-inputs
+ (modify-inputs (package-native-inputs mig)
+ (prepend xgcc xbinutils))))))
(define cross-kernel-headers/implementation
(mlambda (target linux-headers xgcc xbinutils)
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 4/7] gnu: Memozise cross-kernel-headers results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: ludo@HIDDEN, guix-patches@HIDDEN
Resent-Date: Fri, 05 Jan 2024 16:42:03 +0000
Resent-Message-ID: <handler.68266.B68266.170447286515023 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170447286515023
(code B ref 68266); Fri, 05 Jan 2024 16:42:03 +0000
Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:05 +0000
Received: from localhost ([127.0.0.1]:57738 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rLnFo-0003uA-7R
for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:05 -0500
Received: from mira.cbaines.net
([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:41563)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rLnFh-0003s0-5T
for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:00 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 501D827BBEB
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id badd81b3
for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 5 Jan 2024 16:40:46 +0000
Message-ID: <4c720cbfb79b514b7ebae3a2f29998f198aa845f.1704472849.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
To ensure that it just returns a single package record for some given
arguments, as this helps to avoid poor performance of the store connection
object cache.
* gnu/packages/cross-base.scm (cross-kernel-headers*): Move code to
cross-kernel-headers/implementation and call it.
(cross-kernel-headers/implementation) New procedure.
Change-Id: I345604c089e7a8a9884c07f39c95f960760e86db
---
gnu/packages/cross-base.scm | 306 ++++++++++++++++++------------------
1 file changed, 157 insertions(+), 149 deletions(-)
diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
index a4e361b476..f966e2f5ac 100644
--- a/gnu/packages/cross-base.scm
+++ b/gnu/packages/cross-base.scm
@@ -407,10 +407,19 @@ (define* (cross-gcc target
libc))
(define* (cross-kernel-headers . args)
+ "Return headers depending on TARGET."
(if (or (= (length args) 1) (contains-keyword? args))
(apply cross-kernel-headers* args)
(apply cross-kernel-headers/deprecated args)))
+(define* (cross-kernel-headers* target
+ #:key
+ (linux-headers linux-libre-headers)
+ (xgcc (cross-gcc target))
+ (xbinutils (cross-binutils target)))
+ (cross-kernel-headers/implementation target
+ linux-headers xgcc xbinutils))
+
(define* (cross-kernel-headers/deprecated target
#:optional
(linux-headers linux-libre-headers)
@@ -486,159 +495,158 @@ (define* (cross-mig target
(modify-inputs (package-native-inputs mig)
(prepend xgcc xbinutils)))))
-(define* (cross-kernel-headers* target
- #:key
- (linux-headers linux-libre-headers)
- (xgcc (cross-gcc target))
- (xbinutils (cross-binutils target)))
- "Return headers depending on TARGET."
-
- (define xlinux-headers
(package
- (inherit linux-headers)
- (name (string-append (package-name linux-headers)
- "-cross-" target))
(arguments
- (substitute-keyword-arguments
- `(#:implicit-cross-inputs? #f
- ,@(package-arguments linux-headers))
- ((#:phases phases)
- `(modify-phases ,phases
- (replace 'build
- (lambda _
- (setenv "ARCH" ,(platform-linux-architecture
- (lookup-platform-by-target target)))
- (format #t "`ARCH' set to `~a' (cross compiling)~%"
- (getenv "ARCH"))
-
- (invoke "make" ,(system->defconfig target))
- (invoke "make" "mrproper"
- ,@(if (version>=? (package-version linux-headers) "5.3")
- '("headers")
- '("headers_check")))))))))
- (native-inputs `(("cross-gcc" ,xgcc)
- ("cross-binutils" ,xbinutils)
- ,@(package-native-inputs linux-headers)))))
-
- (define xmig
- (cross-mig target #:xgcc xgcc #:xbinutils xbinutils))
-
- (define xgnumach-headers
- (cross-gnumach-headers target #:xgcc xgcc #:xbinutils xbinutils))
-
- (define xhurd-headers
- (package
- (inherit hurd-headers)
- (name (string-append (package-name hurd-headers)
- "-cross-" target))
-
- (arguments
- (substitute-keyword-arguments (package-arguments hurd-headers)
- ((#:configure-flags flags)
- `(cons* ,(string-append "--build=" (%current-system))
- ,(string-append "--host=" target)
- ,flags))))
-
- (native-inputs `(("cross-gcc" ,xgcc)
- ("cross-binutils" ,xbinutils)
- ("cross-mig" ,xmig)
- ,@(alist-delete "mig" (package-native-inputs hurd-headers))))))
-
- (define xglibc/hurd-headers
- (package
- (inherit glibc/hurd-headers)
- (name (string-append (package-name glibc/hurd-headers)
- "-cross-" target))
-
- (arguments
- (substitute-keyword-arguments
- `(#:modules ((guix build gnu-build-system)
- (guix build utils)
- (srfi srfi-26))
- ,@(package-arguments glibc/hurd-headers))
- ((#:phases phases)
- `(modify-phases ,phases
- (add-after 'unpack 'set-cross-headers-path
- (lambda* (#:key inputs #:allow-other-keys)
- (let* ((mach (assoc-ref inputs "gnumach-headers"))
- (hurd (assoc-ref inputs "hurd-headers"))
- (cpath (string-append mach "/include:"
- hurd "/include")))
- (for-each (cut setenv <> cpath)
- ',%gcc-cross-include-paths)
- #t)))))
- ((#:configure-flags flags)
- `(cons* ,(string-append "--build=" (%current-system))
- ,(string-append "--host=" target)
- ,flags))))
-
- (propagated-inputs `(("gnumach-headers" ,xgnumach-headers)
- ("hurd-headers" ,xhurd-headers)))
-
- (native-inputs `(("cross-gcc" ,xgcc)
- ("cross-binutils" ,xbinutils)
- ("cross-mig" ,xmig)
- ,@(alist-delete "mig"(package-native-inputs glibc/hurd-headers))))))
-
- (define xhurd-minimal
- (package
- (inherit hurd-minimal)
- (name (string-append (package-name hurd-minimal)
- "-cross-" target))
- (arguments
- (substitute-keyword-arguments
- `(#:modules ((guix build gnu-build-system)
- (guix build utils)
- (srfi srfi-26))
- ,@(package-arguments hurd-minimal))
- ((#:configure-flags flags)
- `(cons* ,(string-append "--build=" (%current-system))
- ,(string-append "--host=" target)
- ,flags))
- ((#:phases phases)
#~(modify-phases #$phases
- (add-after 'unpack 'delete-shared-target
- ;; Cannot create shared libraries due to missing crt1.o
- (lambda _
- (substitute* "Makeconf"
- (("(targets := \\$\\(libname\\)\\.a) \\$\\(libname\\)\\.so" all static)
- static)
- (("\\$\\(DESTDIR\\)\\$\\(libdir\\)/\\$\\(libname\\)\\.so\\.\\$\\(hurd-version\\)")
- "")
- (("^libs: .*\\.so\\..*" all)
- (string-append "# " all)))))
- (add-before 'configure 'set-cross-headers-path
- (lambda* (#:key inputs #:allow-other-keys)
- (let* ((glibc-headers (assoc-ref inputs "cross-glibc-hurd-headers"))
- (mach-headers (assoc-ref inputs "cross-gnumach-headers"))
- (cpath (string-append glibc-headers "/include"
- ":" mach-headers "/include")))
- (for-each (cut setenv <> cpath)
- '#$%gcc-cross-include-paths)
- #t)))))))
-
- (inputs `(("cross-glibc-hurd-headers" ,xglibc/hurd-headers)
- ("cross-gnumach-headers" ,xgnumach-headers)))
- (native-inputs `(("cross-gcc" ,xgcc)
- ("cross-binutils" ,xbinutils)
- ("cross-mig" ,xmig)
- ,@(alist-delete "mig"
- (package-native-inputs hurd-minimal))))))
-
- (define xhurd-core-headers
- (package
- (inherit hurd-core-headers)
- (name (string-append (package-name hurd-core-headers)
- "-cross-" target))
-
- (inputs `(("gnumach-headers" ,xgnumach-headers)
- ("hurd-headers" ,xhurd-headers)
- ("hurd-minimal" ,xhurd-minimal)))))
-
- (match target
- ((or "i586-pc-gnu" "i586-gnu") xhurd-core-headers)
- (_ xlinux-headers)))
+(define cross-kernel-headers/implementation
+ (mlambda (target linux-headers xgcc xbinutils)
+ (define xlinux-headers
+ (package
+ (inherit linux-headers)
+ (name (string-append (package-name linux-headers)
+ "-cross-" target))
+ (arguments
+ (substitute-keyword-arguments
+ `(#:implicit-cross-inputs? #f
+ ,@(package-arguments linux-headers))
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'build
+ (lambda _
+ (setenv "ARCH" ,(platform-linux-architecture
+ (lookup-platform-by-target target)))
+ (format #t "`ARCH' set to `~a' (cross compiling)~%"
+ (getenv "ARCH"))
+
+ (invoke "make" ,(system->defconfig target))
+ (invoke "make" "mrproper"
+ ,@(if (version>=? (package-version linux-headers) "5.3")
+ '("headers")
+ '("headers_check")))))))))
+ (native-inputs `(("cross-gcc" ,xgcc)
+ ("cross-binutils" ,xbinutils)
+ ,@(package-native-inputs linux-headers)))))
+
+ (define xmig
+ (cross-mig target #:xgcc xgcc #:xbinutils xbinutils))
+
+ (define xgnumach-headers
+ (cross-gnumach-headers target #:xgcc xgcc #:xbinutils xbinutils))
+
+ (define xhurd-headers
+ (package
+ (inherit hurd-headers)
+ (name (string-append (package-name hurd-headers)
+ "-cross-" target))
+
+ (arguments
+ (substitute-keyword-arguments (package-arguments hurd-headers)
+ ((#:configure-flags flags)
+ `(cons* ,(string-append "--build=" (%current-system))
+ ,(string-append "--host=" target)
+ ,flags))))
+
+ (native-inputs `(("cross-gcc" ,xgcc)
+ ("cross-binutils" ,xbinutils)
+ ("cross-mig" ,xmig)
+ ,@(alist-delete "mig" (package-native-inputs hurd-headers))))))
+
+ (define xglibc/hurd-headers
+ (package
+ (inherit glibc/hurd-headers)
+ (name (string-append (package-name glibc/hurd-headers)
+ "-cross-" target))
+
+ (arguments
+ (substitute-keyword-arguments
+ `(#:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (srfi srfi-26))
+ ,@(package-arguments glibc/hurd-headers))
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (add-after 'unpack 'set-cross-headers-path
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((mach (assoc-ref inputs "gnumach-headers"))
+ (hurd (assoc-ref inputs "hurd-headers"))
+ (cpath (string-append mach "/include:"
+ hurd "/include")))
+ (for-each (cut setenv <> cpath)
+ ',%gcc-cross-include-paths)
+ #t)))))
+ ((#:configure-flags flags)
+ `(cons* ,(string-append "--build=" (%current-system))
+ ,(string-append "--host=" target)
+ ,flags))))
+
+ (propagated-inputs `(("gnumach-headers" ,xgnumach-headers)
+ ("hurd-headers" ,xhurd-headers)))
+
+ (native-inputs `(("cross-gcc" ,xgcc)
+ ("cross-binutils" ,xbinutils)
+ ("cross-mig" ,xmig)
+ ,@(alist-delete "mig"(package-native-inputs glibc/hurd-headers))))))
+
+ (define xhurd-minimal
+ (package
+ (inherit hurd-minimal)
+ (name (string-append (package-name hurd-minimal)
+ "-cross-" target))
+ (arguments
+ (substitute-keyword-arguments
+ `(#:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (srfi srfi-26))
+ ,@(package-arguments hurd-minimal))
+ ((#:configure-flags flags)
+ `(cons* ,(string-append "--build=" (%current-system))
+ ,(string-append "--host=" target)
+ ,flags))
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ (add-after 'unpack 'delete-shared-target
+ ;; Cannot create shared libraries due to missing crt1.o
+ (lambda _
+ (substitute* "Makeconf"
+ (("(targets := \\$\\(libname\\)\\.a) \\$\\(libname\\)\\.so" all static)
+ static)
+ (("\\$\\(DESTDIR\\)\\$\\(libdir\\)/\\$\\(libname\\)\\.so\\.\\$\\(hurd-version\\)")
+ "")
+ (("^libs: .*\\.so\\..*" all)
+ (string-append "# " all)))))
+ (add-before 'configure 'set-cross-headers-path
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((glibc-headers (assoc-ref inputs "cross-glibc-hurd-headers"))
+ (mach-headers (assoc-ref inputs "cross-gnumach-headers"))
+ (cpath (string-append glibc-headers "/include"
+ ":" mach-headers "/include")))
+ (for-each (cut setenv <> cpath)
+ '#$%gcc-cross-include-paths)
+ #t)))))))
+
+ (inputs `(("cross-glibc-hurd-headers" ,xglibc/hurd-headers)
+ ("cross-gnumach-headers" ,xgnumach-headers)))
+
+ (native-inputs `(("cross-gcc" ,xgcc)
+ ("cross-binutils" ,xbinutils)
+ ("cross-mig" ,xmig)
+ ,@(alist-delete "mig"
+ (package-native-inputs hurd-minimal))))))
+
+ (define xhurd-core-headers
+ (package
+ (inherit hurd-core-headers)
+ (name (string-append (package-name hurd-core-headers)
+ "-cross-" target))
+
+ (inputs `(("gnumach-headers" ,xgnumach-headers)
+ ("hurd-headers" ,xhurd-headers)
+ ("hurd-minimal" ,xhurd-minimal)))))
+
+ (match target
+ ((or "i586-pc-gnu" "i586-gnu") xhurd-core-headers)
+ (_ xlinux-headers))))
(define* (cross-libc . args)
(if (or (= (length args) 1) (contains-keyword? args))
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 08 Jan 2024 17:23:02 +0000
Resent-Message-ID: <handler.68266.B68266.17047345646473 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Christopher Baines <mail@HIDDEN>
Cc: 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.17047345646473
(code B ref 68266); Mon, 08 Jan 2024 17:23:02 +0000
Received: (at 68266) by debbugs.gnu.org; 8 Jan 2024 17:22:44 +0000
Received: from localhost ([127.0.0.1]:37461 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rMtKm-0001gL-BH
for submit <at> debbugs.gnu.org; Mon, 08 Jan 2024 12:22:44 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:51916)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rMtKk-0001g6-JI
for 68266 <at> debbugs.gnu.org; Mon, 08 Jan 2024 12:22:42 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1rMtKY-0002vN-VZ; Mon, 08 Jan 2024 12:22:30 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=PTQ1mRNSVVdL/eROxysbOX6oock2xMkZ2s1XT18Mpyc=; b=Qx6lE1l5cN9DoYWZaLvs
Gn/xyKEijSK7Uw1jq1Ypo1ycaZhqHnFOs1NHnqOIS66c4yhZlWxlViwDxf43vE+lpFSA7Ts8pAZCW
Y5TjruyxzC8oZY/FcUmVCAtbLqARUQfRFX557oyT216Ev2Up+eKDxbdTexumBc7lA6t52YTGkfgGM
0t9dHrgyX0NoLMycax6hL7XsRYCIfoQXBs8C4+I2UqWDtFXtjC8SaWipILR9YiQYnxXr9r0DggcVd
nTbykwEttxsjak/3ACTFlix0hwk/RdXQmzvW1Q43hAO+LSXwKXHfX0xV+s6BuMjKcLoiRz+9PL2ry
RHVNwT02QvpcWA==;
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
(Christopher Baines's message of "Fri, 5 Jan 2024 16:40:43 +0000")
References: <87plyfrb2x.fsf@HIDDEN>
<13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Nonidi 19 =?UTF-8?Q?Niv=C3=B4se?= an 232 de la
=?UTF-8?Q?R=C3=A9volution,?= jour du Marbre
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 08 Jan 2024 18:22:28 +0100
Message-ID: <87v8833fl7.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi,
Christopher Baines <mail@HIDDEN> skribis:
> To ensure that it just returns a single package record for some given
> arguments, as this helps to avoid poor performance of the store connection
> object cache.
>
> * gnu/packages/base.scm (make-ld-wrapper): Move code to
> make-ld-wrapper/implementation and call it.
> (make-ld-wrapper/implementation) New procedure.
>
> Change-Id: Id6fc805a4a7ffbc5ff0a5174eafcdf2c7c46854d
Do you have figures before and after the change?
The reason I=E2=80=99m asking is that (gnu packages commencement) arranges =
to
not call =E2=80=98make-ld-wrapper=E2=80=99 repeatedly already. For instanc=
e, there=E2=80=99s:
(define-public ld-wrapper
;; The final 'ld' wrapper, which uses the final Guile and Binutils.
(make-ld-wrapper "ld-wrapper"
#:binutils binutils-final
#:guile guile-final
#:bash bash-final))
and from there on we manipulate a single <package> record.
Thanks,
Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 08 Jan 2024 19:10:02 +0000
Resent-Message-ID: <handler.68266.B68266.170474100122277 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170474100122277
(code B ref 68266); Mon, 08 Jan 2024 19:10:02 +0000
Received: (at 68266) by debbugs.gnu.org; 8 Jan 2024 19:10:01 +0000
Received: from localhost ([127.0.0.1]:37587 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rMv0a-0005n8-0c
for submit <at> debbugs.gnu.org; Mon, 08 Jan 2024 14:10:01 -0500
Received: from mira.cbaines.net ([212.71.252.8]:43072)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rMv0U-0005mx-Vu
for 68266 <at> debbugs.gnu.org; Mon, 08 Jan 2024 14:09:58 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 1DBD827BBE2;
Mon, 8 Jan 2024 19:09:47 +0000 (GMT)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id 2dc91849;
Mon, 8 Jan 2024 19:09:45 +0000 (UTC)
References: <87plyfrb2x.fsf@HIDDEN>
<13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<87v8833fl7.fsf@HIDDEN>
User-agent: mu4e 1.10.7; emacs 29.1
From: Christopher Baines <mail@HIDDEN>
Date: Mon, 08 Jan 2024 19:01:58 +0000
In-reply-to: <87v8833fl7.fsf@HIDDEN>
Message-ID: <878r4zr6a0.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Hi,
>
> Christopher Baines <mail@HIDDEN> skribis:
>
>> To ensure that it just returns a single package record for some given
>> arguments, as this helps to avoid poor performance of the store connecti=
on
>> object cache.
>>
>> * gnu/packages/base.scm (make-ld-wrapper): Move code to
>> make-ld-wrapper/implementation and call it.
>> (make-ld-wrapper/implementation) New procedure.
>>
>> Change-Id: Id6fc805a4a7ffbc5ff0a5174eafcdf2c7c46854d
>
> Do you have figures before and after the change?
>
> The reason I=E2=80=99m asking is that (gnu packages commencement) arrange=
s to
> not call =E2=80=98make-ld-wrapper=E2=80=99 repeatedly already. For insta=
nce, there=E2=80=99s:
>
> (define-public ld-wrapper
> ;; The final 'ld' wrapper, which uses the final Guile and Binutils.
> (make-ld-wrapper "ld-wrapper"
> #:binutils binutils-final
> #:guile guile-final
> #:bash bash-final))
>
> and from there on we manipulate a single <package> record.
I believe the reason packages from make-ld-wrapper were showing up
multiple times in the cache for me is linked to it's use in the
cross-base module, as part of the cross-gcc procedure.
A later commit does change cross-gcc to return a single package record
for some given arguments, so that probably resolves the biggest misuse
of make-ld-wrapper.
I think there's other cases (in the llvm and mold modules) where it
looks like it's called multiple times with the same arguments, so maybe
that's an argument for having memoization around make-ld-wrapper even
though it's not needed for all uses.
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWcSHdfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XfZVQ/9EnUCnl48GyAQxKQWWzWUsNcacRgB7HVD
teI7hKFAosZgzESSux7GKzZJDHS9YqR9qWgnfZlE9+/zYt7rIGsp7pAZrAolVR36
h7D72J1gS78wlowUQRSCUWNgp+Un2KJM2HyI+cm5dRap8BTeZQVH0vygVH+Z39vx
k07N3wjeyr+FYhnwW2SN+DCKi+9tkswMeWkNUPgcSbFUIK0V+D/o6f4fFgkmnHAF
lbY2N9RgBjlc4cKJPvD63iGgxXWJE7u89kfGKgj9+x8CHjRw12eU02r2gzpsjsdA
JYD/gCP6oHPbpxtl3YLgL/6SzrN9G/aCLSZuxmsaRW7XrvZlofjvdHViiu9SoCXT
6053TPL9DiuRmF+CZMYfplP8Wct/SMS45S/kYAcJjKZIWSB0xuuuwN+JMoJJwSr5
6p2W0/wzMqvqEJYELvO9ZwkA30XfxwxRJK3Fh8HbjPVqpo8Fm2qL5h4TAHkuW/xv
BL2rjBNDAP6KovnRpcYBtsFo7q6PGkuw6mD9wOjXeNKF/JICLlrptMGhH+c0ML2L
QCZiAQgTlkjEGh3Udq9teWRY7LZvkW3TuxCs0gphs+ImUY6KkaQt0H7frh0O1tlZ
KDLBM8svs5OSAQM3cWSDXV+yqR2dWDaG3B7mVykhzyJKHbx8I6ex7jG5mEYnyH2O
Jx4Ed8eVBUI=
=G7+r
-----END PGP SIGNATURE-----
--=-=-=--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Tue, 09 Jan 2024 23:11:01 +0000
Resent-Message-ID: <handler.68266.B68266.170484182424831 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Christopher Baines <mail@HIDDEN>
Cc: 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170484182424831
(code B ref 68266); Tue, 09 Jan 2024 23:11:01 +0000
Received: (at 68266) by debbugs.gnu.org; 9 Jan 2024 23:10:24 +0000
Received: from localhost ([127.0.0.1]:41254 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rNLEl-0006SR-QP
for submit <at> debbugs.gnu.org; Tue, 09 Jan 2024 18:10:24 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:37558)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rNLEk-0006SD-83
for 68266 <at> debbugs.gnu.org; Tue, 09 Jan 2024 18:10:22 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1rNLEX-0007Pv-KA; Tue, 09 Jan 2024 18:10:09 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=CldhrM5j3uJvBBy8LZ4aN5RSUGHm8zBlvU0kfDe8qjo=; b=TDGRJxbPqrxASc2w9Uql
6VnSimOw7zW2ObokeHOu6KpfFAb8I1tuVqdcKKvdrZ2UYJ0OEVTpol94kR/sCFxnisDqxbg08k5wk
pS9twliHkK2UC3EDzsEzn2/UHPUeXNNtv/1K/Py1uLMdutyFQPzsb926GwH14cigCND4PQpgFjk0u
izF7M86JFLkubU7yH6GMVcAnhf14HQWIRXtPt2S7G1PNydbj4tzvXL5QQSiZItJF+AdUWgEKgeBM3
+69JGS4A3A4VJzEVLKvHYyzcpFhB7vfP2Sn+PdzwFsi8BCfQ5sXxmDwSiRGLZaIWhbv19osRAtJoy
rGc8+hEn53U3Hw==;
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <878r4zr6a0.fsf@HIDDEN> (Christopher Baines's message of
"Mon, 08 Jan 2024 19:01:58 +0000")
References: <87plyfrb2x.fsf@HIDDEN>
<13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<87v8833fl7.fsf@HIDDEN> <878r4zr6a0.fsf@HIDDEN>
Date: Wed, 10 Jan 2024 00:10:04 +0100
Message-ID: <87plyaw1bn.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,
Christopher Baines <mail@HIDDEN> skribis:
> I believe the reason packages from make-ld-wrapper were showing up
> multiple times in the cache for me is linked to it's use in the
> cross-base module, as part of the cross-gcc procedure.
>
> A later commit does change cross-gcc to return a single package record
> for some given arguments, so that probably resolves the biggest misuse
> of make-ld-wrapper.
Oh, I see.
Before resorting to memoization (which doesn=E2=80=99t come for free), we c=
an
for instance make sure we do not create new package records from inputs
fields (since they are thunked, a fresh record would be created each
time the field is accessed). Here=E2=80=99s an example:
--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline
diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
index 6ee7b315d8..ca3b41a4c7 100644
--- a/gnu/packages/cross-base.scm
+++ b/gnu/packages/cross-base.scm
@@ -306,6 +306,11 @@ (define* (cross-gcc target
XGCC as the base compiler. Use XBINUTILS as the associated cross-Binutils.
If LIBC is false, then build a GCC that does not target a libc; otherwise,
target that libc."
+ (define ld-wrapper
+ (make-ld-wrapper (string-append "ld-wrapper-" target)
+ #:target (const target)
+ #:binutils xbinutils))
+
(package
(inherit xgcc)
(name (string-append "gcc-cross-"
@@ -313,8 +318,7 @@ (define* (cross-gcc target
target))
(source
(origin
- (inherit
- (package-source xgcc))
+ (inherit (package-source xgcc))
(patches
(append
(origin-patches (package-source xgcc))
@@ -353,10 +357,7 @@ (define* (cross-gcc target
,@(cross-gcc-arguments target xgcc libc)))
(native-inputs
- `(("ld-wrapper-cross" ,(make-ld-wrapper
- (string-append "ld-wrapper-" target)
- #:target (const target)
- #:binutils xbinutils))
+ `(("ld-wrapper-cross" ,ld-wrapper)
("binutils-cross" ,xbinutils)
,@(let ((inputs (append (package-inputs xgcc)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
This particular one doesn=E2=80=99t have a huge impact on simple cases but =
it
might pay off for bigger graphs:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target=3D=
aarch64-linux-gnu --no-grafts -d
/gnu/store/i8x1gg73x3fvxn4lrrgbcb97sz4qq1yx-hello-2.12.1.drv
Object Cache:
fresh caches: 20
lookups: 5514
hits: 5008 (90.8%)
cache size: 505 entries
$ # before:
$ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target=3D=
aarch64-linux-gnu --no-grafts -d
/gnu/store/kdnh9bwg874aq8bvvw9y2vkl7z94icqa-hello-2.12.1.drv
Object Cache:
fresh caches: 20
lookups: 5406
hits: 4907 (90.8%)
cache size: 498 entries
--8<---------------cut here---------------end--------------->8---
> I think there's other cases (in the llvm and mold modules) where it
> looks like it's called multiple times with the same arguments, so maybe
> that's an argument for having memoization around make-ld-wrapper even
> though it's not needed for all uses.
On guix-devel, Efraim mentioned cargo-build-system packages. Is that
related?
Thanks for looking into this!
Ludo=E2=80=99.
--=-=-=--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 10 Jan 2024 13:57:02 +0000
Resent-Message-ID: <handler.68266.B68266.170489496212923 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170489496212923
(code B ref 68266); Wed, 10 Jan 2024 13:57:02 +0000
Received: (at 68266) by debbugs.gnu.org; 10 Jan 2024 13:56:02 +0000
Received: from localhost ([127.0.0.1]:39296 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rNZ3q-0003ML-65
for submit <at> debbugs.gnu.org; Wed, 10 Jan 2024 08:56:02 -0500
Received: from mira.cbaines.net ([212.71.252.8]:43094)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rNZ3n-0003M3-W9
for 68266 <at> debbugs.gnu.org; Wed, 10 Jan 2024 08:56:00 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 26BCD27BBE2;
Wed, 10 Jan 2024 12:38:31 +0000 (GMT)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id 47b23c40;
Wed, 10 Jan 2024 12:38:30 +0000 (UTC)
References: <87plyfrb2x.fsf@HIDDEN>
<13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<87v8833fl7.fsf@HIDDEN> <878r4zr6a0.fsf@HIDDEN>
<87plyaw1bn.fsf@HIDDEN>
User-agent: mu4e 1.10.7; emacs 29.1
From: Christopher Baines <mail@HIDDEN>
Date: Wed, 10 Jan 2024 12:28:31 +0000
In-reply-to: <87plyaw1bn.fsf@HIDDEN>
Message-ID: <87cyu9gy7w.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Christopher Baines <mail@HIDDEN> skribis:
>
>> I believe the reason packages from make-ld-wrapper were showing up
>> multiple times in the cache for me is linked to it's use in the
>> cross-base module, as part of the cross-gcc procedure.
>>
>> A later commit does change cross-gcc to return a single package record
>> for some given arguments, so that probably resolves the biggest misuse
>> of make-ld-wrapper.
>
> Oh, I see.
>
> Before resorting to memoization (which doesn=E2=80=99t come for free), we=
can
> for instance make sure we do not create new package records from inputs
> fields (since they are thunked, a fresh record would be created each
> time the field is accessed). Here=E2=80=99s an example:
>
> diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm
> index 6ee7b315d8..ca3b41a4c7 100644
> --- a/gnu/packages/cross-base.scm
> +++ b/gnu/packages/cross-base.scm
> @@ -306,6 +306,11 @@ (define* (cross-gcc target
> XGCC as the base compiler. Use XBINUTILS as the associated cross-Binuti=
ls.
> If LIBC is false, then build a GCC that does not target a libc; otherwis=
e,
> target that libc."
> + (define ld-wrapper
> + (make-ld-wrapper (string-append "ld-wrapper-" target)
> + #:target (const target)
> + #:binutils xbinutils))
> +
> (package
> (inherit xgcc)
> (name (string-append "gcc-cross-"
> @@ -313,8 +318,7 @@ (define* (cross-gcc target
> target))
> (source
> (origin
> - (inherit
> - (package-source xgcc))
> + (inherit (package-source xgcc))
> (patches
> (append
> (origin-patches (package-source xgcc))
> @@ -353,10 +357,7 @@ (define* (cross-gcc target
> ,@(cross-gcc-arguments target xgcc libc)))
>=20=20
> (native-inputs
> - `(("ld-wrapper-cross" ,(make-ld-wrapper
> - (string-append "ld-wrapper-" target)
> - #:target (const target)
> - #:binutils xbinutils))
> + `(("ld-wrapper-cross" ,ld-wrapper)
> ("binutils-cross" ,xbinutils)
>=20=20
> ,@(let ((inputs (append (package-inputs xgcc)
>
>
> This particular one doesn=E2=80=99t have a huge impact on simple cases bu=
t it
> might pay off for bigger graphs:
>
> $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target=
=3Daarch64-linux-gnu --no-grafts -d
> /gnu/store/i8x1gg73x3fvxn4lrrgbcb97sz4qq1yx-hello-2.12.1.drv
> Object Cache:
> fresh caches: 20
> lookups: 5514
> hits: 5008 (90.8%)
> cache size: 505 entries
> $ # before:
> $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target=
=3Daarch64-linux-gnu --no-grafts -d
> /gnu/store/kdnh9bwg874aq8bvvw9y2vkl7z94icqa-hello-2.12.1.drv
> Object Cache:
> fresh caches: 20
> lookups: 5406
> hits: 4907 (90.8%)
> cache size: 498 entries
>
>> I think there's other cases (in the llvm and mold modules) where it
>> looks like it's called multiple times with the same arguments, so maybe
>> that's an argument for having memoization around make-ld-wrapper even
>> though it's not needed for all uses.
>
> On guix-devel, Efraim mentioned cargo-build-system packages. Is that
> related?
Looking at the various changes, while rust-sysroot doesn't appear that
many times in the cache, I think it's responsible for most of the
slowness. Maybe that's due in part to what you say above, using the
cross- procedures in the native-inputs field.
Maybe the transformation you suggest above could be applied, but I'd be
worried about how that would work if you set one
%current-system/%current-target-system when you call the procedure, then
another when you compute the derivation.
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWej8NfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xc/1g/7B3RzVXQfEw1WTkDK9Ia3PJfA8EhN4cRr
HVwtVs2SFCjffaM5HTa3LyGzoZnrFV9yItJ1m+2SBZOPdAi26euXnBTEnA72RELg
MB2KK6bpdTZ42iV7ZxWT/RGDrRXhSd0YvzbtpRtMbfdk8wFXBpN7886ubFiOQemX
WDlL7loxI7Ncfqv3XJim+DMkBeeCS3dLiVHGRnBBuWMl8X93atOY4y6/kys/rebc
/yO2cGwwOjZrjVPddFjs0FsAQFov00ESz9f387n5YPU2aEsRSqqYAcyu16oMg+4B
gx029IIIbtFiwCyja5qbpfGuXTIKXNsvxZR9dvV4E2BAllJvHbRaarI+ENzvAdD4
0fddw8KVV5+D2MPUzC+O7INcF+b4yO3IR52Bo9OjyvCCgkWDCc0QBVYXkgCYr3ZN
dOnQqxNoA/oVt3SzTIi1QjL0i57aLjrEXdbDs0goWmxoe7pOEAkUVFfcqCqjfGHg
taawfZhFI+PgosLgnye+Tfe/tRqpfSI+ODPjS3CLfT4mpekOlbzHf91AF6S2oo/L
dDbtLRbNmeobKk+2tIUS99O8GPYoRyQKhTx1EE0sz2meEuohaHCmdWi5pqbcYMlR
aeASq/syYUso3XJdpEK+clYPcpts7mDYMxYHMeQs2kfHFOyI2eB6VMBmfz+wFMDq
PAShmn5JWaU=
=7avI
-----END PGP SIGNATURE-----
--=-=-=--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication.
References: <87plyfrb2x.fsf@HIDDEN>
In-Reply-To: <87plyfrb2x.fsf@HIDDEN>
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN
Resent-Date: Wed, 10 Jan 2024 14:17:01 +0000
Resent-Message-ID: <handler.68266.B68266.17048961631254 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 68266 <at> debbugs.gnu.org
Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
X-Debbugs-Original-Xcc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.17048961631254
(code B ref 68266); Wed, 10 Jan 2024 14:17:01 +0000
Received: (at 68266) by debbugs.gnu.org; 10 Jan 2024 14:16:03 +0000
Received: from localhost ([127.0.0.1]:39381 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rNZNC-0000JT-KS
for submit <at> debbugs.gnu.org; Wed, 10 Jan 2024 09:16:03 -0500
Received: from mira.cbaines.net ([212.71.252.8]:43096)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rNZNA-0000EG-Bt
for 68266 <at> debbugs.gnu.org; Wed, 10 Jan 2024 09:16:00 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id ADD2F27BBE9
for <68266 <at> debbugs.gnu.org>; Wed, 10 Jan 2024 12:57:24 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id 32e0a20a
for <68266 <at> debbugs.gnu.org>; Wed, 10 Jan 2024 12:57:24 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
Date: Wed, 10 Jan 2024 12:57:23 +0000
Message-ID: <89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
This is intended to help with spotting duplication in the object cache, so
where many keys, for example package records map to the same derivation. This
represents an opportunity for improved performance if you can reduce this
duplication in the cache, and better take advantage of the already present
cache entries.
I'm thinking this can be used by the data service, but maybe it could also be
worked in to guix commands.
* guix/store.scm (report-object-cache-duplication): New procedure.
Change-Id: Ia6c816f871d10cae6807543224250110099d764f
---
guix/store.scm | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 59 insertions(+)
diff --git a/guix/store.scm b/guix/store.scm
index 97c4f32a5b..86ca293cac 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -70,6 +70,7 @@ (define-module (guix store)
current-store-protocol-version ;for internal use
cache-lookup-recorder ;for internal use
mcached
+ report-object-cache-duplication
&store-error store-error?
&store-connection-error store-connection-error?
@@ -2037,6 +2038,64 @@ (define-syntax mcached
((_ mvalue object keys ...)
(mcached eq? mvalue object keys ...))))
+(define* (report-object-cache-duplication store #:key (threshold 10)
+ (port (current-error-port)))
+ (define cache-values-to-keys
+ (make-hash-table))
+
+ (define (insert key val)
+ (hash-set!
+ cache-values-to-keys
+ key
+ (or (and=> (hash-ref cache-values-to-keys
+ key)
+ (lambda (existing-values)
+ (cons val existing-values)))
+ (list val))))
+
+ (let* ((cache-size
+ (vhash-fold
+ (lambda (key value result)
+ (match value
+ ((item . keys*)
+ (insert item key)))
+
+ (+ 1 result))
+ 0
+ (store-connection-cache store %object-cache-id)))
+ (cached-values-by-key-count
+ (sort
+ (hash-map->list
+ (lambda (key value)
+ (cons key (length value)))
+ cache-values-to-keys)
+ (lambda (a b)
+ (< (cdr a) (cdr b))))))
+
+ (filter-map
+ (match-lambda
+ ((value . count)
+ (if (>= count threshold)
+ (begin
+ (when port
+ (simple-format port "value ~A cached ~A times\n" value count)
+ (simple-format port "example keys:\n"))
+
+ (let ((keys (hash-ref cache-values-to-keys value)))
+ (when port
+ (for-each
+ (lambda (key)
+ (simple-format #t " - ~A\n" key))
+ (if (> count 10)
+ (take keys 10)
+ keys))
+ (newline port))
+
+ `((value . ,value)
+ (keys . ,keys))))
+ #f)))
+ cached-values-by-key-count)))
+
(define (preserve-documentation original proc)
"Return PROC with documentation taken from ORIGINAL."
(set-object-property! proc 'documentation
base-commit: e541f9593f8bfc84b6140c2408b393243289fae6
--
2.41.0
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 12 Jan 2024 14:14:02 +0000
Resent-Message-ID: <handler.68266.B68266.170506880411593 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Christopher Baines <mail@HIDDEN>
Cc: Efraim Flashner <efraim@HIDDEN>, 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170506880411593
(code B ref 68266); Fri, 12 Jan 2024 14:14:02 +0000
Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 14:13:24 +0000
Received: from localhost ([127.0.0.1]:35248 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rOIHj-00030u-PV
for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:13:24 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:42824)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rOIHg-00030I-Mw
for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:13:21 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1rOIHc-00038Q-7u; Fri, 12 Jan 2024 09:13:16 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=SxuXzKc5P8PsutD3J3SVD96Wo9cxqDPf4RTO1OOEG6U=; b=SAFb1+cTeU8K+Ylp2gRP
OEjFOg+Urz1fhe2KUK48YOt4Yu6bTyGNUm4+kY54pwtwNBovQta+uWvKXPMD0FM9ahDNZqnHL8/pp
hVzUFRloMAybd40Xutbh4cfKkOFGxySBMbChzsmVjVKskA+IqbkLdKidjmaJ/BdTEsYZ1J/Ak8fpz
2NwIb30X/orFyrbU0hj+xB0IDtG7W3sVDhJbLzJfpa8dFNuL9vKyUR9uc0MKrERmolfp9RfWq6ZP5
L5bcXzfYUuvL+5bSuVvCedZVLeYbSX1sld9QAIz9T9CnaLUbY7OlArOW8a1ArduggGXTC1quioBwa
Z6YOtZ6DYbS/FQ==;
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN>
(Christopher Baines's message of "Fri, 5 Jan 2024 16:40:49 +0000")
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN>
Date: Fri, 12 Jan 2024 15:13:12 +0100
Message-ID: <878r4uk5c7.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,
Christopher Baines <mail@HIDDEN> skribis:
> To ensure that it just returns a single package record for some given
> arguments, as this helps to avoid poor performance of the store connection
> object cache.
>
> * gnu/packages/rust.scm (make-rust-sysroot): Move code to
> make-rust-sysroot/implementation.
> (make-rust-sysroot/implementation): New variable.
>
> Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167
[...]
> (define*-public (make-rust-sysroot target)
> - (let ((base-rust rust))
> + (make-rust-sysroot/implementation target rust))
> +
> +(define make-rust-sysroot/implementation
> + (mlambda (target base-rust)
> (package
> (inherit base-rust)
> (name (string-append "rust-sysroot-for-" target))
We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2=80=
=99) with packages as it leads
to deep object comparisons. That=E2=80=99s why for packages we typically
always have one-argument (mlambdaq (package) =E2=80=A6).
But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before, le=
t=E2=80=99s keep it simple
(=E2=80=98diff --ignore-space-change=E2=80=99):
--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 2b86cf3c8b..983a3f0f41 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -62,6 +62,7 @@ (define-module (gnu packages rust)
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix memoization)
#:use-module (guix packages)
#:use-module ((guix build utils) #:select (alist-replace))
#:use-module (guix utils)
@@ -1060,7 +1061,8 @@ (define-public rust
`("procps" ,procps)
(package-native-inputs base-rust))))))
-(define*-public (make-rust-sysroot target)
+(define-public make-rust-sysroot
+ (mlambda (target)
(let ((base-rust rust))
(package
(inherit base-rust)
@@ -1228,7 +1230,7 @@ (define*-public (make-rust-sysroot target)
(cross-binutils target)))))
(properties
`((hidden? . #t)
- ,(package-properties base-rust))))))
+ ,(package-properties base-rust)))))))
(define-public rust-analyzer
(package
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
It should help when cross-compiling several =E2=80=98cargo-build-system=E2=
=80=99 packages:
--8<---------------cut here---------------start------------->8---
$ time GUIX_PROFILING=3Dobject-cache guix build --no-grafts greetd wlgreet =
du-dust circtools --target=3Daarch64-linux-gnu -d # before
/gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv
/gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv
/gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv
/gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv
Object Cache:
fresh caches: 22
lookups: 40435
hits: 36821 (91.1%)
cache size: 3613 entries
real 0m2.964s
user 0m2.925s
sys 0m0.174s
$ time GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build --no-grafts =
greetd wlgreet du-dust circtools --target=3Daarch64-linux-gnu -d # after
/gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv
/gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv
/gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv
/gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv
Object Cache:
fresh caches: 22
lookups: 37942
hits: 34523 (91.0%)
cache size: 3418 entries
real 0m2.980s
user 0m3.300s
sys 0m0.160s
--8<---------------cut here---------------end--------------->8---
(Here we removed ~2.5k nodes from the cache.)
WDYT?
Ludo=E2=80=99.
--=-=-=--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 12 Jan 2024 14:23:01 +0000
Resent-Message-ID: <handler.68266.B68266.170506936022833 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Christopher Baines <mail@HIDDEN>
Cc: Josselin Poiret <dev@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, 68266 <at> debbugs.gnu.org, Ricardo Wurmus <rekado@HIDDEN>, Christopher Baines <guix@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170506936022833
(code B ref 68266); Fri, 12 Jan 2024 14:23:01 +0000
Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 14:22:40 +0000
Received: from localhost ([127.0.0.1]:35260 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rOIQh-0005wC-GQ
for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:22:39 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:39110)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rOIQe-0005vz-RB
for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:22:38 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1rOIQa-0007Lu-Dm; Fri, 12 Jan 2024 09:22:32 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=JNydbxIjPmQqI3QLFUAB6uyyUwBUF7mM/yQwyX9GPdA=; b=ZJ+TyfisP9vav6sXWc7F
ex67mHDaA8GYKqDiiRH3T0HDaR/fnfU7S7cpgJb0Psi3Hx6CPlRYRwuIR1DPuzqvpINlJBNeTsdrU
bW2HgLKgJZXHP7rdLsqRCZQdpecHZQS0HJ960LhLkPBo3xOnFVU44yFpk/ZH8SVPhTXnMgc5rOVV9
A1snS3LMSTrPYzfyVzaEQI5M6Cr1RttrbKp0cr0HXYJ4yuElohhLysRhOjWjpGrhKiDpmKEmOAHl2
KQWou8xdDAH3tim0V59d550nChU8Lgsdbw0GVjdwg8O3PHdD+IM3tDiCiZcFCo8xj7F1Bi5kYQAKq
kg3V8MqYAxI44Q==;
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN>
(Christopher Baines's message of "Wed, 10 Jan 2024 12:57:23 +0000")
References: <87plyfrb2x.fsf@HIDDEN>
<89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN>
Date: Fri, 12 Jan 2024 15:22:07 +0100
Message-ID: <871qamk4xc.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Christopher Baines <mail@HIDDEN> skribis:
> This is intended to help with spotting duplication in the object cache, so
> where many keys, for example package records map to the same derivation. =
This
> represents an opportunity for improved performance if you can reduce this
> duplication in the cache, and better take advantage of the already present
> cache entries.
Another way to detect this is by looking at =E2=80=98add-data-to-store-cach=
e=E2=80=99
stats:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" guix build --no-g=
rafts greetd wlgreet du-dust circtools --target=3Daarch64-linux-gnu -d=20
/gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv
/gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv
/gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv
/gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv
Object Cache:
fresh caches: 22
lookups: 40435
hits: 36821 (91.1%)
cache size: 3613 entries
'add-data-to-store' cache:
lookups: 4090
hits: 958 (23.4%)
.drv files: 3062 (74.9%)
Scheme files: 916 (22.4%)
$ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" ./pre-inst-env gu=
ix build --no-grafts greetd wlgreet du-dust circtools --target=3Daarch64-li=
nux-gnu -d=20
/gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv
/gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv
/gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv
/gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv
Object Cache:
fresh caches: 22
lookups: 37942
hits: 34523 (91.0%)
cache size: 3418 entries
'add-data-to-store' cache:
lookups: 3895
hits: 763 (19.6%)
.drv files: 2957 (75.9%)
Scheme files: 826 (21.2%)
--8<---------------cut here---------------end--------------->8---
Ideally, the hit rate there would be 0% and we could remove it.
If there=E2=80=99s a positive hit rate, it means we keep adding the same .d=
rv
and/or *-builder files to the store, meaning that the object cache was
ineffective.
> +(define* (report-object-cache-duplication store #:key (threshold 10)
> + (port (current-error-port)))
Do you have an example output of this?
How helpful does it look to you in practice?
I guess it we=E2=80=99d need to add a =E2=80=98register-profiling-hook!=E2=
=80=99 call somewhere?
If we were to add this, I=E2=80=99d suggest adding a docstring and followin=
g the
same style as the rest of the code (no long identifiers for local
variables and no car/cdr, for example).
Thanks,
Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 12 Jan 2024 18:05:01 +0000
Resent-Message-ID: <handler.68266.B68266.170508266210509 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: Efraim Flashner <efraim@HIDDEN>, 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170508266210509
(code B ref 68266); Fri, 12 Jan 2024 18:05:01 +0000
Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 18:04:22 +0000
Received: from localhost ([127.0.0.1]:37689 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rOLtG-0002jR-Dd
for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:04:22 -0500
Received: from mira.cbaines.net ([212.71.252.8]:43098)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rOLtE-0002jF-MQ
for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:04:21 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 3F89027BBE2;
Fri, 12 Jan 2024 18:04:21 +0000 (GMT)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id a7995c2e;
Fri, 12 Jan 2024 18:04:20 +0000 (UTC)
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN>
<878r4uk5c7.fsf@HIDDEN>
User-agent: mu4e 1.10.7; emacs 29.1
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 12 Jan 2024 17:57:26 +0000
In-reply-to: <878r4uk5c7.fsf@HIDDEN>
Message-ID: <87zfxafmxq.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Christopher Baines <mail@HIDDEN> skribis:
>
>> To ensure that it just returns a single package record for some given
>> arguments, as this helps to avoid poor performance of the store connecti=
on
>> object cache.
>>
>> * gnu/packages/rust.scm (make-rust-sysroot): Move code to
>> make-rust-sysroot/implementation.
>> (make-rust-sysroot/implementation): New variable.
>>
>> Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167
>
> [...]
>
>> (define*-public (make-rust-sysroot target)
>> - (let ((base-rust rust))
>> + (make-rust-sysroot/implementation target rust))
>> +
>> +(define make-rust-sysroot/implementation
>> + (mlambda (target base-rust)
>> (package
>> (inherit base-rust)
>> (name (string-append "rust-sysroot-for-" target))
>
> We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2=80=
=99) with packages as it leads
> to deep object comparisons. That=E2=80=99s why for packages we typically
> always have one-argument (mlambdaq (package) =E2=80=A6).
>
> But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before, =
let=E2=80=99s keep it simple
> (=E2=80=98diff --ignore-space-change=E2=80=99):
...
> WDYT?
Yeah, that does look good. I pushed my earlier version of this patch
this morning though.
I did have a look at trying to adapt the changes to fit in (guix
build-system cargo) instead, as I noticed that seemed to be a pattern
elsewhere, but I think there's something weird going on with the use of
make-rust-sysroot there since default-rust-sysroot takes an argument,
but doesn't use it. Maybe once that's figured out, we can move the
memoization there and switch to just using the target as the key.
Unfortunately I'm still waiting to see what effect this has on the data
service processing revisions. I'm pretty sure it's going to help, but
I'm concerned it's not going to help enough to make processing revisions
for patches feasible again.
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWhfyFfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xevjw/9GKpOuTwH7hn30C7cFq+lIQqt7Qt5bE9U
DFhXIUrzLc+qJNtUZaxuUsPQwks+F2ilKHR7vdBt7W6MKcYjYQTsaWgen46s4q7i
Gcb/6pe0HhDDRO/HkZedauqfOFupociG2EsR1v2XgpiWS8zg/8O3g99DN7QjjVg6
hTGN/6MkjI0luREESJ5xu0+n2K0RqtUxkR9yiqRvv+3RFCBgXr0EZAX2B4ytjEWs
GA0/LjS78KPO+bKqtZfmoZMwI3fE+2QsjgLhOBWlrGr96pJVRy9DovDJT2OwUbmT
wjviXLLQpMTcvzxkTRYDTSU5JbTlmdRxc2AQrDNWz5hT4XBnLGgSnme10rXeMgPO
iSN8oxBW6nlFhADhmY+JoRehQmyGaoKy7aL6bVaG87fRoeivPz4iCDZygi0SmwIF
qxaPc6IfySAcIuagBI3gXojaLBI7CA+f7iW6VSDZ+k9vPFPixUay5OaeHXh6A5iJ
uRLEA7xggcnK+YXTJ0jiKjEuDdEVPKNQdDz2zF9kODzxJm3ocdsOnkFe7q94jEpn
zgC99kYZPu+XF73eWIzhsVsvA98BaJlKThSHKTC9odqE6fzG8xfDDJ3JzsCOyb3h
Hzcs2PpD++9pKMbG1juwh8yPndKWEWwL68Og9vc/3CxkVAnJtFfkBtnommiSCqx+
rNxZEHNEWPA=
=NTdw
-----END PGP SIGNATURE-----
--=-=-=--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication.
Resent-From: Christopher Baines <mail@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 12 Jan 2024 18:56:02 +0000
Resent-Message-ID: <handler.68266.B68266.170508572114989 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: Josselin Poiret <dev@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, 68266 <at> debbugs.gnu.org, Ricardo Wurmus <rekado@HIDDEN>, Christopher Baines <guix@HIDDEN>
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170508572114989
(code B ref 68266); Fri, 12 Jan 2024 18:56:02 +0000
Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 18:55:21 +0000
Received: from localhost ([127.0.0.1]:37770 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rOMga-0003th-Qy
for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:55:21 -0500
Received: from mira.cbaines.net
([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:51301)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1rOMgY-0003tY-HE
for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:55:19 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id 5F5A827BBE2;
Fri, 12 Jan 2024 18:55:19 +0000 (GMT)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id 61a53281;
Fri, 12 Jan 2024 18:55:18 +0000 (UTC)
References: <87plyfrb2x.fsf@HIDDEN>
<89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN>
<871qamk4xc.fsf@HIDDEN>
User-agent: mu4e 1.10.7; emacs 29.1
From: Christopher Baines <mail@HIDDEN>
Date: Fri, 12 Jan 2024 18:26:24 +0000
In-reply-to: <871qamk4xc.fsf@HIDDEN>
Message-ID: <87v87yfkkr.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Christopher Baines <mail@HIDDEN> skribis:
>
>> This is intended to help with spotting duplication in the object cache, =
so
>> where many keys, for example package records map to the same derivation.=
This
>> represents an opportunity for improved performance if you can reduce this
>> duplication in the cache, and better take advantage of the already prese=
nt
>> cache entries.
>
> Another way to detect this is by looking at =E2=80=98add-data-to-store-ca=
che=E2=80=99
> stats:
>
> $ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" guix build --no=
-grafts greetd wlgreet du-dust circtools --target=3Daarch64-linux-gnu -d=20
> /gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv
> /gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv
> /gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv
> /gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv
> Object Cache:
> fresh caches: 22
> lookups: 40435
> hits: 36821 (91.1%)
> cache size: 3613 entries
>
> 'add-data-to-store' cache:
> lookups: 4090
> hits: 958 (23.4%)
> .drv files: 3062 (74.9%)
> Scheme files: 916 (22.4%)
> $ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" ./pre-inst-env =
guix build --no-grafts greetd wlgreet du-dust circtools --target=3Daarch64-=
linux-gnu -d=20
> /gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv
> /gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv
> /gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv
> /gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv
> Object Cache:
> fresh caches: 22
> lookups: 37942
> hits: 34523 (91.0%)
> cache size: 3418 entries
>
> 'add-data-to-store' cache:
> lookups: 3895
> hits: 763 (19.6%)
> .drv files: 2957 (75.9%)
> Scheme files: 826 (21.2%)
>
> Ideally, the hit rate there would be 0% and we could remove it.
>
> If there=E2=80=99s a positive hit rate, it means we keep adding the same =
.drv
> and/or *-builder files to the store, meaning that the object cache was
> ineffective.
>
>> +(define* (report-object-cache-duplication store #:key (threshold 10)
>> + (port (current-error-port)))
>
> Do you have an example output of this?
>
> How helpful does it look to you in practice?
Yep, so here's some output I get from computing all the cross
derivations to i586-pc-gnu:
value #<derivation
/gnu/store/lqxlksh00cshc816xqfq47r3jjdfj2p9-subversion-1.14.2.drv =3D>
/gnu/store/92avphfdcrcaxx8m5a6ihmw558bj3np8-subversion-1.14.2 7fcfbe5f9=
280>
cached 4174 times
example keys:
- #<package subversion@HIDDEN gnu/packages/version-control.scm:2316 7fc=
fc1c924d0>
- #<package subversion@HIDDEN gnu/packages/version-control.scm:2316 7fc=
fc1c924d0>
- #<file-append #<package subversion@HIDDEN gnu/packages/version-contro=
l.scm:2316 7fcfc1c924d0> "/bin/svn">
- #<file-append #<package subversion@HIDDEN gnu/packages/version-contro=
l.scm:2316 7fcfc1c924d0> "/bin/svn">
So it's not immediately obvious what the issue is, but if you search for
"/bin/svn", then you find the file-append calls in svn-fetch, so there's
a couple of new file-append records added to the cache for each
svn-fetch.
value /gnu/store/qi1km3qlv5hdsql6h3ibwvz6v4z8lqbz-ld-wrapper.in cached 75=
5 times
example keys:
- #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-=
wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package=
s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure=
true (file stat)>>
- #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-=
wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package=
s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure=
true (file stat)>>
- #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-=
wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package=
s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure=
true (file stat)>>
- #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-=
wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package=
s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure=
true (file stat)>>
Similar to file-append, it's not immediately obvious where all these
local-file's are coming from, but searching for ld-wrapper.in suggests
make-ld-wrapper.
My thinking here is that maybe it's worth not caching everything in the
object cache. Particularly for file-append, I'm not sure what the cache
is actually doing. As for local-file, given there's lower level caching,
maybe it's worth leaning on that and not caching local-file either. I
tried this out, and it seemed to not make performance worse at least,
and it did remove the duplication from the cache.
Other things that show up include:
value #<derivation /gnu/store/z28jfsm43wg50cvdfq0548pbf5jfhk8r-binutils-c=
ross-i586-pc-gnu-2.38.drv =3D> /gnu/store/lkmwq399jllig2a5r323v6y9nflpn6gn-=
binutils-cross-i586-pc-gnu-2.38 7fcfbd358000> cached 1228 times
example keys:
- #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm=
:79 7fcfbf3bb8f0>
- #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm=
:79 7fcfbf3b6630>
- #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm=
:79 7fcfbf3b6420>
- #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm=
:79 7fcfbf3b62c0>
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWhixRfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XcrGxAAr+CZGwnbA4IEkRH6kmHCBa1UYeT5mVat
FvOKIINgMqv7es0DMBEPy1g9Yi223BpBEHlqSloE7HUANk3gq/MASNJJd/Aeh6tR
UjRzzWlG9GfuS3QH8u2gmMSPRElCsd6v9OY9WhaFQvw4Zq/EO45jmEw/QbvMzrHW
3K3w/nG0ss+tT1dYB8U+VWgFpzxg+XlAit5QVR1vW0B0r8zuAJ8SLe11iikx3ZmO
X7m/kf6uC4WQU0ptIPVu84oxAjDckmuPMahA4pw+QIY2Ne2ud/fL3rcbUA7d91Ha
3sbdcfO1wClQNJLCsROEEiYUW0+hrfTHZJhlKGHdm4rAP9k6yx4cdiVhNRhu7fwE
HoDmnqreW4Hji4Tgbi+hLu9Bv99l2Y0HJ9KdjKD0WmOzuWkNhhP4WrlOqEW1NWHm
p72CI4zjMXzCOFosdWqCgTw0nuDoXdb4AMbKgegCbddPCizXcnH5n9upRBfhNSA1
VCl1wr/ZhLtT8JEAJp34YXUESWZSKne+pzVHW8F1oHG9H/Uogx5FP2RvSi4OYjhp
3RrzBdIjyMKfQmlp3zQmrq5OrEXXv4WGflKlrZubRCgV4Z1R4O4eK0Q/TDNhbLmY
nvxz9tboQUW1xJAvDbXV/BKZgEf6AjdKEy/Qk3nMUXSx9TFb5BnQ9vBed1B5WTld
h+C2I0Q5Qhk=
=pIg/
-----END PGP SIGNATURE-----
--=-=-=--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results.
Resent-From: Efraim Flashner <efraim@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sat, 13 Jan 2024 16:17:02 +0000
Resent-Message-ID: <handler.68266.B68266.170516256228455 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Christopher Baines <mail@HIDDEN>
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.170516256228455
(code B ref 68266); Sat, 13 Jan 2024 16:17:02 +0000
Received: (at 68266) by debbugs.gnu.org; 13 Jan 2024 16:16:02 +0000
Received: from localhost ([127.0.0.1]:40884 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rOgfx-0007Nx-6L
for submit <at> debbugs.gnu.org; Sat, 13 Jan 2024 11:16:02 -0500
Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:55430)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <efraim.flashner@HIDDEN>) id 1rOgft-0007AQ-Vu
for 68266 <at> debbugs.gnu.org; Sat, 13 Jan 2024 11:15:59 -0500
Received: by mail-wm1-x330.google.com with SMTP id
5b1f17b1804b1-40e68d836bcso11810215e9.2
for <68266 <at> debbugs.gnu.org>; Sat, 13 Jan 2024 08:15:59 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1705162553; x=1705767353; darn=debbugs.gnu.org;
h=in-reply-to:content-disposition:mime-version:references
:mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to
:cc:subject:date:message-id:reply-to;
bh=BSyg6JIW/yI/NlLj1Be8R8KGKgFRZNZJfBSmLfJkVg0=;
b=D3u4xj2a+q2AvHm6m7TMWk60jXcmKMsZgqy528JDpMhRdh+r+/qq5OFJO3lfjbDV23
nFfahQNcUkCIWD4QwKrA5gpNpVX1tMtc5n6/kBV+NGKJI18+DWg2B6Yrb0hOYR1+mgto
Co70iSn4N/rwMaLKP55zlRANVm/hexF0mCHSiAI3uGDrTXhaZodwsqTdLbvcCLFYPilG
kZ18L+oJF/y6xKJxdccx1vXJcaP4MvVYHb+YR7vKv8Kq+sJyku3tsJiom/DUDNZJ6E4a
FjxkACvgmEEckxog2m9bQVrHBJ8njJspMfSRiaoIYvpcEUzMHK4+eT7HUZ08w+bBN9k0
Oftg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1705162553; x=1705767353;
h=in-reply-to:content-disposition:mime-version:references
:mail-followup-to:message-id:subject:cc:to:from:date:sender
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=BSyg6JIW/yI/NlLj1Be8R8KGKgFRZNZJfBSmLfJkVg0=;
b=hK1Qnhna/uwH+IKByAC2I9UIyZjw0DXPLSS6eN9+zM0dq1c2iKC91ZBMgh4/sOmldk
zYYcp8TFTVb8B6YzrEPhuKcZAfQc0zgKliHhtg2P95tT++RvNtnd2AqU7DmlFAQZzLQY
oTEvSGQfXAyc0Y+kiIQFqOsR7GfAqXMnDET5jjobRGVgmT+IjtYQQ+wudbDJxZeXf8Il
O4nVWaRwu8UnzuUU7pCPgkv4KIkR7soDtzWXPK+V3AXtGaV0xH/ztrGu9T5cC0szV7ST
mC7da9A3p+zBoAaEYdJjwtLV1807hxgVhj+DAWT6tIXyJREY4tdyv80pq5Dv8LUy+MCf
7ebA==
X-Gm-Message-State: AOJu0Yz+t025sFZSz5rsS0Tl/GHPpvo+4SYVQiRkoABmHbEjeTmIr1E9
I1eO8/W8ah45o1XETKOWMBk=
X-Google-Smtp-Source: AGHT+IHEhx7gPMhe/DewVPJeVymTbPqd7VAX/XGsqxogcw7BY68UEUWUgwZMxs+ZotgEXPyPNYiVtw==
X-Received: by 2002:a1c:7c0f:0:b0:40c:6e8:610a with SMTP id
x15-20020a1c7c0f000000b0040c06e8610amr1616592wmc.56.1705162552747;
Sat, 13 Jan 2024 08:15:52 -0800 (PST)
Received: from localhost ([141.226.15.142]) by smtp.gmail.com with ESMTPSA id
j8-20020a05600c190800b0040d7b340e07sm9794239wmq.45.2024.01.13.08.15.50
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Sat, 13 Jan 2024 08:15:51 -0800 (PST)
Date: Sat, 13 Jan 2024 18:15:49 +0200
From: Efraim Flashner <efraim@HIDDEN>
Message-ID: <ZaK3Ncr9FRtUGxNE@3900XT>
Mail-Followup-To: Efraim Flashner <efraim@HIDDEN>,
Christopher Baines <mail@HIDDEN>,
Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>,
68266 <at> debbugs.gnu.org
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN>
<878r4uk5c7.fsf@HIDDEN> <87zfxafmxq.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="BE+dJDenWrZcLOkR"
Content-Disposition: inline
In-Reply-To: <87zfxafmxq.fsf@HIDDEN>
X-PGP-Key-ID: 0x41AAE7DCCA3D8351
X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc
X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
X-Spam-Score: 0.5 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)
--BE+dJDenWrZcLOkR
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Fri, Jan 12, 2024 at 05:57:26PM +0000, Christopher Baines wrote:
>=20
> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>=20
> > Christopher Baines <mail@HIDDEN> skribis:
> >
> >> To ensure that it just returns a single package record for some given
> >> arguments, as this helps to avoid poor performance of the store connec=
tion
> >> object cache.
> >>
> >> * gnu/packages/rust.scm (make-rust-sysroot): Move code to
> >> make-rust-sysroot/implementation.
> >> (make-rust-sysroot/implementation): New variable.
> >>
> >> Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167
> >
> > [...]
> >
> >> (define*-public (make-rust-sysroot target)
> >> - (let ((base-rust rust))
> >> + (make-rust-sysroot/implementation target rust))
> >> +
> >> +(define make-rust-sysroot/implementation
> >> + (mlambda (target base-rust)
> >> (package
> >> (inherit base-rust)
> >> (name (string-append "rust-sysroot-for-" target))
> >
> > We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2=
=80=99) with packages as it leads
> > to deep object comparisons. That=E2=80=99s why for packages we typical=
ly
> > always have one-argument (mlambdaq (package) =E2=80=A6).
> >
> > But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before=
, let=E2=80=99s keep it simple
> > (=E2=80=98diff --ignore-space-change=E2=80=99):
>=20
> ...
>=20
> > WDYT?
>=20
> Yeah, that does look good. I pushed my earlier version of this patch
> this morning though.
>=20
> I did have a look at trying to adapt the changes to fit in (guix
> build-system cargo) instead, as I noticed that seemed to be a pattern
> elsewhere, but I think there's something weird going on with the use of
> make-rust-sysroot there since default-rust-sysroot takes an argument,
> but doesn't use it. Maybe once that's figured out, we can move the
> memoization there and switch to just using the target as the key.
>=20
> Unfortunately I'm still waiting to see what effect this has on the data
> service processing revisions. I'm pretty sure it's going to help, but
> I'm concerned it's not going to help enough to make processing revisions
> for patches feasible again.
I looked at the build system a bit and I think it was a combination of
cargo-culting the other cross build implementations that took a target
argument for the cross-compilers and I figured that rust-sysroot would
also need one. The other bit was I think I had in mind the possibility
of choosing seemingly arbitrary targets which were supported by rust but
not known to Guix and having it possible to cross-compile to those.
Given that rust, like go, IIRC doesn't actually need a cross-compiled
compiler to build cross-compiled packages, could it be that parts of
that logic can be rewritten/simplified?
--=20
Efraim Flashner <efraim@HIDDEN> =D7=A8=D7=A0=D7=A9=D7=9C=D7=A4 =
=D7=9D=D7=99=D7=A8=D7=A4=D7=90
GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
--BE+dJDenWrZcLOkR
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmWitzQACgkQQarn3Mo9
g1FMIQ/+NzP0ymT2u5O3czdn7MfaPqy2QawC2fb4SwAr2agAdeViu+BvcmpZ+uqM
n57fviPK0YiLfReQdtHT40GtUCJkXtSZgtViJrwXUKprqe9uWQmzNZ3LWygMTfuc
OJTI7Q6ssgVyLRrqPQ7wQd5MlCnIgCC+DPq692Dr41BBxL1bY3SNH7/NWtqCCmR/
rqt+pKjtUGPDunjkDwkEMINHz4O6306LuVvdlMq0m7V9+lHzCuHPW9OVZBIIfa3e
FGhQEMWwWqB1x7Sq29iZfYehIUECD6bcnjX53dhugp8zSCTQPdbVXKsh4x3Uc9Hm
WF/VWkytBJw4SUncemElECWz/zGB5s9AkQJwAjRfs3mPP0QxxmkoTxycC+zMxIGV
RCnDowmI0LS/3p87uuUsgCeMzS+DTsyYKJirfeoivwfUGwutO91W6uoE5RymqICb
3rB3eG/fUDhzFNw3k5Pn2PW3wFJiELMcmRBRUTds6njlRcBkMStu7DmMUTNihz3/
SXJ1zvQHDItWgk1/kh1+RaW2+vBgTCa5c/1HXq9cbFPPUE1kO38EWPBoyPvrUzZP
gTcRDELw7x6ma3cebp4l3ZPLZKY/4QYuW636l9+wa5xYXE3+X9mypwiwdmHoEf8b
CH/5EV7Dk1+Yf8pDLnbltip4Vr90u/mOppjwedxb+oahSrOPxnY=
=LY7H
-----END PGP SIGNATURE-----
--BE+dJDenWrZcLOkR--
X-Loop: help-debbugs@HIDDEN
Subject: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 15 Jan 2024 16:55:03 +0000
Resent-Message-ID: <handler.68266.B68266.17053376744420 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 68266
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Christopher Baines <mail@HIDDEN>
Cc: Efraim Flashner <efraim@HIDDEN>, 68266 <at> debbugs.gnu.org
Received: via spool by 68266-submit <at> debbugs.gnu.org id=B68266.17053376744420
(code B ref 68266); Mon, 15 Jan 2024 16:55:03 +0000
Received: (at 68266) by debbugs.gnu.org; 15 Jan 2024 16:54:34 +0000
Received: from localhost ([127.0.0.1]:46759 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rPQEL-00019E-Vr
for submit <at> debbugs.gnu.org; Mon, 15 Jan 2024 11:54:34 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:36518)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1rPQEJ-000191-OZ
for 68266 <at> debbugs.gnu.org; Mon, 15 Jan 2024 11:54:32 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1rPQED-0000zl-K6; Mon, 15 Jan 2024 11:54:25 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=q7eLEK7fynHI7ZUMTKTx3pwC2B9iOxTSc5QbHLBPuog=; b=jVWlk83FzpNu90i5imEE
HVTwelQ0yNi3eG+SOnWklw/ehwzj7wgegT1WzOfz9bqdnXzy6B0oLaBnP0IpUXUTeAgbLYMQ/vurc
VE7Y61V0jIN8SdseTs5vxDPYsoN+U8MnlqgerxM8Sc3RhmrPsXzaIpmaLUNV8FcdZ5+A9wYwLQmbI
621hcYvGQkGEYbdw3yBAQGl7tcqnR/ix36xNv+wp/T3oFbLqRvcGd2hIgwdByzG1bTTXeVDB1WC+k
UpU1YtypykPwYPmZJy+ZXLvaNFfC46H9lUveC2ozt79YgKpjWYpOU/JhJNZ591OorVKDPfy4TJZkt
pfevdctQXXWhvA==;
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <87zfxafmxq.fsf@HIDDEN> (Christopher Baines's message of
"Fri, 12 Jan 2024 17:57:26 +0000")
References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN>
<b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN>
<878r4uk5c7.fsf@HIDDEN> <87zfxafmxq.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Sextidi 26 =?UTF-8?Q?Niv=C3=B4se?= an 232 de la
=?UTF-8?Q?R=C3=A9volution,?= jour de
=?UTF-8?Q?l'=C3=89tain?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 15 Jan 2024 17:54:22 +0100
Message-ID: <87h6jeilkx.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hello,
Christopher Baines <mail@HIDDEN> skribis:
> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>
>> Christopher Baines <mail@HIDDEN> skribis:
[...]
>>> (define*-public (make-rust-sysroot target)
>>> - (let ((base-rust rust))
>>> + (make-rust-sysroot/implementation target rust))
>>> +
>>> +(define make-rust-sysroot/implementation
>>> + (mlambda (target base-rust)
>>> (package
>>> (inherit base-rust)
>>> (name (string-append "rust-sysroot-for-" target))
>>
>> We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2=
=80=99) with packages as it leads
>> to deep object comparisons. That=E2=80=99s why for packages we typically
>> always have one-argument (mlambdaq (package) =E2=80=A6).
>>
>> But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before,=
let=E2=80=99s keep it simple
>> (=E2=80=98diff --ignore-space-change=E2=80=99):
>
> ...
>
>> WDYT?
>
> Yeah, that does look good. I pushed my earlier version of this patch
> this morning though.
>
> I did have a look at trying to adapt the changes to fit in (guix
> build-system cargo) instead, as I noticed that seemed to be a pattern
> elsewhere, but I think there's something weird going on with the use of
> make-rust-sysroot there since default-rust-sysroot takes an argument,
What matters is that =E2=80=98make-rust-sysroot=E2=80=99 takes a single arg=
ument,
=E2=80=98target=E2=80=99, so we can safely write:
(define make-rust-sysroot
(mlambda (target)
=E2=80=A6))
I think it=E2=80=99s important to not bring a deep <package> comparison bec=
ause
this kind of cost is then hard to pinpoint.
Perhaps you can adjust =E2=80=98make-rust-sysroot=E2=80=99 along these line=
s?
Thanks,
Ludo=E2=80=99.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.