Received: (at 68266) by debbugs.gnu.org; 15 Jan 2024 16:54:34 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 15 11:54:34 2024 Received: from localhost ([127.0.0.1]:46759 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rPQEL-00019E-Vr for submit <at> debbugs.gnu.org; Mon, 15 Jan 2024 11:54:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:36518) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1rPQEJ-000191-OZ for 68266 <at> debbugs.gnu.org; Mon, 15 Jan 2024 11:54:32 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rPQED-0000zl-K6; Mon, 15 Jan 2024 11:54:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=q7eLEK7fynHI7ZUMTKTx3pwC2B9iOxTSc5QbHLBPuog=; b=jVWlk83FzpNu90i5imEE HVTwelQ0yNi3eG+SOnWklw/ehwzj7wgegT1WzOfz9bqdnXzy6B0oLaBnP0IpUXUTeAgbLYMQ/vurc VE7Y61V0jIN8SdseTs5vxDPYsoN+U8MnlqgerxM8Sc3RhmrPsXzaIpmaLUNV8FcdZ5+A9wYwLQmbI 621hcYvGQkGEYbdw3yBAQGl7tcqnR/ix36xNv+wp/T3oFbLqRvcGd2hIgwdByzG1bTTXeVDB1WC+k UpU1YtypykPwYPmZJy+ZXLvaNFfC46H9lUveC2ozt79YgKpjWYpOU/JhJNZ591OorVKDPfy4TJZkt pfevdctQXXWhvA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results. In-Reply-To: <87zfxafmxq.fsf@HIDDEN> (Christopher Baines's message of "Fri, 12 Jan 2024 17:57:26 +0000") References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN> <878r4uk5c7.fsf@HIDDEN> <87zfxafmxq.fsf@HIDDEN> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 26 =?utf-8?Q?Niv=C3=B4se?= an 232 de la =?utf-8?Q?R=C3=A9volution=2C?= jour de =?utf-8?Q?l'=C3=89tain?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 15 Jan 2024 17:54:22 +0100 Message-ID: <87h6jeilkx.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 68266 Cc: Efraim Flashner <efraim@HIDDEN>, 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hello, Christopher Baines <mail@HIDDEN> skribis: > Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > >> Christopher Baines <mail@HIDDEN> skribis: [...] >>> (define*-public (make-rust-sysroot target) >>> - (let ((base-rust rust)) >>> + (make-rust-sysroot/implementation target rust)) >>> + >>> +(define make-rust-sysroot/implementation >>> + (mlambda (target base-rust) >>> (package >>> (inherit base-rust) >>> (name (string-append "rust-sysroot-for-" target)) >> >> We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2= =80=99) with packages as it leads >> to deep object comparisons. That=E2=80=99s why for packages we typically >> always have one-argument (mlambdaq (package) =E2=80=A6). >> >> But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before,= let=E2=80=99s keep it simple >> (=E2=80=98diff --ignore-space-change=E2=80=99): > > ... > >> WDYT? > > Yeah, that does look good. I pushed my earlier version of this patch > this morning though. > > I did have a look at trying to adapt the changes to fit in (guix > build-system cargo) instead, as I noticed that seemed to be a pattern > elsewhere, but I think there's something weird going on with the use of > make-rust-sysroot there since default-rust-sysroot takes an argument, What matters is that =E2=80=98make-rust-sysroot=E2=80=99 takes a single arg= ument, =E2=80=98target=E2=80=99, so we can safely write: (define make-rust-sysroot (mlambda (target) =E2=80=A6)) I think it=E2=80=99s important to not bring a deep <package> comparison bec= ause this kind of cost is then hard to pinpoint. Perhaps you can adjust =E2=80=98make-rust-sysroot=E2=80=99 along these line= s? Thanks, Ludo=E2=80=99.
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 13 Jan 2024 16:16:02 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 13 11:16:02 2024 Received: from localhost ([127.0.0.1]:40884 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOgfx-0007Nx-6L for submit <at> debbugs.gnu.org; Sat, 13 Jan 2024 11:16:02 -0500 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:55430) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <efraim.flashner@HIDDEN>) id 1rOgft-0007AQ-Vu for 68266 <at> debbugs.gnu.org; Sat, 13 Jan 2024 11:15:59 -0500 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-40e68d836bcso11810215e9.2 for <68266 <at> debbugs.gnu.org>; Sat, 13 Jan 2024 08:15:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705162553; x=1705767353; darn=debbugs.gnu.org; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to :cc:subject:date:message-id:reply-to; bh=BSyg6JIW/yI/NlLj1Be8R8KGKgFRZNZJfBSmLfJkVg0=; b=D3u4xj2a+q2AvHm6m7TMWk60jXcmKMsZgqy528JDpMhRdh+r+/qq5OFJO3lfjbDV23 nFfahQNcUkCIWD4QwKrA5gpNpVX1tMtc5n6/kBV+NGKJI18+DWg2B6Yrb0hOYR1+mgto Co70iSn4N/rwMaLKP55zlRANVm/hexF0mCHSiAI3uGDrTXhaZodwsqTdLbvcCLFYPilG kZ18L+oJF/y6xKJxdccx1vXJcaP4MvVYHb+YR7vKv8Kq+sJyku3tsJiom/DUDNZJ6E4a FjxkACvgmEEckxog2m9bQVrHBJ8njJspMfSRiaoIYvpcEUzMHK4+eT7HUZ08w+bBN9k0 Oftg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705162553; x=1705767353; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BSyg6JIW/yI/NlLj1Be8R8KGKgFRZNZJfBSmLfJkVg0=; b=hK1Qnhna/uwH+IKByAC2I9UIyZjw0DXPLSS6eN9+zM0dq1c2iKC91ZBMgh4/sOmldk zYYcp8TFTVb8B6YzrEPhuKcZAfQc0zgKliHhtg2P95tT++RvNtnd2AqU7DmlFAQZzLQY oTEvSGQfXAyc0Y+kiIQFqOsR7GfAqXMnDET5jjobRGVgmT+IjtYQQ+wudbDJxZeXf8Il O4nVWaRwu8UnzuUU7pCPgkv4KIkR7soDtzWXPK+V3AXtGaV0xH/ztrGu9T5cC0szV7ST mC7da9A3p+zBoAaEYdJjwtLV1807hxgVhj+DAWT6tIXyJREY4tdyv80pq5Dv8LUy+MCf 7ebA== X-Gm-Message-State: AOJu0Yz+t025sFZSz5rsS0Tl/GHPpvo+4SYVQiRkoABmHbEjeTmIr1E9 I1eO8/W8ah45o1XETKOWMBk= X-Google-Smtp-Source: AGHT+IHEhx7gPMhe/DewVPJeVymTbPqd7VAX/XGsqxogcw7BY68UEUWUgwZMxs+ZotgEXPyPNYiVtw== X-Received: by 2002:a1c:7c0f:0:b0:40c:6e8:610a with SMTP id x15-20020a1c7c0f000000b0040c06e8610amr1616592wmc.56.1705162552747; Sat, 13 Jan 2024 08:15:52 -0800 (PST) Received: from localhost ([141.226.15.142]) by smtp.gmail.com with ESMTPSA id j8-20020a05600c190800b0040d7b340e07sm9794239wmq.45.2024.01.13.08.15.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 13 Jan 2024 08:15:51 -0800 (PST) Date: Sat, 13 Jan 2024 18:15:49 +0200 From: Efraim Flashner <efraim@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results. Message-ID: <ZaK3Ncr9FRtUGxNE@3900XT> Mail-Followup-To: Efraim Flashner <efraim@HIDDEN>, Christopher Baines <mail@HIDDEN>, Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 68266 <at> debbugs.gnu.org References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN> <878r4uk5c7.fsf@HIDDEN> <87zfxafmxq.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="BE+dJDenWrZcLOkR" Content-Disposition: inline In-Reply-To: <87zfxafmxq.fsf@HIDDEN> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 68266 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.5 (/) --BE+dJDenWrZcLOkR Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 12, 2024 at 05:57:26PM +0000, Christopher Baines wrote: >=20 > Ludovic Court=C3=A8s <ludo@HIDDEN> writes: >=20 > > Christopher Baines <mail@HIDDEN> skribis: > > > >> To ensure that it just returns a single package record for some given > >> arguments, as this helps to avoid poor performance of the store connec= tion > >> object cache. > >> > >> * gnu/packages/rust.scm (make-rust-sysroot): Move code to > >> make-rust-sysroot/implementation. > >> (make-rust-sysroot/implementation): New variable. > >> > >> Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167 > > > > [...] > > > >> (define*-public (make-rust-sysroot target) > >> - (let ((base-rust rust)) > >> + (make-rust-sysroot/implementation target rust)) > >> + > >> +(define make-rust-sysroot/implementation > >> + (mlambda (target base-rust) > >> (package > >> (inherit base-rust) > >> (name (string-append "rust-sysroot-for-" target)) > > > > We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2= =80=99) with packages as it leads > > to deep object comparisons. That=E2=80=99s why for packages we typical= ly > > always have one-argument (mlambdaq (package) =E2=80=A6). > > > > But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before= , let=E2=80=99s keep it simple > > (=E2=80=98diff --ignore-space-change=E2=80=99): >=20 > ... >=20 > > WDYT? >=20 > Yeah, that does look good. I pushed my earlier version of this patch > this morning though. >=20 > I did have a look at trying to adapt the changes to fit in (guix > build-system cargo) instead, as I noticed that seemed to be a pattern > elsewhere, but I think there's something weird going on with the use of > make-rust-sysroot there since default-rust-sysroot takes an argument, > but doesn't use it. Maybe once that's figured out, we can move the > memoization there and switch to just using the target as the key. >=20 > Unfortunately I'm still waiting to see what effect this has on the data > service processing revisions. I'm pretty sure it's going to help, but > I'm concerned it's not going to help enough to make processing revisions > for patches feasible again. I looked at the build system a bit and I think it was a combination of cargo-culting the other cross build implementations that took a target argument for the cross-compilers and I figured that rust-sysroot would also need one. The other bit was I think I had in mind the possibility of choosing seemingly arbitrary targets which were supported by rust but not known to Guix and having it possible to cross-compile to those. Given that rust, like go, IIRC doesn't actually need a cross-compiled compiler to build cross-compiled packages, could it be that parts of that logic can be rewritten/simplified? --=20 Efraim Flashner <efraim@HIDDEN> =D7=A8=D7=A0=D7=A9=D7=9C=D7=A4 = =D7=9D=D7=99=D7=A8=D7=A4=D7=90 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --BE+dJDenWrZcLOkR Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmWitzQACgkQQarn3Mo9 g1FMIQ/+NzP0ymT2u5O3czdn7MfaPqy2QawC2fb4SwAr2agAdeViu+BvcmpZ+uqM n57fviPK0YiLfReQdtHT40GtUCJkXtSZgtViJrwXUKprqe9uWQmzNZ3LWygMTfuc OJTI7Q6ssgVyLRrqPQ7wQd5MlCnIgCC+DPq692Dr41BBxL1bY3SNH7/NWtqCCmR/ rqt+pKjtUGPDunjkDwkEMINHz4O6306LuVvdlMq0m7V9+lHzCuHPW9OVZBIIfa3e FGhQEMWwWqB1x7Sq29iZfYehIUECD6bcnjX53dhugp8zSCTQPdbVXKsh4x3Uc9Hm WF/VWkytBJw4SUncemElECWz/zGB5s9AkQJwAjRfs3mPP0QxxmkoTxycC+zMxIGV RCnDowmI0LS/3p87uuUsgCeMzS+DTsyYKJirfeoivwfUGwutO91W6uoE5RymqICb 3rB3eG/fUDhzFNw3k5Pn2PW3wFJiELMcmRBRUTds6njlRcBkMStu7DmMUTNihz3/ SXJ1zvQHDItWgk1/kh1+RaW2+vBgTCa5c/1HXq9cbFPPUE1kO38EWPBoyPvrUzZP gTcRDELw7x6ma3cebp4l3ZPLZKY/4QYuW636l9+wa5xYXE3+X9mypwiwdmHoEf8b CH/5EV7Dk1+Yf8pDLnbltip4Vr90u/mOppjwedxb+oahSrOPxnY= =LY7H -----END PGP SIGNATURE----- --BE+dJDenWrZcLOkR--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 18:55:21 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 12 13:55:21 2024 Received: from localhost ([127.0.0.1]:37770 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOMga-0003th-Qy for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:55:21 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:51301) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rOMgY-0003tY-HE for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:55:19 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 5F5A827BBE2; Fri, 12 Jan 2024 18:55:19 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 61a53281; Fri, 12 Jan 2024 18:55:18 +0000 (UTC) References: <87plyfrb2x.fsf@HIDDEN> <89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN> <871qamk4xc.fsf@HIDDEN> User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines <mail@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: Re: [bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication. Date: Fri, 12 Jan 2024 18:26:24 +0000 In-reply-to: <871qamk4xc.fsf@HIDDEN> Message-ID: <87v87yfkkr.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 Cc: Josselin Poiret <dev@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, 68266 <at> debbugs.gnu.org, Ricardo Wurmus <rekado@HIDDEN>, Christopher Baines <guix@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Christopher Baines <mail@HIDDEN> skribis: > >> This is intended to help with spotting duplication in the object cache, = so >> where many keys, for example package records map to the same derivation.= This >> represents an opportunity for improved performance if you can reduce this >> duplication in the cache, and better take advantage of the already prese= nt >> cache entries. > > Another way to detect this is by looking at =E2=80=98add-data-to-store-ca= che=E2=80=99 > stats: > > $ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" guix build --no= -grafts greetd wlgreet du-dust circtools --target=3Daarch64-linux-gnu -d=20 > /gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv > /gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv > /gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv > /gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv > Object Cache: > fresh caches: 22 > lookups: 40435 > hits: 36821 (91.1%) > cache size: 3613 entries > > 'add-data-to-store' cache: > lookups: 4090 > hits: 958 (23.4%) > .drv files: 3062 (74.9%) > Scheme files: 916 (22.4%) > $ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" ./pre-inst-env = guix build --no-grafts greetd wlgreet du-dust circtools --target=3Daarch64-= linux-gnu -d=20 > /gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv > /gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv > /gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv > /gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv > Object Cache: > fresh caches: 22 > lookups: 37942 > hits: 34523 (91.0%) > cache size: 3418 entries > > 'add-data-to-store' cache: > lookups: 3895 > hits: 763 (19.6%) > .drv files: 2957 (75.9%) > Scheme files: 826 (21.2%) > > Ideally, the hit rate there would be 0% and we could remove it. > > If there=E2=80=99s a positive hit rate, it means we keep adding the same = .drv > and/or *-builder files to the store, meaning that the object cache was > ineffective. > >> +(define* (report-object-cache-duplication store #:key (threshold 10) >> + (port (current-error-port))) > > Do you have an example output of this? > > How helpful does it look to you in practice? Yep, so here's some output I get from computing all the cross derivations to i586-pc-gnu: value #<derivation /gnu/store/lqxlksh00cshc816xqfq47r3jjdfj2p9-subversion-1.14.2.drv =3D> /gnu/store/92avphfdcrcaxx8m5a6ihmw558bj3np8-subversion-1.14.2 7fcfbe5f9= 280> cached 4174 times example keys: - #<package subversion@HIDDEN gnu/packages/version-control.scm:2316 7fc= fc1c924d0> - #<package subversion@HIDDEN gnu/packages/version-control.scm:2316 7fc= fc1c924d0> - #<file-append #<package subversion@HIDDEN gnu/packages/version-contro= l.scm:2316 7fcfc1c924d0> "/bin/svn"> - #<file-append #<package subversion@HIDDEN gnu/packages/version-contro= l.scm:2316 7fcfc1c924d0> "/bin/svn"> So it's not immediately obvious what the issue is, but if you search for "/bin/svn", then you find the file-append calls in svn-fetch, so there's a couple of new file-append records added to the cache for each svn-fetch. value /gnu/store/qi1km3qlv5hdsql6h3ibwvz6v4z8lqbz-ld-wrapper.in cached 75= 5 times example keys: - #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-= wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package= s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure= true (file stat)>> - #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-= wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package= s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure= true (file stat)>> - #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-= wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package= s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure= true (file stat)>> - #<<local-file> file: "/home/chris/Projects/Guix/guix/gnu/packages/ld-= wrapper.in" absolute: #<promise "/home/chris/Projects/Guix/guix/gnu/package= s/ld-wrapper.in"> name: "ld-wrapper.in" recursive?: #t select?: #<procedure= true (file stat)>> Similar to file-append, it's not immediately obvious where all these local-file's are coming from, but searching for ld-wrapper.in suggests make-ld-wrapper. My thinking here is that maybe it's worth not caching everything in the object cache. Particularly for file-append, I'm not sure what the cache is actually doing. As for local-file, given there's lower level caching, maybe it's worth leaning on that and not caching local-file either. I tried this out, and it seemed to not make performance worse at least, and it did remove the duplication from the cache. Other things that show up include: value #<derivation /gnu/store/z28jfsm43wg50cvdfq0548pbf5jfhk8r-binutils-c= ross-i586-pc-gnu-2.38.drv =3D> /gnu/store/lkmwq399jllig2a5r323v6y9nflpn6gn-= binutils-cross-i586-pc-gnu-2.38 7fcfbd358000> cached 1228 times example keys: - #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm= :79 7fcfbf3bb8f0> - #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm= :79 7fcfbf3b6630> - #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm= :79 7fcfbf3b6420> - #<package binutils-cross-i586-pc-gnu@HIDDEN gnu/packages/cross-base.scm= :79 7fcfbf3b62c0> --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWhixRfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcrGxAAr+CZGwnbA4IEkRH6kmHCBa1UYeT5mVat FvOKIINgMqv7es0DMBEPy1g9Yi223BpBEHlqSloE7HUANk3gq/MASNJJd/Aeh6tR UjRzzWlG9GfuS3QH8u2gmMSPRElCsd6v9OY9WhaFQvw4Zq/EO45jmEw/QbvMzrHW 3K3w/nG0ss+tT1dYB8U+VWgFpzxg+XlAit5QVR1vW0B0r8zuAJ8SLe11iikx3ZmO X7m/kf6uC4WQU0ptIPVu84oxAjDckmuPMahA4pw+QIY2Ne2ud/fL3rcbUA7d91Ha 3sbdcfO1wClQNJLCsROEEiYUW0+hrfTHZJhlKGHdm4rAP9k6yx4cdiVhNRhu7fwE HoDmnqreW4Hji4Tgbi+hLu9Bv99l2Y0HJ9KdjKD0WmOzuWkNhhP4WrlOqEW1NWHm p72CI4zjMXzCOFosdWqCgTw0nuDoXdb4AMbKgegCbddPCizXcnH5n9upRBfhNSA1 VCl1wr/ZhLtT8JEAJp34YXUESWZSKne+pzVHW8F1oHG9H/Uogx5FP2RvSi4OYjhp 3RrzBdIjyMKfQmlp3zQmrq5OrEXXv4WGflKlrZubRCgV4Z1R4O4eK0Q/TDNhbLmY nvxz9tboQUW1xJAvDbXV/BKZgEf6AjdKEy/Qk3nMUXSx9TFb5BnQ9vBed1B5WTld h+C2I0Q5Qhk= =pIg/ -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 18:04:22 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 12 13:04:22 2024 Received: from localhost ([127.0.0.1]:37689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOLtG-0002jR-Dd for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:04:22 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43098) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rOLtE-0002jF-MQ for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 13:04:21 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 3F89027BBE2; Fri, 12 Jan 2024 18:04:21 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id a7995c2e; Fri, 12 Jan 2024 18:04:20 +0000 (UTC) References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN> <878r4uk5c7.fsf@HIDDEN> User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines <mail@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: Re: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results. Date: Fri, 12 Jan 2024 17:57:26 +0000 In-reply-to: <878r4uk5c7.fsf@HIDDEN> Message-ID: <87zfxafmxq.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 Cc: Efraim Flashner <efraim@HIDDEN>, 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Christopher Baines <mail@HIDDEN> skribis: > >> To ensure that it just returns a single package record for some given >> arguments, as this helps to avoid poor performance of the store connecti= on >> object cache. >> >> * gnu/packages/rust.scm (make-rust-sysroot): Move code to >> make-rust-sysroot/implementation. >> (make-rust-sysroot/implementation): New variable. >> >> Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167 > > [...] > >> (define*-public (make-rust-sysroot target) >> - (let ((base-rust rust)) >> + (make-rust-sysroot/implementation target rust)) >> + >> +(define make-rust-sysroot/implementation >> + (mlambda (target base-rust) >> (package >> (inherit base-rust) >> (name (string-append "rust-sysroot-for-" target)) > > We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2=80= =99) with packages as it leads > to deep object comparisons. That=E2=80=99s why for packages we typically > always have one-argument (mlambdaq (package) =E2=80=A6). > > But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before, = let=E2=80=99s keep it simple > (=E2=80=98diff --ignore-space-change=E2=80=99): ... > WDYT? Yeah, that does look good. I pushed my earlier version of this patch this morning though. I did have a look at trying to adapt the changes to fit in (guix build-system cargo) instead, as I noticed that seemed to be a pattern elsewhere, but I think there's something weird going on with the use of make-rust-sysroot there since default-rust-sysroot takes an argument, but doesn't use it. Maybe once that's figured out, we can move the memoization there and switch to just using the target as the key. Unfortunately I'm still waiting to see what effect this has on the data service processing revisions. I'm pretty sure it's going to help, but I'm concerned it's not going to help enough to make processing revisions for patches feasible again. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWhfyFfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xevjw/9GKpOuTwH7hn30C7cFq+lIQqt7Qt5bE9U DFhXIUrzLc+qJNtUZaxuUsPQwks+F2ilKHR7vdBt7W6MKcYjYQTsaWgen46s4q7i Gcb/6pe0HhDDRO/HkZedauqfOFupociG2EsR1v2XgpiWS8zg/8O3g99DN7QjjVg6 hTGN/6MkjI0luREESJ5xu0+n2K0RqtUxkR9yiqRvv+3RFCBgXr0EZAX2B4ytjEWs GA0/LjS78KPO+bKqtZfmoZMwI3fE+2QsjgLhOBWlrGr96pJVRy9DovDJT2OwUbmT wjviXLLQpMTcvzxkTRYDTSU5JbTlmdRxc2AQrDNWz5hT4XBnLGgSnme10rXeMgPO iSN8oxBW6nlFhADhmY+JoRehQmyGaoKy7aL6bVaG87fRoeivPz4iCDZygi0SmwIF qxaPc6IfySAcIuagBI3gXojaLBI7CA+f7iW6VSDZ+k9vPFPixUay5OaeHXh6A5iJ uRLEA7xggcnK+YXTJ0jiKjEuDdEVPKNQdDz2zF9kODzxJm3ocdsOnkFe7q94jEpn zgC99kYZPu+XF73eWIzhsVsvA98BaJlKThSHKTC9odqE6fzG8xfDDJ3JzsCOyb3h Hzcs2PpD++9pKMbG1juwh8yPndKWEWwL68Og9vc/3CxkVAnJtFfkBtnommiSCqx+ rNxZEHNEWPA= =NTdw -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 14:22:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 12 09:22:39 2024 Received: from localhost ([127.0.0.1]:35260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOIQh-0005wC-GQ for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:22:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39110) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1rOIQe-0005vz-RB for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:22:38 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rOIQa-0007Lu-Dm; Fri, 12 Jan 2024 09:22:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=JNydbxIjPmQqI3QLFUAB6uyyUwBUF7mM/yQwyX9GPdA=; b=ZJ+TyfisP9vav6sXWc7F ex67mHDaA8GYKqDiiRH3T0HDaR/fnfU7S7cpgJb0Psi3Hx6CPlRYRwuIR1DPuzqvpINlJBNeTsdrU bW2HgLKgJZXHP7rdLsqRCZQdpecHZQS0HJ960LhLkPBo3xOnFVU44yFpk/ZH8SVPhTXnMgc5rOVV9 A1snS3LMSTrPYzfyVzaEQI5M6Cr1RttrbKp0cr0HXYJ4yuElohhLysRhOjWjpGrhKiDpmKEmOAHl2 KQWou8xdDAH3tim0V59d550nChU8Lgsdbw0GVjdwg8O3PHdD+IM3tDiCiZcFCo8xj7F1Bi5kYQAKq kg3V8MqYAxI44Q==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication. In-Reply-To: <89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN> (Christopher Baines's message of "Wed, 10 Jan 2024 12:57:23 +0000") References: <87plyfrb2x.fsf@HIDDEN> <89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN> Date: Fri, 12 Jan 2024 15:22:07 +0100 Message-ID: <871qamk4xc.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 68266 Cc: Josselin Poiret <dev@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, 68266 <at> debbugs.gnu.org, Ricardo Wurmus <rekado@HIDDEN>, Christopher Baines <guix@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Christopher Baines <mail@HIDDEN> skribis: > This is intended to help with spotting duplication in the object cache, so > where many keys, for example package records map to the same derivation. = This > represents an opportunity for improved performance if you can reduce this > duplication in the cache, and better take advantage of the already present > cache entries. Another way to detect this is by looking at =E2=80=98add-data-to-store-cach= e=E2=80=99 stats: --8<---------------cut here---------------start------------->8--- $ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" guix build --no-g= rafts greetd wlgreet du-dust circtools --target=3Daarch64-linux-gnu -d=20 /gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv /gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv /gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv /gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv Object Cache: fresh caches: 22 lookups: 40435 hits: 36821 (91.1%) cache size: 3613 entries 'add-data-to-store' cache: lookups: 4090 hits: 958 (23.4%) .drv files: 3062 (74.9%) Scheme files: 916 (22.4%) $ GUIX_PROFILING=3D"add-data-to-store-cache object-cache" ./pre-inst-env gu= ix build --no-grafts greetd wlgreet du-dust circtools --target=3Daarch64-li= nux-gnu -d=20 /gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv /gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv /gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv /gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv Object Cache: fresh caches: 22 lookups: 37942 hits: 34523 (91.0%) cache size: 3418 entries 'add-data-to-store' cache: lookups: 3895 hits: 763 (19.6%) .drv files: 2957 (75.9%) Scheme files: 826 (21.2%) --8<---------------cut here---------------end--------------->8--- Ideally, the hit rate there would be 0% and we could remove it. If there=E2=80=99s a positive hit rate, it means we keep adding the same .d= rv and/or *-builder files to the store, meaning that the object cache was ineffective. > +(define* (report-object-cache-duplication store #:key (threshold 10) > + (port (current-error-port))) Do you have an example output of this? How helpful does it look to you in practice? I guess it we=E2=80=99d need to add a =E2=80=98register-profiling-hook!=E2= =80=99 call somewhere? If we were to add this, I=E2=80=99d suggest adding a docstring and followin= g the same style as the rest of the code (no long identifiers for local variables and no car/cdr, for example). Thanks, Ludo=E2=80=99.
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 12 Jan 2024 14:13:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 12 09:13:24 2024 Received: from localhost ([127.0.0.1]:35248 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOIHj-00030u-PV for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:13:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42824) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1rOIHg-00030I-Mw for 68266 <at> debbugs.gnu.org; Fri, 12 Jan 2024 09:13:21 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rOIHc-00038Q-7u; Fri, 12 Jan 2024 09:13:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=SxuXzKc5P8PsutD3J3SVD96Wo9cxqDPf4RTO1OOEG6U=; b=SAFb1+cTeU8K+Ylp2gRP OEjFOg+Urz1fhe2KUK48YOt4Yu6bTyGNUm4+kY54pwtwNBovQta+uWvKXPMD0FM9ahDNZqnHL8/pp hVzUFRloMAybd40Xutbh4cfKkOFGxySBMbChzsmVjVKskA+IqbkLdKidjmaJ/BdTEsYZ1J/Ak8fpz 2NwIb30X/orFyrbU0hj+xB0IDtG7W3sVDhJbLzJfpa8dFNuL9vKyUR9uc0MKrERmolfp9RfWq6ZP5 L5bcXzfYUuvL+5bSuVvCedZVLeYbSX1sld9QAIz9T9CnaLUbY7OlArOW8a1ArduggGXTC1quioBwa Z6YOtZ6DYbS/FQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results. In-Reply-To: <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN> (Christopher Baines's message of "Fri, 5 Jan 2024 16:40:49 +0000") References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN> Date: Fri, 12 Jan 2024 15:13:12 +0100 Message-ID: <878r4uk5c7.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 68266 Cc: Efraim Flashner <efraim@HIDDEN>, 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Christopher Baines <mail@HIDDEN> skribis: > To ensure that it just returns a single package record for some given > arguments, as this helps to avoid poor performance of the store connection > object cache. > > * gnu/packages/rust.scm (make-rust-sysroot): Move code to > make-rust-sysroot/implementation. > (make-rust-sysroot/implementation): New variable. > > Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167 [...] > (define*-public (make-rust-sysroot target) > - (let ((base-rust rust)) > + (make-rust-sysroot/implementation target rust)) > + > +(define make-rust-sysroot/implementation > + (mlambda (target base-rust) > (package > (inherit base-rust) > (name (string-append "rust-sysroot-for-" target)) We should avoid using =E2=80=98mlambda=E2=80=99 (without =E2=80=98q=E2=80= =99) with packages as it leads to deep object comparisons. That=E2=80=99s why for packages we typically always have one-argument (mlambdaq (package) =E2=80=A6). But since =E2=80=98base-rust=E2=80=99 wasn=E2=80=99t a parameter before, le= t=E2=80=99s keep it simple (=E2=80=98diff --ignore-space-change=E2=80=99): --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm index 2b86cf3c8b..983a3f0f41 100644 --- a/gnu/packages/rust.scm +++ b/gnu/packages/rust.scm @@ -62,6 +62,7 @@ (define-module (gnu packages rust) #:use-module (guix download) #:use-module (guix git-download) #:use-module ((guix licenses) #:prefix license:) + #:use-module (guix memoization) #:use-module (guix packages) #:use-module ((guix build utils) #:select (alist-replace)) #:use-module (guix utils) @@ -1060,7 +1061,8 @@ (define-public rust `("procps" ,procps) (package-native-inputs base-rust)))))) -(define*-public (make-rust-sysroot target) +(define-public make-rust-sysroot + (mlambda (target) (let ((base-rust rust)) (package (inherit base-rust) @@ -1228,7 +1230,7 @@ (define*-public (make-rust-sysroot target) (cross-binutils target))))) (properties `((hidden? . #t) - ,(package-properties base-rust)))))) + ,(package-properties base-rust))))))) (define-public rust-analyzer (package --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable It should help when cross-compiling several =E2=80=98cargo-build-system=E2= =80=99 packages: --8<---------------cut here---------------start------------->8--- $ time GUIX_PROFILING=3Dobject-cache guix build --no-grafts greetd wlgreet = du-dust circtools --target=3Daarch64-linux-gnu -d # before /gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv /gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv /gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv /gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv Object Cache: fresh caches: 22 lookups: 40435 hits: 36821 (91.1%) cache size: 3613 entries real 0m2.964s user 0m2.925s sys 0m0.174s $ time GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build --no-grafts = greetd wlgreet du-dust circtools --target=3Daarch64-linux-gnu -d # after /gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv /gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv /gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv /gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv Object Cache: fresh caches: 22 lookups: 37942 hits: 34523 (91.0%) cache size: 3418 entries real 0m2.980s user 0m3.300s sys 0m0.160s --8<---------------cut here---------------end--------------->8--- (Here we removed ~2.5k nodes from the cache.) WDYT? Ludo=E2=80=99. --=-=-=--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 10 Jan 2024 14:16:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 10 09:16:03 2024 Received: from localhost ([127.0.0.1]:39381 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rNZNC-0000JT-KS for submit <at> debbugs.gnu.org; Wed, 10 Jan 2024 09:16:03 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43096) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rNZNA-0000EG-Bt for 68266 <at> debbugs.gnu.org; Wed, 10 Jan 2024 09:16:00 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id ADD2F27BBE9 for <68266 <at> debbugs.gnu.org>; Wed, 10 Jan 2024 12:57:24 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 32e0a20a for <68266 <at> debbugs.gnu.org>; Wed, 10 Jan 2024 12:57:24 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH v2] guix: store: Add report-object-cache-duplication. Date: Wed, 10 Jan 2024 12:57:23 +0000 Message-ID: <89c875f974d1ad81ddd03f664ef08e397771d224.1704891443.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) This is intended to help with spotting duplication in the object cache, so where many keys, for example package records map to the same derivation. This represents an opportunity for improved performance if you can reduce this duplication in the cache, and better take advantage of the already present cache entries. I'm thinking this can be used by the data service, but maybe it could also be worked in to guix commands. * guix/store.scm (report-object-cache-duplication): New procedure. Change-Id: Ia6c816f871d10cae6807543224250110099d764f --- guix/store.scm | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/guix/store.scm b/guix/store.scm index 97c4f32a5b..86ca293cac 100644 --- a/guix/store.scm +++ b/guix/store.scm @@ -70,6 +70,7 @@ (define-module (guix store) current-store-protocol-version ;for internal use cache-lookup-recorder ;for internal use mcached + report-object-cache-duplication &store-error store-error? &store-connection-error store-connection-error? @@ -2037,6 +2038,64 @@ (define-syntax mcached ((_ mvalue object keys ...) (mcached eq? mvalue object keys ...)))) +(define* (report-object-cache-duplication store #:key (threshold 10) + (port (current-error-port))) + (define cache-values-to-keys + (make-hash-table)) + + (define (insert key val) + (hash-set! + cache-values-to-keys + key + (or (and=> (hash-ref cache-values-to-keys + key) + (lambda (existing-values) + (cons val existing-values))) + (list val)))) + + (let* ((cache-size + (vhash-fold + (lambda (key value result) + (match value + ((item . keys*) + (insert item key))) + + (+ 1 result)) + 0 + (store-connection-cache store %object-cache-id))) + (cached-values-by-key-count + (sort + (hash-map->list + (lambda (key value) + (cons key (length value))) + cache-values-to-keys) + (lambda (a b) + (< (cdr a) (cdr b)))))) + + (filter-map + (match-lambda + ((value . count) + (if (>= count threshold) + (begin + (when port + (simple-format port "value ~A cached ~A times\n" value count) + (simple-format port "example keys:\n")) + + (let ((keys (hash-ref cache-values-to-keys value))) + (when port + (for-each + (lambda (key) + (simple-format #t " - ~A\n" key)) + (if (> count 10) + (take keys 10) + keys)) + (newline port)) + + `((value . ,value) + (keys . ,keys)))) + #f))) + cached-values-by-key-count))) + (define (preserve-documentation original proc) "Return PROC with documentation taken from ORIGINAL." (set-object-property! proc 'documentation base-commit: e541f9593f8bfc84b6140c2408b393243289fae6 -- 2.41.0
guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 10 Jan 2024 13:56:02 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 10 08:56:02 2024 Received: from localhost ([127.0.0.1]:39296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rNZ3q-0003ML-65 for submit <at> debbugs.gnu.org; Wed, 10 Jan 2024 08:56:02 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43094) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rNZ3n-0003M3-W9 for 68266 <at> debbugs.gnu.org; Wed, 10 Jan 2024 08:56:00 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 26BCD27BBE2; Wed, 10 Jan 2024 12:38:31 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 47b23c40; Wed, 10 Jan 2024 12:38:30 +0000 (UTC) References: <87plyfrb2x.fsf@HIDDEN> <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <87v8833fl7.fsf@HIDDEN> <878r4zr6a0.fsf@HIDDEN> <87plyaw1bn.fsf@HIDDEN> User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines <mail@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: Re: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results. Date: Wed, 10 Jan 2024 12:28:31 +0000 In-reply-to: <87plyaw1bn.fsf@HIDDEN> Message-ID: <87cyu9gy7w.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 Cc: 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Christopher Baines <mail@HIDDEN> skribis: > >> I believe the reason packages from make-ld-wrapper were showing up >> multiple times in the cache for me is linked to it's use in the >> cross-base module, as part of the cross-gcc procedure. >> >> A later commit does change cross-gcc to return a single package record >> for some given arguments, so that probably resolves the biggest misuse >> of make-ld-wrapper. > > Oh, I see. > > Before resorting to memoization (which doesn=E2=80=99t come for free), we= can > for instance make sure we do not create new package records from inputs > fields (since they are thunked, a fresh record would be created each > time the field is accessed). Here=E2=80=99s an example: > > diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm > index 6ee7b315d8..ca3b41a4c7 100644 > --- a/gnu/packages/cross-base.scm > +++ b/gnu/packages/cross-base.scm > @@ -306,6 +306,11 @@ (define* (cross-gcc target > XGCC as the base compiler. Use XBINUTILS as the associated cross-Binuti= ls. > If LIBC is false, then build a GCC that does not target a libc; otherwis= e, > target that libc." > + (define ld-wrapper > + (make-ld-wrapper (string-append "ld-wrapper-" target) > + #:target (const target) > + #:binutils xbinutils)) > + > (package > (inherit xgcc) > (name (string-append "gcc-cross-" > @@ -313,8 +318,7 @@ (define* (cross-gcc target > target)) > (source > (origin > - (inherit > - (package-source xgcc)) > + (inherit (package-source xgcc)) > (patches > (append > (origin-patches (package-source xgcc)) > @@ -353,10 +357,7 @@ (define* (cross-gcc target > ,@(cross-gcc-arguments target xgcc libc))) >=20=20 > (native-inputs > - `(("ld-wrapper-cross" ,(make-ld-wrapper > - (string-append "ld-wrapper-" target) > - #:target (const target) > - #:binutils xbinutils)) > + `(("ld-wrapper-cross" ,ld-wrapper) > ("binutils-cross" ,xbinutils) >=20=20 > ,@(let ((inputs (append (package-inputs xgcc) > > > This particular one doesn=E2=80=99t have a huge impact on simple cases bu= t it > might pay off for bigger graphs: > > $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target= =3Daarch64-linux-gnu --no-grafts -d > /gnu/store/i8x1gg73x3fvxn4lrrgbcb97sz4qq1yx-hello-2.12.1.drv > Object Cache: > fresh caches: 20 > lookups: 5514 > hits: 5008 (90.8%) > cache size: 505 entries > $ # before: > $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target= =3Daarch64-linux-gnu --no-grafts -d > /gnu/store/kdnh9bwg874aq8bvvw9y2vkl7z94icqa-hello-2.12.1.drv > Object Cache: > fresh caches: 20 > lookups: 5406 > hits: 4907 (90.8%) > cache size: 498 entries > >> I think there's other cases (in the llvm and mold modules) where it >> looks like it's called multiple times with the same arguments, so maybe >> that's an argument for having memoization around make-ld-wrapper even >> though it's not needed for all uses. > > On guix-devel, Efraim mentioned cargo-build-system packages. Is that > related? Looking at the various changes, while rust-sysroot doesn't appear that many times in the cache, I think it's responsible for most of the slowness. Maybe that's due in part to what you say above, using the cross- procedures in the native-inputs field. Maybe the transformation you suggest above could be applied, but I'd be worried about how that would work if you set one %current-system/%current-target-system when you call the procedure, then another when you compute the derivation. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWej8NfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xc/1g/7B3RzVXQfEw1WTkDK9Ia3PJfA8EhN4cRr HVwtVs2SFCjffaM5HTa3LyGzoZnrFV9yItJ1m+2SBZOPdAi26euXnBTEnA72RELg MB2KK6bpdTZ42iV7ZxWT/RGDrRXhSd0YvzbtpRtMbfdk8wFXBpN7886ubFiOQemX WDlL7loxI7Ncfqv3XJim+DMkBeeCS3dLiVHGRnBBuWMl8X93atOY4y6/kys/rebc /yO2cGwwOjZrjVPddFjs0FsAQFov00ESz9f387n5YPU2aEsRSqqYAcyu16oMg+4B gx029IIIbtFiwCyja5qbpfGuXTIKXNsvxZR9dvV4E2BAllJvHbRaarI+ENzvAdD4 0fddw8KVV5+D2MPUzC+O7INcF+b4yO3IR52Bo9OjyvCCgkWDCc0QBVYXkgCYr3ZN dOnQqxNoA/oVt3SzTIi1QjL0i57aLjrEXdbDs0goWmxoe7pOEAkUVFfcqCqjfGHg taawfZhFI+PgosLgnye+Tfe/tRqpfSI+ODPjS3CLfT4mpekOlbzHf91AF6S2oo/L dDbtLRbNmeobKk+2tIUS99O8GPYoRyQKhTx1EE0sz2meEuohaHCmdWi5pqbcYMlR aeASq/syYUso3XJdpEK+clYPcpts7mDYMxYHMeQs2kfHFOyI2eB6VMBmfz+wFMDq PAShmn5JWaU= =7avI -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 9 Jan 2024 23:10:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 09 18:10:24 2024 Received: from localhost ([127.0.0.1]:41254 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rNLEl-0006SR-QP for submit <at> debbugs.gnu.org; Tue, 09 Jan 2024 18:10:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37558) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1rNLEk-0006SD-83 for 68266 <at> debbugs.gnu.org; Tue, 09 Jan 2024 18:10:22 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rNLEX-0007Pv-KA; Tue, 09 Jan 2024 18:10:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=CldhrM5j3uJvBBy8LZ4aN5RSUGHm8zBlvU0kfDe8qjo=; b=TDGRJxbPqrxASc2w9Uql 6VnSimOw7zW2ObokeHOu6KpfFAb8I1tuVqdcKKvdrZ2UYJ0OEVTpol94kR/sCFxnisDqxbg08k5wk pS9twliHkK2UC3EDzsEzn2/UHPUeXNNtv/1K/Py1uLMdutyFQPzsb926GwH14cigCND4PQpgFjk0u izF7M86JFLkubU7yH6GMVcAnhf14HQWIRXtPt2S7G1PNydbj4tzvXL5QQSiZItJF+AdUWgEKgeBM3 +69JGS4A3A4VJzEVLKvHYyzcpFhB7vfP2Sn+PdzwFsi8BCfQ5sXxmDwSiRGLZaIWhbv19osRAtJoy rGc8+hEn53U3Hw==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results. In-Reply-To: <878r4zr6a0.fsf@HIDDEN> (Christopher Baines's message of "Mon, 08 Jan 2024 19:01:58 +0000") References: <87plyfrb2x.fsf@HIDDEN> <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <87v8833fl7.fsf@HIDDEN> <878r4zr6a0.fsf@HIDDEN> Date: Wed, 10 Jan 2024 00:10:04 +0100 Message-ID: <87plyaw1bn.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 68266 Cc: 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Christopher Baines <mail@HIDDEN> skribis: > I believe the reason packages from make-ld-wrapper were showing up > multiple times in the cache for me is linked to it's use in the > cross-base module, as part of the cross-gcc procedure. > > A later commit does change cross-gcc to return a single package record > for some given arguments, so that probably resolves the biggest misuse > of make-ld-wrapper. Oh, I see. Before resorting to memoization (which doesn=E2=80=99t come for free), we c= an for instance make sure we do not create new package records from inputs fields (since they are thunked, a fresh record would be created each time the field is accessed). Here=E2=80=99s an example: --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm index 6ee7b315d8..ca3b41a4c7 100644 --- a/gnu/packages/cross-base.scm +++ b/gnu/packages/cross-base.scm @@ -306,6 +306,11 @@ (define* (cross-gcc target XGCC as the base compiler. Use XBINUTILS as the associated cross-Binutils. If LIBC is false, then build a GCC that does not target a libc; otherwise, target that libc." + (define ld-wrapper + (make-ld-wrapper (string-append "ld-wrapper-" target) + #:target (const target) + #:binutils xbinutils)) + (package (inherit xgcc) (name (string-append "gcc-cross-" @@ -313,8 +318,7 @@ (define* (cross-gcc target target)) (source (origin - (inherit - (package-source xgcc)) + (inherit (package-source xgcc)) (patches (append (origin-patches (package-source xgcc)) @@ -353,10 +357,7 @@ (define* (cross-gcc target ,@(cross-gcc-arguments target xgcc libc))) (native-inputs - `(("ld-wrapper-cross" ,(make-ld-wrapper - (string-append "ld-wrapper-" target) - #:target (const target) - #:binutils xbinutils)) + `(("ld-wrapper-cross" ,ld-wrapper) ("binutils-cross" ,xbinutils) ,@(let ((inputs (append (package-inputs xgcc) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable This particular one doesn=E2=80=99t have a huge impact on simple cases but = it might pay off for bigger graphs: --8<---------------cut here---------------start------------->8--- $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target=3D= aarch64-linux-gnu --no-grafts -d /gnu/store/i8x1gg73x3fvxn4lrrgbcb97sz4qq1yx-hello-2.12.1.drv Object Cache: fresh caches: 20 lookups: 5514 hits: 5008 (90.8%) cache size: 505 entries $ # before: $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target=3D= aarch64-linux-gnu --no-grafts -d /gnu/store/kdnh9bwg874aq8bvvw9y2vkl7z94icqa-hello-2.12.1.drv Object Cache: fresh caches: 20 lookups: 5406 hits: 4907 (90.8%) cache size: 498 entries --8<---------------cut here---------------end--------------->8--- > I think there's other cases (in the llvm and mold modules) where it > looks like it's called multiple times with the same arguments, so maybe > that's an argument for having memoization around make-ld-wrapper even > though it's not needed for all uses. On guix-devel, Efraim mentioned cargo-build-system packages. Is that related? Thanks for looking into this! Ludo=E2=80=99. --=-=-=--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 8 Jan 2024 19:10:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 08 14:10:01 2024 Received: from localhost ([127.0.0.1]:37587 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rMv0a-0005n8-0c for submit <at> debbugs.gnu.org; Mon, 08 Jan 2024 14:10:01 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43072) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rMv0U-0005mx-Vu for 68266 <at> debbugs.gnu.org; Mon, 08 Jan 2024 14:09:58 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 1DBD827BBE2; Mon, 8 Jan 2024 19:09:47 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 2dc91849; Mon, 8 Jan 2024 19:09:45 +0000 (UTC) References: <87plyfrb2x.fsf@HIDDEN> <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> <87v8833fl7.fsf@HIDDEN> User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines <mail@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: Re: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results. Date: Mon, 08 Jan 2024 19:01:58 +0000 In-reply-to: <87v8833fl7.fsf@HIDDEN> Message-ID: <878r4zr6a0.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 Cc: 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Hi, > > Christopher Baines <mail@HIDDEN> skribis: > >> To ensure that it just returns a single package record for some given >> arguments, as this helps to avoid poor performance of the store connecti= on >> object cache. >> >> * gnu/packages/base.scm (make-ld-wrapper): Move code to >> make-ld-wrapper/implementation and call it. >> (make-ld-wrapper/implementation) New procedure. >> >> Change-Id: Id6fc805a4a7ffbc5ff0a5174eafcdf2c7c46854d > > Do you have figures before and after the change? > > The reason I=E2=80=99m asking is that (gnu packages commencement) arrange= s to > not call =E2=80=98make-ld-wrapper=E2=80=99 repeatedly already. For insta= nce, there=E2=80=99s: > > (define-public ld-wrapper > ;; The final 'ld' wrapper, which uses the final Guile and Binutils. > (make-ld-wrapper "ld-wrapper" > #:binutils binutils-final > #:guile guile-final > #:bash bash-final)) > > and from there on we manipulate a single <package> record. I believe the reason packages from make-ld-wrapper were showing up multiple times in the cache for me is linked to it's use in the cross-base module, as part of the cross-gcc procedure. A later commit does change cross-gcc to return a single package record for some given arguments, so that probably resolves the biggest misuse of make-ld-wrapper. I think there's other cases (in the llvm and mold modules) where it looks like it's called multiple times with the same arguments, so maybe that's an argument for having memoization around make-ld-wrapper even though it's not needed for all uses. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWcSHdfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XfZVQ/9EnUCnl48GyAQxKQWWzWUsNcacRgB7HVD teI7hKFAosZgzESSux7GKzZJDHS9YqR9qWgnfZlE9+/zYt7rIGsp7pAZrAolVR36 h7D72J1gS78wlowUQRSCUWNgp+Un2KJM2HyI+cm5dRap8BTeZQVH0vygVH+Z39vx k07N3wjeyr+FYhnwW2SN+DCKi+9tkswMeWkNUPgcSbFUIK0V+D/o6f4fFgkmnHAF lbY2N9RgBjlc4cKJPvD63iGgxXWJE7u89kfGKgj9+x8CHjRw12eU02r2gzpsjsdA JYD/gCP6oHPbpxtl3YLgL/6SzrN9G/aCLSZuxmsaRW7XrvZlofjvdHViiu9SoCXT 6053TPL9DiuRmF+CZMYfplP8Wct/SMS45S/kYAcJjKZIWSB0xuuuwN+JMoJJwSr5 6p2W0/wzMqvqEJYELvO9ZwkA30XfxwxRJK3Fh8HbjPVqpo8Fm2qL5h4TAHkuW/xv BL2rjBNDAP6KovnRpcYBtsFo7q6PGkuw6mD9wOjXeNKF/JICLlrptMGhH+c0ML2L QCZiAQgTlkjEGh3Udq9teWRY7LZvkW3TuxCs0gphs+ImUY6KkaQt0H7frh0O1tlZ KDLBM8svs5OSAQM3cWSDXV+yqR2dWDaG3B7mVykhzyJKHbx8I6ex7jG5mEYnyH2O Jx4Ed8eVBUI= =G7+r -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 8 Jan 2024 17:22:44 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 08 12:22:44 2024 Received: from localhost ([127.0.0.1]:37461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rMtKm-0001gL-BH for submit <at> debbugs.gnu.org; Mon, 08 Jan 2024 12:22:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51916) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1rMtKk-0001g6-JI for 68266 <at> debbugs.gnu.org; Mon, 08 Jan 2024 12:22:42 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1rMtKY-0002vN-VZ; Mon, 08 Jan 2024 12:22:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=PTQ1mRNSVVdL/eROxysbOX6oock2xMkZ2s1XT18Mpyc=; b=Qx6lE1l5cN9DoYWZaLvs Gn/xyKEijSK7Uw1jq1Ypo1ycaZhqHnFOs1NHnqOIS66c4yhZlWxlViwDxf43vE+lpFSA7Ts8pAZCW Y5TjruyxzC8oZY/FcUmVCAtbLqARUQfRFX557oyT216Ev2Up+eKDxbdTexumBc7lA6t52YTGkfgGM 0t9dHrgyX0NoLMycax6hL7XsRYCIfoQXBs8C4+I2UqWDtFXtjC8SaWipILR9YiQYnxXr9r0DggcVd nTbykwEttxsjak/3ACTFlix0hwk/RdXQmzvW1Q43hAO+LSXwKXHfX0xV+s6BuMjKcLoiRz+9PL2ry RHVNwT02QvpcWA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results. In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> (Christopher Baines's message of "Fri, 5 Jan 2024 16:40:43 +0000") References: <87plyfrb2x.fsf@HIDDEN> <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Nonidi 19 =?utf-8?Q?Niv=C3=B4se?= an 232 de la =?utf-8?Q?R=C3=A9volution=2C?= jour du Marbre X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 08 Jan 2024 18:22:28 +0100 Message-ID: <87v8833fl7.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 68266 Cc: 68266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Christopher Baines <mail@HIDDEN> skribis: > To ensure that it just returns a single package record for some given > arguments, as this helps to avoid poor performance of the store connection > object cache. > > * gnu/packages/base.scm (make-ld-wrapper): Move code to > make-ld-wrapper/implementation and call it. > (make-ld-wrapper/implementation) New procedure. > > Change-Id: Id6fc805a4a7ffbc5ff0a5174eafcdf2c7c46854d Do you have figures before and after the change? The reason I=E2=80=99m asking is that (gnu packages commencement) arranges = to not call =E2=80=98make-ld-wrapper=E2=80=99 repeatedly already. For instanc= e, there=E2=80=99s: (define-public ld-wrapper ;; The final 'ld' wrapper, which uses the final Guile and Binutils. (make-ld-wrapper "ld-wrapper" #:binutils binutils-final #:guile guile-final #:bash bash-final)) and from there on we manipulate a single <package> record. Thanks, Ludo=E2=80=99.
guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:05 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:05 2024 Received: from localhost ([127.0.0.1]:57738 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFo-0003uA-7R for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:05 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:41563) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFh-0003s0-5T for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:00 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 501D827BBEB for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id badd81b3 for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 4/7] gnu: Memozise cross-kernel-headers results. Date: Fri, 5 Jan 2024 16:40:46 +0000 Message-ID: <4c720cbfb79b514b7ebae3a2f29998f198aa845f.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/cross-base.scm (cross-kernel-headers*): Move code to cross-kernel-headers/implementation and call it. (cross-kernel-headers/implementation) New procedure. Change-Id: I345604c089e7a8a9884c07f39c95f960760e86db --- gnu/packages/cross-base.scm | 306 ++++++++++++++++++------------------ 1 file changed, 157 insertions(+), 149 deletions(-) diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm index a4e361b476..f966e2f5ac 100644 --- a/gnu/packages/cross-base.scm +++ b/gnu/packages/cross-base.scm @@ -407,10 +407,19 @@ (define* (cross-gcc target libc)) (define* (cross-kernel-headers . args) + "Return headers depending on TARGET." (if (or (= (length args) 1) (contains-keyword? args)) (apply cross-kernel-headers* args) (apply cross-kernel-headers/deprecated args))) +(define* (cross-kernel-headers* target + #:key + (linux-headers linux-libre-headers) + (xgcc (cross-gcc target)) + (xbinutils (cross-binutils target))) + (cross-kernel-headers/implementation target + linux-headers xgcc xbinutils)) + (define* (cross-kernel-headers/deprecated target #:optional (linux-headers linux-libre-headers) @@ -486,159 +495,158 @@ (define* (cross-mig target (modify-inputs (package-native-inputs mig) (prepend xgcc xbinutils))))) -(define* (cross-kernel-headers* target - #:key - (linux-headers linux-libre-headers) - (xgcc (cross-gcc target)) - (xbinutils (cross-binutils target))) - "Return headers depending on TARGET." - - (define xlinux-headers (package - (inherit linux-headers) - (name (string-append (package-name linux-headers) - "-cross-" target)) (arguments - (substitute-keyword-arguments - `(#:implicit-cross-inputs? #f - ,@(package-arguments linux-headers)) - ((#:phases phases) - `(modify-phases ,phases - (replace 'build - (lambda _ - (setenv "ARCH" ,(platform-linux-architecture - (lookup-platform-by-target target))) - (format #t "`ARCH' set to `~a' (cross compiling)~%" - (getenv "ARCH")) - - (invoke "make" ,(system->defconfig target)) - (invoke "make" "mrproper" - ,@(if (version>=? (package-version linux-headers) "5.3") - '("headers") - '("headers_check"))))))))) - (native-inputs `(("cross-gcc" ,xgcc) - ("cross-binutils" ,xbinutils) - ,@(package-native-inputs linux-headers))))) - - (define xmig - (cross-mig target #:xgcc xgcc #:xbinutils xbinutils)) - - (define xgnumach-headers - (cross-gnumach-headers target #:xgcc xgcc #:xbinutils xbinutils)) - - (define xhurd-headers - (package - (inherit hurd-headers) - (name (string-append (package-name hurd-headers) - "-cross-" target)) - - (arguments - (substitute-keyword-arguments (package-arguments hurd-headers) - ((#:configure-flags flags) - `(cons* ,(string-append "--build=" (%current-system)) - ,(string-append "--host=" target) - ,flags)))) - - (native-inputs `(("cross-gcc" ,xgcc) - ("cross-binutils" ,xbinutils) - ("cross-mig" ,xmig) - ,@(alist-delete "mig" (package-native-inputs hurd-headers)))))) - - (define xglibc/hurd-headers - (package - (inherit glibc/hurd-headers) - (name (string-append (package-name glibc/hurd-headers) - "-cross-" target)) - - (arguments - (substitute-keyword-arguments - `(#:modules ((guix build gnu-build-system) - (guix build utils) - (srfi srfi-26)) - ,@(package-arguments glibc/hurd-headers)) - ((#:phases phases) - `(modify-phases ,phases - (add-after 'unpack 'set-cross-headers-path - (lambda* (#:key inputs #:allow-other-keys) - (let* ((mach (assoc-ref inputs "gnumach-headers")) - (hurd (assoc-ref inputs "hurd-headers")) - (cpath (string-append mach "/include:" - hurd "/include"))) - (for-each (cut setenv <> cpath) - ',%gcc-cross-include-paths) - #t))))) - ((#:configure-flags flags) - `(cons* ,(string-append "--build=" (%current-system)) - ,(string-append "--host=" target) - ,flags)))) - - (propagated-inputs `(("gnumach-headers" ,xgnumach-headers) - ("hurd-headers" ,xhurd-headers))) - - (native-inputs `(("cross-gcc" ,xgcc) - ("cross-binutils" ,xbinutils) - ("cross-mig" ,xmig) - ,@(alist-delete "mig"(package-native-inputs glibc/hurd-headers)))))) - - (define xhurd-minimal - (package - (inherit hurd-minimal) - (name (string-append (package-name hurd-minimal) - "-cross-" target)) - (arguments - (substitute-keyword-arguments - `(#:modules ((guix build gnu-build-system) - (guix build utils) - (srfi srfi-26)) - ,@(package-arguments hurd-minimal)) - ((#:configure-flags flags) - `(cons* ,(string-append "--build=" (%current-system)) - ,(string-append "--host=" target) - ,flags)) - ((#:phases phases) #~(modify-phases #$phases - (add-after 'unpack 'delete-shared-target - ;; Cannot create shared libraries due to missing crt1.o - (lambda _ - (substitute* "Makeconf" - (("(targets := \\$\\(libname\\)\\.a) \\$\\(libname\\)\\.so" all static) - static) - (("\\$\\(DESTDIR\\)\\$\\(libdir\\)/\\$\\(libname\\)\\.so\\.\\$\\(hurd-version\\)") - "") - (("^libs: .*\\.so\\..*" all) - (string-append "# " all))))) - (add-before 'configure 'set-cross-headers-path - (lambda* (#:key inputs #:allow-other-keys) - (let* ((glibc-headers (assoc-ref inputs "cross-glibc-hurd-headers")) - (mach-headers (assoc-ref inputs "cross-gnumach-headers")) - (cpath (string-append glibc-headers "/include" - ":" mach-headers "/include"))) - (for-each (cut setenv <> cpath) - '#$%gcc-cross-include-paths) - #t))))))) - - (inputs `(("cross-glibc-hurd-headers" ,xglibc/hurd-headers) - ("cross-gnumach-headers" ,xgnumach-headers))) - (native-inputs `(("cross-gcc" ,xgcc) - ("cross-binutils" ,xbinutils) - ("cross-mig" ,xmig) - ,@(alist-delete "mig" - (package-native-inputs hurd-minimal)))))) - - (define xhurd-core-headers - (package - (inherit hurd-core-headers) - (name (string-append (package-name hurd-core-headers) - "-cross-" target)) - - (inputs `(("gnumach-headers" ,xgnumach-headers) - ("hurd-headers" ,xhurd-headers) - ("hurd-minimal" ,xhurd-minimal))))) - - (match target - ((or "i586-pc-gnu" "i586-gnu") xhurd-core-headers) - (_ xlinux-headers))) +(define cross-kernel-headers/implementation + (mlambda (target linux-headers xgcc xbinutils) + (define xlinux-headers + (package + (inherit linux-headers) + (name (string-append (package-name linux-headers) + "-cross-" target)) + (arguments + (substitute-keyword-arguments + `(#:implicit-cross-inputs? #f + ,@(package-arguments linux-headers)) + ((#:phases phases) + `(modify-phases ,phases + (replace 'build + (lambda _ + (setenv "ARCH" ,(platform-linux-architecture + (lookup-platform-by-target target))) + (format #t "`ARCH' set to `~a' (cross compiling)~%" + (getenv "ARCH")) + + (invoke "make" ,(system->defconfig target)) + (invoke "make" "mrproper" + ,@(if (version>=? (package-version linux-headers) "5.3") + '("headers") + '("headers_check"))))))))) + (native-inputs `(("cross-gcc" ,xgcc) + ("cross-binutils" ,xbinutils) + ,@(package-native-inputs linux-headers))))) + + (define xmig + (cross-mig target #:xgcc xgcc #:xbinutils xbinutils)) + + (define xgnumach-headers + (cross-gnumach-headers target #:xgcc xgcc #:xbinutils xbinutils)) + + (define xhurd-headers + (package + (inherit hurd-headers) + (name (string-append (package-name hurd-headers) + "-cross-" target)) + + (arguments + (substitute-keyword-arguments (package-arguments hurd-headers) + ((#:configure-flags flags) + `(cons* ,(string-append "--build=" (%current-system)) + ,(string-append "--host=" target) + ,flags)))) + + (native-inputs `(("cross-gcc" ,xgcc) + ("cross-binutils" ,xbinutils) + ("cross-mig" ,xmig) + ,@(alist-delete "mig" (package-native-inputs hurd-headers)))))) + + (define xglibc/hurd-headers + (package + (inherit glibc/hurd-headers) + (name (string-append (package-name glibc/hurd-headers) + "-cross-" target)) + + (arguments + (substitute-keyword-arguments + `(#:modules ((guix build gnu-build-system) + (guix build utils) + (srfi srfi-26)) + ,@(package-arguments glibc/hurd-headers)) + ((#:phases phases) + `(modify-phases ,phases + (add-after 'unpack 'set-cross-headers-path + (lambda* (#:key inputs #:allow-other-keys) + (let* ((mach (assoc-ref inputs "gnumach-headers")) + (hurd (assoc-ref inputs "hurd-headers")) + (cpath (string-append mach "/include:" + hurd "/include"))) + (for-each (cut setenv <> cpath) + ',%gcc-cross-include-paths) + #t))))) + ((#:configure-flags flags) + `(cons* ,(string-append "--build=" (%current-system)) + ,(string-append "--host=" target) + ,flags)))) + + (propagated-inputs `(("gnumach-headers" ,xgnumach-headers) + ("hurd-headers" ,xhurd-headers))) + + (native-inputs `(("cross-gcc" ,xgcc) + ("cross-binutils" ,xbinutils) + ("cross-mig" ,xmig) + ,@(alist-delete "mig"(package-native-inputs glibc/hurd-headers)))))) + + (define xhurd-minimal + (package + (inherit hurd-minimal) + (name (string-append (package-name hurd-minimal) + "-cross-" target)) + (arguments + (substitute-keyword-arguments + `(#:modules ((guix build gnu-build-system) + (guix build utils) + (srfi srfi-26)) + ,@(package-arguments hurd-minimal)) + ((#:configure-flags flags) + `(cons* ,(string-append "--build=" (%current-system)) + ,(string-append "--host=" target) + ,flags)) + ((#:phases phases) + #~(modify-phases #$phases + (add-after 'unpack 'delete-shared-target + ;; Cannot create shared libraries due to missing crt1.o + (lambda _ + (substitute* "Makeconf" + (("(targets := \\$\\(libname\\)\\.a) \\$\\(libname\\)\\.so" all static) + static) + (("\\$\\(DESTDIR\\)\\$\\(libdir\\)/\\$\\(libname\\)\\.so\\.\\$\\(hurd-version\\)") + "") + (("^libs: .*\\.so\\..*" all) + (string-append "# " all))))) + (add-before 'configure 'set-cross-headers-path + (lambda* (#:key inputs #:allow-other-keys) + (let* ((glibc-headers (assoc-ref inputs "cross-glibc-hurd-headers")) + (mach-headers (assoc-ref inputs "cross-gnumach-headers")) + (cpath (string-append glibc-headers "/include" + ":" mach-headers "/include"))) + (for-each (cut setenv <> cpath) + '#$%gcc-cross-include-paths) + #t))))))) + + (inputs `(("cross-glibc-hurd-headers" ,xglibc/hurd-headers) + ("cross-gnumach-headers" ,xgnumach-headers))) + + (native-inputs `(("cross-gcc" ,xgcc) + ("cross-binutils" ,xbinutils) + ("cross-mig" ,xmig) + ,@(alist-delete "mig" + (package-native-inputs hurd-minimal)))))) + + (define xhurd-core-headers + (package + (inherit hurd-core-headers) + (name (string-append (package-name hurd-core-headers) + "-cross-" target)) + + (inputs `(("gnumach-headers" ,xgnumach-headers) + ("hurd-headers" ,xhurd-headers) + ("hurd-minimal" ,xhurd-minimal))))) + + (match target + ((or "i586-pc-gnu" "i586-gnu") xhurd-core-headers) + (_ xlinux-headers)))) (define* (cross-libc . args) (if (or (= (length args) 1) (contains-keyword? args)) -- 2.41.0
ludo@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:04 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:04 2024 Received: from localhost ([127.0.0.1]:57736 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFn-0003tz-Iw for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:04 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:33519) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFh-0003s1-8J for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:59 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 5C84D27BBEC for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id d12c39fb for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 5/7] gnu: Memozise cross-mig results. Date: Fri, 5 Jan 2024 16:40:47 +0000 Message-ID: <e1640e9c30677bb61093dd3b8e409dc0033ebe3a.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/cross-base.scm (cross-mig): Move code to cross-mig/implementation and call it. (cross-mig/implementation) New procedure. Change-Id: Iaf2a69c48664d2f0766b9d2f6e981653e0e3c44c --- gnu/packages/cross-base.scm | 56 +++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm index f966e2f5ac..6c6c6e7636 100644 --- a/gnu/packages/cross-base.scm +++ b/gnu/packages/cross-base.scm @@ -467,37 +467,39 @@ (define* (cross-mig target (xbinutils (cross-binutils target))) "Return a cross-mig for TARGET, where TARGET is a GNU triplet. Use XGCC as the base compiler. Use XBINUTILS as the associated cross-Binutils." - (define xgnumach-headers - (cross-gnumach-headers target - #:xgcc xgcc - #:xbinutils xbinutils)) - (package - (inherit mig) - (name (string-append "mig-cross")) - (arguments - (substitute-keyword-arguments (package-arguments mig) - ((#:configure-flags flags #~'()) - #~(list #$(string-append "--target=" target))) - ((#:tests? _ #f) - #f) - ((#:phases phases #~%standard-phases) - #~(modify-phases #$phases - (add-before 'configure 'set-cross-headers-path - (lambda* (#:key inputs #:allow-other-keys) - (let* ((mach #+xgnumach-headers) - (cpath (string-append mach "/include"))) - (for-each (lambda (variable) - (setenv variable cpath)) - '#$%gcc-cross-include-paths)))))))) - (propagated-inputs - (list xgnumach-headers)) - (native-inputs - (modify-inputs (package-native-inputs mig) - (prepend xgcc xbinutils))))) + (cross-mig/implementation target xgcc xbinutils)) +(define cross-mig/implementation + (mlambda (target xgcc xbinutils) + "Return a cross-mig for TARGET, where TARGET is a GNU triplet. Use XGCC as +the base compiler. Use XBINUTILS as the associated cross-Binutils." + (define xgnumach-headers + (cross-gnumach-headers target + #:xgcc xgcc + #:xbinutils xbinutils)) (package + (inherit mig) + (name (string-append "mig-cross")) (arguments + (substitute-keyword-arguments (package-arguments mig) + ((#:configure-flags flags #~'()) + #~(list #$(string-append "--target=" target))) + ((#:tests? _ #f) + #f) + ((#:phases phases #~%standard-phases) #~(modify-phases #$phases + (add-before 'configure 'set-cross-headers-path + (lambda* (#:key inputs #:allow-other-keys) + (let* ((mach #+xgnumach-headers) + (cpath (string-append mach "/include"))) + (for-each (lambda (variable) + (setenv variable cpath)) + '#$%gcc-cross-include-paths)))))))) + (propagated-inputs + (list xgnumach-headers)) + (native-inputs + (modify-inputs (package-native-inputs mig) + (prepend xgcc xbinutils)))))) (define cross-kernel-headers/implementation (mlambda (target linux-headers xgcc xbinutils) -- 2.41.0
ludo@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:03 2024 Received: from localhost ([127.0.0.1]:57734 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFn-0003tk-3H for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:03 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:55299) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFh-0003s9-Rb for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:59 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 7D6CA27BBEE for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id d075a7ef for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results. Date: Fri, 5 Jan 2024 16:40:49 +0000 Message-ID: <b46982805417342e13a759fb6ac1826962130550.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Efraim Flashner <efraim@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/rust.scm (make-rust-sysroot): Move code to make-rust-sysroot/implementation. (make-rust-sysroot/implementation): New variable. Change-Id: Ibb30c7398328c87c032bb8828635a34ada935167 --- gnu/packages/rust.scm | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm index ea95d27476..64cde7ea14 100644 --- a/gnu/packages/rust.scm +++ b/gnu/packages/rust.scm @@ -60,6 +60,7 @@ (define-module (gnu packages rust) #:use-module (guix build-system gnu) #:use-module (guix build-system trivial) #:use-module (guix download) + #:use-module (guix memoization) #:use-module (guix git-download) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages) @@ -1057,7 +1058,10 @@ (define-public rust (package-native-inputs base-rust)))))) (define*-public (make-rust-sysroot target) - (let ((base-rust rust)) + (make-rust-sysroot/implementation target rust)) + +(define make-rust-sysroot/implementation + (mlambda (target base-rust) (package (inherit base-rust) (name (string-append "rust-sysroot-for-" target)) -- 2.41.0
efraim@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:03 2024 Received: from localhost ([127.0.0.1]:57730 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFm-0003tU-FI for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:03 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFh-0003sA-Tp for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:59 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 7241827BBED for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:52 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id d5431592 for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 6/7] gnu: Memozise cross-libc results. Date: Fri, 5 Jan 2024 16:40:48 +0000 Message-ID: <64ae83ce361b2883e9bdf4bcdaacb03a4fbd9611.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: 1.7 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/cross-base.scm (cross-libc*): Move code to cross-libc/implementation and call it. (cross-libc/implementation) New procedure. Content analysis details: (1.7 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist [URIs: libc.so] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 T_SCC_BODY_TEXT_LINE No description available. 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.7 (/) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/cross-base.scm (cross-libc*): Move code to cross-libc/implementation and call it. (cross-libc/implementation) New procedure. Change-Id: I72f430136860e5d1fd9edeb3274678186b896bd4 --- gnu/packages/cross-base.scm | 165 +++++++++++++++++++----------------- 1 file changed, 85 insertions(+), 80 deletions(-) diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm index 6c6c6e7636..8a40211456 100644 --- a/gnu/packages/cross-base.scm +++ b/gnu/packages/cross-base.scm @@ -651,10 +651,22 @@ (define cross-kernel-headers/implementation (_ xlinux-headers)))) (define* (cross-libc . args) + "Return LIBC cross-built for TARGET, a GNU triplet. Use XGCC and XBINUTILS +and the cross tool chain. If TARGET doesn't have a standard C library #f is +returned." (if (or (= (length args) 1) (contains-keyword? args)) (apply cross-libc* args) (apply cross-libc/deprecated args))) +(define* (cross-libc* target + #:key + (libc (libc-for-target target)) + (xgcc (cross-gcc target)) + (xbinutils (cross-binutils target)) + (xheaders (cross-kernel-headers target))) + (cross-libc/implementation target libc + xgcc xbinutils xheaders)) + (define* (cross-libc/deprecated target #:optional (libc (libc-for-target target)) @@ -668,88 +680,81 @@ (define* (cross-libc/deprecated target #:xbinutils xbinutils #:xheaders xheaders)) -(define* (cross-libc* target - #:key - (libc (libc-for-target target)) - (xgcc (cross-gcc target)) - (xbinutils (cross-binutils target)) - (xheaders (cross-kernel-headers target))) - "Return LIBC cross-built for TARGET, a GNU triplet. Use XGCC and XBINUTILS -and the cross tool chain. If TARGET doesn't have a standard C library #f is -returned." - (match target - ((? target-mingw?) - (let ((machine (substring target 0 (string-index target #\-)))) - (make-mingw-w64 machine - #:xgcc xgcc - #:xbinutils xbinutils))) - ((or (? target-linux?) (? target-hurd?)) - (package - (inherit libc) - (name (string-append "glibc-cross-" target)) - (arguments - (substitute-keyword-arguments - `(;; Disable stripping (see above.) - #:strip-binaries? #f - - ;; This package is used as a target input, but it should not have - ;; the usual cross-compilation inputs since that would include - ;; itself. - #:implicit-cross-inputs? #f - - ;; We need SRFI 26. - #:modules ((guix build gnu-build-system) - (guix build utils) - (srfi srfi-26)) - - ,@(package-arguments libc)) - ((#:configure-flags flags) - `(cons ,(string-append "--host=" target) - ,(if (target-hurd? target) - `(append (list "--disable-werror" - ,@%glibc/hurd-configure-flags) - ,flags) - flags))) - ((#:phases phases) - `(modify-phases ,phases - (add-before 'configure 'set-cross-kernel-headers-path - (lambda* (#:key inputs #:allow-other-keys) - (let* ((kernel (assoc-ref inputs "kernel-headers")) - (cpath (string-append kernel "/include"))) - (for-each (cut setenv <> cpath) - ',%gcc-cross-include-paths) - (setenv "CROSS_LIBRARY_PATH" +(define cross-libc/implementation + (mlambda (target libc xgcc xbinutils xheaders) + (match target + ((? target-mingw?) + (let ((machine (substring target 0 (string-index target #\-)))) + (make-mingw-w64 machine + #:xgcc xgcc + #:xbinutils xbinutils))) + ((or (? target-linux?) (? target-hurd?)) + (package + (inherit libc) + (name (string-append "glibc-cross-" target)) + (arguments + (substitute-keyword-arguments + `(;; Disable stripping (see above.) + #:strip-binaries? #f + + ;; This package is used as a target input, but it should not have + ;; the usual cross-compilation inputs since that would include + ;; itself. + #:implicit-cross-inputs? #f + + ;; We need SRFI 26. + #:modules ((guix build gnu-build-system) + (guix build utils) + (srfi srfi-26)) + + ,@(package-arguments libc)) + ((#:configure-flags flags) + `(cons ,(string-append "--host=" target) + ,(if (target-hurd? target) + `(append (list "--disable-werror" + ,@%glibc/hurd-configure-flags) + ,flags) + flags))) + ((#:phases phases) + `(modify-phases ,phases + (add-before 'configure 'set-cross-kernel-headers-path + (lambda* (#:key inputs #:allow-other-keys) + (let* ((kernel (assoc-ref inputs "kernel-headers")) + (cpath (string-append kernel "/include"))) + (for-each (cut setenv <> cpath) + ',%gcc-cross-include-paths) + (setenv "CROSS_LIBRARY_PATH" (string-append kernel "/lib")) ; for Hurd's libihash #t))) - ,@(if (target-hurd? target) - '((add-after 'install 'augment-libc.so - (lambda* (#:key outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out"))) - (substitute* (string-append out "/lib/libc.so") - (("/[^ ]+/lib/libc.so.0.3") - (string-append out "/lib/libc.so.0.3" - " libmachuser.so libhurduser.so")))) - #t))) - '()))))) - - ;; Shadow the native "kernel-headers" because glibc's recipe expects the - ;; "kernel-headers" input to point to the right thing. - (propagated-inputs `(("kernel-headers" ,xheaders))) - - (native-inputs `(("cross-gcc" ,xgcc) - ("cross-binutils" ,xbinutils) - ,@(if (target-hurd? target) - `(("cross-mig" - ,(cross-mig target - #:xgcc xgcc - #:xbinutils xbinutils))) - '()) - ,@(package-inputs libc) ;FIXME: static-bash - ,@(package-native-inputs libc))))) - ((? target-avr?) - (make-avr-libc #:xbinutils xbinutils - #:xgcc xgcc)) - (else #f))) + ,@(if (target-hurd? target) + '((add-after 'install 'augment-libc.so + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out"))) + (substitute* (string-append out "/lib/libc.so") + (("/[^ ]+/lib/libc.so.0.3") + (string-append out "/lib/libc.so.0.3" + " libmachuser.so libhurduser.so")))) + #t))) + '()))))) + + ;; Shadow the native "kernel-headers" because glibc's recipe expects the + ;; "kernel-headers" input to point to the right thing. + (propagated-inputs `(("kernel-headers" ,xheaders))) + + (native-inputs `(("cross-gcc" ,xgcc) + ("cross-binutils" ,xbinutils) + ,@(if (target-hurd? target) + `(("cross-mig" + ,(cross-mig target + #:xgcc xgcc + #:xbinutils xbinutils))) + '()) + ,@(package-inputs libc) ;FIXME: static-bash + ,@(package-native-inputs libc))))) + ((? target-avr?) + (make-avr-libc #:xbinutils xbinutils + #:xgcc xgcc)) + (else #f)))) (define* (cross-gcc-toolchain/implementation target #:key -- 2.41.0
ludo@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:01 2024 Received: from localhost ([127.0.0.1]:57727 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFk-0003sv-Tz for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:01 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43064) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFg-0003ry-Fo for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:58 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 4344327BBEA for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id e46cabc2 for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 3/7] gnu: Memozise cross-gcc results. Date: Fri, 5 Jan 2024 16:40:45 +0000 Message-ID: <f65514f80f2a29cd40d6fb172101f31d19b65c04.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/cross-base.scm (cross-gcc): Move code to cross-gcc/implementation and call it. (cross-gcc/implementation) New procedure. Change-Id: Ibeafaa4d652fc1d6fd27870b82a309c177b66a05 --- gnu/packages/cross-base.scm | 183 +++++++++++++++++++----------------- 1 file changed, 95 insertions(+), 88 deletions(-) diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm index a04e4f9c9e..a4e361b476 100644 --- a/gnu/packages/cross-base.scm +++ b/gnu/packages/cross-base.scm @@ -301,6 +301,97 @@ (define (cross-gcc-search-paths target) ,(string-append target "/include"))))))) %gcc-cross-include-paths))) +(define cross-gcc/implementation + (mlambda (target xgcc xbinutils libc) + (package + (inherit xgcc) + (name (string-append "gcc-cross-" + (if libc "" "sans-libc-") + target)) + (source + (origin + (inherit + (package-source xgcc)) + (patches + (append + (origin-patches (package-source xgcc)) + (append (cond + ((version>=? (package-version xgcc) "12.0") + (search-patches "gcc-12-cross-environment-variables.patch" + "gcc-cross-gxx-include-dir.patch")) + ((version>=? (package-version xgcc) "10.0") + (search-patches "gcc-10-cross-environment-variables.patch" + "gcc-cross-gxx-include-dir.patch")) + ((version>=? (package-version xgcc) "8.0") + (search-patches "gcc-8-cross-environment-variables.patch")) + ((version>=? (package-version xgcc) "6.0") + (search-patches "gcc-7-cross-toolexeclibdir.patch" + "gcc-6-cross-environment-variables.patch")) + (else + (search-patches "gcc-cross-environment-variables.patch"))) + (cross-gcc-patches xgcc target)))) + (modules '((guix build utils))) + (snippet + (cross-gcc-snippet target)))) + + (outputs '("out" "lib")) + + (arguments + `(#:implicit-inputs? #f + #:imported-modules ((gnu build cross-toolchain) + ,@%gnu-build-system-modules) + #:modules ((guix build gnu-build-system) + (guix build utils) + (gnu build cross-toolchain) + (srfi srfi-1) + (srfi srfi-26) + (ice-9 regex)) + + ,@(cross-gcc-arguments target xgcc libc))) + + (native-inputs + `(("ld-wrapper-cross" ,(make-ld-wrapper + (string-append "ld-wrapper-" target) + #:target (const target) + #:binutils xbinutils)) + ("binutils-cross" ,xbinutils) + + ,@(let ((inputs (append (package-inputs xgcc) + (fold alist-delete (%final-inputs) + '("libc" "libc:static")) + + ;; Call it differently so that the builder can + ;; check whether the "libc" input is #f. + `(("libc-native" + ,@(assoc-ref (%final-inputs) "libc")) + ("libc-native:static" + ,@(assoc-ref (%final-inputs) + "libc:static")))))) + (cond + ((target-mingw? target) + (if libc + `(,@inputs + ("libc" ,libc)) + `(,@inputs + ("mingw-source" ,(package-source mingw-w64))))) + ((and libc (target-avr? target)) + `(,@inputs + ("libc" ,libc))) + (libc + `(,@inputs + ("libc" ,libc) + ("libc:static" ,libc "static") + ("xkernel-headers" ;the target headers + ,@(assoc-ref (package-propagated-inputs libc) + "kernel-headers")))) + (else inputs))))) + + (inputs '()) + + ;; Only search target inputs, not host inputs. + (search-paths (cross-gcc-search-paths target)) + (native-search-paths '())))) + (define* (cross-gcc target #:key (xgcc %xgcc) @@ -310,94 +401,10 @@ (define* (cross-gcc target XGCC as the base compiler. Use XBINUTILS as the associated cross-Binutils. If LIBC is false, then build a GCC that does not target a libc; otherwise, target that libc." - (package - (inherit xgcc) - (name (string-append "gcc-cross-" - (if libc "" "sans-libc-") - target)) - (source - (origin - (inherit - (package-source xgcc)) - (patches - (append - (origin-patches (package-source xgcc)) - (append (cond - ((version>=? (package-version xgcc) "12.0") - (search-patches "gcc-12-cross-environment-variables.patch" - "gcc-cross-gxx-include-dir.patch")) - ((version>=? (package-version xgcc) "10.0") - (search-patches "gcc-10-cross-environment-variables.patch" - "gcc-cross-gxx-include-dir.patch")) - ((version>=? (package-version xgcc) "8.0") - (search-patches "gcc-8-cross-environment-variables.patch")) - ((version>=? (package-version xgcc) "6.0") - (search-patches "gcc-7-cross-toolexeclibdir.patch" - "gcc-6-cross-environment-variables.patch")) - (else - (search-patches "gcc-cross-environment-variables.patch"))) - (cross-gcc-patches xgcc target)))) - (modules '((guix build utils))) - (snippet - (cross-gcc-snippet target)))) - - (outputs '("out" "lib")) - - (arguments - `(#:implicit-inputs? #f - #:imported-modules ((gnu build cross-toolchain) - ,@%gnu-build-system-modules) - #:modules ((guix build gnu-build-system) - (guix build utils) - (gnu build cross-toolchain) - (srfi srfi-1) - (srfi srfi-26) - (ice-9 regex)) - - ,@(cross-gcc-arguments target xgcc libc))) - - (native-inputs - `(("ld-wrapper-cross" ,(make-ld-wrapper - (string-append "ld-wrapper-" target) - #:target (const target) - #:binutils xbinutils)) - ("binutils-cross" ,xbinutils) - - ,@(let ((inputs (append (package-inputs xgcc) - (fold alist-delete (%final-inputs) - '("libc" "libc:static")) - - ;; Call it differently so that the builder can - ;; check whether the "libc" input is #f. - `(("libc-native" - ,@(assoc-ref (%final-inputs) "libc")) - ("libc-native:static" - ,@(assoc-ref (%final-inputs) - "libc:static")))))) - (cond - ((target-mingw? target) - (if libc - `(,@inputs - ("libc" ,libc)) - `(,@inputs - ("mingw-source" ,(package-source mingw-w64))))) - ((and libc (target-avr? target)) - `(,@inputs - ("libc" ,libc))) - (libc - `(,@inputs - ("libc" ,libc) - ("libc:static" ,libc "static") - ("xkernel-headers" ;the target headers - ,@(assoc-ref (package-propagated-inputs libc) - "kernel-headers")))) - (else inputs))))) - - (inputs '()) - - ;; Only search target inputs, not host inputs. - (search-paths (cross-gcc-search-paths target)) - (native-search-paths '()))) + (cross-gcc/implementation target + xgcc + xbinutils + libc)) (define* (cross-kernel-headers . args) (if (or (= (length args) 1) (contains-keyword? args)) -- 2.41.0
ludo@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:00 2024 Received: from localhost ([127.0.0.1]:57725 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFk-0003si-64 for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:00 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:57531) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFg-0003rx-EC for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:58 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 37E7C27BBE9 for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 8ed7c751 for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 2/7] gnu: Memozise cross-binutils results. Date: Fri, 5 Jan 2024 16:40:44 +0000 Message-ID: <cc0b70d8f909618989aa669c8e9689c466210932.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 In-Reply-To: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> References: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/cross-base.scm (cross-binutils*): Move code to cross-binutils/implementation and call it. (cross-binutils/implementation) New procedure. (cross-binutils/deprecated): Call cross-binutils/implementation. Change-Id: Ic7a493177026c7a699108ab6d75482ff3c189340 --- gnu/packages/cross-base.scm | 92 +++++++++++++++++++------------------ 1 file changed, 48 insertions(+), 44 deletions(-) diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm index 6ee7b315d8..a04e4f9c9e 100644 --- a/gnu/packages/cross-base.scm +++ b/gnu/packages/cross-base.scm @@ -89,56 +89,60 @@ (define (contains-keyword? args) (find keyword? args)) (define* (cross-binutils . args) + "Return a cross-Binutils for TARGET using BINUTILS." (if (or (= (length args) 1) (contains-keyword? args)) (apply cross-binutils* args) (apply cross-binutils/deprecated args))) +(define* (cross-binutils* target #:key (binutils binutils)) + (cross-binutils/implementation target binutils)) + (define* (cross-binutils/deprecated target #:optional (binutils binutils)) (warning (G_ "'cross-binutils' must be used with keyword arguments~%")) - (cross-binutils* target #:binutils binutils)) - -(define* (cross-binutils* target #:key (binutils binutils)) - "Return a cross-Binutils for TARGET using BINUTILS." - (let ((binutils (package - (inherit binutils) - (arguments - (substitute-keyword-arguments (package-arguments - binutils) - ((#:configure-flags flags) - ;; Build with `--with-sysroot' so that ld honors - ;; DT_RUNPATH entries when searching for a needed - ;; library. This works because as a side effect - ;; `genscripts.sh' sets `USE_LIBPATH=yes', which tells - ;; elf32.em to use DT_RUNPATH in its search list. - ;; See <http://sourceware.org/ml/binutils/2013-05/msg00312.html>. - ;; - ;; In theory choosing / as the sysroot could lead ld - ;; to pick up native libs instead of target ones. In - ;; practice the RUNPATH of target libs only refers to - ;; target libs, not native libs, so this is safe. - `(cons "--with-sysroot=/" ,flags))))))) - - ;; For Xtensa, apply Qualcomm's patch. - (cross (cond ((string-prefix? "xtensa-" target) - (package-with-patches binutils - (search-patches - "ath9k-htc-firmware-binutils.patch"))) - ((target-mingw? target) - (package-with-extra-patches - (package-with-extra-configure-variable - ;; mingw binutils does not work correctly when configured - ;; with `--enable-compressed-debug-sections`. An error - ;; like the following will occur whenever you try to link: - ;; - ;; x86_64-w64-mingw32-ld: final link failed: bad value - ;; - ;; TODO: This seems like a deeper problem that warrants - ;; deeper investigation. - binutils "--enable-compressed-debug-sections" "no") - (search-patches "binutils-mingw-w64-timestamp.patch" - "binutils-mingw-w64-deterministic.patch"))) - (else binutils)) - target))) + (cross-binutils/implementation target binutils)) + +(define cross-binutils/implementation + (mlambda (target binutils) + (let ((binutils (package + (inherit binutils) + (arguments + (substitute-keyword-arguments (package-arguments + binutils) + ((#:configure-flags flags) + ;; Build with `--with-sysroot' so that ld honors + ;; DT_RUNPATH entries when searching for a needed + ;; library. This works because as a side effect + ;; `genscripts.sh' sets `USE_LIBPATH=yes', which tells + ;; elf32.em to use DT_RUNPATH in its search list. + ;; See <http://sourceware.org/ml/binutils/2013-05/msg00312.html>. + ;; + ;; In theory choosing / as the sysroot could lead ld + ;; to pick up native libs instead of target ones. In + ;; practice the RUNPATH of target libs only refers to + ;; target libs, not native libs, so this is safe. + `(cons "--with-sysroot=/" ,flags))))))) + + ;; For Xtensa, apply Qualcomm's patch. + (cross (cond ((string-prefix? "xtensa-" target) + (package-with-patches binutils + (search-patches + "ath9k-htc-firmware-binutils.patch"))) + ((target-mingw? target) + (package-with-extra-patches + (package-with-extra-configure-variable + ;; mingw binutils does not work correctly when configured + ;; with `--enable-compressed-debug-sections`. An error + ;; like the following will occur whenever you try to link: + ;; + ;; x86_64-w64-mingw32-ld: final link failed: bad value + ;; + ;; TODO: This seems like a deeper problem that warrants + ;; deeper investigation. + binutils "--enable-compressed-debug-sections" "no") + (search-patches "binutils-mingw-w64-timestamp.patch" + "binutils-mingw-w64-deterministic.patch"))) + (else binutils)) + target)))) (define (cross-gcc-arguments target xgcc libc) "Return build system arguments for a cross-gcc for TARGET, using XGCC as the -- 2.41.0
ludo@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at 68266) by debbugs.gnu.org; 5 Jan 2024 16:41:00 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:41:00 2024 Received: from localhost ([127.0.0.1]:57723 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnFj-0003sO-0f for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:41:00 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:50159) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnFg-0003rw-EA for 68266 <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:40:57 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 2BEF927BBE2 for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:51 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 019e4bde for <68266 <at> debbugs.gnu.org>; Fri, 5 Jan 2024 16:40:50 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> To: 68266 <at> debbugs.gnu.org Subject: [PATCH 1/7] gnu: Memozise make-ld-wrapper results. Date: Fri, 5 Jan 2024 16:40:43 +0000 Message-ID: <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@HIDDEN> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN> Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68266 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) To ensure that it just returns a single package record for some given arguments, as this helps to avoid poor performance of the store connection object cache. * gnu/packages/base.scm (make-ld-wrapper): Move code to make-ld-wrapper/implementation and call it. (make-ld-wrapper/implementation) New procedure. Change-Id: Id6fc805a4a7ffbc5ff0a5174eafcdf2c7c46854d --- gnu/packages/base.scm | 126 ++++++++++++++++++++++-------------------- 1 file changed, 66 insertions(+), 60 deletions(-) diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm index 8b25af6a5e..929bf9f422 100644 --- a/gnu/packages/base.scm +++ b/gnu/packages/base.scm @@ -66,6 +66,7 @@ (define-module (gnu packages base) #:use-module (guix gexp) #:use-module (guix packages) #:use-module (guix download) + #:use-module (guix memoization) #:use-module (guix git-download) #:use-module (guix build-system gnu) #:use-module (guix build-system trivial) @@ -715,68 +716,73 @@ (define* (make-ld-wrapper name #:key wrapper for the cross-linker for that target, called 'TARGET-ld'. To use a different linker than the default \"ld\", such as \"ld.gold\" the linker name can be provided via the LINKER argument." - ;; Note: #:system->target-triplet is a procedure so that the evaluation of - ;; its result can be delayed until the 'arguments' field is evaluated, thus - ;; in a context where '%current-system' is accurate. - (package - (name name) - (version "0") - (source #f) - (build-system trivial-build-system) - (inputs `(("binutils" ,binutils) - ("guile" ,guile) - ("bash" ,bash) - ("wrapper" ,(search-path %load-path - "gnu/packages/ld-wrapper.in")))) - (arguments - (let ((target (target (%current-system)))) - `(#:guile ,guile-for-build - #:modules ((guix build utils)) - #:builder (begin - (use-modules (guix build utils) - (system base compile)) - - (let* ((out (assoc-ref %outputs "out")) - (bin (string-append out "/bin")) - (ld ,(if target - `(string-append bin "/" ,target "-" - ,linker) - `(string-append bin "/" ,linker))) - (go (string-append ld ".go"))) - - (setvbuf (current-output-port) - (cond-expand (guile-2.0 _IOLBF) - (else 'line))) - (format #t "building ~s/bin/ld wrapper in ~s~%" - (assoc-ref %build-inputs "binutils") - out) - - (mkdir-p bin) - (copy-file (assoc-ref %build-inputs "wrapper") ld) - (substitute* ld - (("@SELF@") - ld) - (("@GUILE@") - (string-append (assoc-ref %build-inputs "guile") - "/bin/guile")) - (("@BASH@") - (string-append (assoc-ref %build-inputs "bash") - "/bin/bash")) - (("@LD@") - (string-append (assoc-ref %build-inputs "binutils") - ,(if target - (string-append "/bin/" - target "-" linker) - (string-append "/bin/" linker))))) - (chmod ld #o555) - (compile-file ld #:output-file go)))))) - (synopsis "The linker wrapper") - (description - "The linker wrapper (or @code{ld-wrapper}) wraps the linker to add any + (make-ld-wrapper/implementation name target binutils linker + guile bash guile-for-build)) + +(define make-ld-wrapper/implementation + (mlambda (name target binutils linker guile bash guile-for-build) + ;; Note: #:system->target-triplet is a procedure so that the evaluation of + ;; its result can be delayed until the 'arguments' field is evaluated, + ;; thus in a context where '%current-system' is accurate. + (package + (name name) + (version "0") + (source #f) + (build-system trivial-build-system) + (inputs `(("binutils" ,binutils) + ("guile" ,guile) + ("bash" ,bash) + ("wrapper" ,(search-path %load-path + "gnu/packages/ld-wrapper.in")))) + (arguments + (let ((target (target (%current-system)))) + `(#:guile ,guile-for-build + #:modules ((guix build utils)) + #:builder (begin + (use-modules (guix build utils) + (system base compile)) + + (let* ((out (assoc-ref %outputs "out")) + (bin (string-append out "/bin")) + (ld ,(if target + `(string-append bin "/" ,target "-" + ,linker) + `(string-append bin "/" ,linker))) + (go (string-append ld ".go"))) + + (setvbuf (current-output-port) + (cond-expand (guile-2.0 _IOLBF) + (else 'line))) + (format #t "building ~s/bin/ld wrapper in ~s~%" + (assoc-ref %build-inputs "binutils") + out) + + (mkdir-p bin) + (copy-file (assoc-ref %build-inputs "wrapper") ld) + (substitute* ld + (("@SELF@") + ld) + (("@GUILE@") + (string-append (assoc-ref %build-inputs "guile") + "/bin/guile")) + (("@BASH@") + (string-append (assoc-ref %build-inputs "bash") + "/bin/bash")) + (("@LD@") + (string-append (assoc-ref %build-inputs "binutils") + ,(if target + (string-append "/bin/" + target "-" linker) + (string-append "/bin/" linker))))) + (chmod ld #o555) + (compile-file ld #:output-file go)))))) + (synopsis "The linker wrapper") + (description + "The linker wrapper (or @code{ld-wrapper}) wraps the linker to add any missing @code{-rpath} flags, and to detect any misuse of libraries outside of the store.") - (home-page "https://www.gnu.org/software/guix//") - (license gpl3+))) + (home-page "https://www.gnu.org/software/guix//") + (license gpl3+)))) (define-public %glibc/hurd-configure-flags ;; 'configure' in glibc 2.35 omits to pass '-ffreestanding' when detecting base-commit: 5279bd453f354cbbaafff44e46c6fa03a39bc10a -- 2.41.0
ludo@HIDDEN, guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.Received: (at submit) by debbugs.gnu.org; 5 Jan 2024 16:37:10 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 05 11:37:10 2024 Received: from localhost ([127.0.0.1]:57711 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rLnC2-0003lg-El for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:37:10 -0500 Received: from lists.gnu.org ([2001:470:142::17]:32926) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1rLnC0-0003lG-WC for submit <at> debbugs.gnu.org; Fri, 05 Jan 2024 11:37:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1rLnBr-0004tJ-BC for guix-patches@HIDDEN; Fri, 05 Jan 2024 11:36:59 -0500 Received: from mira.cbaines.net ([212.71.252.8]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1rLnBp-0007mr-SH for guix-patches@HIDDEN; Fri, 05 Jan 2024 11:36:59 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 10CD727BBE2 for <guix-patches@HIDDEN>; Fri, 5 Jan 2024 16:36:55 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id bd3839a5 for <guix-patches@HIDDEN>; Fri, 5 Jan 2024 16:36:54 +0000 (UTC) User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines <mail@HIDDEN> To: guix-patches@HIDDEN Subject: [PATCH 0/7] Memoize packages associated with cross building. Date: Fri, 05 Jan 2024 16:35:25 +0000 Message-ID: <87plyfrb2x.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@HIDDEN; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.1 (/) --=-=-= Content-Type: text/plain This comes from dumping the contents of the store connection object cache after computing derivations targeting i586-pc-gnu, and looking at what packages appear multiple times as cache keys. Christopher Baines (7): gnu: Memozise make-ld-wrapper results. gnu: Memozise cross-binutils results. gnu: Memozise cross-gcc results. gnu: Memozise cross-kernel-headers results. gnu: Memozise cross-mig results. gnu: Memozise cross-libc results. packages: rust: Memoize make-rust-sysroot results. gnu/packages/base.scm | 126 +++--- gnu/packages/cross-base.scm | 804 +++++++++++++++++++----------------- gnu/packages/rust.scm | 6 +- 3 files changed, 486 insertions(+), 450 deletions(-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWYMCZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XewgBAAj9aTBQ3J1mNMlCZ0aFsIWTxOEQFusvol V3IOeUboC8w1Oe2wxGZtgtQBsArcZivfo8Wt9gSsL1z0o6d3i/m88/6NpIBwaAMq hhP3f2b8zjngI+YYY3Tz6oEzG5GUGPLSQSb31bnPnmGEfkFTracvdb1r0/gshm9X sLy+RaSy5z7UMLaSELEBH5G/a2GfRv++2WGYasftCKMludVFw99db9XkhsA8SO6b TIH3bMNHapr1B43gRqc3AstxuAXjgXyPCMrJq/8gFezIoqjTdi+BOLLn0xk3G0nZ WZoqPRNJdWfp5NPmYinLpU+ylUYCicXhxgWRRbfMjP6h+F1zmGZezI9E+h74m26G /gQotLAQi/Y74jezsWcaPWEByB9Omouv+LncTClF26J5EB8ZaIf/sG19J/pggOWJ 5AY+hHDzgzZsME8UhCU5TvMXz+0brV39cHvOSJYQPKIP7ml9RtWe6GumhYTZdUrw ClQGm5DycXXuFY10NUX+Ms7oEEOfYzWoWMIV/Q+T2MSTaeii9TYFutjG4OJDVzkx Pn76Ik18eRWhgaNY4jJ9TsBTaZeU+3zbxoFYnRSLgsS7Xal6VAbZGmZ9qHciBwN2 9ktBORdciRnjP2b2venuJ27PqGu2K8nxrZXLvYcwye1HLk1FF5xf6Kk623nKTOqT 6yzwOXCRZd0= =ElRU -----END PGP SIGNATURE----- --=-=-=--
Christopher Baines <mail@HIDDEN>
:guix-patches@HIDDEN
.
Full text available.guix-patches@HIDDEN
:bug#68266
; Package guix-patches
.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.