GNU bug report logs - #65866
[PATCH 0/8] Add built-in builder for Git checkouts

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: Ludovic Courtès <ludo@HIDDEN>; Keywords: patch; dated Mon, 11 Sep 2023 14:25:01 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 14:04:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 10:04:48 2023
Received: from localhost ([127.0.0.1]:46360 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkmCa-0002Iy-6f
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:47 -0400
Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:40252)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qkmC1-0002HK-R9
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:14 -0400
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-4053e6e8ca7so15908215e9.1
 for <65866 <at> debbugs.gnu.org>; Mon, 25 Sep 2023 07:03:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695650631; x=1696255431; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=CSbztDokeJkNLVIljXqWkC6PbS1wDJHjSafZ8y+Rebg=;
 b=dQTAy8DkdSXfqomdpT7Nxnk+WWoZ3igDb/Ec4gs+gx7g4PD/xyS7S3cXaTZXa+0o5a
 L4KFSbr2LPb3xQkhxs9HfDqKPLH8+pOqDFGYOFko5AUUHQIVU5IaYbKiAh6Z/bRpS5Sl
 uP+S8cfcjLK1cVDvPbBjqG7UolTNovDkwi9ZlteQr7w6gC8yrQXG8g5QfOAr6A0alsMw
 YKCY881y+Ld6fDiQ29e+3jfq00XByFFCeHGCY7H17d6S9kYt6KCCdnAT/I3nO3yT1jFV
 zFsE/OdpAUjOxw/VXxCRbMAxsweE8ERQLUQRjXd2L5mFQy+K4dj05b078SSbR+V/jh7C
 4Otg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695650631; x=1696255431;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=CSbztDokeJkNLVIljXqWkC6PbS1wDJHjSafZ8y+Rebg=;
 b=QyiT6h/jEyoC6Bcp6PVSYIK33ttgFzUAMSa1ETRMKSY6DEU2LRBY0zG7yK73grk4mQ
 Rn9e56UowXv4iRz/f3DmfnALJvkCq40S2f3fXXUgi6O4VecYbpjr83QWqNzcCUcSLBEB
 3FsJEVZl6aa2dsd75mlitta8Tv/RozmhcEuvXOKlJWN0U3akxy7U90r6DrBhY+wH3QxM
 WsZsy5qYWf2urNl/SUifn8ocBUiPUpMd0tSqWWzzzpF5Oq1RNhZ9TFWks4kFE4ZU7z4c
 Plcx04CIvki9oMP6Csy2JA7U2MyGg3e5aJeoZqd0Dkeiam+I6qk2xC18IWdMW03Uz5be
 EsjA==
X-Gm-Message-State: AOJu0Yx0SzrUNiWRbZ8KBkDsJtqewwJbpKtsB57K8a3oavMjmkJAfm9A
 YeDtgBoN3TfV9cRzJauqmAI=
X-Google-Smtp-Source: AGHT+IGb0+s78NNpgbS9uy+z/1JyfSjBUOIK+LEY8uX2Bs2n5X2M9T5R4/EASITaEfcZ1UXizqmtRg==
X-Received: by 2002:a05:6000:1c7:b0:317:3d36:b2c1 with SMTP id
 t7-20020a05600001c700b003173d36b2c1mr6490711wrx.7.1695650631329; 
 Mon, 25 Sep 2023 07:03:51 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 z8-20020adfd0c8000000b00317a29af4b2sm12005738wrh.68.2023.09.25.07.03.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 25 Sep 2023 07:03:51 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 65866 <at> debbugs.gnu.org
Subject: Re: [bug#65866] [PATCH v2 5/8] build: Add dependency on Git.
In-Reply-To: <8ec588483525275aac1eb057543a903bea707ead.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <8ec588483525275aac1eb057543a903bea707ead.1695421391.git.ludo@HIDDEN>
Date: Mon, 25 Sep 2023 15:59:17 +0200
Message-ID: <87sf724ami.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Sat, 23 Sep 2023 at 00:28, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
> * configure.ac: Check for =E2=80=98git=E2=80=99 and substitute =E2=80=98G=
IT=E2=80=99.
> * guix/config.scm.in (%git): New variable.
> * guix/self.scm (compiled-guix): Define =E2=80=98git=E2=80=99 and pass it=
 to
> =E2=80=98make-config.scm=E2=80=99.
> (make-config.scm): Add #:git; emit a =E2=80=98%git=E2=80=99 variable.
> * doc/guix.texi (Requirements): Add it.

Moving the Git dependency to a daemon dependency tweaks a bit what we
control when =E2=80=9Cbootstrapping=E2=80=9D, no?  Maybe I misread or misun=
derstand a
point.

Currently, the full bootstrap story requires the binary seed (well
documented in the manual :-)), running a Linux kernel and a Guix daemon.
And then, everything is built one after the other, resolving the chain
of dependencies.

And obviously, there is another chicken-or-the-egg problem barely
discussed.  For building something, we first need to communicate with
the world for fetching the source code to build. :-)

It is not Git specific and also happens with =E2=80=99url-fetch=E2=80=99 =
=E2=80=93 as reported
very early (see #22774 from 2016).  For instance, TLS is required before
Guix has built GnuTLS.  The chicken-or-the-egg had been solved by hiding
this dependency as a dependency of the daemon.  A hack.

Therefore, it means that the Reduced Binary Seed bootstrap is somehow
enlarged by what the Guix daemon depends on.

I mean, breaking the dependency cycles by introducing a dependency on
Git (as for fixing #63331) is not free of other dependencies.  It is
another hack when it could be avoided since the issue is about
Guile-GnuTLS.

It means we are doing a step back for a full bootstrap story, IMHO,
having a =E2=80=9Cbuiltin:git-download=E2=80=9D makes less clear what are t=
he hard
dependencies to what Guix is able to build from source starting from
almost nothing.

Cheers,
simon






Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 14:04:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 10:04:12 2023
Received: from localhost ([127.0.0.1]:46356 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkmC4-0002I0-8Q
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:12 -0400
Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:40103)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qkmC0-0002HH-Si
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:10 -0400
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40471c054f9so20547935e9.0
 for <65866 <at> debbugs.gnu.org>; Mon, 25 Sep 2023 07:03:56 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695650631; x=1696255431; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=bJ+iocxi13fD3PtIDOeWZLCoZBUMcBeZr1abisfsMmY=;
 b=EAx04Xil1s3A++vx1E2EQ1ajkeHlhT7aqrehPxu6x0s7wUfoQ1SnpO7mZN6+tn724/
 zQn52aeCMf+o7J3GRzJIJdqKfjkv3eJZked7/tFaVdPUIZDbVLxRFAC1dwTBxPlP2kIP
 ki7AJlOrfm8xA6pNZBZWctZaOwkG7PX+MMkHaJChXjrCdh6oBsXxlftJPPjjBUa38+tX
 ozD3Qz9Yjh9jMlZHNwNvRC6O0kAbVGOAHivYaYRvb1dEM9ruQ4oYU6Yy9XceDBVIRvkK
 gaizL4/Y8EY1b/BuTKpx1mcD269D4orlUciP7bwrIaSMlEy5zTbW6BIK0bVAx6klB5mm
 3lvA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695650631; x=1696255431;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=bJ+iocxi13fD3PtIDOeWZLCoZBUMcBeZr1abisfsMmY=;
 b=BCBXScu2Lrid2eSx4r94LucUdi2PTO4vL8w7iofvDnNISDgmzRvheewXBHtMGj+vVy
 fkZboxmhYpFIUzeBKYUZgdKpUXLSTQvK9BZgA7zRoJfkdxrRvvExTPVddX7H4FbqDOj6
 AkTscaQlvtTz0sByJZjOMSiL0M00C6y1bSqTEIhfT1BM0j4Xq9c8E4C9sQ2EajeDNbbK
 o1gAyBqaGNjltBsPawpTeU8jwTK6ABZ0AejRwf5JtYNVyCWZHrCMLykiZKkYOc72B1q1
 mbjxQhdk1KJW/+EYUPJfXVfdeb3ono08M69Pom3M9IPwp6rJ6TVfxIw4ay/SE5TLZkRv
 1V8A==
X-Gm-Message-State: AOJu0YxSpMqyNF8NZh56KI9ulUcRQXqI68ZA7F+VPrTXPNaxLWzkfkgv
 nkd60PiSRCkcV3E3VHREzu27LupJo4A=
X-Google-Smtp-Source: AGHT+IG8r/62kCwFB+3QtkyvrQHtQrvI4TI4HgKkFXxTPKlZ+vLLUyu6vRysXtxPh8ZnUDJjdjuWxw==
X-Received: by 2002:a05:600c:4797:b0:3ff:8617:672b with SMTP id
 k23-20020a05600c479700b003ff8617672bmr6007657wmo.2.1695650630490; 
 Mon, 25 Sep 2023 07:03:50 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 m13-20020a7bcb8d000000b003ff3b964a9asm15225576wmi.39.2023.09.25.07.03.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 25 Sep 2023 07:03:50 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts
In-Reply-To: <87zg1a7gjd.fsf_-_@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <b0a3a92d6adda6582cd3aa61d3be30a90d91c96b.1695421391.git.ludo@HIDDEN>
 <87a5ta7iua.fsf@HIDDEN> <87zg1a7gjd.fsf_-_@HIDDEN>
Date: Mon, 25 Sep 2023 14:48:04 +0200
Message-ID: <874jji5shn.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Re,

On Mon, 25 Sep 2023 at 11:23, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

>>> +  "Return a fixed-output derivation without any dependency that perfor=
ms a Git
>>> +checkout of REF, using the \"builtin:git-download\" derivation builder=
."
>>
>> I do not understand what means =E2=80=9Cwithout any dependency=E2=80=9D =
here.
>
> It means the derivation does not depend on anything (it has zero
> =E2=80=9Csources=E2=80=9D and zero =E2=80=9Cinput derivations=E2=80=9D). =
 That=E2=80=99s fundamental here, which
> is why I made it explicit.

For instance, in =E2=80=99built-in-download=E2=80=99, the docstring reads:

--8<---------------cut here---------------start------------->8---
This is an \"out-of-band\" download in that the returned derivation does not
explicitly depend on Guile, GnuTLS, etc.  Instead, the daemon performs the
download by itself using its own dependencies.
--8<---------------cut here---------------end--------------->8---

which I find =E2=80=9Cclearer=E2=80=9D than =E2=80=9Cwithout any dependency=
=E2=80=9D.

Well, enough for nitpicking. :-)

Cheers,
simon





Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 14:04:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 10:04:12 2023
Received: from localhost ([127.0.0.1]:46354 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkmC3-0002Hu-Tc
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:12 -0400
Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]:39184)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qkmC0-0002HF-3R
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:10 -0400
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40594a6d1ecso3187325e9.1
 for <65866 <at> debbugs.gnu.org>; Mon, 25 Sep 2023 07:03:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695650630; x=1696255430; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=vwRCDT1m5+vVh35VrdEws4DAQlShzV6zImUK+Wys9Zw=;
 b=OUKPGqodRZ1n/kqz3m1iXcaov7e4W+an9xnB22xIpz/URMl32BGwvcGkj3ZTo1o0gJ
 1w2+6lL19+t0tBxeKnMJ7RQNHvnsd7XbA4Z92NLTZ9vM7gbdq0BQad+vHwqvmyffAINL
 9zQOxOhPi6aB4VEfVWY6k1pXTG2uFBOpKmpvJ0vsg7cQn+UzawD3l+Sn7kGo/S3fdc3a
 oUoU/tC5gLwBVl3ffJtkSmHYfNPs/Vb2NQjkg3JbjV/xU3cSSDLfZk4YJcLOe91N/pl/
 8HVswE0PaDOhqySfu4x3M7JtYNqni9U3tEXwNVtWN0rXuIjK/N5MWNhw3PLCOn8UKvrB
 OCGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695650630; x=1696255430;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=vwRCDT1m5+vVh35VrdEws4DAQlShzV6zImUK+Wys9Zw=;
 b=DDcm42dArpA6u3fPrPgmz80+YcTw7tUMinxC7b0XjqUujKKeHaJqQYLG8vilMa4Vmu
 mZsH4vnyGYQkApcJIWAVGovwc4mw6BqsMIVrK2L4DourrJjqyppAoCH4M34HU8cOR/5Y
 kiJgU14SAmsQHFi1CJ7iJG8yqxETB/Rq1lfz2OzcCkc43h12UdQhFs1UMnVG7JVxiCcZ
 dPaq5uKuTOzZZHjjUxDoRdEwpr79dVfxDutFXt7OIeQJ0XTScVey98dTjhsvlD8bxbaA
 VfGD97R9egYxTXK0Tl6I/W0yunikrNg5LgE7KwA3k54MzbuioNCHjHbWjeJ8MKEUPwv1
 ZtDg==
X-Gm-Message-State: AOJu0YyMF5SXdoqtZThEaEOyeTbndLV6NFE8iGHcsPgXfQcIdv/s4wel
 ze3omQiQ/UrYsaEv9gPbg4U=
X-Google-Smtp-Source: AGHT+IGPHngZJFlfvv+6elYnAUB86LDpkFb+OpUtQjsrkXSr9LBtI3sBQ5DWWjZEp9iDSZHRJ8RCEQ==
X-Received: by 2002:a05:600c:1c2a:b0:404:7606:a871 with SMTP id
 j42-20020a05600c1c2a00b004047606a871mr6475351wms.2.1695650629566; 
 Mon, 25 Sep 2023 07:03:49 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 c10-20020a5d4f0a000000b0031f3ad17b2csm11955954wru.52.2023.09.25.07.03.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 25 Sep 2023 07:03:49 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts
In-Reply-To: <87zg1a7gjd.fsf_-_@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <b0a3a92d6adda6582cd3aa61d3be30a90d91c96b.1695421391.git.ludo@HIDDEN>
 <87a5ta7iua.fsf@HIDDEN> <87zg1a7gjd.fsf_-_@HIDDEN>
Date: Mon, 25 Sep 2023 14:37:17 +0200
Message-ID: <87bkdq5szm.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On Mon, 25 Sep 2023 at 11:23, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
> Simon Tournier <zimon.toutoune@HIDDEN> skribis:
>
>> On Sat, 23 Sep 2023 at 00:28, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>>
>>> +(define* (git-fetch/built-in ref hash-algo hash
>>> +                             #:optional name
>>> +                             #:key (system (%current-system)))
>>> +  "Return a fixed-output derivation without any dependency that perfor=
ms a Git
>>> +checkout of REF, using the \"builtin:git-download\" derivation builder=
."
>>
>> I do not understand what means =E2=80=9Cwithout any dependency=E2=80=9D =
here.
>
> It means the derivation does not depend on anything (it has zero
> =E2=80=9Csources=E2=80=9D and zero =E2=80=9Cinput derivations=E2=80=9D). =
 That=E2=80=99s fundamental here, which
> is why I made it explicit.

Aaah :-)  Yeah that makes sense.

Well, from my understanding, =C2=AB fixed-output derivation without any
dependency =C2=BB is not clear at first (we are always dependent on somethi=
ng
else ;-)) and I am sure that weeks or months later, I will not remember
that it means: =C2=AB zero =E2=80=9Csources=E2=80=9D and zero =E2=80=9Cinpu=
t derivations=E2=80=9D =C2=BB.

Hum, once it is known, it is hard to find a better wording though. ;-)

I do not know if it is better, at least, it appears clearer from my
point of view:

      "Return a fixed-output derivation that performs a Git checkout of REF,
    using the \"builtin:git-download\" derivation builder, thus without any
    dependency other than from builtin."


Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 14:04:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 10:04:11 2023
Received: from localhost ([127.0.0.1]:46352 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkmC3-0002Hq-HV
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:11 -0400
Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]:37913)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qkmBz-0002HD-3A
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 10:04:09 -0400
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-405d70d19bcso209155e9.0
 for <65866 <at> debbugs.gnu.org>; Mon, 25 Sep 2023 07:03:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695650629; x=1696255429; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=af414lC9IjnF3z/dlz62yytq7zJtWYzrtNo4VcbAWS4=;
 b=G8Sl2JKmgq9avFbMKlo6hUkQYVSGhOgYxWpHLWcxrLO+FUX5D2/oxoURpLsmNl4J2y
 ZmWAIikAIzABdHBXfqAoaPUJHYZF0nEYgah00fvy5jfvTgI+LNRSPl392Ysz/L4wlZrS
 k1P3apta2+P+dn5D9G9qaBLRUUQcFp6OfTrQHCOFn93+VemFb9UliW8t8vcgvUPEOxV6
 yOxyRanr/4LEDTjN4BYcSihFauyXEKZFVK89PMCRfRxt9k3/hGkZVhstDmKR9qL/XgU5
 B6Os3jVxwKR+yCvDJ7LSd7RIiNmqOxmvmeTtOFYrn9itkPyYjYCibyFIhrch7s1MgWhT
 dIMA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695650629; x=1696255429;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=af414lC9IjnF3z/dlz62yytq7zJtWYzrtNo4VcbAWS4=;
 b=XYYAS1OUAoeXTL+HJR82muuc3GZVOdx34qJmoyck4yQ1JFhgGQzXIO226CXlp1S2rC
 o5EzPLVl05FkojtkbdGIEG1KsRheNUn3N0bzRpYRtnOpgGD4Mi8c0+ItOkne7ud5a5w+
 TVRyMJxt6c6u9Iny3PoufKMNQAl4p2Nz6B4/TQ5wKr3zLmTMqS1qm+AXvhtJa/JHUpaR
 ETu1yWgF/3fLi5TNMlOcGrZR9p3r7P0C4FNeIiH5HPA6lukwsb0MtGIsT0uWJaYMzC+N
 Ktq4VP0w/vXpe9WcyrVOAzjXkvZHmeUcchY8uPGIxuXDVB+hlDMkA7/E3TvLWon8AN/Y
 7TvQ==
X-Gm-Message-State: AOJu0YzO1h3viil0LfdN0lIukoHFLa8H2UJNOyl8niaPwJe398cqayhT
 1huGwQtE0AHUsgC7GXAM5Ug=
X-Google-Smtp-Source: AGHT+IHy0P6K6a4RgxyGwve5wcwifSnx3AVpxjejQam/WycP1/kDiIe5p1HZRWl0pGlINd1mVEXt5A==
X-Received: by 2002:a05:600c:4a12:b0:404:75cc:62e6 with SMTP id
 c18-20020a05600c4a1200b0040475cc62e6mr5687175wmp.3.1695650628526; 
 Mon, 25 Sep 2023 07:03:48 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 q25-20020a7bce99000000b003fefcbe7fa8sm105205wmj.28.2023.09.25.07.03.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 25 Sep 2023 07:03:47 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts
In-Reply-To: <87ttri7gha.fsf_-_@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <765792320d841e1e33a11f119ad687094ca2766f.1695421391.git.ludo@HIDDEN>
 <87edim7jom.fsf@HIDDEN> <87ttri7gha.fsf_-_@HIDDEN>
Date: Mon, 25 Sep 2023 14:13:46 +0200
Message-ID: <87il7y5u2t.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On Mon, 25 Sep 2023 at 11:24, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

>> If raising a warning here is not appropriate, maybe document in the
>> docstring the returned value as #f.
>
> I agree, but let=E2=80=99s discuss that separately from this review if yo=
u don=E2=80=99t
> mind (the code here is not new).

Since this change introduces a new procedure publicly exposed, maybe a
line in the docstring about the =E2=80=9Ccontract=E2=80=9D would help.

--8<---------------cut here---------------start------------->8---
  "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall back to
alternative methods when fetching from URL fails: attempt to download a nar,
and if that also fails, download from the Software Heritage archive;
recursive checkouts are not supported when falling back to Software
Heritage, return #false if all methods fail."
--8<---------------cut here---------------end--------------->8---

Well, if that=E2=80=99s inappropriate, let address it separately. :-)

Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 09:24:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 05:24:56 2023
Received: from localhost ([127.0.0.1]:44212 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkhpo-0008Oz-9X
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 05:24:56 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:52672)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qkhpm-0008Ol-Hv
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 05:24:54 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qkhpU-0006OL-2f; Mon, 25 Sep 2023 05:24:36 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=SwQiVdubWzktFMpdcMkShTbf+GUTrNtv8UtbhpQAO0Q=; b=gB9xzgfWew8x2tt4NyZI
 tfWwnkDcAdomrzRoSDs9oD4NpJLJK0cGFggctzo6NSc35G3W0qKwUbvxMwHU6B9qCObxEm5Qu+KCR
 5ju7M0yCTaTwBkoPR8D3aKf2SReHpkuetblDyJQPxnbTquhbHnB1g4KtElC3v5ZaAeAR8b7a72NzS
 HHeQL+7OHoqknkSDDS30aWevyO+0CW3RxGAFYxnBDubnLCkxAi99dmMsprDc7kiGL4ml0oU4ZH2FD
 D+InPo9cbXT2kwcwwS2naUw/4FLn8UFSGJy5c/vMnnHm4jpA5Oc7u6g9J0kBg4JOj2NS+nBR8B3Hi
 9ixp8JWhofo9YQ==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Simon Tournier <zimon.toutoune@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <765792320d841e1e33a11f119ad687094ca2766f.1695421391.git.ludo@HIDDEN>
 <87edim7jom.fsf@HIDDEN>
Date: Mon, 25 Sep 2023 11:24:33 +0200
In-Reply-To: <87edim7jom.fsf@HIDDEN> (Simon Tournier's message of "Mon, 25
 Sep 2023 10:15:21 +0200")
Message-ID: <87ttri7gha.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Simon Tournier <zimon.toutoune@HIDDEN> skribis:

> On Sat, 23 Sep 2023 at 00:27, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>
>> +      ;; XXX: Currently recursive checkouts are not supported.
>> +      (and (not recursive?)
>
> Naively, and similarly as Maxim=E2=80=99s remark, maybe we could raise a =
warning
> when the case is not supported instead of silently return #f.
>
> If raising a warning here is not appropriate, maybe document in the
> docstring the returned value as #f.

I agree, but let=E2=80=99s discuss that separately from this review if you =
don=E2=80=99t
mind (the code here is not new).

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 09:23:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 05:23:45 2023
Received: from localhost ([127.0.0.1]:44207 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkhoe-0008NA-Si
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 05:23:45 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:51246)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qkhoc-0008Mx-EL
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 05:23:43 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qkhoJ-0006G3-N3; Mon, 25 Sep 2023 05:23:23 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=AngKLiG5JPvpkaSQN1vCRs9CtS269mRyWlDbSsXaV+o=; b=l87OdVtDrmI6mZavvrpN
 FnNAc9DWVk4T3Wlp4g7d0atPdZ/WRJMnMI1BzqFwY8e6mAYXl+ZEdgWGPmDTnUSOV2/rSBsp471h0
 XDXMqdwuy+wmQltzgwvbzISsMbHueXwseC1/x2+OiO5SYiku7/ZXdK/MQq8pwyhIYqwNJgrL4MzaG
 XDPlJJpO/jTXbZ4RqwgyxBQciIfZu/7iazOcm7rGUVY8CA6tXMxsgBYJsZ5V0L79+LZ6ExYmDhiuf
 YJdSt4DK7at58vN8fCy0+HBIIcEPuz4kGI7/XYm14BacUZeoZP+CKqk8JfAQPsaCPgsFTkz1wBnte
 EngROu28MuFRRg==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Simon Tournier <zimon.toutoune@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <b0a3a92d6adda6582cd3aa61d3be30a90d91c96b.1695421391.git.ludo@HIDDEN>
 <87a5ta7iua.fsf@HIDDEN>
Date: Mon, 25 Sep 2023 11:23:18 +0200
In-Reply-To: <87a5ta7iua.fsf@HIDDEN> (Simon Tournier's message of "Mon, 25
 Sep 2023 10:33:33 +0200")
Message-ID: <87zg1a7gjd.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Simon Tournier <zimon.toutoune@HIDDEN> skribis:

> On Sat, 23 Sep 2023 at 00:28, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>
>> +(define* (git-fetch/built-in ref hash-algo hash
>> +                             #:optional name
>> +                             #:key (system (%current-system)))
>> +  "Return a fixed-output derivation without any dependency that perform=
s a Git
>> +checkout of REF, using the \"builtin:git-download\" derivation builder."
>
> I do not understand what means =E2=80=9Cwithout any dependency=E2=80=9D h=
ere.

It means the derivation does not depend on anything (it has zero
=E2=80=9Csources=E2=80=9D and zero =E2=80=9Cinput derivations=E2=80=9D).  T=
hat=E2=80=99s fundamental here, which
is why I made it explicit.

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 08:42:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 04:42:50 2023
Received: from localhost ([127.0.0.1]:44126 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkhB3-0007GR-QO
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 04:42:50 -0400
Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]:40139)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qkhAk-0007FC-Pj
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 04:42:31 -0400
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-40471c054f9so19041805e9.0
 for <65866 <at> debbugs.gnu.org>; Mon, 25 Sep 2023 01:42:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695631333; x=1696236133; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=fKiDRIZd2ylu12fAhgY35khWKZe7hlXVGSINHOpBFeo=;
 b=CMAEE1D8GZvOyvmGuR1yDm0D9ct4fqHoBigt0I7NdieKpAzmNXJdKMHbM8TOu7Z7FQ
 /POM54LdDEM7dXWxKAG7njVtFXInPqJTmevW2R4uSwtt42QuPVqYAmMaGmWFJ8aoGtj+
 ZDbRC6rbqbaCaZw/wJFwdmvNvocGlsChqg85ucCLm5cnsFz0pdl9zCOIhYcaXGla1P0A
 Umde07pkqR5jq7c8ojgcR6DuH2OOntnyuiXBK899QA/LKCGaoWrksbNmqHq0TgWJN0Ml
 C2yvHvM7w7cfKAlKEerHY+MYD/CtOpRQ2nXU1PbZf3u1YTbnNhbXbLSbvunvgiwcCNs6
 razw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695631333; x=1696236133;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=fKiDRIZd2ylu12fAhgY35khWKZe7hlXVGSINHOpBFeo=;
 b=kIf0jDTNMYKHB56kn1ORT57reDucM4sAtG+9w/ZT+21lUfZpRtyrtuZJvSgRhI2xvW
 Os4YTJ1Rr9tW0/nceAK9l9rMTHvhn/zhkUVXnRkliTfSPB7B5e/2FrzFxpL7CnPDwK53
 B1aBj/6uz38dDcv7Ku/Kv7sE3acQHXwPnkArG8dmtR0DYTW6Yl/aOk1j+3obMTtyonjk
 6zycIJgVy2dnv70WcbFEe34YhyRiC4/S3XyqScQKs05Zr9rVtdsfyYfcDjSjMu7Cd0JA
 fBDGSpL9DUrpngKMRDio736hyfK/vIA5JpKBinJRKfXlgIcglPDtZjC3aQT7fW6S4kcs
 M7hw==
X-Gm-Message-State: AOJu0YyJDUpi3scZ3Ti4f0qIDkMSHwJlBPb6u5Wf/ngRl5ZednkxCN0m
 PpKX6M7Bm5/OPmqHPVUC7eNXfn/3K5k=
X-Google-Smtp-Source: AGHT+IFGdskWjj0eDfv+DDB6vZVFfVAmrozhoW0bmD9p9ZfG7YsENvMjV4baI+9n27lIj8OSK6uE0g==
X-Received: by 2002:a5d:46d0:0:b0:316:ef5f:7d8f with SMTP id
 g16-20020a5d46d0000000b00316ef5f7d8fmr5052225wrs.3.1695631332767; 
 Mon, 25 Sep 2023 01:42:12 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 w12-20020a5d608c000000b003179d5aee67sm11230692wrt.94.2023.09.25.01.42.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 25 Sep 2023 01:42:12 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 65866 <at> debbugs.gnu.org
Subject: Re: [bug#65866] [PATCH v2 7/8] git-download: Use
 =?utf-8?Q?=E2=80=9Cbuiltin=3Agit-download=E2=80=9D?= when available.
In-Reply-To: <b0a3a92d6adda6582cd3aa61d3be30a90d91c96b.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <b0a3a92d6adda6582cd3aa61d3be30a90d91c96b.1695421391.git.ludo@HIDDEN>
Date: Mon, 25 Sep 2023 10:33:33 +0200
Message-ID: <87a5ta7iua.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Sat, 23 Sep 2023 at 00:28, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> +(define* (git-fetch/built-in ref hash-algo hash
> +                             #:optional name
> +                             #:key (system (%current-system)))
> +  "Return a fixed-output derivation without any dependency that performs=
 a Git
> +checkout of REF, using the \"builtin:git-download\" derivation builder."

I do not understand what means =E2=80=9Cwithout any dependency=E2=80=9D her=
e.  I would
drop it.

Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 25 Sep 2023 08:42:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 25 04:42:31 2023
Received: from localhost ([127.0.0.1]:44122 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qkhAl-0007Fr-Gt
	for submit <at> debbugs.gnu.org; Mon, 25 Sep 2023 04:42:31 -0400
Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:40393)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qkhAj-0007FB-UA
 for 65866 <at> debbugs.gnu.org; Mon, 25 Sep 2023 04:42:30 -0400
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-32163c3ece5so584898f8f.1
 for <65866 <at> debbugs.gnu.org>; Mon, 25 Sep 2023 01:42:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695631332; x=1696236132; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=XjP9SZTDm+8QhGSODKIH3+xx0ng6cc7UfGZv44cPEa0=;
 b=IMGxRJNMHrYGeLiUGP5BQJyEH3isK3hmMGChND87cnLAQuftwV6NVVQZmH4l29RyNF
 MbqQvorGHlkEAVhjhqV2NY6u2ThLcIle0Q2sy9mxi97+Na6l42cEdYpwLufoOg+OLHXV
 47NG6+15WJ5Y/klrYaTGLmN+iOPC+mYf02v2W5eoRkidnbA9sbd4RHIEn0xFZiJgg+SP
 tF+lxfvadvisCo03S+mbOoVBMQrzDowarWgIrQZHx9acSfho5r5aYcJjO6j9To8BSBll
 DVGjfnw6BTUVyt1WVYmkUuXIO7iwaSu+pDWMynFN0Bsw8FdyBPWWAsyBQ2+7xsJ6wyHE
 K0+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695631332; x=1696236132;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=XjP9SZTDm+8QhGSODKIH3+xx0ng6cc7UfGZv44cPEa0=;
 b=i683GiCkYV7RFAeKNuDUwtRwulJxK3FSU1cl9qMyJQIyN+NpTaPVZQkv6DKwIwg4Ix
 xsAVfJDSltN+qpeH8HgF49WqbbK47nsB9C81C8D7cxZ3dk8AbbWGkIlKIHdd2tIbqP67
 HK0Ok0JGMnSXITWnTzPh0mUkli3O5WnE3ZbeMCvthUJ7jMv+mPBsrOAX3qVDs51/eS4X
 Ekp3XXgKLXTmqCB068QjMcZYkW/0WvCu74UDwv+/ynUwbq55uq8+AWV3IzG7SgfImk7i
 VWcigvCQ5qoCW2okZvRxdVvYSD0NtqAfqgKiMYnW/fIpC3tuNKrerumFrwpPmtPDo52z
 W5mw==
X-Gm-Message-State: AOJu0YwTKROV8Fyh4xmlab2rAVxMf1uQLf+UpE72fnfbOWtIr4zcc7b8
 EPqrvXJJjGZNr0KctcZn5LK0mGXEbXQ=
X-Google-Smtp-Source: AGHT+IGID9KNTAYk1iisJ2JKzqS6ZbR4of7pgbUugXIHVB2y/PQ6Jkvxj+sX7XKLlDo5TELIlUmVzQ==
X-Received: by 2002:a5d:46d0:0:b0:316:ef5f:7d8f with SMTP id
 g16-20020a5d46d0000000b00316ef5f7d8fmr5052197wrs.3.1695631331744; 
 Mon, 25 Sep 2023 01:42:11 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 n11-20020a5d4c4b000000b0031fbbe347ebsm11141843wrt.22.2023.09.25.01.42.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 25 Sep 2023 01:42:11 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 65866 <at> debbugs.gnu.org
Subject: Re: [bug#65866] [PATCH v2 1/8] git-download: Move fallback code to
 (guix build git).
In-Reply-To: <765792320d841e1e33a11f119ad687094ca2766f.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
 <765792320d841e1e33a11f119ad687094ca2766f.1695421391.git.ludo@HIDDEN>
Date: Mon, 25 Sep 2023 10:15:21 +0200
Message-ID: <87edim7jom.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Sat, 23 Sep 2023 at 00:27, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> +      ;; XXX: Currently recursive checkouts are not supported.
> +      (and (not recursive?)

Naively, and similarly as Maxim=E2=80=99s remark, maybe we could raise a wa=
rning
when the case is not supported instead of silently return #f.

If raising a warning here is not appropriate, maybe document in the
docstring the returned value as #f.
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20

> -      (with-extensions (list guile-json gnutls   ;for (guix swh)
> +      (with-extensions (list guile-json gnutls    ;for (guix swh)

Nitpick: Since the change is complex, I would suggest to avoid this
cosmetic.


Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:34 2023
Received: from localhost ([127.0.0.1]:37437 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoeT-0006mi-RW
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:34 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:43670)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodt-0006jS-Bh
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodd-0005mf-72; Fri, 22 Sep 2023 18:28:41 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=u9+hTSEMumwCQQ/u1GucMUnU5Oqp6LOTsklYGo7TPpc=; b=MqvUHkYeKrbMXHVycTw7
 yGAM3/S11inHOI1uPvc5skPN2dmhrK3nigH7a7ixhmcGbMB6gyS7EoYj0vUEisx+o1ozErVqIgWdx
 y92cLVwCryPrwUUxkTPHfzrrpI41Dj2Gp+UdTu1cxV5wYJ0b6Cbf8Rayj254ybpCe6ApPoxR6TWf+
 /coMjd7arAv1yCZHtOWcgbTwoKxzQBelkENs4qvqACVBfWdKFXNAYm20iEXfPzpjMmcbu1Rh4Ayyy
 8pCc5rpq1fJ4H8AsU0OGlduFzN7jb/YyuN62IGpqyrypXCYNgkH2XNqRuje0XUu0usgQ9II9JQzx/
 1Sk7AwYc1tDi/w==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 8/8] =?UTF-8?q?tests:=20Assume=20=E2=80=98git=E2=80=99?=
 =?UTF-8?q?=20is=20always=20available.?=
Date: Sat, 23 Sep 2023 00:28:04 +0200
Message-ID: <9a69aaf117119f603cc02587f540298fe579df6c.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* tests/channels.scm (gpg+git-available?): Check for ‘gpg-command’
only.
Remove all ‘test-skip’ statements.
* tests/derivations.scm: Likewise.
* tests/git-authenticate.scm: Likewise.
* tests/git.scm: Likewise.
* tests/import-git.scm: Likewise.
---
 tests/channels.scm         |  7 +------
 tests/derivations.scm      |  6 +-----
 tests/git-authenticate.scm |  1 -
 tests/git.scm              | 10 ----------
 tests/import-git.scm       | 18 ------------------
 5 files changed, 2 insertions(+), 40 deletions(-)

diff --git a/tests/channels.scm b/tests/channels.scm
index 62312e240c..6c4276deb4 100644
--- a/tests/channels.scm
+++ b/tests/channels.scm
@@ -50,7 +50,7 @@ (define-module (test-channels)
   #:use-module (ice-9 match))
 
 (define (gpg+git-available?)
-  (and (which (git-command))
+  (and #t                                         ;'git' is always available
        (which (gpg-command)) (which (gpgconf-command))))
 
 (define commit-id-string
@@ -196,7 +196,6 @@ (define channel-metadata-dependencies
                                           "abc1234")))
                          instances)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-channel-instances #:validate-pull"
   'descendant
 
@@ -306,7 +305,6 @@ (define channel-metadata-dependencies
                (depends? drv3
                          (list drv2 drv0) (list))))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "channel-news, no news"
   '()
   (with-temporary-git-repository directory
@@ -318,7 +316,6 @@ (define channel-metadata-dependencies
             (latest  (reference-name->oid repository "HEAD")))
         (channel-news-for-commit channel (oid->string latest))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "channel-news, one entry"
   (with-temporary-git-repository directory
       `((add ".guix-channel"
@@ -406,7 +403,6 @@ (define channel-metadata-dependencies
                          (channel-news-for-commit channel commit5 commit1))
                     '(#f "tag-for-first-news-entry")))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "channel-news, annotated tag"
   (with-temporary-git-repository directory
       `((add ".guix-channel"
@@ -453,7 +449,6 @@ (define channel-metadata-dependencies
                          (channel-news-for-commit channel commit2))
                     (list commit1)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "latest-channel-instances, missing introduction for 'guix'"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
diff --git a/tests/derivations.scm b/tests/derivations.scm
index e1312bd46b..0e87778981 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -29,7 +29,7 @@ (define-module (test-derivations)
   #:use-module (guix tests git)
   #:use-module (guix tests http)
   #:use-module ((guix packages) #:select (package-derivation base32))
-  #:use-module ((guix build utils) #:select (executable-file? which))
+  #:use-module ((guix build utils) #:select (executable-file?))
   #:use-module ((guix hash) #:select (file-hash*))
   #:use-module ((git oid) #:select (oid->string))
   #:use-module ((git reference) #:select (reference-name->oid))
@@ -295,8 +295,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
                          get-string-all)
                        text))))))
 
-;; 'with-temporary-git-repository' relies on the 'git' command.
-(unless (which (git-command)) (test-skip 1))
 (test-equal "'git-download' built-in builder"
   `(("/a.txt" . "AAA")
     ("/b.scm" . "#t"))
@@ -325,7 +323,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
         (build-derivations %store (list drv))
         (directory-contents (derivation->output-path drv) get-string-all)))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "'git-download' built-in builder, invalid hash"
   (with-temporary-git-repository directory
       `((add "a.txt" "AAA")
@@ -349,7 +346,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
         (build-derivations %store (list drv))
         #f))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "'git-download' built-in builder, invalid commit"
   (with-temporary-git-repository directory
       `((add "a.txt" "AAA")
diff --git a/tests/git-authenticate.scm b/tests/git-authenticate.scm
index c063920c12..4de223d422 100644
--- a/tests/git-authenticate.scm
+++ b/tests/git-authenticate.scm
@@ -44,7 +44,6 @@ (define (gpg+git-available?)
 
 (test-begin "git-authenticate")
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "unsigned commits"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
diff --git a/tests/git.scm b/tests/git.scm
index 9c944d65b1..ad43435b67 100644
--- a/tests/git.scm
+++ b/tests/git.scm
@@ -21,7 +21,6 @@ (define-module (test-git)
   #:use-module (git)
   #:use-module (guix git)
   #:use-module (guix tests git)
-  #:use-module (guix build utils)
   #:use-module ((guix utils) #:select (call-with-temporary-directory))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-64)
@@ -33,8 +32,6 @@ (define-module (test-git)
 
 (test-begin "git")
 
-;; 'with-temporary-git-repository' relies on the 'git' command.
-(unless (which (git-command)) (test-skip 1))
 (test-assert "commit-difference, linear history"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
@@ -61,7 +58,6 @@ (define-module (test-git)
              ;; empty list.
              (null? (commit-difference commit1 commit4)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "commit-difference, fork"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
@@ -101,7 +97,6 @@ (define-module (test-git)
              (lset= eq? (commit-difference master4 master2)
                     (list master4 merge master3 devel1 devel2)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "commit-difference, excluded commits"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
@@ -126,7 +121,6 @@ (define-module (test-git)
                     (list commit4))
              (null? (commit-difference commit4 commit1 (list commit5))))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "commit-relation"
   '(self                                          ;master3 master3
     ancestor                                      ;master1 master3
@@ -166,7 +160,6 @@ (define-module (test-git)
               (commit-relation master1 merge)
               (commit-relation merge master1))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "commit-descendant?"
   '((master3 master3 => #t)
     (master1 master3 => #f)
@@ -216,7 +209,6 @@ (define-module (test-git)
                   (master1 merge)
                   (merge master1)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "remote-refs"
   '("refs/heads/develop" "refs/heads/master"
     "refs/tags/v1.0" "refs/tags/v1.1")
@@ -231,7 +223,6 @@ (define-module (test-git)
         (tag "v1.1" "release-1.1"))
     (remote-refs directory)))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "remote-refs: only tags"
  '("refs/tags/v1.0" "refs/tags/v1.1")
   (with-temporary-git-repository directory
@@ -243,7 +234,6 @@ (define-module (test-git)
         (tag "v1.1" "Release 1.1"))
     (remote-refs directory #:tags? #t)))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "update-cached-checkout, tag"
   (call-with-temporary-directory
    (lambda (cache)
diff --git a/tests/import-git.scm b/tests/import-git.scm
index f1bce154bb..20255dedb3 100644
--- a/tests/import-git.scm
+++ b/tests/import-git.scm
@@ -24,7 +24,6 @@ (define-module (test-import-git)
   #:use-module (guix import git)
   #:use-module (guix git-download)
   #:use-module (guix tests git)
-  #:use-module (guix build utils)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-64))
 
@@ -46,7 +45,6 @@ (define* (make-package directory version #:optional (properties '()))
         (base32
          "0000000000000000000000000000000000000000000000000000"))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: no custom prefix, suffix, and delimiter"
   "1.0.1"
   (with-temporary-git-repository directory
@@ -56,7 +54,6 @@ (define* (make-package directory version #:optional (properties '()))
     (let ((package (make-package directory "1.0.0")))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom prefix, no suffix and delimiter"
   "1.0.1"
   (with-temporary-git-repository directory
@@ -67,7 +64,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-prefix . "prefix-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom suffix, no prefix and delimiter"
   "1.0.1"
   (with-temporary-git-repository directory
@@ -78,7 +74,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-suffix . "-suffix-[0-9]*")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom delimiter, no prefix and suffix"
   "2021.09.07"
   (with-temporary-git-repository directory
@@ -89,7 +84,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-version-delimiter . "-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: empty delimiter, no prefix and suffix"
   "20210907"
   (with-temporary-git-repository directory
@@ -100,7 +94,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-version-delimiter . "")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom prefix and suffix, no delimiter"
   "2.0.0"
   (with-temporary-git-repository directory
@@ -112,7 +105,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-suffix . "suffix-[0-9]")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom prefix, suffix, and delimiter"
   "2.0.0"
   (with-temporary-git-repository directory
@@ -125,7 +117,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "_")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: only pre-releases available"
   #f
   (with-temporary-git-repository directory
@@ -135,7 +126,6 @@ (define* (make-package directory version #:optional (properties '()))
     (let ((package (make-package directory "1.0.0")))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases"
   "2.0.0-rc1"
   (with-temporary-git-repository directory
@@ -146,7 +136,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((accept-pre-releases? . #t)))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom prefix"
   "2.0.0-rc1"
   (with-temporary-git-repository directory
@@ -158,7 +147,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-prefix . "version-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix"
   "2.0.0-rc1"
   (with-temporary-git-repository directory
@@ -170,7 +158,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-suffix . "-suffix")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, delimiter conflicts with pre-release part"
   "2.0.0_alpha"
   (with-temporary-git-repository directory
@@ -182,7 +169,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "_")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix and prefix"
   "2.0.0-alpha"
   (with-temporary-git-repository directory
@@ -195,7 +181,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-suffix . "-suffix")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix, prefix, and delimiter"
   "2.0.0-alpha"
   (with-temporary-git-repository directory
@@ -209,7 +194,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, no delimiter, and custom suffix, prefix"
   "2alpha"
   (with-temporary-git-repository directory
@@ -223,7 +207,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: no tags found"
   #f
   (with-temporary-git-repository directory
@@ -232,7 +215,6 @@ (define* (make-package directory version #:optional (properties '()))
     (let ((package (make-package directory "1.0.0")))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: no valid tags found"
   #f
   (with-temporary-git-repository directory
-- 
2.41.0





Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:33 2023
Received: from localhost ([127.0.0.1]:37435 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoeQ-0006mX-En
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:33 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45546)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodq-0006jM-Nr
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjoda-0005mD-JP; Fri, 22 Sep 2023 18:28:38 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=+TlMDh70EYvfg4cVz5uy++mbsJSSzAW/iTqtp0ssVZw=; b=WJgW7p5eJx4CL3cxclQi
 yeiT9P59rCMyoY4v9ovfAShcqixERvZcfbET82wj35DOwho++rnYSQP53Wqx0A6NABQkftqktDSWs
 2PgoIip03B9Wd0AB3LUBJBSw5nvBethQxa4XSoNjjtCi8Amtj971d52hpSXwuMJhSlYTQd9r1hkiy
 xkZqxr/lLGgZOHHEpJDq+SH9ESUM50hJaS60XZAeXHD2whjqbkJCVAGP7bBWrB98DJdu7cedqU6t3
 dX1lokFb1h87FcRIlK2NSOn9shQS3bA6IOIGiCFssDQkEtmxy+eT9Tu++84VNbe5O1P8y5JMayZiN
 0tVKwfHT2AuXpw==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 5/8] build: Add dependency on Git.
Date: Sat, 23 Sep 2023 00:28:01 +0200
Message-ID: <8ec588483525275aac1eb057543a903bea707ead.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* configure.ac: Check for ‘git’ and substitute ‘GIT’.
* guix/config.scm.in (%git): New variable.
* guix/self.scm (compiled-guix): Define ‘git’ and pass it to
‘make-config.scm’.
(make-config.scm): Add #:git; emit a ‘%git’ variable.
* doc/guix.texi (Requirements): Add it.
---
 configure.ac       |  7 +++++++
 doc/guix.texi      |  1 +
 guix/config.scm.in |  6 +++++-
 guix/self.scm      | 10 +++++++++-
 4 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index 92dede8014..d817f620cf 100644
--- a/configure.ac
+++ b/configure.ac
@@ -201,6 +201,13 @@ AC_SUBST([GZIP])
 AC_SUBST([BZIP2])
 AC_SUBST([XZ])
 
+dnl Git is now required for the "builtin:git-download" derivation builder.
+AC_PATH_PROG([GIT], [git])
+if test "x$GIT" = "x"; then
+  AC_MSG_ERROR([Git is missing; please install it.])
+fi
+AC_SUBST([GIT])
+
 LIBGCRYPT_LIBDIR="no"
 LIBGCRYPT_PREFIX="no"
 
diff --git a/doc/guix.texi b/doc/guix.texi
index 50c4984d71..8812e42e99 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1011,6 +1011,7 @@ Requirements
 @item
 @uref{https://gitlab.com/guile-git/guile-git, Guile-Git}, version 0.5.0
 or later;
+@item @uref{https://git-scm.com, Git} (yes, both!);
 @item @uref{https://savannah.nongnu.org/projects/guile-json/, Guile-JSON}
 4.3.0 or later;
 @item @url{https://www.gnu.org/software/make/, GNU Make}.
diff --git a/guix/config.scm.in b/guix/config.scm.in
index d582d91d74..62e15dd713 100644
--- a/guix/config.scm.in
+++ b/guix/config.scm.in
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2018, 2019, 2021 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2012-2016, 2018-2019, 2021, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;; Copyright © 2017 Caleb Ristvedt <caleb.ristvedt@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -35,6 +35,7 @@ (define-module (guix config)
             %config-directory
 
             %system
+            %git
             %gzip
             %bzip2
             %xz))
@@ -109,6 +110,9 @@ (define %config-directory
 (define %system
   "@guix_system@")
 
+(define %git
+  "@GIT@")
+
 (define %gzip
   "@GZIP@")
 
diff --git a/guix/self.scm b/guix/self.scm
index d2300052d8..9eaddc7a29 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -69,6 +69,7 @@ (define %packages
       ("gzip"               . ,(ref 'compression 'gzip))
       ("bzip2"              . ,(ref 'compression 'bzip2))
       ("xz"                 . ,(ref 'compression 'xz))
+      ("git-minimal"        . ,(ref 'version-control 'git-minimal))
       ("po4a"               . ,(ref 'gettext 'po4a))
       ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
       ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
@@ -826,6 +827,9 @@ (define* (compiled-guix source #:key
   (define guile-lzma
     (specification->package "guile-lzma"))
 
+  (define git
+    (specification->package "git-minimal"))
+
   (define dependencies
     (append-map transitive-package-dependencies
                 (list guile-gcrypt guile-gnutls guile-git guile-avahi
@@ -999,6 +1003,7 @@ (define* (compiled-guix source #:key
                     => ,(make-config.scm #:gzip gzip
                                          #:bzip2 bzip2
                                          #:xz xz
+                                         #:git git
                                          #:package-name
                                          %guix-package-name
                                          #:package-version
@@ -1104,7 +1109,7 @@ (define %default-config-variables
     (%storedir . "/gnu/store")
     (%sysconfdir . "/etc")))
 
-(define* (make-config.scm #:key gzip xz bzip2
+(define* (make-config.scm #:key gzip xz bzip2 git
                           (package-name "GNU Guix")
                           (package-version "0")
                           (channel-metadata #f)
@@ -1134,6 +1139,7 @@ (define* (make-config.scm #:key gzip xz bzip2
                                %state-directory
                                %store-database-directory
                                %config-directory
+                               %git
                                %gzip
                                %bzip2
                                %xz))
@@ -1176,6 +1182,8 @@ (define* (make-config.scm #:key gzip xz bzip2
                      ;; information is used by (guix describe).
                      '#$channel-metadata)
 
+                   (define %git
+                     #+(and git (file-append git "/bin/git")))
                    (define %gzip
                      #+(and gzip (file-append gzip "/bin/gzip")))
                    (define %bzip2
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:30 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:30 2023
Received: from localhost ([127.0.0.1]:37433 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoeP-0006mQ-Qx
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:30 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:43664)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjods-0006jR-E0
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:57 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodc-0005mW-A5; Fri, 22 Sep 2023 18:28:40 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=ShZItNlvYjaRrNo4JPlFc63xoz7be61ooWEzJHPU0v4=; b=JaNFm80+YsFQs4iTSECj
 facs5A7BhK5Q3NjkvVQ1br4KKqPyEEtnlgTAwoBXPgXv3D31A+UnwZZo/8TXHmshX2BkeU31rH+E6
 Lfxke1q6uxsslMuFLY2rqE+zD3waofcJyQfCzqjJhYgrRHF2xVU7/5wE+kbrT4ggHSsy+aWul9GkM
 fEWD3wAId7AcFfPNNKuHc/9IcmmDH0NbpY4gKuljT4E7jn+RI/kf1VByyVJc8NZL0Bt4koJMrjNXP
 QWt9iYYuydAdzDAig/3iR6kvWeqsqG/66bzyrD3SkwqSDH3srBDy+sSjb8kzE4b8H+cRfUxahdGQH
 ZgOUZcqfVY6W+g==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 7/8] =?UTF-8?q?git-download:=20Use=20=E2=80=9Cbuiltin:g?=
 =?UTF-8?q?it-download=E2=80=9D=20when=20available.?=
Date: Sat, 23 Sep 2023 00:28:03 +0200
Message-ID: <b0a3a92d6adda6582cd3aa61d3be30a90d91c96b.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Fixes <https://issues.guix.gnu.org/63331>.

Longer-term this will remove Git from the derivation graph when its sole
use is to perform a checkout for a fixed-output derivation, thereby
breaking dependency cycles that can arise in these situations.

* guix/git-download.scm (git-fetch): Rename to…
(git-fetch/in-band): … this.  Deal with GIT or GUILE being #f.
(git-fetch/built-in, built-in-builders*, git-fetch): New procedures.
* tests/builders.scm ("git-fetch, file URI"): New test.
---
 guix/git-download.scm | 68 +++++++++++++++++++++++++++++++++++++------
 tests/builders.scm    | 29 +++++++++++++++++-
 2 files changed, 87 insertions(+), 10 deletions(-)

diff --git a/guix/git-download.scm b/guix/git-download.scm
index f1f19397c6..505dff0a89 100644
--- a/guix/git-download.scm
+++ b/guix/git-download.scm
@@ -27,6 +27,7 @@ (define-module (guix git-download)
   #:use-module (guix records)
   #:use-module (guix packages)
   #:use-module (guix modules)
+  #:use-module ((guix derivations) #:select (raw-derivation))
   #:autoload   (guix build-system gnu) (standard-packages)
   #:autoload   (guix download) (%download-fallback-test)
   #:autoload   (git bindings)   (libgit2-init!)
@@ -78,15 +79,19 @@ (define (git-package)
   (let ((distro (resolve-interface '(gnu packages version-control))))
     (module-ref distro 'git-minimal)))
 
-(define* (git-fetch ref hash-algo hash
-                    #:optional name
-                    #:key (system (%current-system)) (guile (default-guile))
-                    (git (git-package)))
-  "Return a fixed-output derivation that fetches REF, a <git-reference>
-object.  The output is expected to have recursive hash HASH of type
-HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if #f."
+(define* (git-fetch/in-band ref hash-algo hash
+                            #:optional name
+                            #:key (system (%current-system))
+                            (guile (default-guile))
+                            (git (git-package)))
+  "Return a fixed-output derivation that performs a Git checkout of REF, using
+GIT and GUILE (thus, said derivation depends on GIT and GUILE).
+
+This method is deprecated in favor of the \"builtin:git-download\" builder.
+It will be removed when versions of guix-daemon implementing
+\"builtin:git-download\" will be sufficiently widespread."
   (define inputs
-    `(("git" ,git)
+    `(("git" ,(or git (git-package)))
 
       ;; When doing 'git clone --recursive', we need sed, grep, etc. to be
       ;; available so that 'git submodule' works.
@@ -154,7 +159,8 @@ (define* (git-fetch ref hash-algo hash
                                      #:recursive? recursive?
                                      #:git-command "git")))))
 
-  (mlet %store-monad ((guile (package->derivation guile system)))
+  (mlet %store-monad ((guile (package->derivation (or guile (default-guile))
+                                                  system)))
     (gexp->derivation (or name "git-checkout") build
 
                       ;; Use environment variables and a fixed script name so
@@ -181,6 +187,50 @@ (define* (git-fetch ref hash-algo hash
                       #:recursive? #t
                       #:guile-for-build guile)))
 
+(define* (git-fetch/built-in ref hash-algo hash
+                             #:optional name
+                             #:key (system (%current-system)))
+  "Return a fixed-output derivation without any dependency that performs a Git
+checkout of REF, using the \"builtin:git-download\" derivation builder."
+  (raw-derivation (or name "git-checkout") "builtin:git-download" '()
+                  #:system system
+                  #:hash-algo hash-algo
+                  #:hash hash
+                  #:recursive? #t
+                  #:env-vars
+                  `(("url" . ,(object->string
+                               (match (%download-fallback-test)
+                                 ('content-addressed-mirrors
+                                  "https://example.org/does-not-exist")
+                                 (_
+                                  (git-reference-url ref)))))
+                    ("commit" . ,(git-reference-commit ref))
+                    ("recursive?" . ,(object->string
+                                      (git-reference-recursive? ref))))
+                  #:leaked-env-vars '("http_proxy" "https_proxy"
+                                      "LC_ALL" "LC_MESSAGES" "LANG"
+                                      "COLUMNS")
+                  #:local-build? #t))
+
+(define built-in-builders*
+  (store-lift built-in-builders))
+
+(define* (git-fetch ref hash-algo hash
+                    #:optional name
+                    #:key (system (%current-system))
+                    guile git)
+  "Return a fixed-output derivation that fetches REF, a <git-reference>
+object.  The output is expected to have recursive hash HASH of type
+HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if #f."
+  (mlet %store-monad ((builtins (built-in-builders*)))
+    (if (member "git-download" builtins)
+        (git-fetch/built-in ref hash-algo hash name
+                            #:system system)
+        (git-fetch/in-band ref hash-algo hash name
+                           #:system system
+                           #:guile guile
+                           #:git git))))
+
 (define (git-version version revision commit)
   "Return the version string for packages using git-download."
   ;; git-version is almost exclusively executed while modules are being loaded.
diff --git a/tests/builders.scm b/tests/builders.scm
index 0b5577c7a3..619caa5f31 100644
--- a/tests/builders.scm
+++ b/tests/builders.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2018, 2019, 2021 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2012-2015, 2018-2019, 2021, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;; Copyright © 2021 Lars-Dominik Braun <lars@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -20,6 +20,7 @@
 
 (define-module (tests builders)
   #:use-module (guix download)
+  #:use-module (guix git-download)
   #:use-module (guix build-system)
   #:use-module (guix build-system gnu)
   #:use-module (guix build gnu-build-system)
@@ -31,9 +32,12 @@ (define-module (tests builders)
   #:use-module (guix base32)
   #:use-module (guix derivations)
   #:use-module (gcrypt hash)
+  #:use-module ((guix hash) #:select (file-hash*))
   #:use-module (guix tests)
+  #:use-module (guix tests git)
   #:use-module (guix packages)
   #:use-module (gnu packages bootstrap)
+  #:use-module ((ice-9 ftw) #:select (scandir))
   #:use-module (ice-9 match)
   #:use-module (ice-9 textual-ports)
   #:use-module (srfi srfi-1)
@@ -84,6 +88,29 @@ (define url-fetch*
     (and (file-exists? out)
          (valid-path? %store out))))
 
+(test-equal "git-fetch, file URI"
+  '("." ".." "a.txt" "b.scm")
+  (let ((nonce (random-text)))
+    (with-temporary-git-repository directory
+        `((add "a.txt" ,nonce)
+          (add "b.scm" "#t")
+          (commit "Commit.")
+          (tag "v1.0.0" "The tag."))
+      (run-with-store %store
+        (mlet* %store-monad ((hash
+                              -> (file-hash* directory
+                                             #:algorithm (hash-algorithm sha256)
+                                             #:recursive? #t))
+                             (drv (git-fetch
+                                   (git-reference
+                                    (url (string-append "file://" directory))
+                                    (commit "v1.0.0"))
+                                   'sha256 hash
+                                   "git-fetch-test")))
+          (mbegin %store-monad
+            (built-derivations (list drv))
+            (return (scandir (derivation->output-path drv)))))))))
+
 (test-assert "gnu-build-system"
   (build-system? gnu-build-system))
 
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:29 2023
Received: from localhost ([127.0.0.1]:37431 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoeM-0006mE-2k
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:29 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45556)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodq-0006jN-RA
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:56 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodZ-0005m8-Ma; Fri, 22 Sep 2023 18:28:37 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=WG9PJCFhQ2r3BrsfN+53baekgsye0ORiyoUrEAdMA+w=; b=j8iM+swSnP/Om4P+Yl31
 tJVYR1aobXf3/nxJi/IkIT3qbS0lc7fNu0RFQyPbzxe4RC6vm8xDzqSPpQtnNHuMoONVxBH5ejV9W
 qXOMgd3brUXEBzByQTBEts5PDQ8T0bhM8KE6wSuY10aqUEWEvTz9jpSvcIej7tfBGzAamn+bsGtoB
 fG34tT0tkZmtiDj/kbAU726hPLBoL2Mp00/oKOXOLeElE9duyQPOj4AEiY+8sFkitKlMm8ESV4TUX
 s0BkB7KfDEDdWkO/PxAetTxVb7fuc7NA64s2BqjZ+tkDhAMOxTnQQsBYYNAxi7E+5mmjzkEgjganu
 80IW4MtQIFbPpQ==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 4/8] =?UTF-8?q?daemon:=20Add=20=E2=80=9Cgit-download?=
 =?UTF-8?q?=E2=80=9D=20built-in=20builder.?=
Date: Sat, 23 Sep 2023 00:28:00 +0200
Message-ID: <72b8f3ee7ef875273240f5dcbc8e9b1a5bd75515.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludovic.courtes@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

From: Ludovic Courtès <ludovic.courtes@HIDDEN>

The new builder makes it possible to break cycles that occurs when the
fixed-output derivation for the source of a dependency of ‘git’ would
itself depend on ‘git’.

* guix/scripts/perform-download.scm (perform-git-download): New
procedure.
(perform-download): Move fixed-output derivation check to…
(guix-perform-download): … here.  Invoke ‘perform-download’ or
‘perform-git-download’ depending on what ‘derivation-builder’ returns.
* nix/libstore/builtins.cc (builtins): Add “git-download”.
* tests/derivations.scm ("built-in-builders"): Update.
("'git-download' built-in builder")
("'git-download' built-in builder, invalid hash")
("'git-download' built-in builder, invalid commit")
("'git-download' built-in builder, not found"): New tests.
---
 guix/scripts/perform-download.scm |  49 ++++++++++++---
 nix/libstore/builtins.cc          |   5 +-
 tests/derivations.scm             | 100 +++++++++++++++++++++++++++++-
 3 files changed, 142 insertions(+), 12 deletions(-)

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index 3b29a3c81d..bb1e51aa30 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2016, 2017, 2018, 2020 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2016-2018, 2020, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,7 +21,8 @@ (define-module (guix scripts perform-download)
   #:use-module (guix scripts)
   #:use-module (guix derivations)
   #:use-module ((guix store) #:select (derivation-path? store-path?))
-  #:use-module (guix build download)
+  #:autoload   (guix build download) (url-fetch)
+  #:autoload   (guix build git) (git-fetch-with-fallback)
   #:use-module (ice-9 match)
   #:export (guix-perform-download))
 
@@ -61,10 +62,6 @@ (define* (perform-download drv output
            (drv-output (assoc-ref (derivation-outputs drv) "out"))
            (algo       (derivation-output-hash-algo drv-output))
            (hash       (derivation-output-hash drv-output)))
-      (unless (and algo hash)
-        (leave (G_ "~a is not a fixed-output derivation~%")
-               (derivation-file-name drv)))
-
       ;; We're invoked by the daemon, which gives us write access to OUTPUT.
       (when (url-fetch url output
                        #:print-build-trace? print-build-trace?
@@ -89,6 +86,30 @@ (define* (perform-download drv output
         (when (and executable (string=? executable "1"))
           (chmod output #o755))))))
 
+(define* (perform-git-download drv output
+                               #:key print-build-trace?)
+  "Perform the download described by DRV, a fixed-output derivation, to
+OUTPUT.
+
+Note: OUTPUT may differ from the 'out' value of DRV, notably for 'bmCheck' or
+'bmRepair' builds."
+  (derivation-let drv ((url "url")
+                       (commit "commit")
+                       (recursive? "recursive?"))
+    (unless url
+      (leave (G_ "~a: missing Git URL~%") (derivation-file-name drv)))
+    (unless commit
+      (leave (G_ "~a: missing Git commit~%") (derivation-file-name drv)))
+
+    (let* ((url        (call-with-input-string url read))
+           (recursive? (and recursive?
+                            (call-with-input-string recursive? read)))
+           (drv-output (assoc-ref (derivation-outputs drv) "out"))
+           (algo       (derivation-output-hash-algo drv-output))
+           (hash       (derivation-output-hash drv-output)))
+      (git-fetch-with-fallback url commit output
+                               #:recursive? recursive?))))
+
 (define (assert-low-privileges)
   (when (zero? (getuid))
     (leave (G_ "refusing to run with elevated privileges (UID ~a)~%")
@@ -117,8 +138,20 @@ (define-command (guix-perform-download . args)
     (match args
       (((? derivation-path? drv) (? store-path? output))
        (assert-low-privileges)
-       (let ((drv (read-derivation-from-file drv)))
-         (perform-download drv output #:print-build-trace? print-build-trace?)))
+       (let* ((drv (read-derivation-from-file drv))
+              (download (match (derivation-builder drv)
+                          ("builtin:download" perform-download)
+                          ("builtin:git-download" perform-git-download)
+                          (unknown (leave (G_ "~a: unknown builtin builder")
+                                          unknown))))
+              (drv-output (assoc-ref (derivation-outputs drv) "out"))
+              (algo       (derivation-output-hash-algo drv-output))
+              (hash       (derivation-output-hash drv-output)))
+         (unless (and hash algo)
+           (leave (G_ "~a is not a fixed-output derivation~%")
+                  (derivation-file-name drv)))
+
+         (download drv output #:print-build-trace? print-build-trace?)))
       (("--version")
        (show-version-and-exit))
       (x
diff --git a/nix/libstore/builtins.cc b/nix/libstore/builtins.cc
index 4111ac4760..6bf467354a 100644
--- a/nix/libstore/builtins.cc
+++ b/nix/libstore/builtins.cc
@@ -1,5 +1,5 @@
 /* GNU Guix --- Functional package management for GNU
-   Copyright (C) 2016, 2017, 2018, 2019 Ludovic Courtès <ludo@HIDDEN>
+   Copyright (C) 2016-2019, 2023 Ludovic Courtès <ludo@HIDDEN>
 
    This file is part of GNU Guix.
 
@@ -58,7 +58,8 @@ static void builtinDownload(const Derivation &drv,
 
 static const std::map<std::string, derivationBuilder> builtins =
 {
-    { "download", builtinDownload }
+    { "download", builtinDownload },
+    { "git-download", builtinDownload }
 };
 
 derivationBuilder lookupBuiltinBuilder(const std::string & name)
diff --git a/tests/derivations.scm b/tests/derivations.scm
index 66c777cfe7..e1312bd46b 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -24,10 +24,15 @@ (define-module (test-derivations)
   #:use-module (guix utils)
   #:use-module ((gcrypt hash) #:prefix gcrypt:)
   #:use-module (guix base32)
+  #:use-module ((guix git) #:select (with-repository))
   #:use-module (guix tests)
+  #:use-module (guix tests git)
   #:use-module (guix tests http)
   #:use-module ((guix packages) #:select (package-derivation base32))
-  #:use-module ((guix build utils) #:select (executable-file?))
+  #:use-module ((guix build utils) #:select (executable-file? which))
+  #:use-module ((guix hash) #:select (file-hash*))
+  #:use-module ((git oid) #:select (oid->string))
+  #:use-module ((git reference) #:select (reference-name->oid))
   #:use-module (gnu packages bootstrap)
   #:use-module ((gnu packages guile) #:select (guile-1.8))
   #:use-module (srfi srfi-1)
@@ -195,7 +200,7 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
                    (stat:ino (lstat file2))))))))
 
 (test-equal "built-in-builders"
-  '("download")
+  '("download" "git-download")
   (built-in-builders %store))
 
 (test-assert "unknown built-in builder"
@@ -290,6 +295,97 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
                          get-string-all)
                        text))))))
 
+;; 'with-temporary-git-repository' relies on the 'git' command.
+(unless (which (git-command)) (test-skip 1))
+(test-equal "'git-download' built-in builder"
+  `(("/a.txt" . "AAA")
+    ("/b.scm" . "#t"))
+  (let ((nonce (random-text)))
+    (with-temporary-git-repository directory
+        `((add "a.txt" "AAA")
+          (add "b.scm" "#t")
+          (commit ,nonce))
+      (let* ((commit (with-repository directory repository
+                       (oid->string
+                        (reference-name->oid repository "HEAD"))))
+             (drv (derivation %store "git-download"
+                              "builtin:git-download" '()
+                              #:env-vars
+                              `(("url"
+                                 . ,(object->string
+                                     (string-append "file://" directory)))
+                                ("commit" . ,commit))
+                              #:hash-algo 'sha256
+                              #:hash (file-hash* directory
+                                                 #:algorithm
+                                                 (gcrypt:hash-algorithm
+                                                  gcrypt:sha256)
+                                                 #:recursive? #t)
+                              #:recursive? #t)))
+        (build-derivations %store (list drv))
+        (directory-contents (derivation->output-path drv) get-string-all)))))
+
+(unless (which (git-command)) (test-skip 1))
+(test-assert "'git-download' built-in builder, invalid hash"
+  (with-temporary-git-repository directory
+      `((add "a.txt" "AAA")
+        (add "b.scm" "#t")
+        (commit "Commit!"))
+    (let* ((commit (with-repository directory repository
+                     (oid->string
+                      (reference-name->oid repository "HEAD"))))
+           (drv (derivation %store "git-download"
+                            "builtin:git-download" '()
+                            #:env-vars
+                            `(("url"
+                               . ,(object->string
+                                   (string-append "file://" directory)))
+                              ("commit" . ,commit))
+                            #:hash-algo 'sha256
+                            #:hash (gcrypt:sha256 #vu8())
+                            #:recursive? #t)))
+      (guard (c ((store-protocol-error? c)
+                 (string-contains (store-protocol-error-message c) "failed")))
+        (build-derivations %store (list drv))
+        #f))))
+
+(unless (which (git-command)) (test-skip 1))
+(test-assert "'git-download' built-in builder, invalid commit"
+  (with-temporary-git-repository directory
+      `((add "a.txt" "AAA")
+        (add "b.scm" "#t")
+        (commit "Commit!"))
+    (let* ((drv (derivation %store "git-download"
+                            "builtin:git-download" '()
+                            #:env-vars
+                            `(("url"
+                               . ,(object->string
+                                   (string-append "file://" directory)))
+                              ("commit"
+                               . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"))
+                            #:hash-algo 'sha256
+                            #:hash (gcrypt:sha256 #vu8())
+                            #:recursive? #t)))
+      (guard (c ((store-protocol-error? c)
+                 (string-contains (store-protocol-error-message c) "failed")))
+        (build-derivations %store (list drv))
+        #f))))
+
+(test-assert "'git-download' built-in builder, not found"
+  (let* ((drv (derivation %store "git-download"
+                          "builtin:git-download" '()
+                          #:env-vars
+                          `(("url" . "file:///does-not-exist.git")
+                            ("commit"
+                             . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"))
+                          #:hash-algo 'sha256
+                          #:hash (gcrypt:sha256 #vu8())
+                          #:recursive? #t)))
+    (guard (c ((store-protocol-error? c)
+               (string-contains (store-protocol-error-message c) "failed")))
+      (build-derivations %store (list drv))
+      #f)))
+
 (test-equal "derivation-name"
   "foo-0.0"
   (let ((drv (derivation %store "foo-0.0" %bash '())))
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:26 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:26 2023
Received: from localhost ([127.0.0.1]:37429 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoeL-0006mB-Js
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:26 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45572)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodr-0006jQ-Ir
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:56 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodb-0005mN-Ed; Fri, 22 Sep 2023 18:28:39 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=WBBIHCG4OPF6Tvp3S31BnKTGe7cjx3lYg1Q29cAJBkU=; b=fBGu0w49cQ75OJISdB8o
 JTljVpOZI03jZHlX+th1oUTwO3l8DJN+ePXDi/R3jPimHIRadcbSMdYR++l7mrm9LOOkTK6rui2ij
 4q/cr1C6WP5aiY/dgSIndYMdRidMXQB3eJe6N6wAE2qa8tRcSuLchXpBuWN/0KXgmRAk8RLaJtRac
 dtwN1DKmJOxF+YDZt2OP8HPKr0uYdWvbrcYwkP0+WoLF1DSTt+eII417YuDlE+8YiEnE0QbK0PEez
 DIbQn5jOXcyM5szX9tpIq9n7zJYC7LCcc0vDlo+xiRoTmFwU/g+blyjkW3DzweKWQz3YfqsWfpFm8
 E0r+2psdeuECLw==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 6/8] =?UTF-8?q?perform-download:=20Use=20the=20?=
 =?UTF-8?q?=E2=80=98git=E2=80=99=20command=20captured=20at=20configure=20t?=
 =?UTF-8?q?ime.?=
Date: Sat, 23 Sep 2023 00:28:02 +0200
Message-ID: <c77239b7ac84022c6740b94a541ab241621fc80f.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* guix/scripts/perform-download.scm (perform-git-download): Pass #:git-command
to ‘git-fetch-with-fallback’.
---
 guix/scripts/perform-download.scm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index bb1e51aa30..045dd84ad6 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -23,6 +23,7 @@ (define-module (guix scripts perform-download)
   #:use-module ((guix store) #:select (derivation-path? store-path?))
   #:autoload   (guix build download) (url-fetch)
   #:autoload   (guix build git) (git-fetch-with-fallback)
+  #:autoload   (guix config) (%git)
   #:use-module (ice-9 match)
   #:export (guix-perform-download))
 
@@ -108,7 +109,8 @@ (define* (perform-git-download drv output
            (algo       (derivation-output-hash-algo drv-output))
            (hash       (derivation-output-hash drv-output)))
       (git-fetch-with-fallback url commit output
-                               #:recursive? recursive?))))
+                               #:recursive? recursive?
+                               #:git-command %git))))
 
 (define (assert-low-privileges)
   (when (zero? (getuid))
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:02 2023
Received: from localhost ([127.0.0.1]:37408 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjodx-0006kk-Ur
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:02 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45538)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodp-0006jJ-5C
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:55 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodZ-0005m1-0G; Fri, 22 Sep 2023 18:28:37 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=gNug78rp8gIAX876iiouNa+KGe0CY0p3NljM9It7aTE=; b=qTu/Q3cgDVIDaCLUgU+8
 GtQ95DcpHgT4y8kVXhPSiyMkOvw8q8IrNVqoJFPb5iKK5UM9W9PfcVx8Avm00tT5K8RQpp4qEovmH
 DnTTX3s6/vZpTz2dsxxlvS1BHH7Ahe+SNUxTtk1GAIthKBzo/3JtqGF+OacE4N9dfMQhUWCCjlJt+
 tZCWPsOjNVkTK3NnJIqudWGjA9Y2k430CseeYQ+mCJYeIuaOySKq6KxHM1Foy2YydR5Fh/38k0eYJ
 dC9KcYoZNbwGGmnJL0zwGll0JYKcJXmoKCioumqApoFG3zC6FFnk5rrzupUro+1WtVKl4o9ynkihc
 g7/whf4uVciR0A==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 3/8] perform-download: Remove unused one-argument clause.
Date: Sat, 23 Sep 2023 00:27:59 +0200
Message-ID: <f35b5a56fc9a06a690f91e20d125df9fe0396819.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Code in ‘builtins.cc’ only ever invokes ‘guix perform-download’ with two
arguments.

* guix/scripts/perform-download.scm (guix-perform-download): Remove
unused one-argument clause.
(perform-download): Make ‘output’ parameter mandatory; remove ‘output*’
variable.
---
 guix/scripts/perform-download.scm | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index 6889bcef79..3b29a3c81d 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -42,16 +42,14 @@ (define %user-module
     (module-use! module (resolve-interface '(guix base32)))
     module))
 
-(define* (perform-download drv #:optional output
+(define* (perform-download drv output
                            #:key print-build-trace?)
   "Perform the download described by DRV, a fixed-output derivation, to
 OUTPUT.
 
-Note: Unless OUTPUT is #f, we don't read the value of 'out' in DRV since the
-actual output is different from that when we're doing a 'bmCheck' or
-'bmRepair' build."
+Note: OUTPUT may differ from the 'out' value of DRV, notably for 'bmCheck' or
+'bmRepair' builds."
   (derivation-let drv ((url "url")
-                       (output* "out")
                        (executable "executable")
                        (mirrors "mirrors")
                        (content-addressed-mirrors "content-addressed-mirrors")
@@ -59,8 +57,7 @@ (define* (perform-download drv #:optional output
     (unless url
       (leave (G_ "~a: missing URL~%") (derivation-file-name drv)))
 
-    (let* ((output     (or output output*))
-           (url        (call-with-input-string url read))
+    (let* ((url        (call-with-input-string url read))
            (drv-output (assoc-ref (derivation-outputs drv) "out"))
            (algo       (derivation-output-hash-algo drv-output))
            (hash       (derivation-output-hash drv-output)))
@@ -120,13 +117,8 @@ (define-command (guix-perform-download . args)
     (match args
       (((? derivation-path? drv) (? store-path? output))
        (assert-low-privileges)
-       (perform-download (read-derivation-from-file drv)
-                         output
-                         #:print-build-trace? print-build-trace?))
-      (((? derivation-path? drv))                 ;backward compatibility
-       (assert-low-privileges)
-       (perform-download (read-derivation-from-file drv)
-                         #:print-build-trace? print-build-trace?))
+       (let ((drv (read-derivation-from-file drv)))
+         (perform-download drv output #:print-build-trace? print-build-trace?)))
       (("--version")
        (show-version-and-exit))
       (x
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:29:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:29:01 2023
Received: from localhost ([127.0.0.1]:37406 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjodt-0006kK-V3
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:29:01 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45518)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodn-0006jE-7L
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:55 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodW-0005lU-5W; Fri, 22 Sep 2023 18:28:34 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=lk47iO4kQzqe0QpNVs9Y830SM9FGaIUAEeD/pw9Aqbo=; b=hKBU41HT/Sg8QDjk9/Kk
 3WDTAxsYDmVv5ynGBwGNHkaOafcf6fPTlTIsmIeu4tz5uCBCD4uyBvajGEuW/D/akftXcpN0kByOH
 PDO+FSW7vHDEzXrw+iDxS05rcZhcbf8fzoLO2wet1iBeB84xKEFjzjtFjyzLTvSd8YT2Nhnl134R+
 /gf0nnmMQbppZi7HbTT1agiW8z29qwdzqXPZWk7K/E4avFmQinH7RIvPgOxSVJ9pPh/SJtVMU63H3
 50EE+feos3RKdf3fIzQlhmStf7Za9ZESQa9nl8AnmZMaR8jRzb/bs0BdVPVcoOe3vbs1/7sLQ8kil
 BEeWLyyazrlYZA==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 0/8] Add built-in builder for Git checkouts
Date: Sat, 23 Sep 2023 00:27:56 +0200
Message-ID: <cover.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <87o7hwas61.fsf@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>,
 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hello,

Changes compared to v1:

  • The ‘output’ parameter of the ‘perform-download’ and
    ‘perform-git-download’ procedures is now mandatory.
    Consequently, the confusing ‘output*’ variable is gone.

  • The docstring of these two procedures has been clarified
    accordingly.

  • I think there’s no third item.

Let me know if I missed something!

Thanks,
Ludo’.

Ludovic Courtès (8):
  git-download: Move fallback code to (guix build git).
  git-download: Honor the ‘GUIX_DOWNLOAD_FALLBACK_TEST’ environment
    variable.
  perform-download: Remove unused one-argument clause.
  daemon: Add “git-download” built-in builder.
  build: Add dependency on Git.
  perform-download: Use the ‘git’ command captured at configure time.
  git-download: Use “builtin:git-download” when available.
  tests: Assume ‘git’ is always available.

 configure.ac                      |   7 ++
 doc/guix.texi                     |   1 +
 guix/build/git.scm                |  44 ++++++++++-
 guix/config.scm.in                |   6 +-
 guix/git-download.scm             | 122 ++++++++++++++++++------------
 guix/scripts/perform-download.scm |  67 +++++++++++-----
 guix/self.scm                     |  10 ++-
 nix/libstore/builtins.cc          |   5 +-
 tests/builders.scm                |  29 ++++++-
 tests/channels.scm                |   7 +-
 tests/derivations.scm             |  94 ++++++++++++++++++++++-
 tests/git-authenticate.scm        |   1 -
 tests/git.scm                     |  10 ---
 tests/import-git.scm              |  18 -----
 14 files changed, 309 insertions(+), 112 deletions(-)


base-commit: 3d8d67ef6928f5d81118c97f03372cd341eab8b0
-- 
2.41.0





Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:28:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:28:57 2023
Received: from localhost ([127.0.0.1]:37402 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjodt-0006k8-Aq
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:57 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45528)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodo-0006jI-Fp
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:55 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodX-0005lu-Vm; Fri, 22 Sep 2023 18:28:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=lbJIuSS34XKou1KAAp0q0w8yQNp1j9kF2Ik0svE2RAw=; b=kB+jDDNPZ+oZ+91BsT5H
 6vW2NEQYpnDQ10waBWfUMBlYK4M/RET3kJnTKE5ih3Twm3QRPPusGaZeP5pRjS5CdcDURgjxt4RqS
 aYzp6FsIaACaILu3+76VqbxjfsE8If1iYOIr8O8WQ4Cb8H5sSYd4FeDPbOR56Z9oTVwfgRJRi5OsS
 XPGtQdqI9J9KxCIQIa/XxOvjXdWSAB5y2+yHw2bdWVKkCEUwMc3sMNo7z2aZLAWnDE+bz/DWMl/hw
 p/TdFcWnrdTFf2wDiifOEzG2CpO7cVx92AvkBkIcHR5egwy1T84aBFyQXl/k/B7cpVJa/OqPPKPdg
 4KKI1UKMBmBKYA==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 2/8] =?UTF-8?q?git-download:=20Honor=20the=20=E2=80=98G?=
 =?UTF-8?q?UIX=5FDOWNLOAD=5FFALLBACK=5FTEST=E2=80=99=20environment=20varia?=
 =?UTF-8?q?ble.?=
Date: Sat, 23 Sep 2023 00:27:58 +0200
Message-ID: <0c78818e6b49331c108195cdab1f37eff3d56dde.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* guix/git-download.scm (git-fetch): Honor ‘%download-fallback-test’.
---
 guix/git-download.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/guix/git-download.scm b/guix/git-download.scm
index 8989b1b463..f1f19397c6 100644
--- a/guix/git-download.scm
+++ b/guix/git-download.scm
@@ -28,6 +28,7 @@ (define-module (guix git-download)
   #:use-module (guix packages)
   #:use-module (guix modules)
   #:autoload   (guix build-system gnu) (standard-packages)
+  #:autoload   (guix download) (%download-fallback-test)
   #:autoload   (git bindings)   (libgit2-init!)
   #:autoload   (git repository) (repository-open
                                  repository-close!
@@ -161,7 +162,11 @@ (define* (git-fetch ref hash-algo hash
                       ;; downloads.
                       #:script-name "git-download"
                       #:env-vars
-                      `(("git url" . ,(git-reference-url ref))
+                      `(("git url" . ,(match (%download-fallback-test)
+                                        ('content-addressed-mirrors
+                                         "https://example.org/does-not-exist")
+                                        (_
+                                         (git-reference-url ref))))
                         ("git commit" . ,(git-reference-commit ref))
                         ("git recursive?" . ,(object->string
                                               (git-reference-recursive? ref))))
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:28:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:28:57 2023
Received: from localhost ([127.0.0.1]:37400 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjods-0006jy-Gy
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:57 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45520)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjodn-0006jF-8k
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:28:54 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjodX-0005lj-23; Fri, 22 Sep 2023 18:28:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=NhkD4LZUlsDDA0ql2tBJERZrS5hCR8P2YozzIJfod8I=; b=AswgeUpg0QCfzZNE2vV2
 wgoMscITs749Z21M4txMlu34foKVujOHAwFiuNrPQsgEKHUYMfhdDXi2WVGnhIi7qtVD1xD9JAlUb
 bJ3TP7+CV+0N6yYNS9wVJzUeyp4OnFgCRsV2Zp+Waengs+Mo93x2vybuBqSLrOaz9I8AXHwQJ9yKI
 IfYidi7Cu9SMI/FSsfOJEZQIyZf/9/Ao9M6LHo2rGjTZg12Qhs3yhPnpTTNmyJXUyS/nipVArZwpF
 J95tgfmxo8bdx6ZcAUwZ6iSI540kOcSr6M+i/IpoMyfESG16TZLJSIxYHXls7mOjTBFovmnAdXwKV
 FakwHR5jxEWH2A==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH v2 1/8] git-download: Move fallback code to (guix build git).
Date: Sat, 23 Sep 2023 00:27:57 +0200
Message-ID: <765792320d841e1e33a11f119ad687094ca2766f.1695421391.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1695421391.git.ludo@HIDDEN>
References: <87o7hwas61.fsf@HIDDEN> <cover.1695421391.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* guix/build/git.scm (git-fetch-with-fallback): New procedure, with code
taken from…
* guix/git-download.scm (git-fetch): … here.
[modules]: Remove modules that are no longer directly used in ‘build’.
[build]: Use ‘git-fetch-with-fallback’.
---
 guix/build/git.scm    | 44 ++++++++++++++++++++++++++++++++++++++--
 guix/git-download.scm | 47 ++++++++-----------------------------------
 2 files changed, 50 insertions(+), 41 deletions(-)

diff --git a/guix/build/git.scm b/guix/build/git.scm
index deda10fee8..0ff263c81b 100644
--- a/guix/build/git.scm
+++ b/guix/build/git.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2016, 2019 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2014, 2016, 2019, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -18,9 +18,12 @@
 
 (define-module (guix build git)
   #:use-module (guix build utils)
+  #:autoload   (guix build download-nar) (download-nar)
+  #:autoload   (guix swh) (%verify-swh-certificate? swh-download)
   #:use-module (srfi srfi-34)
   #:use-module (ice-9 format)
-  #:export (git-fetch))
+  #:export (git-fetch
+            git-fetch-with-fallback))
 
 ;;; Commentary:
 ;;;
@@ -76,4 +79,41 @@ (define* (git-fetch url commit directory
       (delete-file-recursively ".git")
       #t)))
 
+
+(define* (git-fetch-with-fallback url commit directory
+                                  #:key (git-command "git") recursive?)
+  "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall back to
+alternative methods when fetching from URL fails: attempt to download a nar,
+and if that also fails, download from the Software Heritage archive."
+  (or (git-fetch url commit directory
+                 #:recursive? recursive?
+                 #:git-command git-command)
+      (download-nar directory)
+
+      ;; As a last resort, attempt to download from Software Heritage.
+      ;; Disable X.509 certificate verification to avoid depending
+      ;; on nss-certs--we're authenticating the checkout anyway.
+      ;; XXX: Currently recursive checkouts are not supported.
+      (and (not recursive?)
+           (parameterize ((%verify-swh-certificate? #f))
+             (format (current-error-port)
+                     "Trying to download from Software Heritage...~%")
+
+             (swh-download url commit directory)
+             (when (file-exists?
+                    (string-append directory "/.gitattributes"))
+               ;; Perform CR/LF conversion and other changes
+               ;; specificied by '.gitattributes'.
+               (invoke git-command "-C" directory "init")
+               (invoke git-command "-C" directory "config" "--local"
+                       "user.email" "you@HIDDEN")
+               (invoke git-command "-C" directory "config" "--local"
+                       "user.name" "Your Name")
+               (invoke git-command "-C" directory "add" ".")
+               (invoke git-command "-C" directory "commit" "-am" "init")
+               (invoke git-command "-C" directory "read-tree" "--empty")
+               (invoke git-command "-C" directory "reset" "--hard")
+               (delete-file-recursively
+                (string-append directory "/.git")))))))
+
 ;;; git.scm ends here
diff --git a/guix/git-download.scm b/guix/git-download.scm
index d88f4c40ee..8989b1b463 100644
--- a/guix/git-download.scm
+++ b/guix/git-download.scm
@@ -116,19 +116,16 @@ (define* (git-fetch ref hash-algo hash
   (define modules
     (delete '(guix config)
             (source-module-closure '((guix build git)
-                                     (guix build utils)
-                                     (guix build download-nar)
-                                     (guix swh)))))
+                                     (guix build utils)))))
 
   (define build
     (with-imported-modules modules
-      (with-extensions (list guile-json gnutls   ;for (guix swh)
+      (with-extensions (list guile-json gnutls    ;for (guix swh)
                              guile-lzlib)
         #~(begin
             (use-modules (guix build git)
-                         (guix build utils)
-                         (guix build download-nar)
-                         (guix swh)
+                         ((guix build utils)
+                          #:select (set-path-environment-variable))
                          (ice-9 match))
 
             (define recursive?
@@ -151,38 +148,10 @@ (define* (git-fetch ref hash-algo hash
             (setvbuf (current-output-port) 'line)
             (setvbuf (current-error-port) 'line)
 
-            (or (git-fetch (getenv "git url") (getenv "git commit")
-                           #$output
-                           #:recursive? recursive?
-                           #:git-command "git")
-                (download-nar #$output)
-
-                ;; As a last resort, attempt to download from Software Heritage.
-                ;; Disable X.509 certificate verification to avoid depending
-                ;; on nss-certs--we're authenticating the checkout anyway.
-                ;; XXX: Currently recursive checkouts are not supported.
-                (and (not recursive?)
-                     (parameterize ((%verify-swh-certificate? #f))
-                       (format (current-error-port)
-                               "Trying to download from Software Heritage...~%")
-
-                       (swh-download (getenv "git url") (getenv "git commit")
-                                     #$output)
-                       (when (file-exists?
-                              (string-append #$output "/.gitattributes"))
-                         ;; Perform CR/LF conversion and other changes
-                         ;; specificied by '.gitattributes'.
-                         (invoke "git" "-C" #$output "init")
-                         (invoke "git" "-C" #$output "config" "--local"
-                                 "user.email" "you@HIDDEN")
-                         (invoke "git" "-C" #$output "config" "--local"
-                                 "user.name" "Your Name")
-                         (invoke "git" "-C" #$output "add" ".")
-                         (invoke "git" "-C" #$output "commit" "-am" "init")
-                         (invoke "git" "-C" #$output "read-tree" "--empty")
-                         (invoke "git" "-C" #$output "reset" "--hard")
-                         (delete-file-recursively
-                          (string-append #$output "/.git"))))))))))
+            (git-fetch-with-fallback (getenv "git url") (getenv "git commit")
+                                     #$output
+                                     #:recursive? recursive?
+                                     #:git-command "git")))))
 
   (mlet %store-monad ((guile (package->derivation guile system)))
     (gexp->derivation (or name "git-checkout") build
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 22:01:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 18:01:04 2023
Received: from localhost ([127.0.0.1]:37379 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoCp-00066F-PY
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:01:04 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:49250)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjoCk-00065w-Ox
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 18:00:59 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjoCU-0001NG-KD; Fri, 22 Sep 2023 18:00:38 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=/LMn6fykmjCo9uVXIBb1fg4DKUn9WjRH3XVZ0bxWEzs=; b=rcO8rx9xqmJXhDZ8iry0
 eRcuCR1u2ISTTj5f+Mwh8Yd0VsZwpt5nrJQfnq018Yx9e3eP99HX6nf+tF+OT6ajqCyWfpI1BmDBz
 7UGf/DZE0yDx3VjX1ozghIouLKr9+tsF6k76iyEdE+8vO3G88WIdgv41OBpcJHVI1DUS/z0LMtVTX
 D1BDeNAIuquEsjCKmSPhd1sUCkmRvvH7EUv06Q6rh2cxMZrftPyldqW1E/4Rp6w0lNo9tRS88zPxD
 BiOQ1tkBhzwGS+Az9vUmsTMACQ9C+oh4XAuk6q2MeADLnhY8B0Kj+T0NYtt0+eYtZtme6ZuSuQjOY
 D6cs7V7JyYuTYg==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <4eca94501c2c1e9986e1f718eeccb3eb9276dcd4.1694441831.git.ludo@HIDDEN>
 <87pm2c4tie.fsf_-_@HIDDEN>
Date: Sat, 23 Sep 2023 00:00:35 +0200
In-Reply-To: <87pm2c4tie.fsf_-_@HIDDEN> (Maxim Cournoyer's message of "Wed, 
 20 Sep 2023 13:57:45 -0400")
Message-ID: <87a5tdamws.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>
>> * configure.ac: Check for =E2=80=98git=E2=80=99 and substitute =E2=80=98=
GIT=E2=80=99.
>> * guix/config.scm.in (%git): New variable.
>> * guix/self.scm (compiled-guix): Define =E2=80=98git=E2=80=99 and pass i=
t to
>> =E2=80=98make-config.scm=E2=80=99.
>> (make-config.scm): Add #:git; emit a =E2=80=98%git=E2=80=99 variable.
>> * doc/guix.texi (Requirements): Add it.
>
> I'm a bit confused; we *both* capture git from the build environment,
> and reference git from the git-minimal Guix package -- why can't we
> strictly rely on the captured Git from the environment?

That=E2=80=99s because we have two build systems: Autotools and (guix self).
It=E2=80=99s the same change semantically, but for each of these build syst=
ems.

> Nitpick: this commit should be ordered before the daemon changes that
> requires it.

I believe that=E2=80=99s the case.

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 21:58:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 17:58:49 2023
Received: from localhost ([127.0.0.1]:37374 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjoAj-00061I-6z
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 17:58:49 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:47506)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjoAg-000615-NV
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 17:58:47 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjoAQ-0000vF-8H; Fri, 22 Sep 2023 17:58:30 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=Zx4yZ6pXoK9nHzG2+mHrMKyGNWpj8auU94W1jtJ1bXM=; b=TUcqe6m1UY5lNRtgHBgu
 jeSYvcfpZC/SZ+1msMEtFcQ5IsX8iLYqCplae3399fVRl7M1mgmaVF5GYWf8fuoqQz6X6CdzucNOB
 y9X/KG06/qNOfa+4lWbj//B9uDjLXPvpgS2QXcXsDW4jvrUH8iqmULUOSW+XhSMTrhH4rW9YDfdHK
 Jv+jSfCj6ga6NVPhODkamvPaEKcJ0EsQ03aFcMjPbXyWOxsvuQx1SZ/CCwP8PvaKwI3w8i7sfz/NE
 4v/j2nT6YFfbwrebsJGTm4koBhOpMTUZ+0kbOFfw8eejn6QgE+xzbPWzqjlLKRsha7ubGTK8OZy0p
 H+hswD3rJ+nbvQ==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@HIDDEN>
 <87ttro4tv2.fsf_-_@HIDDEN>
Date: Fri, 22 Sep 2023 23:58:27 +0200
In-Reply-To: <87ttro4tv2.fsf_-_@HIDDEN> (Maxim Cournoyer's message of "Wed, 
 20 Sep 2023 13:50:09 -0400")
Message-ID: <87fs35an0c.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>
>> Fixes <https://issues.guix.gnu.org/63331>.
>>
>> Longer-term this will remove Git from the derivation graph when its sole
>> use is to perform a checkout for a fixed-output derivation, thereby
>> breaking dependency cycles that can arise in these situations.
>>
>> * guix/git-download.scm (git-fetch): Rename to=E2=80=A6
>> (git-fetch/in-band): =E2=80=A6 this.  Deal with GIT or GUILE being #f.
>
> Nitpick, but I find this usage of dynamic default argument on top of
> default arguments inelegant; see my comments below for an
> alternative.

Ah, let me explain=E2=80=A6

>> +(define* (git-fetch/in-band ref hash-algo hash
>> +                            #:optional name
>> +                            #:key (system (%current-system))
>> +                            (guile (default-guile))
>> +                            (git (git-package)))
>> +  "Return a fixed-output derivation that performs a Git checkout of REF=
, using
>> +GIT and GUILE (thus, said derivation depends on GIT and GUILE).
>> +
>> +This method is deprecated in favor of the \"builtin:git-download\" buil=
der.
>> +It will be removed when versions of guix-daemon implementing
>> +\"builtin:git-download\" will be sufficiently widespread."
>>    (define inputs
>> -    `(("git" ,git)
>> +    `(("git" ,(or git (git-package)))
>
> Instead of using 'or' here to ensure git has a value, the default values
> should have been copied to the new definition of git-fetch.

[...]

>> +(define* (git-fetch ref hash-algo hash
>> +                    #:optional name
>> +                    #:key (system (%current-system))
>> +                    guile git)
>
> As mentioned above, I'd have kept the default values for guile and git
> here.

The reason =E2=80=98guile=E2=80=99 and =E2=80=98git=E2=80=99 default to #f =
here is because we don=E2=80=99t need
them in what we expect to be the common case eventually:

>> +  "Return a fixed-output derivation that fetches REF, a <git-reference>
>> +object.  The output is expected to have recursive hash HASH of type
>> +HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if =
#f."
>> +  (mlet %store-monad ((builtins (built-in-builders*)))
>> +    (if (member "git-download" builtins)
>> +        (git-fetch/built-in ref hash-algo hash name
>> +                            #:system system)

So it=E2=80=99s an optimization to avoid module lookups when they=E2=80=99re
unnecessary.

I hope that makes sense!

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 22 Sep 2023 21:53:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 22 17:53:57 2023
Received: from localhost ([127.0.0.1]:37369 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjo61-0005u8-FV
	for submit <at> debbugs.gnu.org; Fri, 22 Sep 2023 17:53:57 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:39028)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjo5w-0005tr-NO
 for 65866 <at> debbugs.gnu.org; Fri, 22 Sep 2023 17:53:56 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjo5e-0007cX-Uu; Fri, 22 Sep 2023 17:53:34 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=lZNWiYSFhZ16FAoPN2X25fUw4sTLU2Ga4L2tjmKybDA=; b=GRFcPpvb/rnwe/CF45AZ
 bfq0GopqjpLuCAtRoC3k5E6NFfnEVC5w9MW454COQCvKaHPIKggbJh3dx2SNREZnzzgDj/Z5u3miT
 k3EtIDEB+g7o2zYUuVsCIGa7r9kRyUidKOnR+ejiORSRPQdPdBp+JEST8epzagZKSUxY4A6izYJT8
 nOVUdK8X1DCx0VhEx+riMbuppK5ldqqQ5S+MK/dAhNYphsCOVuHqOgV1NBDZ0WFBSn4QbS5eskC3B
 LinUzmI9eUGWfbzwKatcnSzkHUA0pwW5qE1gwrcpmUlgjVJjgQiP4UZZ1cxdq4LAp8PRuAdCu8qto
 Z1AtALEvPhWG+Q==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <b25aa2dd644ac85ed72dabf3cb2098fd8c6358d0.1694441831.git.ludo@HIDDEN>
 <87fs386da4.fsf_-_@HIDDEN>
Date: Fri, 22 Sep 2023 23:53:31 +0200
In-Reply-To: <87fs386da4.fsf_-_@HIDDEN> (Maxim Cournoyer's message of "Wed, 
 20 Sep 2023 12:05:23 -0400")
Message-ID: <87jzshan8k.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

>> +(define* (git-fetch-with-fallback url commit directory
>> +                                  #:key (git-command "git") recursive?)
>> +  "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall bac=
k to
>> +alternative methods when fetching from URL fails: attempt to download a=
 nar,
>> +and if that also fails, download from the Software Heritage archive."
>> +  (or (git-fetch url commit directory
>> +                 #:recursive? recursive?
>> +                 #:git-command git-command)
>> +      (download-nar directory)
>> +
>> +      ;; As a last resort, attempt to download from Software Heritage.
>> +      ;; Disable X.509 certificate verification to avoid depending
>> +      ;; on nss-certs--we're authenticating the checkout anyway.
>> +      ;; XXX: Currently recursive checkouts are not supported.
>> +      (and (not recursive?)
>
> I know this is code moved from elsewhere, but it seems it'd be useful to
> fail hard here with a proper error instead of returning #f silently?  Or
> add support for recursive clones; was is missing to enable that?

Note that this is for the SWH fallback.  The SWH Vault doesn=E2=80=99t quite
support submodules; apparently there=E2=80=99s some work in that direction=
=C2=B9 but
it=E2=80=99s not there yet (though perhaps we could still implement it using
additional API endpoints, I=E2=80=99m not sure).

Ludo=E2=80=99.

=C2=B9 https://gitlab.softwareheritage.org/swh/devel/swh-vault/-/issues/4349




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 21 Sep 2023 07:43:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 21 03:43:08 2023
Received: from localhost ([127.0.0.1]:60970 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qjEL5-0006u0-V6
	for submit <at> debbugs.gnu.org; Thu, 21 Sep 2023 03:43:08 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:38656)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qjEL4-0006tP-0d
 for 65866 <at> debbugs.gnu.org; Thu, 21 Sep 2023 03:43:06 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qjEKk-0007Xp-BD; Thu, 21 Sep 2023 03:42:47 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=r7jgmg7Tbqs/pkl3ag7avRFLnhRgE1sSWRSD9nHg3+E=; b=i4ImfnwWsp9UU+7eDVW/
 XLJzSd8bahPsg5/kddtmEb6rJgLwGZyu+Sh1gG3rN0i2uTDyrlww2AxWErG5pj2XMe2TPZkqMCS3n
 AC7espyJLghVjOsQE10LcD4spAnFo5qdrIRI/MXo3Is5tPOYt+IsRdwuS9DuiXGLnK1LyvCmtj8Kz
 KkGU1sq0gxH9janNZEnpEiqcJ6QitnkIrymriQv+tjow9LdCVqHmvqzfCJ6R0wAMTP0iE/3oF5QB9
 GxsHih90jDLzoMS4eGEMOys2uqPMng7yA+nNNQuO6M/kjCvToLzonpprge9GSoz/vtDIfLcg3g/F6
 IEKR9QQCVQoZIQ==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <3c42634cb47dd7eaa81a198bc2d097ca74a973ed.1694441831.git.ludo@HIDDEN>
 <8734z8698q.fsf_-_@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Quintidi 5 jour =?utf-8?Q?compl=C3=A9mentaire?= an 231
 de la
 =?utf-8?Q?R=C3=A9volution=2C?= jour des =?utf-8?Q?R=C3=A9compenses?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Thu, 21 Sep 2023 09:42:30 +0200
In-Reply-To: <8734z8698q.fsf_-_@HIDDEN> (Maxim Cournoyer's message of "Wed, 
 20 Sep 2023 13:32:37 -0400")
Message-ID: <87o7hwas61.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

>> +(define* (perform-git-download drv #:optional output
>> +                               #:key print-build-trace?)
>> +  "Perform the download described by DRV, a fixed-output derivation, to
>> +OUTPUT.
>> +
>> +Note: Unless OUTPUT is #f, we don't read the value of 'out' in DRV sinc=
e the
>> +actual output is different from that when we're doing a 'bmCheck' or
>
> I'd drop the 'we's and use impersonal imperative tense or at least
> 's/when we're doing/when doing/'.

Noted.  (That=E2=80=99s actually copied from =E2=80=98perform-download=E2=
=80=99; I=E2=80=99ll fix it
there as well.)

>> +'bmRepair' build."
>> +  (derivation-let drv ((output* "out")
>
> I'd name this variable just 'out', for consistency with the others.

No because there=E2=80=99s also a parameter called =E2=80=98output=E2=80=99=
 and there=E2=80=99s
(or output output*).  But lemme see, I should remove this optional
=E2=80=98output=E2=80=99 parameter.

>> +;; 'with-temporary-git-repository' relies on the 'git' command.
>> +(unless (which (git-command)) (test-skip 1))
>
> I'd expect the 'git' command to now be required by Autoconf at build
> time, which should mean checking it here is not useful/required?

That comes in a subsequent patch.

>> +(test-assert "'git-download' built-in builder, not found"
>> +  (let* ((drv (derivation %store "git-download"
>> +                          "builtin:git-download" '()
>> +                          #:env-vars
>> +                          `(("url" . "file:///does-not-exist.git")
>> +                            ("commit"
>> +                             . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=
a"))
>> +                          #:hash-algo 'sha256
>> +                          #:hash (gcrypt:sha256 #vu8())
>> +                          #:recursive? #t)))
>> +    (guard (c ((store-protocol-error? c)
>> +               (string-contains (store-protocol-error-message c) "faile=
d")))
>> +      (build-derivations %store (list drv))
>> +      #f)))
>> +
>
> Maybe the error message compared could be more precised, if it already
> contains the necessary details?

Unfortunately it doesn=E2=80=99t (same strategy as with the existing
=E2=80=9Cbuiltin:download=E2=80=9D tests.)

Thanks for your feedback!

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 17:59:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 13:59:34 2023
Received: from localhost ([127.0.0.1]:60339 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qj1U6-0006bM-J4
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:59:34 -0400
Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]:62650)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qj1U4-0006b9-Ce
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:59:32 -0400
Received: by mail-qt1-x831.google.com with SMTP id
 d75a77b69052e-4121788397fso6020581cf.0
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 10:59:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695232757; x=1695837557; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=o7UWrNj4H8tdWLZpyMgiFTr1EVUziNJl45u/SZwBFzE=;
 b=nKeU3Apgzb+UlCt2phwXCoo/2Z8H6i3l0bGmyZ8QgfTtltCCWBTPrx1HPifBOfwDM7
 CDPW2ZhwcVL8irhqC4V1W9pKLdUpI7VwLMgwDq2Y3g+hXXflmXyOxqhP9Ykn2b8uyqbK
 FJEK2qRapcfbrllA9OuPlT5oTqCIoatR89DpB8mb2PQx6rJhjS+w76ONF0MGhn+5sdke
 qMJn8XRNlK6BPPiiq8Nl8Jl+XLvGaYUQSAGk9oUDJcqaElgPw0KRN8kGhi45dYanE4U5
 KZa1Bx06PPgD4XwezCzyQOW7AoZXpq7pda9IxR9qo+EFlXYOR4eDwOPp1o5pZdYQFslG
 cwxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695232757; x=1695837557;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=o7UWrNj4H8tdWLZpyMgiFTr1EVUziNJl45u/SZwBFzE=;
 b=eNfTB5mOlfMPOBb0jBu4f8Oe+BKNiRKBltU6D5v3j4db6+E7WX2u3GwX5y9KdPFuL6
 FD9GuyJEvYksTQ/E7wTZKzPkz00tTUBLnPrmHAvcWC6CxUXw31t0V02wAL5d1O37RWaB
 qwbmltAKREPtjoXuS10Hvws5ZCXV8YYUHpCX0zSR71omqUj2ZJdoAwcNX5AEDAMSAoRh
 oPO7bUjp1ejpxODsskJD4vJPYvYxWns9ItC998itK7oeEwmdYAa4CWOf0XBd1hG0B6Ag
 tu+en46e+mqJVyquzjqgSsRq+IIFb1kn+IhdlVIwjj9FStMaRK4vmOVLAspABbMNYGWh
 qdgQ==
X-Gm-Message-State: AOJu0YzsGoV2LD99PPLP6eG3T3bIOj8bP1k2pdY48e84lnzW25AlCzX1
 ClS6WTRVy629pc34IW/6My1KObEzvbg=
X-Google-Smtp-Source: AGHT+IEJR3i5w3Xbd3dFy1iHQn+p06af7SkiNrfNjU9jLCm06hodF8xqr6m5OYoiB1yTrNpg20iTsg==
X-Received: by 2002:ac8:5d4b:0:b0:417:97ea:4276 with SMTP id
 g11-20020ac85d4b000000b0041797ea4276mr4603143qtx.13.1695232756788; 
 Wed, 20 Sep 2023 10:59:16 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 l24-20020ac84a98000000b0041519614354sm4753284qtq.7.2023.09.20.10.59.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 10:59:16 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <1488a25806497439ab0b586df11ee689dd7fb446.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 13:59:15 -0400
In-Reply-To: <1488a25806497439ab0b586df11ee689dd7fb446.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:26 +0200")
Message-ID: <87led04tfw.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: 65866 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> * tests/channels.scm (gpg+git-available?): Check for =E2=80=98gpg-command=
=E2=80=99
> only.
> Remove all =E2=80=98test-skip=E2=80=99 statements.
> * tests/derivations.scm: Likewise.
> * tests/git-authenticate.scm: Likewise.
> * tests/git.scm: Likewise.
> * tests/import-git.scm: Likewise.

Ah!  This invalidates my previous comment about doing what this commit
does :-).

LGTM.

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 17:58:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 13:58:11 2023
Received: from localhost ([127.0.0.1]:60335 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qj1Sh-0006Yh-EL
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:58:11 -0400
Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]:51542)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qj1Sc-0006Y3-5W
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:58:06 -0400
Received: by mail-qv1-xf2c.google.com with SMTP id
 6a1803df08f44-6563022e3d3so294326d6.3
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 10:57:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695232667; x=1695837467; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=P9VzaDxZbwB0S3KMv+Hg5Yq8MWa/mgrefr1nHgDH3rk=;
 b=Y4O2jxjl0xETtsFrl8NQ0gCvwXs5oRKUPcFvf0Uk5mbZFrZt7RHtyO4dou5k329QEl
 8OkM3LF7H/6Onh97qjQq/eD2+qCk8JFvatWe+KQnl8L5SLJv7VlzNil+piPCiWs7US8X
 u/bhixzV3Bt0iPjYFeYuMtzR1sgYsdUYpvS05AK/bUMZaG3dzuDm8+mZzIVecVPWd+S/
 E23/HRo2l+jZXhi8XTAUX6ZAT39vAIXuLNhnn6RSYgiGKzA7L5zXTRBXUY8nRzZLpRFB
 3WvrARHReD+tZh9nz/zSM6DCSrnV8KVc9QdBWvBrepVZb6ra0PC1H1YV/BoyVkzXwXWh
 IoxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695232667; x=1695837467;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=P9VzaDxZbwB0S3KMv+Hg5Yq8MWa/mgrefr1nHgDH3rk=;
 b=Ltq4eiZdmVqI2+m1spcwKtwCYOiTJ2bw8Z8N26eV/8xqiCcHiollRje/x2AY5lccBe
 Zc/PJvdjJ+M18Vt5WtdUwYNJlXSHLDe5g+xIHZ3z/X0rhp94XPC9sLyjw/YTWqKRHTPz
 fkxzVHt6w8Et0H9qKhA8vyEgdQIpl6x6k8/lp6oRpsSWo2saqOGApu2VItnj8WCRnLzg
 iA27Vj7XEWkNMZK5RMsSrPW9yNeqzRAmS3xw7Hhw8Gq4qxxNpUQoHWU5c6tPlA9aAHTS
 ZVUH1/wAkgUE6X6mNlGnMaqWDrYTNeFAPSvnMS70plEQp4U3ejENEnSuNp/YucKeCiQ5
 qbDQ==
X-Gm-Message-State: AOJu0YzCW3Yf/Snad+zSp3TZBuxRdP/7IYqoIJujXn/dizEOUnzZB7uN
 JghEXvABL8B5pGBK3bucXhs=
X-Google-Smtp-Source: AGHT+IEHk9CZi1J8RsOt5i8b2rU4wfH9G/DjFPEzb2mI9o7iBFmzSfRu8b69LGim9Bw3ataxcBTgSQ==
X-Received: by 2002:a05:6214:1930:b0:655:d260:42b8 with SMTP id
 es16-20020a056214193000b00655d26042b8mr2933097qvb.19.1695232666949; 
 Wed, 20 Sep 2023 10:57:46 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 r9-20020a0ccc09000000b00655ebd053dcsm4772136qvk.82.2023.09.20.10.57.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 10:57:46 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <4eca94501c2c1e9986e1f718eeccb3eb9276dcd4.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 13:57:45 -0400
In-Reply-To: <4eca94501c2c1e9986e1f718eeccb3eb9276dcd4.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:23 +0200")
Message-ID: <87pm2c4tie.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> * configure.ac: Check for =E2=80=98git=E2=80=99 and substitute =E2=80=98G=
IT=E2=80=99.
> * guix/config.scm.in (%git): New variable.
> * guix/self.scm (compiled-guix): Define =E2=80=98git=E2=80=99 and pass it=
 to
> =E2=80=98make-config.scm=E2=80=99.
> (make-config.scm): Add #:git; emit a =E2=80=98%git=E2=80=99 variable.
> * doc/guix.texi (Requirements): Add it.

I'm a bit confused; we *both* capture git from the build environment,
and reference git from the git-minimal Guix package -- why can't we
strictly rely on the captured Git from the environment?

Nitpick: this commit should be ordered before the daemon changes that
requires it.

> ---
>  configure.ac       |  7 +++++++
>  doc/guix.texi      |  1 +
>  guix/config.scm.in |  6 +++++-
>  guix/self.scm      | 10 +++++++++-
>  4 files changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/configure.ac b/configure.ac
> index 92dede8014..d817f620cf 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -201,6 +201,13 @@ AC_SUBST([GZIP])
>  AC_SUBST([BZIP2])
>  AC_SUBST([XZ])
>=20=20
> +dnl Git is now required for the "builtin:git-download" derivation builde=
r.
> +AC_PATH_PROG([GIT], [git])
> +if test "x$GIT" =3D "x"; then
> +  AC_MSG_ERROR([Git is missing; please install it.])
> +fi
> +AC_SUBST([GIT])

Since git is now a hard requirement, the conditional checks to disable
tests relying on git in the test suite, as added in previous commits of
this series are no longer needed and should be removed.

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 17:50:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 13:50:38 2023
Received: from localhost ([127.0.0.1]:60322 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qj1LO-0006Kh-TJ
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:50:38 -0400
Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]:49464)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qj1LG-0006KI-C6
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:50:34 -0400
Received: by mail-qk1-x731.google.com with SMTP id
 af79cd13be357-770ef903e80so714185a.3
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 10:50:16 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695232211; x=1695837011; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=O2KeWgLd1w7IvPAAi7O9f8xdI1sZ7M0ORJqXKJHEbv4=;
 b=JklLIp3zzTAYFCYIy9pmzr35KOtKLfDCyBjIX3CQQeyqmxbE19suWK0/q4hjX1sIYK
 smIySxKH+sbDtmuLJ0zA3ciJqY/FqWhr1Wg/weyF783Pim93uKgb4UvcFx2XfH6Vdg3R
 vDzey6hNV8tVz0+MBNWm++Z6tyyeu9kBdMhvvirWyygLPkv7LnFuA2JOsi2fDE/lTOq2
 rQLutahSidOWiurIahOJ2svoU3BwDsRlXKBn51JAK805LcJyNKoeH8dpoLS2tRYHC81w
 kI8izp1vQ8XVih5lzylgi+fA/od2cdKm1iW/FYUF0tH7BsrFJgJDaT7+PYuBLuUCB3wT
 47tQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695232211; x=1695837011;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=O2KeWgLd1w7IvPAAi7O9f8xdI1sZ7M0ORJqXKJHEbv4=;
 b=lSuI+T+xilEnSa6PcTlvrPHNN7ynSqjS/ijU6mlGl73HTX1pNMA78ILLI0LckA3J3S
 HVCCMMLKZenAD+zX1QBsCHKOZCaN4ApLXdO6aS+RavNZ7MKarYqBekH7nZQNIXDTJICQ
 i3d8PGb0pW6SyZEg++m5qOSTxA6DKaLVmki8TtYinjzbV3j6JMeXhi0KzvDjfIkL3SJP
 mjxwCQro+eFhZ/feH8ZoVMgKe8x7Qwni7GBK7WM/5zJMSvqRv5OMqUd2owpJDt+qC1XC
 cAoduBiAAV/oijy0pPoxqf16kBcceMbFn7UMeOxBkriuT4mQ5yh2nCaGoZB0BIF5urof
 sb4w==
X-Gm-Message-State: AOJu0YwZgfc0LppcHLuViITRmLYuG+vqHmGr1Xe+WqmYy6AeokrJzayE
 T/QBx7rf/Sb3Cgb8gGsYFF4=
X-Google-Smtp-Source: AGHT+IHQEEYj43akXkemqMKQqmS2n9HVW5Ag6GFiV86fxYBKpQLFfougbH2eS5vw4phjCY/BoT/mxA==
X-Received: by 2002:a05:620a:384f:b0:770:f1e1:202a with SMTP id
 po15-20020a05620a384f00b00770f1e1202amr2697056qkn.66.1695232211181; 
 Wed, 20 Sep 2023 10:50:11 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 hz11-20020a05622a678b00b0040fcf8c0aaasm4674753qtb.54.2023.09.20.10.50.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 10:50:10 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 13:50:09 -0400
In-Reply-To: <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:25 +0200")
Message-ID: <87ttro4tv2.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello!

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> Fixes <https://issues.guix.gnu.org/63331>.
>
> Longer-term this will remove Git from the derivation graph when its sole
> use is to perform a checkout for a fixed-output derivation, thereby
> breaking dependency cycles that can arise in these situations.
>
> * guix/git-download.scm (git-fetch): Rename to=E2=80=A6
> (git-fetch/in-band): =E2=80=A6 this.  Deal with GIT or GUILE being #f.

Nitpick, but I find this usage of dynamic default argument on top of
default arguments inelegant; see my comments below for an
alternative.

> (git-fetch/built-in, built-in-builders*, git-fetch): New procedures.
> * tests/builders.scm ("git-fetch, file URI"): New test.
> ---
>  guix/git-download.scm | 68 +++++++++++++++++++++++++++++++++++++------
>  tests/builders.scm    | 29 +++++++++++++++++-
>  2 files changed, 87 insertions(+), 10 deletions(-)
>
> diff --git a/guix/git-download.scm b/guix/git-download.scm
> index f1f19397c6..505dff0a89 100644
> --- a/guix/git-download.scm
> +++ b/guix/git-download.scm
> @@ -27,6 +27,7 @@ (define-module (guix git-download)
>    #:use-module (guix records)
>    #:use-module (guix packages)
>    #:use-module (guix modules)
> +  #:use-module ((guix derivations) #:select (raw-derivation))
>    #:autoload   (guix build-system gnu) (standard-packages)
>    #:autoload   (guix download) (%download-fallback-test)
>    #:autoload   (git bindings)   (libgit2-init!)
> @@ -78,15 +79,19 @@ (define (git-package)
>    (let ((distro (resolve-interface '(gnu packages version-control))))
>      (module-ref distro 'git-minimal)))
>=20=20
> -(define* (git-fetch ref hash-algo hash
> -                    #:optional name
> -                    #:key (system (%current-system)) (guile (default-gui=
le))
> -                    (git (git-package)))
> -  "Return a fixed-output derivation that fetches REF, a <git-reference>
> -object.  The output is expected to have recursive hash HASH of type
> -HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if #=
f."
> +(define* (git-fetch/in-band ref hash-algo hash
> +                            #:optional name
> +                            #:key (system (%current-system))
> +                            (guile (default-guile))
> +                            (git (git-package)))
> +  "Return a fixed-output derivation that performs a Git checkout of REF,=
 using
> +GIT and GUILE (thus, said derivation depends on GIT and GUILE).
> +
> +This method is deprecated in favor of the \"builtin:git-download\" build=
er.
> +It will be removed when versions of guix-daemon implementing
> +\"builtin:git-download\" will be sufficiently widespread."
>    (define inputs
> -    `(("git" ,git)
> +    `(("git" ,(or git (git-package)))

Instead of using 'or' here to ensure git has a value, the default values
should have been copied to the new definition of git-fetch.

>=20=20
>        ;; When doing 'git clone --recursive', we need sed, grep, etc. to =
be
>        ;; available so that 'git submodule' works.
> @@ -154,7 +159,8 @@ (define* (git-fetch ref hash-algo hash
>                                       #:recursive? recursive?
>                                       #:git-command "git")))))
>=20=20
> -  (mlet %store-monad ((guile (package->derivation guile system)))
> +  (mlet %store-monad ((guile (package->derivation (or guile (default-gui=
le))
> +                                                  system)))
>      (gexp->derivation (or name "git-checkout") build
>=20=20
>                        ;; Use environment variables and a fixed script na=
me so
> @@ -181,6 +187,50 @@ (define* (git-fetch ref hash-algo hash
>                        #:recursive? #t
>                        #:guile-for-build guile)))
>=20=20
> +(define* (git-fetch/built-in ref hash-algo hash
> +                             #:optional name
> +                             #:key (system (%current-system)))
> +  "Return a fixed-output derivation without any dependency that performs=
 a Git
> +checkout of REF, using the \"builtin:git-download\" derivation builder."
> +  (raw-derivation (or name "git-checkout") "builtin:git-download" '()
> +                  #:system system
> +                  #:hash-algo hash-algo
> +                  #:hash hash
> +                  #:recursive? #t
> +                  #:env-vars
> +                  `(("url" . ,(object->string
> +                               (match (%download-fallback-test)
> +                                 ('content-addressed-mirrors
> +                                  "https://example.org/does-not-exist")
> +                                 (_
> +                                  (git-reference-url ref)))))
> +                    ("commit" . ,(git-reference-commit ref))
> +                    ("recursive?" . ,(object->string
> +                                      (git-reference-recursive? ref))))
> +                  #:leaked-env-vars '("http_proxy" "https_proxy"
> +                                      "LC_ALL" "LC_MESSAGES" "LANG"
> +                                      "COLUMNS")
> +                  #:local-build? #t))
> +
> +(define built-in-builders*
> +  (store-lift built-in-builders))
> +
> +(define* (git-fetch ref hash-algo hash
> +                    #:optional name
> +                    #:key (system (%current-system))
> +                    guile git)

As mentioned above, I'd have kept the default values for guile and git
here.

> +  "Return a fixed-output derivation that fetches REF, a <git-reference>
> +object.  The output is expected to have recursive hash HASH of type
> +HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if #=
f."
> +  (mlet %store-monad ((builtins (built-in-builders*)))
> +    (if (member "git-download" builtins)
> +        (git-fetch/built-in ref hash-algo hash name
> +                            #:system system)
> +        (git-fetch/in-band ref hash-algo hash name
> +                           #:system system
> +                           #:guile guile
> +                           #:git git))))
> +
>  (define (git-version version revision commit)
>    "Return the version string for packages using git-download."
>    ;; git-version is almost exclusively executed while modules are being =
loaded.
> diff --git a/tests/builders.scm b/tests/builders.scm
> index 0b5577c7a3..619caa5f31 100644
> --- a/tests/builders.scm
> +++ b/tests/builders.scm
> @@ -1,5 +1,5 @@
>  ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2018, 2019, 2021 Ludovic Co=
urt=C3=A8s <ludo@HIDDEN>
> +;;; Copyright =C2=A9 2012-2015, 2018-2019, 2021, 2023 Ludovic Court=C3=
=A8s <ludo@HIDDEN>
>  ;;; Copyright =C2=A9 2021 Lars-Dominik Braun <lars@HIDDEN>
>  ;;;
>  ;;; This file is part of GNU Guix.
> @@ -20,6 +20,7 @@
>=20=20
>  (define-module (tests builders)
>    #:use-module (guix download)
> +  #:use-module (guix git-download)
>    #:use-module (guix build-system)
>    #:use-module (guix build-system gnu)
>    #:use-module (guix build gnu-build-system)
> @@ -31,9 +32,12 @@ (define-module (tests builders)
>    #:use-module (guix base32)
>    #:use-module (guix derivations)
>    #:use-module (gcrypt hash)
> +  #:use-module ((guix hash) #:select (file-hash*))
>    #:use-module (guix tests)
> +  #:use-module (guix tests git)
>    #:use-module (guix packages)
>    #:use-module (gnu packages bootstrap)
> +  #:use-module ((ice-9 ftw) #:select (scandir))
>    #:use-module (ice-9 match)
>    #:use-module (ice-9 textual-ports)
>    #:use-module (srfi srfi-1)
> @@ -84,6 +88,29 @@ (define url-fetch*
>      (and (file-exists? out)
>           (valid-path? %store out))))
>=20=20
> +(test-equal "git-fetch, file URI"
> +  '("." ".." "a.txt" "b.scm")
> +  (let ((nonce (random-text)))
> +    (with-temporary-git-repository directory
> +        `((add "a.txt" ,nonce)
> +          (add "b.scm" "#t")
> +          (commit "Commit.")
> +          (tag "v1.0.0" "The tag."))
> +      (run-with-store %store
> +        (mlet* %store-monad ((hash
> +                              -> (file-hash* directory
> +                                             #:algorithm (hash-algorithm=
 sha256)
> +                                             #:recursive? #t))
> +                             (drv (git-fetch
> +                                   (git-reference
> +                                    (url (string-append "file://" direct=
ory))
> +                                    (commit "v1.0.0"))
> +                                   'sha256 hash
> +                                   "git-fetch-test")))
> +          (mbegin %store-monad
> +            (built-derivations (list drv))
> +            (return (scandir (derivation->output-path drv)))))))))
> +
>  (test-assert "gnu-build-system"
>    (build-system? gnu-build-system))

Pretty neat test!  LGTM.  You can add a 'Reviewed-by:' git trailer in
Magit easily with 'C-u C-c C-r' :-)

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 17:34:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 13:34:20 2023
Received: from localhost ([127.0.0.1]:60310 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qj15g-0005mP-CC
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:34:20 -0400
Received: from mail-vs1-xe2d.google.com ([2607:f8b0:4864:20::e2d]:61948)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qj15d-0005m8-F1
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:34:19 -0400
Received: by mail-vs1-xe2d.google.com with SMTP id
 ada2fe7eead31-4528bd2c3b2so33984137.3
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 10:34:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695231242; x=1695836042; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=+5ONVYpeUtp9uYJUambS8jcwx9xtUE+xoZZ78gFlKz4=;
 b=C1OBHdV2vTOzXmSUhEavZNTY2RQc7OvMaoDecZrszc4gIvj8PO+/3XmgHrDIU8HDY6
 QBf8YUTJom/2iXr7b766/8NLaQmykZjEu1nOoGONsIwmKeB5SlZwf7BTNYBSK0lyQtUA
 C4IAyMvJEHrBHKmk3beILLkj97h8htPuu6onGbeM17uuNMWOCoYw3jfRcae4ZqqslRrI
 juyIXwiqkcyj2YClQhZXkdt4lf7kc8MQXWXBJytiNgzOsYI7SazDEp/TpohKbqDoG4IC
 D/2xi3cAfAJIZNdj8ods9hgUlzmu05mMxOSqJv61ZRF+7KL4G/qextlHe/ilc6IL2vx0
 84kQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695231242; x=1695836042;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=+5ONVYpeUtp9uYJUambS8jcwx9xtUE+xoZZ78gFlKz4=;
 b=gAZcRRxyQagl1yv1/aMCvTTUhd0sA3Vq2pcvVNkcOb6K7evd3dp3A2bqsSQuzlM93p
 11Uk5zJ4+YXJgyKXRrg4L56/fgRKumguv5LhJiAteU2eOjrgn6v9x9DRqM2bdxp2BL0L
 NgSc2udugyND2eVxggmHSyhmj1m5RsCWrOOVdSE/ZgWJQw8JMXKiskTZsFV7IHJrPAAQ
 xX/D1yjjX/xy0QSGl4OM6ROqwcHe6wRNcmOSMHEAD/bUV82SC1eCB2PshoQIc8ZT1LhA
 EGBSRe6ESaab/P98qz16SqQAX9OTZlcvpRkJlsSVITBnl2kU9teM8XeL8shjm0qzLfmO
 xHuw==
X-Gm-Message-State: AOJu0Yyewb2FhkgMGn94g7iL9iNiV3CBT6LbDRFQmU4FBNpbul1JjpsK
 73nDvFEbnSV1k9FMS/aRBAA=
X-Google-Smtp-Source: AGHT+IFqUKfum/LlbP2HcIkoGyAmB/5wLdUKDqm3gfj4nEWvAnCPHpSjzJhfWECPC63SU7H+UUXoeA==
X-Received: by 2002:a67:fa4d:0:b0:452:6366:b0ed with SMTP id
 j13-20020a67fa4d000000b004526366b0edmr3501182vsq.13.1695231242218; 
 Wed, 20 Sep 2023 10:34:02 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 h4-20020a0cf204000000b0064f66ae4be3sm3546731qvk.88.2023.09.20.10.34.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 10:34:01 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <52b761e331150a637bddf696469c5e265646d461.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 13:34:00 -0400
In-Reply-To: <52b761e331150a637bddf696469c5e265646d461.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:24 +0200")
Message-ID: <87y1h04ulz.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> * guix/scripts/perform-download.scm (perform-git-download): Pass #:git-co=
mmand
> to =E2=80=98git-fetch-with-fallback=E2=80=99.
> ---
>  guix/scripts/perform-download.scm | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

LGTM!

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 17:32:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 13:32:59 2023
Received: from localhost ([127.0.0.1]:60306 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qj14M-0005jy-FW
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:32:59 -0400
Received: from mail-qv1-xf29.google.com ([2607:f8b0:4864:20::f29]:54696)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qj14J-0005ji-8e
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 13:32:56 -0400
Received: by mail-qv1-xf29.google.com with SMTP id
 6a1803df08f44-6563bca1b38so235336d6.1
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 10:32:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695231160; x=1695835960; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=58DlJNOwOK3ij5mCkfPhsbgAUOv4gXR0C+7hSWmKqKI=;
 b=MKuqvBZZt/rxLMLM6YhTJ3KqBDLD4yUIjeNrKoNenTXdn0cPphR0RBQRIoSrVxP/pW
 XOuJaCZKB1gkkJp5LeTOPj1UWuVhCvWK5N/CxCQKgAqvnZAdfYPmXsL+8sMedQ95Q92C
 BAH01HbPsQEB0jZviPcwkgfRgJs/yJQyB6tz/PqUdTGZUwPRfHd/4spX+GgBijvAMQkL
 1IW4lLSdtv+fZUA8yxMuDxOWqw+KMWMKuMNZFgzIfvPHSkMuFdNYToaL16YPiF0U66Td
 HiR0lqswlsp8FiFSjMNJnXGof7pz2uBcHLCgjxtM3KCPxLKprybhQaIb8hXynLTX5PDH
 cJ+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695231160; x=1695835960;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=58DlJNOwOK3ij5mCkfPhsbgAUOv4gXR0C+7hSWmKqKI=;
 b=gVLNdkCULCMQUFlRCs7qRVsAXIaNsfbxunM0Z8IRYYcH+w3lxwbDNpXprhwNxlU7BK
 9C64lILrDPWnvQN0lO8RvRV25Pm/VG8vlgTE4ZYVHPKo3XR9g31/Yel0g9hB2+3pmuiX
 xPBmSg2+1HLIF37JUcWkQ9cucMwiExa1N32Pf6RuQreeVlG7jra0HSk6QmzNRe93G4VB
 hYP7rrZa8qcE7vG/K9jnSYZxDH5BrWJEp6gNc2Jv1eF9SnA/lxAO1RVwGqt8Kt2N8+bT
 V+PbSZaDlWb/9b/Ay7m6sWx2ctuzVDWKW71p1WtRL6Rtawe8/CnzgXptJG0rIxctEbRz
 h9kQ==
X-Gm-Message-State: AOJu0YxmJW7wZCxxvE6bVx+09O/erKthnEIV1q38DbLW3+oEGi8WO5nc
 ++HhmFXmR32XdrQrdqetOXw=
X-Google-Smtp-Source: AGHT+IGdqtDnGEt3rSomdn5BZv1kyDXl8iXXKmYGYtfZ3xujLozpPWxj9tCpNufVHd8L6UR0Cs9CwQ==
X-Received: by 2002:a05:6214:ab1:b0:651:69d7:3d6a with SMTP id
 ew17-20020a0562140ab100b0065169d73d6amr2488553qvb.15.1695231160036; 
 Wed, 20 Sep 2023 10:32:40 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 r11-20020a0ce28b000000b00646e0411e8csm3788061qvl.30.2023.09.20.10.32.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 10:32:39 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <3c42634cb47dd7eaa81a198bc2d097ca74a973ed.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 13:32:37 -0400
In-Reply-To: <3c42634cb47dd7eaa81a198bc2d097ca74a973ed.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:22 +0200")
Message-ID: <8734z8698q.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>,
 Ludovic =?utf-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN>,
 Ricardo Wurmus <rekado@HIDDEN>, 65866 <at> debbugs.gnu.org,
 Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> From: Ludovic Court=C3=A8s <ludovic.courtes@HIDDEN>
>
> The new builder makes it possible to break cycles that occurs when the
> fixed-output derivation for the source of a dependency of =E2=80=98git=E2=
=80=99 would
> itself depend on =E2=80=98git=E2=80=99.
>
> * guix/scripts/perform-download.scm (perform-git-download): New
> procedure.
> (perform-download): Move fixed-output derivation check to=E2=80=A6
> (guix-perform-download): =E2=80=A6 here.  Invoke =E2=80=98perform-downloa=
d=E2=80=99 or
> =E2=80=98perform-git-download=E2=80=99 depending on what =E2=80=98derivat=
ion-builder=E2=80=99 returns.
> * nix/libstore/builtins.cc (builtins): Add =E2=80=9Cgit-download=E2=80=9D.
> * tests/derivations.scm ("built-in-builders"): Update.
> ("'git-download' built-in builder")
> ("'git-download' built-in builder, invalid hash")
> ("'git-download' built-in builder, invalid commit")
> ("'git-download' built-in builder, not found"): New tests.
> ---
>  guix/scripts/perform-download.scm |  52 +++++++++++++---
>  nix/libstore/builtins.cc          |   5 +-
>  tests/derivations.scm             | 100 +++++++++++++++++++++++++++++-
>  3 files changed, 145 insertions(+), 12 deletions(-)
>
> diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-dow=
nload.scm
> index c8f044e82e..a287e97528 100644
> --- a/guix/scripts/perform-download.scm
> +++ b/guix/scripts/perform-download.scm
> @@ -1,5 +1,5 @@
>  ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright =C2=A9 2016, 2017, 2018, 2020 Ludovic Court=C3=A8s <ludo@g=
nu.org>
> +;;; Copyright =C2=A9 2016-2018, 2020, 2023 Ludovic Court=C3=A8s <ludo@gn=
u.org>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -21,7 +21,8 @@ (define-module (guix scripts perform-download)
>    #:use-module (guix scripts)
>    #:use-module (guix derivations)
>    #:use-module ((guix store) #:select (derivation-path? store-path?))
> -  #:use-module (guix build download)
> +  #:autoload   (guix build download) (url-fetch)
> +  #:autoload   (guix build git) (git-fetch-with-fallback)
>    #:use-module (ice-9 match)
>    #:export (guix-perform-download))
>=20=20
> @@ -64,10 +65,6 @@ (define* (perform-download drv #:optional output
>             (drv-output (assoc-ref (derivation-outputs drv) "out"))
>             (algo       (derivation-output-hash-algo drv-output))
>             (hash       (derivation-output-hash drv-output)))
> -      (unless (and algo hash)
> -        (leave (G_ "~a is not a fixed-output derivation~%")
> -               (derivation-file-name drv)))
> -
>        ;; We're invoked by the daemon, which gives us write access to OUT=
PUT.
>        (when (url-fetch url output
>                         #:print-build-trace? print-build-trace?
> @@ -92,6 +89,33 @@ (define* (perform-download drv #:optional output
>          (when (and executable (string=3D? executable "1"))
>            (chmod output #o755))))))
>=20=20
> +(define* (perform-git-download drv #:optional output
> +                               #:key print-build-trace?)
> +  "Perform the download described by DRV, a fixed-output derivation, to
> +OUTPUT.
> +
> +Note: Unless OUTPUT is #f, we don't read the value of 'out' in DRV since=
 the
> +actual output is different from that when we're doing a 'bmCheck' or

I'd drop the 'we's and use impersonal imperative tense or at least
's/when we're doing/when doing/'.

> +'bmRepair' build."
> +  (derivation-let drv ((output* "out")

I'd name this variable just 'out', for consistency with the others.

> +                       (url "url")
> +                       (commit "commit")
> +                       (recursive? "recursive?"))
> +    (unless url
> +      (leave (G_ "~a: missing Git URL~%") (derivation-file-name drv)))
> +    (unless commit
> +      (leave (G_ "~a: missing Git commit~%") (derivation-file-name drv)))
> +
> +    (let* ((output     (or output output*))
>=20
> +           (url        (call-with-input-string url read))
> +           (recursive? (and recursive?
> +                            (call-with-input-string recursive? read)))
> +           (drv-output (assoc-ref (derivation-outputs drv) "out"))
> +           (algo       (derivation-output-hash-algo drv-output))
> +           (hash       (derivation-output-hash drv-output)))
> +      (git-fetch-with-fallback url commit output
> +                               #:recursive? recursive?))))
> +
>  (define (assert-low-privileges)
>    (when (zero? (getuid))
>      (leave (G_ "refusing to run with elevated privileges (UID ~a)~%")
> @@ -120,8 +144,20 @@ (define-command (guix-perform-download . args)
>      (match args
>        (((? derivation-path? drv) (? store-path? output))
>         (assert-low-privileges)
> -       (let ((drv (read-derivation-from-file drv)))
> -         (perform-download drv output #:print-build-trace? print-build-t=
race?)))
> +       (let* ((drv (read-derivation-from-file drv))
> +              (download (match (derivation-builder drv)
> +                          ("builtin:download" perform-download)
> +                          ("builtin:git-download" perform-git-download)
> +                          (unknown (leave (G_ "~a: unknown builtin build=
er")
> +                                          unknown))))
> +              (drv-output (assoc-ref (derivation-outputs drv) "out"))
> +              (algo       (derivation-output-hash-algo drv-output))
> +              (hash       (derivation-output-hash drv-output)))
> +         (unless (and hash algo)
> +           (leave (G_ "~a is not a fixed-output derivation~%")
> +                  (derivation-file-name drv)))
> +
> +         (download drv output #:print-build-trace? print-build-trace?)))
>        (("--version")
>         (show-version-and-exit))
>        (x
> diff --git a/nix/libstore/builtins.cc b/nix/libstore/builtins.cc
> index 4111ac4760..6bf467354a 100644
> --- a/nix/libstore/builtins.cc
> +++ b/nix/libstore/builtins.cc
> @@ -1,5 +1,5 @@
>  /* GNU Guix --- Functional package management for GNU
> -   Copyright (C) 2016, 2017, 2018, 2019 Ludovic Court=C3=A8s <ludo@HIDDEN=
rg>
> +   Copyright (C) 2016-2019, 2023 Ludovic Court=C3=A8s <ludo@HIDDEN>
>=20=20
>     This file is part of GNU Guix.
>=20=20
> @@ -58,7 +58,8 @@ static void builtinDownload(const Derivation &drv,
>=20=20
>  static const std::map<std::string, derivationBuilder> builtins =3D
>  {
> -    { "download", builtinDownload }
> +    { "download", builtinDownload },
> +    { "git-download", builtinDownload }
>  };
>=20=20
>  derivationBuilder lookupBuiltinBuilder(const std::string & name)
> diff --git a/tests/derivations.scm b/tests/derivations.scm
> index 66c777cfe7..e1312bd46b 100644
> --- a/tests/derivations.scm
> +++ b/tests/derivations.scm
> @@ -24,10 +24,15 @@ (define-module (test-derivations)
>    #:use-module (guix utils)
>    #:use-module ((gcrypt hash) #:prefix gcrypt:)
>    #:use-module (guix base32)
> +  #:use-module ((guix git) #:select (with-repository))
>    #:use-module (guix tests)
> +  #:use-module (guix tests git)
>    #:use-module (guix tests http)
>    #:use-module ((guix packages) #:select (package-derivation base32))
> -  #:use-module ((guix build utils) #:select (executable-file?))
> +  #:use-module ((guix build utils) #:select (executable-file? which))
> +  #:use-module ((guix hash) #:select (file-hash*))
> +  #:use-module ((git oid) #:select (oid->string))
> +  #:use-module ((git reference) #:select (reference-name->oid))
>    #:use-module (gnu packages bootstrap)
>    #:use-module ((gnu packages guile) #:select (guile-1.8))
>    #:use-module (srfi srfi-1)
> @@ -195,7 +200,7 @@ (define* (directory-contents dir #:optional (slurp ge=
t-bytevector-all))
>                     (stat:ino (lstat file2))))))))
>=20=20
>  (test-equal "built-in-builders"
> -  '("download")
> +  '("download" "git-download")
>    (built-in-builders %store))
>=20=20
>  (test-assert "unknown built-in builder"
> @@ -290,6 +295,97 @@ (define* (directory-contents dir #:optional (slurp g=
et-bytevector-all))
>                           get-string-all)
>                         text))))))
>=20=20
> +;; 'with-temporary-git-repository' relies on the 'git' command.
> +(unless (which (git-command)) (test-skip 1))

I'd expect the 'git' command to now be required by Autoconf at build
time, which should mean checking it here is not useful/required?

> +(test-equal "'git-download' built-in builder"
> +  `(("/a.txt" . "AAA")
> +    ("/b.scm" . "#t"))
> +  (let ((nonce (random-text)))
> +    (with-temporary-git-repository directory
> +        `((add "a.txt" "AAA")
> +          (add "b.scm" "#t")
> +          (commit ,nonce))
> +      (let* ((commit (with-repository directory repository
> +                       (oid->string
> +                        (reference-name->oid repository "HEAD"))))
> +             (drv (derivation %store "git-download"
> +                              "builtin:git-download" '()
> +                              #:env-vars
> +                              `(("url"
> +                                 . ,(object->string
> +                                     (string-append "file://" directory)=
))
> +                                ("commit" . ,commit))
> +                              #:hash-algo 'sha256
> +                              #:hash (file-hash* directory
> +                                                 #:algorithm
> +                                                 (gcrypt:hash-algorithm
> +                                                  gcrypt:sha256)
> +                                                 #:recursive? #t)
> +                              #:recursive? #t)))
> +        (build-derivations %store (list drv))
> +        (directory-contents (derivation->output-path drv) get-string-all=
)))))
> +
> +(unless (which (git-command)) (test-skip 1))
> +(test-assert "'git-download' built-in builder, invalid hash"
> +  (with-temporary-git-repository directory
> +      `((add "a.txt" "AAA")
> +        (add "b.scm" "#t")
> +        (commit "Commit!"))
> +    (let* ((commit (with-repository directory repository
> +                     (oid->string
> +                      (reference-name->oid repository "HEAD"))))
> +           (drv (derivation %store "git-download"
> +                            "builtin:git-download" '()
> +                            #:env-vars
> +                            `(("url"
> +                               . ,(object->string
> +                                   (string-append "file://" directory)))
> +                              ("commit" . ,commit))
> +                            #:hash-algo 'sha256
> +                            #:hash (gcrypt:sha256 #vu8())
> +                            #:recursive? #t)))
> +      (guard (c ((store-protocol-error? c)
> +                 (string-contains (store-protocol-error-message c) "fail=
ed")))
> +        (build-derivations %store (list drv))
> +        #f))))
> +
> +(unless (which (git-command)) (test-skip 1))
> +(test-assert "'git-download' built-in builder, invalid commit"
> +  (with-temporary-git-repository directory
> +      `((add "a.txt" "AAA")
> +        (add "b.scm" "#t")
> +        (commit "Commit!"))
> +    (let* ((drv (derivation %store "git-download"
> +                            "builtin:git-download" '()
> +                            #:env-vars
> +                            `(("url"
> +                               . ,(object->string
> +                                   (string-append "file://" directory)))
> +                              ("commit"
> +                               . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=
aa"))
> +                            #:hash-algo 'sha256
> +                            #:hash (gcrypt:sha256 #vu8())
> +                            #:recursive? #t)))
> +      (guard (c ((store-protocol-error? c)
> +                 (string-contains (store-protocol-error-message c) "fail=
ed")))
> +        (build-derivations %store (list drv))
> +        #f))))
> +
> +(test-assert "'git-download' built-in builder, not found"
> +  (let* ((drv (derivation %store "git-download"
> +                          "builtin:git-download" '()
> +                          #:env-vars
> +                          `(("url" . "file:///does-not-exist.git")
> +                            ("commit"
> +                             . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=
"))
> +                          #:hash-algo 'sha256
> +                          #:hash (gcrypt:sha256 #vu8())
> +                          #:recursive? #t)))
> +    (guard (c ((store-protocol-error? c)
> +               (string-contains (store-protocol-error-message c) "failed=
")))
> +      (build-derivations %store (list drv))
> +      #f)))
> +

Maybe the error message compared could be more precised, if it already
contains the necessary details?

Otherwise, well done!  LGTM with my above comments.

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 16:41:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 12:41:12 2023
Received: from localhost ([127.0.0.1]:60272 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qj0GG-0004Jm-Il
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:41:12 -0400
Received: from mail-qk1-x72f.google.com ([2607:f8b0:4864:20::72f]:38014)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qj0GC-0004JW-6Q
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:41:10 -0400
Received: by mail-qk1-x72f.google.com with SMTP id
 af79cd13be357-76f05b89e56so94578585a.0
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 09:40:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695228053; x=1695832853; darn=debbugs.gnu.org;
 h=cc:to:subject:message-id:date:from:in-reply-to:references
 :mime-version:from:to:cc:subject:date:message-id:reply-to;
 bh=Wfef2XG6HSemulimFO8WJkavKhqez9L8encQq70NjSk=;
 b=O+P6HeT7IZ3bvIsVnIo2Z/fAWdEQWiYYFex8SYJgr+iD1hK0PMe7yEJxLXTwgiqK5q
 U1LP8LaLC7jHqs5zUgyLAW0Fjcy1352+GzIP0PnpKeOtaRCk+nZ/a2FCob9vy5zBhWZr
 LAp+FMmpSxJguxz/qr2ZQc9eqkvMdwHY0HdSEnWG7dtaGjqpkG7KU9yClGE+d5SwIIp5
 xDn/Dlhq0cxTyHBL4nh75EUNO0S1emCvWt26E7cQ8X4WBIccFo91Ja7cECv856m6pMGb
 ii9SGqR1zFCaNlBcJB5femcuGsq+19MNSAu/61w8Cv8flkJNzQ6z4wBN6iM2KfpS6XpN
 1MoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695228053; x=1695832853;
 h=cc:to:subject:message-id:date:from:in-reply-to:references
 :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
 :reply-to;
 bh=Wfef2XG6HSemulimFO8WJkavKhqez9L8encQq70NjSk=;
 b=gyG+rN+E2cM/WQyEAtYr2mXixjw742FEoXkY7ApGDvl+lczLqHgetnq9lDVK0S6iZW
 vluKyJX44Km1nQ5mV9k31+pgprJIxIQ15/umf/dODhdYHabLsxYV8Six/1aD/5vG+jKx
 gv14wXJKAIEpm53+MLtjbfshzHkcRUVh9YxQp2f2rQrPkuzc/LyCxuaGVUMXNM6QOalH
 wYDeegDW5xvHQmGpaNWfeMQ9FnCZOSMoq7tLP9SqZMrB9Vc2YDG5kWRHNY5d1DDjnVJw
 q9DXnG1/11WkEZf2rgnwQj0GydUUWr1DPf+Ym3YxivzkwLWUpF0t8iRWvkpW0NauLsIR
 G9tw==
X-Gm-Message-State: AOJu0Yywk+rhbDinE4moRq0vHpUO2Jk6v4hKyEjaRweQnPkpBPWKPk80
 zqgd5Q+PzpWn0GT+qz81KjXdhzYKp/bGbNmjF9o=
X-Google-Smtp-Source: AGHT+IHjW72pU7paO7RAROPPgcGc9AX6hT2BhYv6XR2BneqWGxNLspv1aTxeIzuW9pRHBsvucIqMB5LR5Ne/p+1bh3o=
X-Received: by 2002:a05:6214:518b:b0:656:2e07:94cf with SMTP id
 kl11-20020a056214518b00b006562e0794cfmr2947000qvb.3.1695228053004; Wed, 20
 Sep 2023 09:40:53 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1694441830.git.ludo@HIDDEN>
 <b25aa2dd644ac85ed72dabf3cb2098fd8c6358d0.1694441831.git.ludo@HIDDEN>
 <87fs386da4.fsf_-_@HIDDEN>
In-Reply-To: <87fs386da4.fsf_-_@HIDDEN>
From: Simon Tournier <zimon.toutoune@HIDDEN>
Date: Wed, 20 Sep 2023 18:40:41 +0200
Message-ID: <CAJ3okZ3aJTbBZTB9_qYjBwSjDLcwo7=-z9VuMKtNJnkB00ERgg@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Maxim,

On Wed, 20 Sept 2023 at 18:05, Maxim Cournoyer
<maxim.cournoyer@HIDDEN> wrote:

> > +           (parameterize ((%verify-swh-certificate? #f))
> > +             (format (current-error-port)
> > +                     "Trying to download from Software Heritage...~%")
> > +
> > +             (swh-download url commit directory)
> > +             (when (file-exists?
> > +                    (string-append directory "/.gitattributes"))
> > +               ;; Perform CR/LF conversion and other changes
> > +               ;; specificied by '.gitattributes'.
> > +               (invoke git-command "-C" directory "init")
> > +               (invoke git-command "-C" directory "config" "--local"
> > +                       "user.email" "you@HIDDEN")
> > +               (invoke git-command "-C" directory "config" "--local"
> > +                       "user.name" "Your Name")
> > +               (invoke git-command "-C" directory "add" ".")
> > +               (invoke git-command "-C" directory "commit" "-am" "init")
> > +               (invoke git-command "-C" directory "read-tree" "--empty")
> > +               (invoke git-command "-C" directory "reset" "--hard")
> > +               (delete-file-recursively
> > +                (string-append directory "/.git")))))))
>
> I'm not familiar with this code, but was wondering why we need to do
> this post processing and handle .gitattributes.  I never care about this
> on my GNU/Linux machine when using 'git clone'.  Perhaps 'git fetch' is
> used directly, which is why?
>
> Time passes...  Ah!  I misread -- that's peculiar to Software Heritage.

We need to post-process .gitattributes because it depends on how the
remote host serves the files.  And yeah it mainly comes from SWH. :-)
They store the files with an uniform normalization and so without
applying .gitattributes, we do not necessary get the correct checksum.

To my knowledge, we cannot do better than these sequential Git commands.

Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 16:09:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 12:09:38 2023
Received: from localhost ([127.0.0.1]:60239 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qizle-0003U9-5D
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:09:37 -0400
Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]:46495)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qizlc-0003Tx-Dh
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:09:32 -0400
Received: by mail-qk1-x734.google.com with SMTP id
 af79cd13be357-76ef80a503fso457821785a.1
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 09:09:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695226157; x=1695830957; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=c2zcAwiJLwKM90T38l6tL6cntNTFZRkiIMXbPjMip8Y=;
 b=gFaF02Glg20Bu2s0FeKoKKErYCtk3NucqScxOhAd8IU/wzY7H7xCxtYCgFPmH/ktQF
 EHFHzAoNUE09NDtUExm6e7kJFClYfJParaFjfR10bIPey0YfeO7cBC10kUBhRTTOEBrr
 lz+O6bh2QGpiJc8O2jY76QQ8qgVujhCSuuxHytz99Zbn7mgKYCdjqL8GQOKKTV0WFbl0
 FmUpssbFqPdHOpzJqFNvRz4GzZgNtUJxhsdjbcANY3EfnpvPTOzLscJ8CGn43To9kRSv
 GgR11+kDSAVN0qNWE2fP6f4N7WCofCmMGnKdYTG5Wajp83AjbivQwcOelOBgZb0CzNOo
 3NSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695226157; x=1695830957;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=c2zcAwiJLwKM90T38l6tL6cntNTFZRkiIMXbPjMip8Y=;
 b=AHyAwiIXoM/t2n/BamA5y1Nfe2gJSSQ2oXpy4SqIMd27woMlHVLVjB1Y6Eof2Hfglm
 bt1nsvswQ7c/S+QTxR5tTmFyBG0d7/59SkF6LR4NlZah9yBPEF917iTepp0ZV0y/YhH8
 SYNoWsrNPXllzA9+mvzVnsgqoqAQi9iwevGSBJfPBaT9L0QqLHVfLdFtjFZabbw8N7T4
 g86kSt30Td6q04mFwXNClwOJVon7aI8HJinf7HyalbxDtG3rP9mVC674Nq/Im2DK/chn
 QlwMEjyyq/OSfyzD1nhQ9040zhJoQ/hk6eJk9kDZexZb0duxALami78qJkTH+2ge4GGz
 MpKQ==
X-Gm-Message-State: AOJu0YymuB1oXd0TWqcNHZmxGp9A/iB5oQNXsQnkW+6qouH9dCNGtjBh
 Z2AcUXd9JgkWG2WXiIr1miY=
X-Google-Smtp-Source: AGHT+IHh1Vpsr7Nqr2OXVdMwhgnSvH5XRqHoETHFesOIff3U2EcGLUFBdqWIE5cL9MoYVst9N0DYhw==
X-Received: by 2002:a0c:e38e:0:b0:658:50c1:b1d6 with SMTP id
 a14-20020a0ce38e000000b0065850c1b1d6mr3215328qvl.1.1695226157337; 
 Wed, 20 Sep 2023 09:09:17 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 j14-20020a0cf30e000000b0064f43efc844sm3378556qvl.32.2023.09.20.09.09.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 09:09:16 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <cd7c4b170a3063778a5c65b3b63e19b22037db07.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 12:09:15 -0400
In-Reply-To: <cd7c4b170a3063778a5c65b3b63e19b22037db07.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:21 +0200")
Message-ID: <877cok6d3o.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> Code in =E2=80=98builtins.cc=E2=80=99 only ever invokes =E2=80=98guix per=
form-download=E2=80=99 with two
> arguments.
>
> * guix/scripts/perform-download.scm (guix-perform-download): Remove
> unused one-argument clause.

LGTM!

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 16:08:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 12:08:00 2023
Received: from localhost ([127.0.0.1]:60231 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qizk8-0003RD-Cz
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:08:00 -0400
Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]:46490)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qizk5-0003Qs-I2
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:07:59 -0400
Received: by mail-qk1-x731.google.com with SMTP id
 af79cd13be357-76ef80a503fso457712285a.1
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 09:07:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695226061; x=1695830861; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=m6aKTZeK6FuANr+9cdWqgj9ypzQG8EodhwCV9YsdozM=;
 b=dAu4CPQpfKsviZG304gUkyZPKTNQjTe+Lw9JYL9vPC7mVWRl45waeVZaGhfJI6gG7d
 jobV2HGqJ23iodDiqTqkiaf8HtJ7qvzpRQhT5+HmzuZ7krOI7AJUz7pwH8dAcsBRnEkG
 q++ytShRBaPCTtBJthZIa7TLeuEe9RyTQbTDd7hi/ax11YHdFh4DiPfUX/uMRO1vVARW
 boNQ6dAG8PzFyAqrGZzAr5mLBpSOklEn9QnVJy8Dt1B+q22rNXVJwxnZTCihb/V3v9hr
 getftO/ETo6Oxbjqwewy/zVYtYer7CalTPM47kFPKGNOjGRPOTDe6Obaoel7jW9iSVyo
 vLRw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695226061; x=1695830861;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=m6aKTZeK6FuANr+9cdWqgj9ypzQG8EodhwCV9YsdozM=;
 b=UoVQGn9cwaAmssf0dt2PSdtlESw/+dkWtNzV6ARQwWBXzmS30HdwOT+D2pCiLZUUNO
 GlaWkHgKTBKBXzN9UkS/QoPe3cLUO/4h2DjzUUsKQCHjVpwhdo4b0zXnybop/NuCRtqt
 uNp5cTiLnIuOFDCcum/a338O03hdu8nvbUfzks2tt2iYv+S8fW0N5LC1vYBmiAH3Crf3
 kpPF/NFsCMyaYLI7yKVTg454euKY7zgQUN3tNEnqhc1cHHlDszKwIPlTQsgaWgud7TV2
 cBrMMXJPbcI4F96aWBGI+H7xl8RXzFUzDJKiDi+NpEAsSJ5+JkOTr3EZ1oofxK1hUC5v
 IHOg==
X-Gm-Message-State: AOJu0YxOnkMqSTd7QNmATJ6f1298DqpN/eL93WPG+5dqVucNSUtQv42+
 BeN7dVqIwZfe6OImrwFNTac=
X-Google-Smtp-Source: AGHT+IHiljZnshlvBKg8G6/regkWReMVUvVeJv0Kd6ysl9FOYEXvuUn+JukkQk4fjRZQVGm1u2c6lw==
X-Received: by 2002:a05:620a:1996:b0:76d:9f64:f638 with SMTP id
 bm22-20020a05620a199600b0076d9f64f638mr3481457qkb.51.1695226061318; 
 Wed, 20 Sep 2023 09:07:41 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 g6-20020ae9e106000000b007726002d69esm4906552qkm.10.2023.09.20.09.07.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 09:07:40 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <2cc7a7d7d2d1c801ec4529b18f2a526d8d2b07ef.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 12:07:39 -0400
In-Reply-To: <2cc7a7d7d2d1c801ec4529b18f2a526d8d2b07ef.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:20 +0200")
Message-ID: <87bkdw6d6c.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> * guix/git-download.scm (git-fetch): Honor =E2=80=98%download-fallback-te=
st=E2=80=99.

LGTM!

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 20 Sep 2023 16:05:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 20 12:05:47 2023
Received: from localhost ([127.0.0.1]:60227 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qizhy-0003Nf-F9
	for submit <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:05:47 -0400
Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]:48223)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qizhs-0003NM-Vi
 for 65866 <at> debbugs.gnu.org; Wed, 20 Sep 2023 12:05:45 -0400
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-773eee01525so84198685a.0
 for <65866 <at> debbugs.gnu.org>; Wed, 20 Sep 2023 09:05:31 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695225925; x=1695830725; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=WGxod3K5Czw+c+EKyMajxhcmfAUzy65FpA9HXcfn7a4=;
 b=mXzD1AFgK2lrRJRwQqWSci5VfxdpuD8nRb8VF5eOksF4XJ6iYyLfTvgpoNYwRcSr6k
 SNP+Ut18CX/Rb6K57Z/H10IJsBrx5bP7ZlmDh3lvtxHIj9klJxqOPJKRMNETIc0dsHKv
 Ku7KNMVPbVac9/2qmgbmYu55uvkR97+B4OBXMs4VlLJ83DUEHwUV8aAS3+Bpo2E/3dsQ
 T+EyzfKU7t22MA3BS7Tof4Y5ALqZ5TTbrYIKUOQQCkPy2hW56R3XW7fnq7ZWHZznMKjF
 +QqdU7LtshzUWhu9OB5Bg4zWYmaCXHQs5hu9Gy2JQBXMBXDwBYD2re+DeyDYtzHgotZl
 t9QA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695225925; x=1695830725;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=WGxod3K5Czw+c+EKyMajxhcmfAUzy65FpA9HXcfn7a4=;
 b=mk5ULVbpH+h1dnpmH0VNuRKHC9dbJgutHGYEA47QEorMbyjdPT8v4Lu1OSpsgQbp9x
 csMEUoxKKC2rZg6/rP0d1wL5VvoTtRxXAocX53zFUHXD8k8V1oYtHpOnAhRgO0oMoBKN
 sUibF3GB3/sViobunOlKOGJvU1SMwdfqZkW7fqhGRsCEwNmMF1HFBK79OKOwbTkEFFuK
 rvP8I5LZGLHZMXm74t0hbJMFaVdERJpXQuQYAtfGt/c4L1HbRLFAdOJn6y8ymQ6HlbB9
 9Y0rtaOYhiQoNLtJwmkENsc0F9L5qWPZBJsmHM74CXIIAJuAx7+8WTWszfxpxQK7qAs8
 IXXQ==
X-Gm-Message-State: AOJu0YwGQZgmwwnrHJ8XYl66SvSQN4UMAmaaRYkizAA7xFWb7zlk8QzJ
 EkofP6l9JaonrZCQDLwK8jQ=
X-Google-Smtp-Source: AGHT+IGRqLVEFQbkBmj+/EeDMhBL4Ua+2EiwW887oTDlxzsuNgnPZnc1WuPWRBHlP34ejzIH/WcA1A==
X-Received: by 2002:a05:620a:2495:b0:773:d4a1:c5c4 with SMTP id
 i21-20020a05620a249500b00773d4a1c5c4mr3584850qkn.54.1695225925483; 
 Wed, 20 Sep 2023 09:05:25 -0700 (PDT)
Received: from hurd (dsl-10-134-200.b2b2c.ca. [72.10.134.200])
 by smtp.gmail.com with ESMTPSA id
 p9-20020a05620a132900b0077241440be8sm4910813qkj.7.2023.09.20.09.05.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 20 Sep 2023 09:05:24 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <cover.1694441830.git.ludo@HIDDEN>
 <b25aa2dd644ac85ed72dabf3cb2098fd8c6358d0.1694441831.git.ludo@HIDDEN>
Date: Wed, 20 Sep 2023 12:05:23 -0400
In-Reply-To: <b25aa2dd644ac85ed72dabf3cb2098fd8c6358d0.1694441831.git.ludo@HIDDEN>
 ("Ludovic =?utf-8?Q?Court=C3=A8s=22's?= message of "Mon, 11 Sep 2023
 16:25:19 +0200")
Message-ID: <87fs386da4.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Josselin Poiret <dev@HIDDEN>,
 Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
 Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
 65866 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludovic,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> * guix/build/git.scm (git-fetch-with-fallback): New procedure, with code
> taken from=E2=80=A6
> * guix/git-download.scm (git-fetch): =E2=80=A6 here.
> [modules]: Remove modules that are no longer directly used in =E2=80=98bu=
ild=E2=80=99.
> [build]: Use =E2=80=98git-fetch-with-fallback=E2=80=99.

[...]

> +
> +(define* (git-fetch-with-fallback url commit directory
> +                                  #:key (git-command "git") recursive?)
> +  "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall back=
 to
> +alternative methods when fetching from URL fails: attempt to download a =
nar,
> +and if that also fails, download from the Software Heritage archive."
> +  (or (git-fetch url commit directory
> +                 #:recursive? recursive?
> +                 #:git-command git-command)
> +      (download-nar directory)
> +
> +      ;; As a last resort, attempt to download from Software Heritage.
> +      ;; Disable X.509 certificate verification to avoid depending
> +      ;; on nss-certs--we're authenticating the checkout anyway.
> +      ;; XXX: Currently recursive checkouts are not supported.
> +      (and (not recursive?)

I know this is code moved from elsewhere, but it seems it'd be useful to
fail hard here with a proper error instead of returning #f silently?  Or
add support for recursive clones; was is missing to enable that?  It's
at least easy from the git CLI.

> +           (parameterize ((%verify-swh-certificate? #f))
> +             (format (current-error-port)
> +                     "Trying to download from Software Heritage...~%")
> +
> +             (swh-download url commit directory)
> +             (when (file-exists?
> +                    (string-append directory "/.gitattributes"))
> +               ;; Perform CR/LF conversion and other changes
> +               ;; specificied by '.gitattributes'.
> +               (invoke git-command "-C" directory "init")
> +               (invoke git-command "-C" directory "config" "--local"
> +                       "user.email" "you@HIDDEN")
> +               (invoke git-command "-C" directory "config" "--local"
> +                       "user.name" "Your Name")
> +               (invoke git-command "-C" directory "add" ".")
> +               (invoke git-command "-C" directory "commit" "-am" "init")
> +               (invoke git-command "-C" directory "read-tree" "--empty")
> +               (invoke git-command "-C" directory "reset" "--hard")
> +               (delete-file-recursively
> +                (string-append directory "/.git")))))))

I'm not familiar with this code, but was wondering why we need to do
this post processing and handle .gitattributes.  I never care about this
on my GNU/Linux machine when using 'git clone'.  Perhaps 'git fetch' is
used directly, which is why?

Time passes...  Ah!  I misread -- that's peculiar to Software Heritage.

>  ;;; git.scm ends here
> diff --git a/guix/git-download.scm b/guix/git-download.scm
> index d88f4c40ee..8989b1b463 100644
> --- a/guix/git-download.scm
> +++ b/guix/git-download.scm
> @@ -116,19 +116,16 @@ (define* (git-fetch ref hash-algo hash
>    (define modules
>      (delete '(guix config)
>              (source-module-closure '((guix build git)
> -                                     (guix build utils)
> -                                     (guix build download-nar)
> -                                     (guix swh)))))
> +                                     (guix build utils)))))
>=20=20
>    (define build
>      (with-imported-modules modules
> -      (with-extensions (list guile-json gnutls   ;for (guix swh)
> +      (with-extensions (list guile-json gnutls    ;for (guix swh)
>                               guile-lzlib)
>          #~(begin
>              (use-modules (guix build git)
> -                         (guix build utils)
> -                         (guix build download-nar)
> -                         (guix swh)
> +                         ((guix build utils)
> +                          #:select (set-path-environment-variable))
>                           (ice-9 match))
>=20=20
>              (define recursive?
> @@ -151,38 +148,10 @@ (define* (git-fetch ref hash-algo hash
>              (setvbuf (current-output-port) 'line)
>              (setvbuf (current-error-port) 'line)
>=20=20
> -            (or (git-fetch (getenv "git url") (getenv "git commit")
> -                           #$output
> -                           #:recursive? recursive?
> -                           #:git-command "git")
> -                (download-nar #$output)
> -
> -                ;; As a last resort, attempt to download from Software H=
eritage.
> -                ;; Disable X.509 certificate verification to avoid depen=
ding
> -                ;; on nss-certs--we're authenticating the checkout anywa=
y.
> -                ;; XXX: Currently recursive checkouts are not supported.
> -                (and (not recursive?)
> -                     (parameterize ((%verify-swh-certificate? #f))
> -                       (format (current-error-port)
> -                               "Trying to download from Software Heritag=
e...~%")
> -
> -                       (swh-download (getenv "git url") (getenv "git com=
mit")
> -                                     #$output)
> -                       (when (file-exists?
> -                              (string-append #$output "/.gitattributes"))
> -                         ;; Perform CR/LF conversion and other changes
> -                         ;; specificied by '.gitattributes'.
> -                         (invoke "git" "-C" #$output "init")
> -                         (invoke "git" "-C" #$output "config" "--local"
> -                                 "user.email" "you@HIDDEN")
> -                         (invoke "git" "-C" #$output "config" "--local"
> -                                 "user.name" "Your Name")
> -                         (invoke "git" "-C" #$output "add" ".")
> -                         (invoke "git" "-C" #$output "commit" "-am" "ini=
t")
> -                         (invoke "git" "-C" #$output "read-tree" "--empt=
y")
> -                         (invoke "git" "-C" #$output "reset" "--hard")
> -                         (delete-file-recursively
> -                          (string-append #$output "/.git"))))))))))
> +            (git-fetch-with-fallback (getenv "git url") (getenv "git com=
mit")
> +                                     #$output
> +                                     #:recursive? recursive?
> +                                     #:git-command "git")))))
>=20=20
>    (mlet %store-monad ((guile (package->derivation guile system)))
>      (gexp->derivation (or name "git-checkout") build

LGTM.

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 19 Sep 2023 17:21:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 19 13:21:35 2023
Received: from localhost ([127.0.0.1]:57751 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qiePn-0004w5-HP
	for submit <at> debbugs.gnu.org; Tue, 19 Sep 2023 13:21:35 -0400
Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]:39402)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qiePk-0004vp-Vx
 for 65866 <at> debbugs.gnu.org; Tue, 19 Sep 2023 13:21:33 -0400
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-404724ec0dcso15373205e9.1
 for <65866 <at> debbugs.gnu.org>; Tue, 19 Sep 2023 10:21:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695144077; x=1695748877; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=qbgFrX+QD+Ix2ug6Kc3Z79JzIO2z/BlOsypprecaw28=;
 b=EThzSGE3cpJRUKq59J8wRS5tsmO8NEndsL8A8hGBtnraIFLz11SgVut82dEWZRdKwc
 5EJW5rAKcqtTEH9bSUTBt25Zp8YOvuLsf0NrBSyUkzCz7ZJv6hm2c2wLfYuwQQpYHFQH
 dRUuWRbctlE1wG4rGG793F7t1/3k/r0JHF910EZQELhQbaEZVERefuna2uXU4V/4Uvg2
 El3/YnEGRaN1KRY0iO+lkdmGIybSzt0tXOdszYKquAnz4VsFobCR1+H5SwXwo/npPiCT
 o4OMeKwUFnx5UQAQ1X6miDYKdlGbTjIidu9bx9rs8mN5f/qNMykpPVvaswGxpazvTi6H
 CzPg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695144077; x=1695748877;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=qbgFrX+QD+Ix2ug6Kc3Z79JzIO2z/BlOsypprecaw28=;
 b=VHEyMvIXuoUb4cxnmqlJT+ZsI+mwHfvFoxUXjw6daFOkIteG8MaM/U8Xw681ZdoiJZ
 7zeb5gd0QX93E1vnOTjcOvUgTZMb5itDUG/wT2KJBGY09N3OkzecI0acRq1v+EMx6lPj
 Me+2OPvZIl+6kORDseER+0OuWex8CqfHam48Vz7hUc76thFxZPApKnfFWiGztu1K1Q66
 DYlGqPH56JTy6ruzP/FKdZ7J8ZMhYyUF9qiLvCgoRzedMNsnJKa7bT9ACbonyaa/jvra
 QXz9mJZCdOXOBsntsMmH2ijAGHFybPC2SwkEZAOgBZwY2zr3S5KxNOXB1Be8Has5ch/0
 WodQ==
X-Gm-Message-State: AOJu0Yy0ZpQF8qdq97Xdf9hs2u8DUGi3pEiOVb+E7Ejc7WbmH9wlLT8R
 xucsJacSb4FDoVEIBMsIiSw=
X-Google-Smtp-Source: AGHT+IFQYjhHmwfW0gBJhAKyzGUcc9C/ljx4Yon1liF0AR1btHGCFHBV6wOOp0ySmxEpGFxnx4jATA==
X-Received: by 2002:a05:600c:3b82:b0:401:c717:ec69 with SMTP id
 n2-20020a05600c3b8200b00401c717ec69mr332639wms.4.1695144077664; 
 Tue, 19 Sep 2023 10:21:17 -0700 (PDT)
Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id
 l12-20020a1ced0c000000b003fed4fa0c19sm18579557wmh.5.2023.09.19.10.21.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 19 Sep 2023 10:21:17 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
In-Reply-To: <87ttrqcjft.fsf_-_@HIDDEN>
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
 <878r987l5t.fsf@HIDDEN> <87wmwp8rxy.fsf@HIDDEN>
 <87wmwnh9fe.fsf@HIDDEN>
 <CAJ3okZ2WSMNQ7vWViMLvGtT4Ku-fR3QKGhMC5DQEQ4w35jCWKA@HIDDEN>
 <87ttrqcjft.fsf_-_@HIDDEN>
Date: Tue, 19 Sep 2023 19:09:22 +0200
Message-ID: <87bkdy9jjx.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Tue, 19 Sep 2023 at 16:43, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

>>>   1. Merge <https://issues.guix.gnu.org/65866> with the hard Git
>>>      dependency.
>>
>> Is #65866 fixing bug#63331 (Guile-GnuTLS/Git circular dependency) [1]?
>
> Yes, as written in the cover letter.

[...]

>                            As I wrote in the cover letter, this patch
> series is the complete fix for <https://issues.guix.gnu.org/63331>.

Thanks for clarifying the cover letter:

        This patch series is a first step towards getting Git out of
        derivation graphs when it=E2=80=99s only used to fetch source code
        (origins with =E2=80=98git-fetch=E2=80=99), with the goal of fixing:

          https://issues.guix.gnu.org/63331

Because I am not native, my dictionary says, Goal: Something that is
your goal is something that you hope to achieve, especially when much
time and effort will be needed.

Sorry if, from the cover letter and my vague understanding of the code,
it was not obvious for me that merging #65866 directly close #63331.
From my understanding, #65866 was one step toward closing #63331 and not
the complete fix.

Anyway. :-)


> I suggest that we focus on the various sub-problems we=E2=80=99re trying =
to
> solve without losing sight of the big picture, yet without conflating
> them all.

The way we are trying to focus or solve these various sub-problems
depends on what we have at hand (the big picture).  Having an hard
dependency of Git means these immediate improvements by drop-in
replacements:

 + git clone =3D> 3x faster for full Guix repository
 + shallow clone =3D> 25% of improvements
 + git fetch =3D> no worry much about gc
 + commit-relation =3D> 35x faster

for an increase of the closure between 9% and 12%.  All these numbers
are for my machine and I guess they would be the order on average.

That said, I expressed my concerns about the =E2=80=9Cbig picture=E2=80=9D =
and
libgit2. :-)

Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 19 Sep 2023 14:44:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 19 10:44:21 2023
Received: from localhost ([127.0.0.1]:57640 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qibxc-0000Xw-W1
	for submit <at> debbugs.gnu.org; Tue, 19 Sep 2023 10:44:21 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:34316)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qibxa-0000Xe-LR
 for 65866 <at> debbugs.gnu.org; Tue, 19 Sep 2023 10:44:19 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qibxM-0001XJ-FV; Tue, 19 Sep 2023 10:44:04 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=8T2kdzOGwHKamCjo9hN2Cv51N1SnkhPrujUmPkF16Js=; b=OBqaPIjlD5xuhjGYh3dN
 yBfoswic0M2ZSfh3+b5FzPfB5P52hcSFTCxMuEmS/c2H40fK8M1k9CnXJrtxK3ujDlKM7+TIhTLv6
 ogUVnL9LXBdqt+TJebF6mVJUjegvGmXWej9/QjQAS7vNI/q3JRkhCtL4X8trmfm1vHHXfgoADY1Md
 ks9HIa/Js0FEb/+v9TyRRAYrR+Kz4IVBCbBw9cyqRUzWVfF2on5mVvrFPKE8sounuGuJ3JI/2dBRh
 g7lQDBylfb1FXFBjwvIkkIk+7MT8AhfOzl6/hKxDCesi8+Xb3/QLTv3qOrCL4zcwlIDp/7xq2ArB9
 WyMGKUM9jCoVbw==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Simon Tournier <zimon.toutoune@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
 <878r987l5t.fsf@HIDDEN> <87wmwp8rxy.fsf@HIDDEN>
 <87wmwnh9fe.fsf@HIDDEN>
 <CAJ3okZ2WSMNQ7vWViMLvGtT4Ku-fR3QKGhMC5DQEQ4w35jCWKA@HIDDEN>
Date: Tue, 19 Sep 2023 16:43:34 +0200
In-Reply-To: <CAJ3okZ2WSMNQ7vWViMLvGtT4Ku-fR3QKGhMC5DQEQ4w35jCWKA@HIDDEN>
 (Simon Tournier's message of "Mon, 18 Sep 2023 16:45:00 +0200")
Message-ID: <87ttrqcjft.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi Simon,

Simon Tournier <zimon.toutoune@HIDDEN> skribis:

> On Mon, 18 Sept 2023 at 15:56, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>
>> Anyhow, how about this plan:
>>
>>   1. Merge <https://issues.guix.gnu.org/65866> with the hard Git
>>      dependency.
>
> Is #65866 fixing bug#63331 (Guile-GnuTLS/Git circular dependency) [1]?

Yes, as written in the cover letter.

[...]

>>   2. When libgit2 1.7 with shallow clones is available in Guix, work on
>>      a patch to use Guile-Git for clones and evaluate it.
>
> ...we could also suggest to continue and have a complete fix of #63331
> before merging #65866.

Sorry, I don=E2=80=99t understand.  As I wrote in the cover letter, this pa=
tch
series is the complete fix for <https://issues.guix.gnu.org/63331>.

> It avoids to introduce a hard dependency which will be difficult to
> remove and let the time for this evaluation of libgit-2.1.7, no?

What this patch series sets in stone is =E2=80=9Cbuiltin:git-download=E2=80=
=9D and its
semantics.

Its implementation can change over time though: it can switch to
libgit2, to OCaml-Git, or anything that pleases us.  These are
implementation details not visible from the outside.

>> As I wrote, as an example, I don=E2=80=99t think that there could be a p=
ractical
>> implementation of (guix git-authenticate) shelling out to =E2=80=98git=
=E2=80=99.
>
> [...]
>
>> PS: I don=E2=80=99t buy the =E2=80=9Clibgit2 will disappear from Guix=E2=
=80=9D argument because
>>     it=E2=80=99s not a natural phenomenon that we=E2=80=99re observing b=
ut a willful
>>     construction.
>
> As I wrote elsewhere, Git-Annex (or Magit) are shelling out to 'git',
> IIRC.  Well, personally I do not consider that Git-Annex is slow or
> that Git-Annex does not implement features as complex as (guix
> git-authenticate).
>
> After reading [2],
>
>      I cannot imagine a viable implementation of things like =E2=80=98com=
mit-closure=E2=80=99
>      and =E2=80=98commit-relation=E2=80=99 from (guix git) done by shelli=
ng out to =E2=80=98git=E2=80=99.
>      I=E2=80=99m quite confident this would be slow and brittle.
>
> wolf came 3 days later [3] with a first rough implementation for
> 'commit-relation' using Git plumbing which is much more faster than
> the one implemented with Guile-Git.

Yes, point taken.  It=E2=80=99s not so much about whether Git-Annex is =E2=
=80=9Cless
complex=E2=80=9D, it=E2=80=99s about the level of integration needed.  But =
you don=E2=80=99t
have to take my word for it.

We=E2=80=99ve spent lots of words on the issue of a dependency on Git, and =
yet
this patch series doesn=E2=80=99t actually change much in that regard:
=E2=80=98git-fetch=E2=80=99 already uses Git.

I suggest that we focus on the various sub-problems we=E2=80=99re trying to
solve without losing sight of the big picture, yet without conflating
them all.

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 18 Sep 2023 14:45:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 18 10:45:29 2023
Received: from localhost ([127.0.0.1]:53999 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qiFVA-0007CF-Qe
	for submit <at> debbugs.gnu.org; Mon, 18 Sep 2023 10:45:29 -0400
Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]:41012)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qiFV8-0007Bq-0U
 for 65866 <at> debbugs.gnu.org; Mon, 18 Sep 2023 10:45:27 -0400
Received: by mail-qv1-xf2c.google.com with SMTP id
 6a1803df08f44-637aaaf27f1so12267476d6.0
 for <65866 <at> debbugs.gnu.org>; Mon, 18 Sep 2023 07:45:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1695048312; x=1695653112; darn=debbugs.gnu.org;
 h=content-transfer-encoding:cc:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:from:to:cc:subject:date
 :message-id:reply-to;
 bh=87G98ELrSjEDBZ3S7d7TG8bY7a7vyPWEJKzD/2IVtKk=;
 b=Y3nsPOJBEoCl5L4ptKb1oaHK34ZkfxJ77wF5F0PO/7AH+iRCvaFaaUkI1JARvQRLBG
 k/ndcvrLYJj7F+tFvFNo9n73EoYm7WRZBVOcPNvTbn44OPbZUNuF5KdidHcq1u3ZCKcs
 +Enz7ka0VpGeBBTDwldsjiwbOoVISkU2PL6G3ms4o9XQlaYKdI352wRY79fqpcyj8t2Q
 byhLSafeRMmOPNKC9XWYdFwwKzTNF0Cp5wSN47NO6GmCgBENI/zK2YNIlK85QgtkJ8sA
 xLF+t4IRkBg6530z0Mkztui/INuirhJHOioyGKKhmtyWuFCMigWNlVLIy7ew7R6q9iMt
 5DBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1695048312; x=1695653112;
 h=content-transfer-encoding:cc:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=87G98ELrSjEDBZ3S7d7TG8bY7a7vyPWEJKzD/2IVtKk=;
 b=dzTpUsR4U3HV65VY47F4qLWmYtlXXAwjzSJHumRIw8CGv/H33Ns6Zt8cIllqV6Nu3G
 UTBB+I6uLZlSi9IUf0b6J/acVUa1g3tb6eq5zli+aFdCRSPMSWlVrx4gFyhFAnngEa8F
 YARRjOvDr3w5adz+z6KIp+bU0sLr5Vot4camxbl6qdFYSRHO4qmspPX8S7Q1pwcdmjuA
 NPvroE6U/Tp+S6lGTFkfe1ZIjJ+sBG7hEOjncocVw6FJ+w76m6PQuUIwGx/ljSU3Tg0J
 EQeFmSReSDkhqKdem6wdj0RTHzUUn6Pw/6FI1qzD+Y41xDW0rCdoHSEzy2OK/6cyuZxj
 TJXQ==
X-Gm-Message-State: AOJu0YwjkqRCUk6+U4qtd+nQSNeauMUOuQmDyvLe/r/KnaZiQac9AFIQ
 lKkxbHMcLHIEKjaap4bUjYpjv+cXi2HJFapTa2U=
X-Google-Smtp-Source: AGHT+IEzaDmo98995KIOR8XLKG7rrMMq6fP0MGYgtgyQhNkNKxMEpdAleIcmHZDOJbVHb2nqxao0ZZHcGErNoyhOh78=
X-Received: by 2002:a05:6214:4519:b0:658:30c4:206 with SMTP id
 oo25-20020a056214451900b0065830c40206mr1203436qvb.0.1695048311971; Mon, 18
 Sep 2023 07:45:11 -0700 (PDT)
MIME-Version: 1.0
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
 <878r987l5t.fsf@HIDDEN> <87wmwp8rxy.fsf@HIDDEN> <87wmwnh9fe.fsf@HIDDEN>
In-Reply-To: <87wmwnh9fe.fsf@HIDDEN>
From: Simon Tournier <zimon.toutoune@HIDDEN>
Date: Mon, 18 Sep 2023 16:45:00 +0200
Message-ID: <CAJ3okZ2WSMNQ7vWViMLvGtT4Ku-fR3QKGhMC5DQEQ4w35jCWKA@HIDDEN>
Subject: Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in
 builder for Git checkouts)
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Mon, 18 Sept 2023 at 15:56, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> Anyhow, how about this plan:
>
>   1. Merge <https://issues.guix.gnu.org/65866> with the hard Git
>      dependency.

Is #65866 fixing bug#63331 (Guile-GnuTLS/Git circular dependency) [1]?

Does the merge of #65866 lead to close #63331?  Because you wrote,

    This patch series is a first step towards getting Git out of
    derivation graphs when it=E2=80=99s only used to fetch source code
    (origins with =E2=80=98git-fetch=E2=80=99), with the goal of fixing:

so...

>   2. When libgit2 1.7 with shallow clones is available in Guix, work on
>      a patch to use Guile-Git for clones and evaluate it.

...we could also suggest to continue and have a complete fix of #63331
before merging #65866.  It avoids to introduce a hard dependency which
will be difficult to remove and let the time for this evaluation of
libgit-2.1.7, no?

For what my opinion is worth, I have nothing for introducing a hard
dependency to Git but we have to be clear that 1. once introduced it
will hard to remove, 2. we will merge patches using faster Git
plumbing equivalent implementation and so 3. it will push out
Guile-Git.


> As I wrote, as an example, I don=E2=80=99t think that there could be a pr=
actical
> implementation of (guix git-authenticate) shelling out to =E2=80=98git=E2=
=80=99.

[...]

> PS: I don=E2=80=99t buy the =E2=80=9Clibgit2 will disappear from Guix=E2=
=80=9D argument because
>     it=E2=80=99s not a natural phenomenon that we=E2=80=99re observing bu=
t a willful
>     construction.

As I wrote elsewhere, Git-Annex (or Magit) are shelling out to 'git',
IIRC.  Well, personally I do not consider that Git-Annex is slow or
that Git-Annex does not implement features as complex as (guix
git-authenticate).

After reading [2],

     I cannot imagine a viable implementation of things like =E2=80=98commi=
t-closure=E2=80=99
     and =E2=80=98commit-relation=E2=80=99 from (guix git) done by shelling=
 out to =E2=80=98git=E2=80=99.
     I=E2=80=99m quite confident this would be slow and brittle.

wolf came 3 days later [3] with a first rough implementation for
'commit-relation' using Git plumbing which is much more faster than
the one implemented with Guile-Git.  Even, speaking specifically about
'commit-relation', I challenge whoever to beat "git merge-base
--is-ancestor"; life is risky: I bet my round of beers in Brussels in
the next Guix Days. :-)  Reading the C implementation of
"merge-base.c" [4] and following the various procedures, it appears to
me impossible to beat it; bah using Guile-Git cumulates various
penalties from talking to libgit2 to the Garbage Collector of Guile.

The question does not appear to me if you buy it or not. :-)  The
question is instead: do we merge code that uses Git plumbing shelling
out that is faster than the current implementation using Guile-Git?

Cheers,
simon

1: https://issues.guix.gnu.org/issue/63331

2: bug#65720: Guile-Git-managed checkouts grow way too much
Ludovic Court=C3=A8s <ludo@HIDDEN>
Fri, 08 Sep 2023 19:08:05 +0200
id:87pm2s385m.fsf@HIDDEN
https://issues.guix.gnu.org//65720
https://issues.guix.gnu.org/msgid/87pm2s385m.fsf@HIDDEN
https://yhetil.org/guix/87pm2s385m.fsf@HIDDEN

3: bug#65720: Guile-Git-managed checkouts grow way too much
wolf <wolf@HIDDEN>
Mon, 11 Sep 2023 16:42:59 +0200
id:ZP8nc1m8rN_34XV-@ws
https://issues.guix.gnu.org//65720
https://issues.guix.gnu.org/msgid/ZP8nc1m8rN_34XV-@ws
https://yhetil.org/guix/ZP8nc1m8rN_34XV-@ws

4: https://github.com/git/git/blob/bda494f4043963b9ec9a1ecd4b19b7d1cd9a0518=
/builtin/merge-base.c#L103-L115




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 18 Sep 2023 13:57:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 18 09:57:16 2023
Received: from localhost ([127.0.0.1]:53931 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qiEkU-0005Xc-PR
	for submit <at> debbugs.gnu.org; Mon, 18 Sep 2023 09:57:16 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45558)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qiEkL-0005Wr-4X
 for 65866 <at> debbugs.gnu.org; Mon, 18 Sep 2023 09:57:13 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qiEk5-0001fj-K5; Mon, 18 Sep 2023 09:56:50 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=FxtiTTB1022HVZqFFCZzqHD77WnUrInwpWkbz7H0i9o=; b=GyBu67oSkkoogWEjLMPg
 BgMy2xQxFCa9pZMlDFRlM8BsREliHkSPCcE2jPYyMb5JEgYCZiifmD6hnL3wbbw6haYwW7XzM+xuG
 tZzjd4xRzN6GSCt7qBXh84+dx93+rOsgO1h7RM2PjHkp6u6/Op+qk0LmKtoHr1x1ThUwOJ1Yt7Kio
 sHdw5TZKiNbBRlOzn+Uh/aaS85WxfsFo2UPiiBOVhj0k8+TmXfD3IKQTloh9qM5v4SAFkuLUxGszD
 s/SEIuhGuJCXUob50aCfTjZgCqUyZzkgeLFcfTDt2oFxj9ppeuWa5+JvWGN9akoJPgfg0sEX17GWZ
 gNNl0F1udA1j7A==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add
 built-in builder for Git checkouts)
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
 <878r987l5t.fsf@HIDDEN> <87wmwp8rxy.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Duodi 2 jour =?utf-8?Q?compl=C3=A9mentaire?= an 231 de
 la =?utf-8?Q?R=C3=A9volution=2C?=
 jour du =?utf-8?Q?G=C3=A9nie?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 18 Sep 2023 15:56:21 +0200
In-Reply-To: <87wmwp8rxy.fsf@HIDDEN> (Maxim Cournoyer's message of "Sat, 16
 Sep 2023 22:16:41 -0400")
Message-ID: <87wmwnh9fe.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Simon Tournier <zimon.toutoune@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hello!

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

> Yeah, 'git gc' is lacking from libgit2.  I'm not against adding
> dependency on the real 'git' CLI, but at that point, as Simon mentioned,
> I see little reason to keep libgit2 around for much longer, given it
> performs worst than git CLI in every aspect.  I doubt forking processes
> on GNU/Linux would cause a performance hit compared to using libgit2,
> especially given how optimized git appears to be (at least compared to
> libgit2).

As I wrote, as an example, I don=E2=80=99t think that there could be a prac=
tical
implementation of (guix git-authenticate) shelling out to =E2=80=98git=E2=
=80=99.

Anyhow, how about this plan:

  1. Merge <https://issues.guix.gnu.org/65866> with the hard Git
     dependency.

  2. When libgit2 1.7 with shallow clones is available in Guix, work on
     a patch to use Guile-Git for clones and evaluate it.

  3. Brainstorm on ways to address lack of GC support based on a closer
     analysis of disk usage for Guix=E2=80=99s cached checkouts.

Deal?

Ludo=E2=80=99.

PS: I don=E2=80=99t buy the =E2=80=9Clibgit2 will disappear from Guix=E2=80=
=9D argument because
    it=E2=80=99s not a natural phenomenon that we=E2=80=99re observing but =
a willful
    construction.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 17 Sep 2023 02:17:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Sep 16 22:17:02 2023
Received: from localhost ([127.0.0.1]:48774 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qhhLK-0007Q2-1z
	for submit <at> debbugs.gnu.org; Sat, 16 Sep 2023 22:17:02 -0400
Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]:62572)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1qhhLF-0007Pc-7o
 for 65866 <at> debbugs.gnu.org; Sat, 16 Sep 2023 22:17:01 -0400
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-773b18cf900so68057285a.0
 for <65866 <at> debbugs.gnu.org>; Sat, 16 Sep 2023 19:16:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1694917004; x=1695521804; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=o7NM8hOquEDbtmgyvDiJ3cIgVdWydSiNPX5KWr+O4g8=;
 b=GJUF8luLlwxHQf4J0t/dyig/qcqgbxcoyiXKeZGmujvuTKxbWOr4bvP2vDqpHxwCtT
 BJ1fOWwyqA3dCS5NI0IlaD4o0EJIBIXgRHAwQQfoFilzu+qRVw2zkvv7gODVv8Hcv0hb
 MdspvGkCq+plxCABnMnDsUG900jXat5Rs/RJZpfgk4PDrQNEN0aFyQ3OpOlaT0B85E4q
 /WsZs/1mUnI1n/GJl86ilFJeZw1s1ef/6DMrWE1H+UhcusmInGATJiWOBSV/0CMoQZkK
 f2C5NhJDc8yfjpyP+0a0X03xw9Ad20eTJIC/jXywvjswlsuI6ByzW6sV1cAimk4CqFPt
 iHHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1694917004; x=1695521804;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=o7NM8hOquEDbtmgyvDiJ3cIgVdWydSiNPX5KWr+O4g8=;
 b=YXDtyV5nOsqTYgTz5dSSHtzI3yhUQEc3h7Qcr0VB2nteg0+35vj66PxX7xzyyW2BNj
 y7I+wIK4Zz3oqEyCR/uIeyAVrgtSJH65yztvKzsc8o7q8UkmD/noqad1MMuNAPbz6vjy
 dy3UVCkxm6p9QiFEogmDKkDKoedApxIjrkypur5jvc43m4EVbkCc0vKzSeWuqQG3Wyex
 i210SGDe8YmcS+gCuFdp6FD4wJ8eFeYxYUsf82pUsa/Zv4aLezrmdAEpQYuiSNsXNbq7
 9X3b51tgoUvuJJWPgfng8wEaXMhmFeRe/OnvOpwdK9cs9RkljJQsHrtEZfKcwGcoKusz
 TVTg==
X-Gm-Message-State: AOJu0Yx5rtQVaevkGMrOu0hBG3ouON6QK1T0ZTek01yzRzvhRy4LZ+nT
 oOxcdwtZThqENpQ/jlEUIB69KWhsPfE=
X-Google-Smtp-Source: AGHT+IFpTBEqPy2BoLy5ukdf/st4thXbQNhTcDDXwdAgDnCsn9Z9pfyso9jEUmTbRRl/MKRuccBnhA==
X-Received: by 2002:ad4:5b8b:0:b0:63f:9130:4e9c with SMTP id
 11-20020ad45b8b000000b0063f91304e9cmr9341063qvp.26.1694917003899; 
 Sat, 16 Sep 2023 19:16:43 -0700 (PDT)
Received: from hurd (dsl-10-149-131.b2b2c.ca. [72.10.149.131])
 by smtp.gmail.com with ESMTPSA id
 r1-20020a0c9e81000000b00653589babcbsm1621482qvd.87.2023.09.16.19.16.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 16 Sep 2023 19:16:43 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add
 built-in builder for Git checkouts)
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
 <878r987l5t.fsf@HIDDEN>
Date: Sat, 16 Sep 2023 22:16:41 -0400
In-Reply-To: <878r987l5t.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
 =?utf-8?Q?s?= message of "Thu, 14 Sep 2023 18:51:42 +0200")
Message-ID: <87wmwp8rxy.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Simon Tournier <zimon.toutoune@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludovic,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> Hi!
>
> Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:
>
>> So given there's no technical reasons not to use libgit2, I'd use that
>> and keep the closure size down.
>
> For the record, that=E2=80=99s a 6% increase:
>
> $ guix size guix | tail -1
> total: 633.0 MiB
> $ guix size guix git-minimal | tail -1
> total: 675.7 MiB
>
> (Of course it all adds up; I=E2=80=99m not saying we can dismiss it.)

As Simon pointed out, it'd be more after wrapping 'git' with coreutils
and possible util-linux on its PATH.

> In the context of <https://issues.guix.gnu.org/65866> plus the lack of
> GC in libgit2 discussed in <https://issues.guix.gnu.org/65720>, my
> inclination is to include that hard dependency on Git.
>
> That=E2=80=99s not a happy choice for me, but it has the advantage of sol=
ving
> two immediate problems.
>
> I would revisit it as soon as libgit2 supports shallow clones (which is
> coming, as you write)

This isn't "coming", it's already been released :-).

> and GC (or a workaround to that effect).  SHA256 may also soon be a
> requirement: we=E2=80=99ll need to be able to clone repos that use it.
>
> How does that sound?

Yeah, 'git gc' is lacking from libgit2.  I'm not against adding
dependency on the real 'git' CLI, but at that point, as Simon mentioned,
I see little reason to keep libgit2 around for much longer, given it
performs worst than git CLI in every aspect.  I doubt forking processes
on GNU/Linux would cause a performance hit compared to using libgit2,
especially given how optimized git appears to be (at least compared to
libgit2).

So, I think we need to agree on the future of libgit2 in the big picture
and decide to invest in it or let it in favor of just using git.

--=20
Thanks,
Maxim




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 14 Sep 2023 17:28:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 14 13:28:33 2023
Received: from localhost ([127.0.0.1]:41032 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qgq8n-0001cF-EN
	for submit <at> debbugs.gnu.org; Thu, 14 Sep 2023 13:28:33 -0400
Received: from mail-lj1-x230.google.com ([2a00:1450:4864:20::230]:39066)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1qgq8k-0001bu-8t
 for 65866 <at> debbugs.gnu.org; Thu, 14 Sep 2023 13:28:31 -0400
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2be4bc1ae53so4061411fa.0
 for <65866 <at> debbugs.gnu.org>; Thu, 14 Sep 2023 10:28:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20221208; t=1694712498; x=1695317298; darn=debbugs.gnu.org;
 h=content-transfer-encoding:cc:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:from:to:cc:subject:date
 :message-id:reply-to;
 bh=C8BlSTojt/4l6Kcqcls8ku91AkOC2MyYO6XR0UGoaeI=;
 b=Nh5cJvWk6LKY4YvO8bzWX9bNSBwc0rdpDnlSzE+JWO/ySE2w79VWe0A5nHIsuLuIFa
 Xi8bW7vAP7kqZ96fWR1eEZretiKiRec0+39UWVafneToqzm2WiI3XRh+tCLdqNzf44Hb
 XTv5XVceoEKA3IbEdjC1uluwEVQEm8FLtB5M9EmKCDAUoey09YPbHTNktl9FhB/HY3e7
 sa2J1J7cFFyu2+COnAlTwduHgvVk1QeCN4ef9zwElGEO/FE4RQhjPl2VZ2o3VGo7sNnH
 Mr1G3yaNT5S/nsw18ayTtjdon6BHgG/vQXTxcE69umh9+kgluMSWOTaoSGtihYTyWLQl
 5FgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1694712498; x=1695317298;
 h=content-transfer-encoding:cc:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=C8BlSTojt/4l6Kcqcls8ku91AkOC2MyYO6XR0UGoaeI=;
 b=hPsyXPtfY5B2DAzZCdzhNz3C57DBE6bi9Hi0kI8iY4EJvqhYccrQ95npvyEZwaf3ha
 amFdBqDZRGlNxSpQzmR+jPtCm3Z7SC3PbcCGnoTsfrA0W2u4CAsSrBVQFuXgJxw1I+cD
 uus4+UDm+dxzqvSzXUEpV8Kf07b8bX4sXCL9ikd0xTIPmKkvJcOd4McRLdMdl2J6h3oZ
 4jvVLsumJDuNAkn5PGFMaKr9gv5ch200H7HvRaaTeyR81+Oxjey64938m66GoiUed91V
 etN7rQdgrt1riM9irlTapooM1poTixSUnJjrPfFGB+o8YyxD4DRfOSb18QjH+9v1kwRW
 F5xg==
X-Gm-Message-State: AOJu0YyoZLlDDsLauctWcZaCvqpTYgRae2iC0Ve521Nrrh7lEUjPlSit
 6bH5eHsLPqYF9llOYnVy3AzQ92m+OYhmy0EmcK8=
X-Google-Smtp-Source: AGHT+IFQqjgwi8rWAVLAtzEdTOtoOL0UNGLYy3AcpvLDi1HJ/9Eg+XN9E8It+bSpoVVqvHPnuvLZxjaUXf+3SC1XS7M=
X-Received: by 2002:a2e:a48e:0:b0:2bc:e36a:9e32 with SMTP id
 h14-20020a2ea48e000000b002bce36a9e32mr4607588lji.5.1694712497848; Thu, 14 Sep
 2023 10:28:17 -0700 (PDT)
MIME-Version: 1.0
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
 <878r987l5t.fsf@HIDDEN>
In-Reply-To: <878r987l5t.fsf@HIDDEN>
From: Simon Tournier <zimon.toutoune@HIDDEN>
Date: Thu, 14 Sep 2023 19:28:06 +0200
Message-ID: <CAJ3okZ0hzimVNtTcSsJKR-x=WKpPVtHYxshznGzecqxNHFWC5Q@HIDDEN>
Subject: Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in
 builder for Git checkouts)
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On Thu, 14 Sept 2023 at 18:51, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> For the record, that=E2=80=99s a 6% increase:
>
> --8<---------------cut here---------------start------------->8---
> $ guix size guix | tail -1
> total: 633.0 MiB
> $ guix size guix git-minimal | tail -1
> total: 675.7 MiB
> --8<---------------cut here---------------end--------------->8---
>
> (Of course it all adds up; I=E2=80=99m not saying we can dismiss it.)
>
> In the context of <https://issues.guix.gnu.org/65866> plus the lack of
> GC in libgit2 discussed in <https://issues.guix.gnu.org/65720>, my
> inclination is to include that hard dependency on Git.

And considering bug#65924 [1], it is not 6% but more.  Because
currently git-minimal is broken and coreutils and potentially
util-linux would also be part of the closure.

--8<---------------cut here---------------start------------->8---
$ guix size guix | tail -1
total: 633.0 MiB
$ guix size guix git-minimal coreutils | tail -1
total: 692.7 MiB
$ guix size guix git-minimal coreutils util-linux | tail -1
total: 706.6 MiB
--8<---------------cut here---------------end--------------->8---

Therefore. it is 9.4% or worse 11.6%.

1: bug#65924: git searches coreutils and util-linux commands in PATH
Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Wed, 13 Sep 2023 14:00:09 -0400
id:87fs3iuf6e.fsf@HIDDEN
https://yhetil.org/guix/87fs3iuf6e.fsf@HIDDEN
https://issues.guix.gnu.org/msgid/87fs3iuf6e.fsf@HIDDEN


> That=E2=80=99s not a happy choice for me, but it has the advantage of sol=
ving
> two immediate problems.

Three. :-)

Once git-minimal and plumbing Git commands around, some slow
procedures using libgit2 will be replaced by faster ones.  And I also
have in mind some Git repository normalization that differs from SWH;
having plain Git commands would also easy that part.

Just to point that the introduction of git-minimal as hard dependency
of Guix means that libgit2 will be slowly removed from the picture,
IMHO.

Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 14 Sep 2023 16:52:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 14 12:52:01 2023
Received: from localhost ([127.0.0.1]:40997 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qgpZR-0000R8-2x
	for submit <at> debbugs.gnu.org; Thu, 14 Sep 2023 12:52:01 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:43972)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qgpZN-0000Qg-BC
 for 65866 <at> debbugs.gnu.org; Thu, 14 Sep 2023 12:51:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qgpZC-00046k-1A; Thu, 14 Sep 2023 12:51:46 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=ej+RWM1R2CZTHsJg6J/jBSDXePUVe/ByfcQ+ZmWrtcs=; b=X+aX+WPzrldd4AtJ7Y8y
 ioTBv9KAQLPLMpttUi2s4c9RtpkWjWnZKKgTqhHldXNq72+tDa1XJHIm/wat52+im6C05cQtGtP4D
 4Ne5vcNrHzWF1ZhyiUbpPlM4KzpvsLWHQVP69zWLcHTIZLiLn/K7h0xZ2AWKrJRgoOVK6WXoI+1eA
 iBWJpMJNqeJHvAE4q9sH+6+gBOrwd+o6Y7eYFwLdG/AbAV55roDecbj52jCSPwTbQKterJusfUxpK
 mfFB8okfQj7p3pDRn/yeYWPtMjyfHFjt9Q5NDJMzI3f3uwZ06TfNKAOYkVN6FsPG1CxhJxDDfR71R
 nRPjqgRBhqhE1g==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add
 built-in builder for Git checkouts)
References: <871qf4ivsr.fsf@HIDDEN> <87r0n4wp0d.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Octidi 28 Fructidor an 231 de la =?utf-8?Q?R=C3=A9vo?=
 =?utf-8?Q?lution=2C?= jour du =?utf-8?Q?Ma=C3=AFs?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Thu, 14 Sep 2023 18:51:42 +0200
In-Reply-To: <87r0n4wp0d.fsf@HIDDEN> (Maxim Cournoyer's message of "Mon, 11
 Sep 2023 14:20:18 -0400")
Message-ID: <878r987l5t.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: guix-devel@HIDDEN, 65866 <at> debbugs.gnu.org,
 Simon Tournier <zimon.toutoune@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi!

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

> So given there's no technical reasons not to use libgit2, I'd use that
> and keep the closure size down.

For the record, that=E2=80=99s a 6% increase:

--8<---------------cut here---------------start------------->8---
$ guix size guix | tail -1
total: 633.0 MiB
$ guix size guix git-minimal | tail -1
total: 675.7 MiB
--8<---------------cut here---------------end--------------->8---

(Of course it all adds up; I=E2=80=99m not saying we can dismiss it.)

In the context of <https://issues.guix.gnu.org/65866> plus the lack of
GC in libgit2 discussed in <https://issues.guix.gnu.org/65720>, my
inclination is to include that hard dependency on Git.

That=E2=80=99s not a happy choice for me, but it has the advantage of solvi=
ng
two immediate problems.

I would revisit it as soon as libgit2 supports shallow clones (which is
coming, as you write) and GC (or a workaround to that effect).  SHA256
may also soon be a requirement: we=E2=80=99ll need to be able to clone repos
that use it.

How does that sound?

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 20:57:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 16:57:58 2023
Received: from localhost ([127.0.0.1]:54944 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfnyo-0004ve-6T
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 16:57:58 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:38566)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>)
 id 1qfnyj-0004vH-2F; Mon, 11 Sep 2023 16:57:57 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfnyW-0002LK-PM; Mon, 11 Sep 2023 16:57:40 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=Cs4AeZoUVoyPBu0Yy3lQgTV4tC/vJhwazmGLrJMrzWg=; b=pqX1259XHVMsa3tMzKLf
 +bSyZ1bfXnZxzJiwhqOPQqiNHLiI1KfnuoklwJsniJubRbPockcFQX9DOL2mLmhHviJ+XsymDFuXF
 JZOdSPLbmwaarOaRKovESDTnVLYBR9e0J6nrPwFQo6PiYUv0bJy9XbwBe6EEcAwKKV3WqwpBno+Xl
 m2rWCYBt6TbP5Y95bOJZ522V+QYRRwhqyrLmwwSNpSCV5Fwqy1iRMMHdqfWLxr+dwRjPSZYONsnYX
 hhZLG7fAJY9Y6fhm5U4D4apJCrU3UNQxf/h0O6HuwuP4B+AOnN5Oy6Y67fpchC6PqrXbjLLhWFKKF
 tZRCuJbMYG/6Iw==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Vivien Kraus <vivien@HIDDEN>
Subject: Re: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts
References: <877ctljs0m.fsf@HIDDEN> <87pm2osrot.fsf@HIDDEN>
 <1ae8a6379d508bf5eb69e3540f6dba5a2c0374de.camel@HIDDEN>
Date: Mon, 11 Sep 2023 22:57:35 +0200
In-Reply-To: <1ae8a6379d508bf5eb69e3540f6dba5a2c0374de.camel@HIDDEN>
 (Vivien Kraus's message of "Mon, 11 Sep 2023 17:16:14 +0200")
Message-ID: <87pm2oqvgg.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: Simon Josefsson <simon@HIDDEN>, 65866 <at> debbugs.gnu.org,
 63331 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Vivien Kraus <vivien@HIDDEN> skribis:

> Le lundi 11 septembre 2023 =C3=A0 16:36 +0200, Ludovic Court=C3=A8s a =C3=
=A9crit=C2=A0:
>> Eventually, when users are all running recent versions of
>> =E2=80=98guix-daemon=E2=80=99 with support for =E2=80=9Cbuiltin:git-down=
load=E2=80=9D (2=E2=80=934
>> years from now?), we=E2=80=99ll be able to use =E2=80=9Cbuiltin:git-down=
load=E2=80=9D
>> unconditionally and thus be sure there are no risks of
>> derivation cycles.
>
> Do foreign distros need to update their guix package as well? If that
> is the case, the provided time frame might be optimistic.

At some point, we can change clients to print a warning saying that
their daemon is outdated if it lacks =E2=80=9Cbuiltin:git-download=E2=80=9D=
.  That
should help speed things up.

>> Note that the patch series adds a hard dependency on Git.
>> This is because the existing =E2=80=98git-fetch=E2=80=99 code depends on=
 Git
>
> I applaud the switch to the regular git program from libgit2, as I
> would then be able to pull from my cgit "dumb" server instead of having
> to maintain a mirror.

Nothing changes here: =E2=80=98git-fetch=E2=80=99 already uses Git.

Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 15:11:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 11:11:31 2023
Received: from localhost ([127.0.0.1]:54406 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfiZX-0007yq-7Y
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 11:11:31 -0400
Received: from planete-kraus.eu ([89.234.140.182]:53852)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <vivien@HIDDEN>)
 id 1qfiZT-0007yX-Nk; Mon, 11 Sep 2023 11:11:29 -0400
Received: from planete-kraus.eu (localhost.lan [127.0.0.1])
 by planete-kraus.eu (OpenSMTPD) with ESMTP id d0031dfb;
 Mon, 11 Sep 2023 15:11:20 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=planete-kraus.eu; h=
 message-id:subject:from:to:cc:date:in-reply-to:references
 :content-type:content-transfer-encoding:mime-version; s=
 albinoniA; bh=bjOMfe2tOPAmK7WUDrAeEeNviz0=; b=XW9It09c4PfM0y+0wY
 qz2WS87En+jMVpQZE5VlVa8W+gCYwJgK+aLNuN2Gun2u2msjsb6FoukQbF9fFm3C
 1wRXVGrcqh+TNoMcbwtJrYBofoVmKbilj3FOJ51oCpG/W4ZfwL7mH/UNgov+2ntY
 5Srd/bhJRwYTqbjmxILpLLSOJ7L/28/NjjcmYxeq6GbuosOryTkzk4JojOyKog5P
 HIno2OVv1hd0r24rR8SkICNCal6F6y3rGt8fweCpM9D/hJLmI05r4kQY4DBf1gzM
 TPVuKBNHlMxlMgz+w0rdMvK9QKqKBgz6ivf0iqByVovLlFPeJExZtwvb/tYkPST8
 trQA==
Received: by planete-kraus.eu (OpenSMTPD) with ESMTPSA id e002078c
 (TLSv1.3:TLS_CHACHA20_POLY1305_SHA256:256:NO); 
 Mon, 11 Sep 2023 15:11:20 +0000 (UTC)
Message-ID: <1ae8a6379d508bf5eb69e3540f6dba5a2c0374de.camel@HIDDEN>
Subject: Re: bug#63331: Guile-GnuTLS/Git circular dependency and built-in
 git checkouts
From: Vivien Kraus <vivien@HIDDEN>
To: Ludovic =?ISO-8859-1?Q?Court=E8s?= <ludo@HIDDEN>, 63331 <at> debbugs.gnu.org
Date: Mon, 11 Sep 2023 17:16:14 +0200
In-Reply-To: <87pm2osrot.fsf@HIDDEN>
References: <877ctljs0m.fsf@HIDDEN> <87pm2osrot.fsf@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.48.4 
MIME-Version: 1.0
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 65866
Cc: Simon Josefsson <simon@HIDDEN>, 65866 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello!

Le lundi 11 septembre 2023 =C3=A0 16:36 +0200, Ludovic Court=C3=A8s a =C3=
=A9crit=C2=A0:
> Eventually, when users are all running recent versions of
> =E2=80=98guix-daemon=E2=80=99 with support for =E2=80=9Cbuiltin:git-downl=
oad=E2=80=9D (2=E2=80=934
> years from now?), we=E2=80=99ll be able to use =E2=80=9Cbuiltin:git-downl=
oad=E2=80=9D
> unconditionally and thus be sure there are no risks of
> derivation cycles.

Do foreign distros need to update their guix package as well? If that
is the case, the provided time frame might be optimistic.

> Note that the patch series adds a hard dependency on Git.
> This is because the existing =E2=80=98git-fetch=E2=80=99 code depends on =
Git

I applaud the switch to the regular git program from libgit2, as I
would then be able to pull from my cgit "dumb" server instead of having
to maintain a mirror.

Vivien




Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:26:07 2023
Received: from localhost ([127.0.0.1]:54319 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhra-0006iO-Au
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:26:07 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:55028)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrP-0006fU-Df
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:56 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrG-00034U-1C; Mon, 11 Sep 2023 10:25:46 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=u9+hTSEMumwCQQ/u1GucMUnU5Oqp6LOTsklYGo7TPpc=; b=qTADEbxmokHSswl6hdhb
 N8E4qb8u1SltapQ0dBY9TvnTYsGIoySbLMXkJHERyV9M2DZYY5RkmtMZcbpNsWGQXXlX4MTAfT8d7
 gRAwysyPTxcUHugqGYZcY1CrCcZb3KG73dkgXxWLgsLrOfSwOYHQlGyG1I5Bso+4JAYdN6qaGCcL1
 OacqJoVaGOBqN7rZAJMl7fijWOziv8ivkTfoiHqVucAAEOFQUpOUaKpicLyHWYDGJWUGqToHFpKey
 QJgpnDipHm2m+uY0Bq02CDjdvWi+ZE9mvYjzhIpPoJ5JHFWiCcIm+xi9fMz5K0cjqEvezLYir6CFX
 1EUdRmY0QF4m1w==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 8/8] =?UTF-8?q?tests:=20Assume=20=E2=80=98git=E2=80=99=20i?=
 =?UTF-8?q?s=20always=20available.?=
Date: Mon, 11 Sep 2023 16:25:26 +0200
Message-ID: <1488a25806497439ab0b586df11ee689dd7fb446.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* tests/channels.scm (gpg+git-available?): Check for ‘gpg-command’
only.
Remove all ‘test-skip’ statements.
* tests/derivations.scm: Likewise.
* tests/git-authenticate.scm: Likewise.
* tests/git.scm: Likewise.
* tests/import-git.scm: Likewise.
---
 tests/channels.scm         |  7 +------
 tests/derivations.scm      |  6 +-----
 tests/git-authenticate.scm |  1 -
 tests/git.scm              | 10 ----------
 tests/import-git.scm       | 18 ------------------
 5 files changed, 2 insertions(+), 40 deletions(-)

diff --git a/tests/channels.scm b/tests/channels.scm
index 62312e240c..6c4276deb4 100644
--- a/tests/channels.scm
+++ b/tests/channels.scm
@@ -50,7 +50,7 @@ (define-module (test-channels)
   #:use-module (ice-9 match))
 
 (define (gpg+git-available?)
-  (and (which (git-command))
+  (and #t                                         ;'git' is always available
        (which (gpg-command)) (which (gpgconf-command))))
 
 (define commit-id-string
@@ -196,7 +196,6 @@ (define channel-metadata-dependencies
                                           "abc1234")))
                          instances)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-channel-instances #:validate-pull"
   'descendant
 
@@ -306,7 +305,6 @@ (define channel-metadata-dependencies
                (depends? drv3
                          (list drv2 drv0) (list))))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "channel-news, no news"
   '()
   (with-temporary-git-repository directory
@@ -318,7 +316,6 @@ (define channel-metadata-dependencies
             (latest  (reference-name->oid repository "HEAD")))
         (channel-news-for-commit channel (oid->string latest))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "channel-news, one entry"
   (with-temporary-git-repository directory
       `((add ".guix-channel"
@@ -406,7 +403,6 @@ (define channel-metadata-dependencies
                          (channel-news-for-commit channel commit5 commit1))
                     '(#f "tag-for-first-news-entry")))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "channel-news, annotated tag"
   (with-temporary-git-repository directory
       `((add ".guix-channel"
@@ -453,7 +449,6 @@ (define channel-metadata-dependencies
                          (channel-news-for-commit channel commit2))
                     (list commit1)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "latest-channel-instances, missing introduction for 'guix'"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
diff --git a/tests/derivations.scm b/tests/derivations.scm
index e1312bd46b..0e87778981 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -29,7 +29,7 @@ (define-module (test-derivations)
   #:use-module (guix tests git)
   #:use-module (guix tests http)
   #:use-module ((guix packages) #:select (package-derivation base32))
-  #:use-module ((guix build utils) #:select (executable-file? which))
+  #:use-module ((guix build utils) #:select (executable-file?))
   #:use-module ((guix hash) #:select (file-hash*))
   #:use-module ((git oid) #:select (oid->string))
   #:use-module ((git reference) #:select (reference-name->oid))
@@ -295,8 +295,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
                          get-string-all)
                        text))))))
 
-;; 'with-temporary-git-repository' relies on the 'git' command.
-(unless (which (git-command)) (test-skip 1))
 (test-equal "'git-download' built-in builder"
   `(("/a.txt" . "AAA")
     ("/b.scm" . "#t"))
@@ -325,7 +323,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
         (build-derivations %store (list drv))
         (directory-contents (derivation->output-path drv) get-string-all)))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "'git-download' built-in builder, invalid hash"
   (with-temporary-git-repository directory
       `((add "a.txt" "AAA")
@@ -349,7 +346,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
         (build-derivations %store (list drv))
         #f))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "'git-download' built-in builder, invalid commit"
   (with-temporary-git-repository directory
       `((add "a.txt" "AAA")
diff --git a/tests/git-authenticate.scm b/tests/git-authenticate.scm
index c063920c12..4de223d422 100644
--- a/tests/git-authenticate.scm
+++ b/tests/git-authenticate.scm
@@ -44,7 +44,6 @@ (define (gpg+git-available?)
 
 (test-begin "git-authenticate")
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "unsigned commits"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
diff --git a/tests/git.scm b/tests/git.scm
index 9c944d65b1..ad43435b67 100644
--- a/tests/git.scm
+++ b/tests/git.scm
@@ -21,7 +21,6 @@ (define-module (test-git)
   #:use-module (git)
   #:use-module (guix git)
   #:use-module (guix tests git)
-  #:use-module (guix build utils)
   #:use-module ((guix utils) #:select (call-with-temporary-directory))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-64)
@@ -33,8 +32,6 @@ (define-module (test-git)
 
 (test-begin "git")
 
-;; 'with-temporary-git-repository' relies on the 'git' command.
-(unless (which (git-command)) (test-skip 1))
 (test-assert "commit-difference, linear history"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
@@ -61,7 +58,6 @@ (define-module (test-git)
              ;; empty list.
              (null? (commit-difference commit1 commit4)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "commit-difference, fork"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
@@ -101,7 +97,6 @@ (define-module (test-git)
              (lset= eq? (commit-difference master4 master2)
                     (list master4 merge master3 devel1 devel2)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "commit-difference, excluded commits"
   (with-temporary-git-repository directory
       '((add "a.txt" "A")
@@ -126,7 +121,6 @@ (define-module (test-git)
                     (list commit4))
              (null? (commit-difference commit4 commit1 (list commit5))))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "commit-relation"
   '(self                                          ;master3 master3
     ancestor                                      ;master1 master3
@@ -166,7 +160,6 @@ (define-module (test-git)
               (commit-relation master1 merge)
               (commit-relation merge master1))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "commit-descendant?"
   '((master3 master3 => #t)
     (master1 master3 => #f)
@@ -216,7 +209,6 @@ (define-module (test-git)
                   (master1 merge)
                   (merge master1)))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "remote-refs"
   '("refs/heads/develop" "refs/heads/master"
     "refs/tags/v1.0" "refs/tags/v1.1")
@@ -231,7 +223,6 @@ (define-module (test-git)
         (tag "v1.1" "release-1.1"))
     (remote-refs directory)))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "remote-refs: only tags"
  '("refs/tags/v1.0" "refs/tags/v1.1")
   (with-temporary-git-repository directory
@@ -243,7 +234,6 @@ (define-module (test-git)
         (tag "v1.1" "Release 1.1"))
     (remote-refs directory #:tags? #t)))
 
-(unless (which (git-command)) (test-skip 1))
 (test-assert "update-cached-checkout, tag"
   (call-with-temporary-directory
    (lambda (cache)
diff --git a/tests/import-git.scm b/tests/import-git.scm
index f1bce154bb..20255dedb3 100644
--- a/tests/import-git.scm
+++ b/tests/import-git.scm
@@ -24,7 +24,6 @@ (define-module (test-import-git)
   #:use-module (guix import git)
   #:use-module (guix git-download)
   #:use-module (guix tests git)
-  #:use-module (guix build utils)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-64))
 
@@ -46,7 +45,6 @@ (define* (make-package directory version #:optional (properties '()))
         (base32
          "0000000000000000000000000000000000000000000000000000"))))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: no custom prefix, suffix, and delimiter"
   "1.0.1"
   (with-temporary-git-repository directory
@@ -56,7 +54,6 @@ (define* (make-package directory version #:optional (properties '()))
     (let ((package (make-package directory "1.0.0")))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom prefix, no suffix and delimiter"
   "1.0.1"
   (with-temporary-git-repository directory
@@ -67,7 +64,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-prefix . "prefix-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom suffix, no prefix and delimiter"
   "1.0.1"
   (with-temporary-git-repository directory
@@ -78,7 +74,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-suffix . "-suffix-[0-9]*")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom delimiter, no prefix and suffix"
   "2021.09.07"
   (with-temporary-git-repository directory
@@ -89,7 +84,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-version-delimiter . "-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: empty delimiter, no prefix and suffix"
   "20210907"
   (with-temporary-git-repository directory
@@ -100,7 +94,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((release-tag-version-delimiter . "")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom prefix and suffix, no delimiter"
   "2.0.0"
   (with-temporary-git-repository directory
@@ -112,7 +105,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-suffix . "suffix-[0-9]")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: custom prefix, suffix, and delimiter"
   "2.0.0"
   (with-temporary-git-repository directory
@@ -125,7 +117,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "_")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: only pre-releases available"
   #f
   (with-temporary-git-repository directory
@@ -135,7 +126,6 @@ (define* (make-package directory version #:optional (properties '()))
     (let ((package (make-package directory "1.0.0")))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases"
   "2.0.0-rc1"
   (with-temporary-git-repository directory
@@ -146,7 +136,6 @@ (define* (make-package directory version #:optional (properties '()))
                                  '((accept-pre-releases? . #t)))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom prefix"
   "2.0.0-rc1"
   (with-temporary-git-repository directory
@@ -158,7 +147,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-prefix . "version-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix"
   "2.0.0-rc1"
   (with-temporary-git-repository directory
@@ -170,7 +158,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-suffix . "-suffix")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, delimiter conflicts with pre-release part"
   "2.0.0_alpha"
   (with-temporary-git-repository directory
@@ -182,7 +169,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "_")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix and prefix"
   "2.0.0-alpha"
   (with-temporary-git-repository directory
@@ -195,7 +181,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-suffix . "-suffix")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix, prefix, and delimiter"
   "2.0.0-alpha"
   (with-temporary-git-repository directory
@@ -209,7 +194,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "-")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: accept pre-releases, no delimiter, and custom suffix, prefix"
   "2alpha"
   (with-temporary-git-repository directory
@@ -223,7 +207,6 @@ (define* (make-package directory version #:optional (properties '()))
                                    (release-tag-version-delimiter . "")))))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: no tags found"
   #f
   (with-temporary-git-repository directory
@@ -232,7 +215,6 @@ (define* (make-package directory version #:optional (properties '()))
     (let ((package (make-package directory "1.0.0")))
       (latest-git-tag-version package))))
 
-(unless (which (git-command)) (test-skip 1))
 (test-equal "latest-git-tag-version: no valid tags found"
   #f
   (with-temporary-git-repository directory
-- 
2.41.0





Information forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:26:06 2023
Received: from localhost ([127.0.0.1]:54310 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrZ-0006iE-Ea
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:26:06 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54996)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrM-0006fE-Nl
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:56 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrD-000312-BV; Mon, 11 Sep 2023 10:25:43 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=kdp9llTbF8+3aSxiW2tzTYl/MmzrtCqW5SxqJyuizdM=; b=d+aomIVl1LJSbNczrvGW
 UQj43n9l5aVNFvL6AtcDLyJEFBi80mjkZio1zbGYuMqSOttlIU0LvZ4tKPwukb0UpmRmYS/0whxHt
 ZM3yyz54PxzpkbPd/7Oe25qx1HFWMsTP6MbPkEiIsJS7ukZgF39SH/XEK6TdoSZVeLp+PQBw5MyR9
 M+lVLxuSYv6M01hKDkAy4SG87iiLc0jhcwH/e/leVuEswr3UIC2o45f6TYTTb8b87Pq0Ila07X2cC
 MXN4YIM5p4GMM0u+JnEV7Ai0TO0U3lYZgmfOqA4wlpteXvmd0bBywPlJcT3lXQ4OiyEggCaZGd69M
 gbHb4AFHsd7ZWg==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 5/8] build: Add dependency on Git.
Date: Mon, 11 Sep 2023 16:25:23 +0200
Message-ID: <4eca94501c2c1e9986e1f718eeccb3eb9276dcd4.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* configure.ac: Check for ‘git’ and substitute ‘GIT’.
* guix/config.scm.in (%git): New variable.
* guix/self.scm (compiled-guix): Define ‘git’ and pass it to
‘make-config.scm’.
(make-config.scm): Add #:git; emit a ‘%git’ variable.
* doc/guix.texi (Requirements): Add it.
---
 configure.ac       |  7 +++++++
 doc/guix.texi      |  1 +
 guix/config.scm.in |  6 +++++-
 guix/self.scm      | 10 +++++++++-
 4 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index 92dede8014..d817f620cf 100644
--- a/configure.ac
+++ b/configure.ac
@@ -201,6 +201,13 @@ AC_SUBST([GZIP])
 AC_SUBST([BZIP2])
 AC_SUBST([XZ])
 
+dnl Git is now required for the "builtin:git-download" derivation builder.
+AC_PATH_PROG([GIT], [git])
+if test "x$GIT" = "x"; then
+  AC_MSG_ERROR([Git is missing; please install it.])
+fi
+AC_SUBST([GIT])
+
 LIBGCRYPT_LIBDIR="no"
 LIBGCRYPT_PREFIX="no"
 
diff --git a/doc/guix.texi b/doc/guix.texi
index 339dcb2a41..a2520ce89d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1011,6 +1011,7 @@ Requirements
 @item
 @uref{https://gitlab.com/guile-git/guile-git, Guile-Git}, version 0.5.0
 or later;
+@item @uref{https://git-scm.com, Git} (yes, both!);
 @item @uref{https://savannah.nongnu.org/projects/guile-json/, Guile-JSON}
 4.3.0 or later;
 @item @url{https://www.gnu.org/software/make/, GNU Make}.
diff --git a/guix/config.scm.in b/guix/config.scm.in
index d582d91d74..62e15dd713 100644
--- a/guix/config.scm.in
+++ b/guix/config.scm.in
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2018, 2019, 2021 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2012-2016, 2018-2019, 2021, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;; Copyright © 2017 Caleb Ristvedt <caleb.ristvedt@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -35,6 +35,7 @@ (define-module (guix config)
             %config-directory
 
             %system
+            %git
             %gzip
             %bzip2
             %xz))
@@ -109,6 +110,9 @@ (define %config-directory
 (define %system
   "@guix_system@")
 
+(define %git
+  "@GIT@")
+
 (define %gzip
   "@GZIP@")
 
diff --git a/guix/self.scm b/guix/self.scm
index 81a36e007f..41c5f40786 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -68,6 +68,7 @@ (define %packages
       ("gzip"               . ,(ref 'compression 'gzip))
       ("bzip2"              . ,(ref 'compression 'bzip2))
       ("xz"                 . ,(ref 'compression 'xz))
+      ("git-minimal"        . ,(ref 'version-control 'git-minimal))
       ("po4a"               . ,(ref 'gettext 'po4a))
       ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
       ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
@@ -825,6 +826,9 @@ (define* (compiled-guix source #:key
   (define guile-lzma
     (specification->package "guile-lzma"))
 
+  (define git
+    (specification->package "git-minimal"))
+
   (define dependencies
     (append-map transitive-package-dependencies
                 (list guile-gcrypt guile-gnutls guile-git guile-avahi
@@ -998,6 +1002,7 @@ (define* (compiled-guix source #:key
                     => ,(make-config.scm #:gzip gzip
                                          #:bzip2 bzip2
                                          #:xz xz
+                                         #:git git
                                          #:package-name
                                          %guix-package-name
                                          #:package-version
@@ -1103,7 +1108,7 @@ (define %default-config-variables
     (%storedir . "/gnu/store")
     (%sysconfdir . "/etc")))
 
-(define* (make-config.scm #:key gzip xz bzip2
+(define* (make-config.scm #:key gzip xz bzip2 git
                           (package-name "GNU Guix")
                           (package-version "0")
                           (channel-metadata #f)
@@ -1133,6 +1138,7 @@ (define* (make-config.scm #:key gzip xz bzip2
                                %state-directory
                                %store-database-directory
                                %config-directory
+                               %git
                                %gzip
                                %bzip2
                                %xz))
@@ -1175,6 +1181,8 @@ (define* (make-config.scm #:key gzip xz bzip2
                      ;; information is used by (guix describe).
                      '#$channel-metadata)
 
+                   (define %git
+                     #+(and git (file-append git "/bin/git")))
                    (define %gzip
                      #+(and gzip (file-append gzip "/bin/gzip")))
                    (define %bzip2
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:26:00 2023
Received: from localhost ([127.0.0.1]:54280 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrT-0006ge-QM
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:26:00 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:55022)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrO-0006fJ-G1
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:55 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrF-000344-3n; Mon, 11 Sep 2023 10:25:45 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=ShZItNlvYjaRrNo4JPlFc63xoz7be61ooWEzJHPU0v4=; b=oHz5EoLWBmpAliEnOeLs
 QaqCICjXjhsBGU2R5y13a00M9MZTquQrf2G/MwdumOXgCo6WVwW6H+LNQoeStmf8/3jxuZM1EVI+B
 ZdJvDOI8CYZ1LzKNLec4T3mcScJkNSrl22/p9/mx7BiQR3SyiTx8hw/WAI8TcPO6AMpYy08ykLz0T
 vgXZ6oWBJfcjOrlj6M+eDr3chUCZDHUqY/Ul0sdGh9QbSNJOltn3GyKWQbVcSgODlB65odEHlClzu
 dumDECqVnrin3wRDY1pYuSB+n1Eq/dR0d/uhhi/2gt7cW7UXNIpsgkAY+FfR5dag4wQA6r3/mhod3
 VKh1/qCMbTKJYQ==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 7/8] =?UTF-8?q?git-download:=20Use=20=E2=80=9Cbuiltin:git-?=
 =?UTF-8?q?download=E2=80=9D=20when=20available.?=
Date: Mon, 11 Sep 2023 16:25:25 +0200
Message-ID: <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Fixes <https://issues.guix.gnu.org/63331>.

Longer-term this will remove Git from the derivation graph when its sole
use is to perform a checkout for a fixed-output derivation, thereby
breaking dependency cycles that can arise in these situations.

* guix/git-download.scm (git-fetch): Rename to…
(git-fetch/in-band): … this.  Deal with GIT or GUILE being #f.
(git-fetch/built-in, built-in-builders*, git-fetch): New procedures.
* tests/builders.scm ("git-fetch, file URI"): New test.
---
 guix/git-download.scm | 68 +++++++++++++++++++++++++++++++++++++------
 tests/builders.scm    | 29 +++++++++++++++++-
 2 files changed, 87 insertions(+), 10 deletions(-)

diff --git a/guix/git-download.scm b/guix/git-download.scm
index f1f19397c6..505dff0a89 100644
--- a/guix/git-download.scm
+++ b/guix/git-download.scm
@@ -27,6 +27,7 @@ (define-module (guix git-download)
   #:use-module (guix records)
   #:use-module (guix packages)
   #:use-module (guix modules)
+  #:use-module ((guix derivations) #:select (raw-derivation))
   #:autoload   (guix build-system gnu) (standard-packages)
   #:autoload   (guix download) (%download-fallback-test)
   #:autoload   (git bindings)   (libgit2-init!)
@@ -78,15 +79,19 @@ (define (git-package)
   (let ((distro (resolve-interface '(gnu packages version-control))))
     (module-ref distro 'git-minimal)))
 
-(define* (git-fetch ref hash-algo hash
-                    #:optional name
-                    #:key (system (%current-system)) (guile (default-guile))
-                    (git (git-package)))
-  "Return a fixed-output derivation that fetches REF, a <git-reference>
-object.  The output is expected to have recursive hash HASH of type
-HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if #f."
+(define* (git-fetch/in-band ref hash-algo hash
+                            #:optional name
+                            #:key (system (%current-system))
+                            (guile (default-guile))
+                            (git (git-package)))
+  "Return a fixed-output derivation that performs a Git checkout of REF, using
+GIT and GUILE (thus, said derivation depends on GIT and GUILE).
+
+This method is deprecated in favor of the \"builtin:git-download\" builder.
+It will be removed when versions of guix-daemon implementing
+\"builtin:git-download\" will be sufficiently widespread."
   (define inputs
-    `(("git" ,git)
+    `(("git" ,(or git (git-package)))
 
       ;; When doing 'git clone --recursive', we need sed, grep, etc. to be
       ;; available so that 'git submodule' works.
@@ -154,7 +159,8 @@ (define* (git-fetch ref hash-algo hash
                                      #:recursive? recursive?
                                      #:git-command "git")))))
 
-  (mlet %store-monad ((guile (package->derivation guile system)))
+  (mlet %store-monad ((guile (package->derivation (or guile (default-guile))
+                                                  system)))
     (gexp->derivation (or name "git-checkout") build
 
                       ;; Use environment variables and a fixed script name so
@@ -181,6 +187,50 @@ (define* (git-fetch ref hash-algo hash
                       #:recursive? #t
                       #:guile-for-build guile)))
 
+(define* (git-fetch/built-in ref hash-algo hash
+                             #:optional name
+                             #:key (system (%current-system)))
+  "Return a fixed-output derivation without any dependency that performs a Git
+checkout of REF, using the \"builtin:git-download\" derivation builder."
+  (raw-derivation (or name "git-checkout") "builtin:git-download" '()
+                  #:system system
+                  #:hash-algo hash-algo
+                  #:hash hash
+                  #:recursive? #t
+                  #:env-vars
+                  `(("url" . ,(object->string
+                               (match (%download-fallback-test)
+                                 ('content-addressed-mirrors
+                                  "https://example.org/does-not-exist")
+                                 (_
+                                  (git-reference-url ref)))))
+                    ("commit" . ,(git-reference-commit ref))
+                    ("recursive?" . ,(object->string
+                                      (git-reference-recursive? ref))))
+                  #:leaked-env-vars '("http_proxy" "https_proxy"
+                                      "LC_ALL" "LC_MESSAGES" "LANG"
+                                      "COLUMNS")
+                  #:local-build? #t))
+
+(define built-in-builders*
+  (store-lift built-in-builders))
+
+(define* (git-fetch ref hash-algo hash
+                    #:optional name
+                    #:key (system (%current-system))
+                    guile git)
+  "Return a fixed-output derivation that fetches REF, a <git-reference>
+object.  The output is expected to have recursive hash HASH of type
+HASH-ALGO (a symbol).  Use NAME as the file name, or a generic name if #f."
+  (mlet %store-monad ((builtins (built-in-builders*)))
+    (if (member "git-download" builtins)
+        (git-fetch/built-in ref hash-algo hash name
+                            #:system system)
+        (git-fetch/in-band ref hash-algo hash name
+                           #:system system
+                           #:guile guile
+                           #:git git))))
+
 (define (git-version version revision commit)
   "Return the version string for packages using git-download."
   ;; git-version is almost exclusively executed while modules are being loaded.
diff --git a/tests/builders.scm b/tests/builders.scm
index 0b5577c7a3..619caa5f31 100644
--- a/tests/builders.scm
+++ b/tests/builders.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2018, 2019, 2021 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2012-2015, 2018-2019, 2021, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;; Copyright © 2021 Lars-Dominik Braun <lars@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -20,6 +20,7 @@
 
 (define-module (tests builders)
   #:use-module (guix download)
+  #:use-module (guix git-download)
   #:use-module (guix build-system)
   #:use-module (guix build-system gnu)
   #:use-module (guix build gnu-build-system)
@@ -31,9 +32,12 @@ (define-module (tests builders)
   #:use-module (guix base32)
   #:use-module (guix derivations)
   #:use-module (gcrypt hash)
+  #:use-module ((guix hash) #:select (file-hash*))
   #:use-module (guix tests)
+  #:use-module (guix tests git)
   #:use-module (guix packages)
   #:use-module (gnu packages bootstrap)
+  #:use-module ((ice-9 ftw) #:select (scandir))
   #:use-module (ice-9 match)
   #:use-module (ice-9 textual-ports)
   #:use-module (srfi srfi-1)
@@ -84,6 +88,29 @@ (define url-fetch*
     (and (file-exists? out)
          (valid-path? %store out))))
 
+(test-equal "git-fetch, file URI"
+  '("." ".." "a.txt" "b.scm")
+  (let ((nonce (random-text)))
+    (with-temporary-git-repository directory
+        `((add "a.txt" ,nonce)
+          (add "b.scm" "#t")
+          (commit "Commit.")
+          (tag "v1.0.0" "The tag."))
+      (run-with-store %store
+        (mlet* %store-monad ((hash
+                              -> (file-hash* directory
+                                             #:algorithm (hash-algorithm sha256)
+                                             #:recursive? #t))
+                             (drv (git-fetch
+                                   (git-reference
+                                    (url (string-append "file://" directory))
+                                    (commit "v1.0.0"))
+                                   'sha256 hash
+                                   "git-fetch-test")))
+          (mbegin %store-monad
+            (built-derivations (list drv))
+            (return (scandir (derivation->output-path drv)))))))))
+
 (test-assert "gnu-build-system"
   (build-system? gnu-build-system))
 
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:25:59 2023
Received: from localhost ([127.0.0.1]:54278 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrT-0006gX-G4
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:59 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:55018)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrN-0006fH-Jd
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:54 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrE-00033D-7r; Mon, 11 Sep 2023 10:25:44 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=2EsfrtzZMQfiz6o+ksU68WCBCRlX5ph0lDFzZywzfbI=; b=d9AICfwDAeCc9S32AQW8
 1aqVhm5KnPbH8gt6xDnOID5QSa6JtfCO0c3M2tKO34bFpssaUL4kcZZAB6BRDItHC/9Mb+6xAHWFn
 P6QUfggP4GjOWo2Ry+i/9HU/E/1HGD9/kXOm3/4iDUBUEBRyTkUXGHpyZ73AsC3q9W6aQbhG8RQkU
 7beL21wOKdQsVydNzAHlg+Ku5QJMaDM04enXvov8Z/jtFDlwJnaytQbnPtmQ8X4kpXanUEl0xKGyd
 kUJXRslqzJj/HR64WJy9/uPrzMLGHSPS+x8sYr0qB4E8y2a+yT/8ibyqAvnB7FkaF0w4FfRtBbIHL
 3Rp3th/Ng/TVmQ==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 6/8] =?UTF-8?q?perform-git-download:=20Use=20the=20?=
 =?UTF-8?q?=E2=80=98git=E2=80=99=20command=20captured=20at=20configure=20t?=
 =?UTF-8?q?ime.?=
Date: Mon, 11 Sep 2023 16:25:24 +0200
Message-ID: <52b761e331150a637bddf696469c5e265646d461.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* guix/scripts/perform-download.scm (perform-git-download): Pass #:git-command
to ‘git-fetch-with-fallback’.
---
 guix/scripts/perform-download.scm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index a287e97528..e1c584fbda 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -23,6 +23,7 @@ (define-module (guix scripts perform-download)
   #:use-module ((guix store) #:select (derivation-path? store-path?))
   #:autoload   (guix build download) (url-fetch)
   #:autoload   (guix build git) (git-fetch-with-fallback)
+  #:autoload   (guix config) (%git)
   #:use-module (ice-9 match)
   #:export (guix-perform-download))
 
@@ -114,7 +115,8 @@ (define* (perform-git-download drv #:optional output
            (algo       (derivation-output-hash-algo drv-output))
            (hash       (derivation-output-hash drv-output)))
       (git-fetch-with-fallback url commit output
-                               #:recursive? recursive?))))
+                               #:recursive? recursive?
+                               #:git-command %git))))
 
 (define (assert-low-privileges)
   (when (zero? (getuid))
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:25:59 2023
Received: from localhost ([127.0.0.1]:54276 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrS-0006gP-MT
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:59 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:55002)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrM-0006fF-QW
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:53 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrC-0002zo-HX; Mon, 11 Sep 2023 10:25:42 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=99DJrUUViq7vxNIOv5rl+/tDT4n8RnWxwe8vYvOdTYY=; b=SmO+MULr6zuErUGBxYWQ
 4NFu1jaKbhmVJdtar2NLvbY3+gAaBY6E1lgfxnSz7ZaS9jI03sNI+vQ+8cfJawun82NxH/LGDHItc
 YSgcwzszb9CW8hZS7qZ+gPW64CZgGPU9L5s6tvGURPcRW1BQL5ZJ9eCfUjvX/QaAOTycGYHJuKXem
 5z1L4n7m7SmlnyGgq9qWGJQayds/9l5zu7eLujHraSbQz00RgDndmLSNBCUHsIInht13zTHXQaVSk
 /2u9qDe+bX3nEdaBbg6ozJs2ixYzX0OqL6dZQv2micbTZXqldF/kXyUONWPhX4DzgiJqt7WbQn7XB
 d2cbcJNN8HokxQ==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 4/8] =?UTF-8?q?daemon:=20Add=20=E2=80=9Cgit-download?=
 =?UTF-8?q?=E2=80=9D=20built-in=20builder.?=
Date: Mon, 11 Sep 2023 16:25:22 +0200
Message-ID: <3c42634cb47dd7eaa81a198bc2d097ca74a973ed.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludovic.courtes@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

From: Ludovic Courtès <ludovic.courtes@HIDDEN>

The new builder makes it possible to break cycles that occurs when the
fixed-output derivation for the source of a dependency of ‘git’ would
itself depend on ‘git’.

* guix/scripts/perform-download.scm (perform-git-download): New
procedure.
(perform-download): Move fixed-output derivation check to…
(guix-perform-download): … here.  Invoke ‘perform-download’ or
‘perform-git-download’ depending on what ‘derivation-builder’ returns.
* nix/libstore/builtins.cc (builtins): Add “git-download”.
* tests/derivations.scm ("built-in-builders"): Update.
("'git-download' built-in builder")
("'git-download' built-in builder, invalid hash")
("'git-download' built-in builder, invalid commit")
("'git-download' built-in builder, not found"): New tests.
---
 guix/scripts/perform-download.scm |  52 +++++++++++++---
 nix/libstore/builtins.cc          |   5 +-
 tests/derivations.scm             | 100 +++++++++++++++++++++++++++++-
 3 files changed, 145 insertions(+), 12 deletions(-)

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index c8f044e82e..a287e97528 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2016, 2017, 2018, 2020 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2016-2018, 2020, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,7 +21,8 @@ (define-module (guix scripts perform-download)
   #:use-module (guix scripts)
   #:use-module (guix derivations)
   #:use-module ((guix store) #:select (derivation-path? store-path?))
-  #:use-module (guix build download)
+  #:autoload   (guix build download) (url-fetch)
+  #:autoload   (guix build git) (git-fetch-with-fallback)
   #:use-module (ice-9 match)
   #:export (guix-perform-download))
 
@@ -64,10 +65,6 @@ (define* (perform-download drv #:optional output
            (drv-output (assoc-ref (derivation-outputs drv) "out"))
            (algo       (derivation-output-hash-algo drv-output))
            (hash       (derivation-output-hash drv-output)))
-      (unless (and algo hash)
-        (leave (G_ "~a is not a fixed-output derivation~%")
-               (derivation-file-name drv)))
-
       ;; We're invoked by the daemon, which gives us write access to OUTPUT.
       (when (url-fetch url output
                        #:print-build-trace? print-build-trace?
@@ -92,6 +89,33 @@ (define* (perform-download drv #:optional output
         (when (and executable (string=? executable "1"))
           (chmod output #o755))))))
 
+(define* (perform-git-download drv #:optional output
+                               #:key print-build-trace?)
+  "Perform the download described by DRV, a fixed-output derivation, to
+OUTPUT.
+
+Note: Unless OUTPUT is #f, we don't read the value of 'out' in DRV since the
+actual output is different from that when we're doing a 'bmCheck' or
+'bmRepair' build."
+  (derivation-let drv ((output* "out")
+                       (url "url")
+                       (commit "commit")
+                       (recursive? "recursive?"))
+    (unless url
+      (leave (G_ "~a: missing Git URL~%") (derivation-file-name drv)))
+    (unless commit
+      (leave (G_ "~a: missing Git commit~%") (derivation-file-name drv)))
+
+    (let* ((output     (or output output*))
+           (url        (call-with-input-string url read))
+           (recursive? (and recursive?
+                            (call-with-input-string recursive? read)))
+           (drv-output (assoc-ref (derivation-outputs drv) "out"))
+           (algo       (derivation-output-hash-algo drv-output))
+           (hash       (derivation-output-hash drv-output)))
+      (git-fetch-with-fallback url commit output
+                               #:recursive? recursive?))))
+
 (define (assert-low-privileges)
   (when (zero? (getuid))
     (leave (G_ "refusing to run with elevated privileges (UID ~a)~%")
@@ -120,8 +144,20 @@ (define-command (guix-perform-download . args)
     (match args
       (((? derivation-path? drv) (? store-path? output))
        (assert-low-privileges)
-       (let ((drv (read-derivation-from-file drv)))
-         (perform-download drv output #:print-build-trace? print-build-trace?)))
+       (let* ((drv (read-derivation-from-file drv))
+              (download (match (derivation-builder drv)
+                          ("builtin:download" perform-download)
+                          ("builtin:git-download" perform-git-download)
+                          (unknown (leave (G_ "~a: unknown builtin builder")
+                                          unknown))))
+              (drv-output (assoc-ref (derivation-outputs drv) "out"))
+              (algo       (derivation-output-hash-algo drv-output))
+              (hash       (derivation-output-hash drv-output)))
+         (unless (and hash algo)
+           (leave (G_ "~a is not a fixed-output derivation~%")
+                  (derivation-file-name drv)))
+
+         (download drv output #:print-build-trace? print-build-trace?)))
       (("--version")
        (show-version-and-exit))
       (x
diff --git a/nix/libstore/builtins.cc b/nix/libstore/builtins.cc
index 4111ac4760..6bf467354a 100644
--- a/nix/libstore/builtins.cc
+++ b/nix/libstore/builtins.cc
@@ -1,5 +1,5 @@
 /* GNU Guix --- Functional package management for GNU
-   Copyright (C) 2016, 2017, 2018, 2019 Ludovic Courtès <ludo@HIDDEN>
+   Copyright (C) 2016-2019, 2023 Ludovic Courtès <ludo@HIDDEN>
 
    This file is part of GNU Guix.
 
@@ -58,7 +58,8 @@ static void builtinDownload(const Derivation &drv,
 
 static const std::map<std::string, derivationBuilder> builtins =
 {
-    { "download", builtinDownload }
+    { "download", builtinDownload },
+    { "git-download", builtinDownload }
 };
 
 derivationBuilder lookupBuiltinBuilder(const std::string & name)
diff --git a/tests/derivations.scm b/tests/derivations.scm
index 66c777cfe7..e1312bd46b 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -24,10 +24,15 @@ (define-module (test-derivations)
   #:use-module (guix utils)
   #:use-module ((gcrypt hash) #:prefix gcrypt:)
   #:use-module (guix base32)
+  #:use-module ((guix git) #:select (with-repository))
   #:use-module (guix tests)
+  #:use-module (guix tests git)
   #:use-module (guix tests http)
   #:use-module ((guix packages) #:select (package-derivation base32))
-  #:use-module ((guix build utils) #:select (executable-file?))
+  #:use-module ((guix build utils) #:select (executable-file? which))
+  #:use-module ((guix hash) #:select (file-hash*))
+  #:use-module ((git oid) #:select (oid->string))
+  #:use-module ((git reference) #:select (reference-name->oid))
   #:use-module (gnu packages bootstrap)
   #:use-module ((gnu packages guile) #:select (guile-1.8))
   #:use-module (srfi srfi-1)
@@ -195,7 +200,7 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
                    (stat:ino (lstat file2))))))))
 
 (test-equal "built-in-builders"
-  '("download")
+  '("download" "git-download")
   (built-in-builders %store))
 
 (test-assert "unknown built-in builder"
@@ -290,6 +295,97 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all))
                          get-string-all)
                        text))))))
 
+;; 'with-temporary-git-repository' relies on the 'git' command.
+(unless (which (git-command)) (test-skip 1))
+(test-equal "'git-download' built-in builder"
+  `(("/a.txt" . "AAA")
+    ("/b.scm" . "#t"))
+  (let ((nonce (random-text)))
+    (with-temporary-git-repository directory
+        `((add "a.txt" "AAA")
+          (add "b.scm" "#t")
+          (commit ,nonce))
+      (let* ((commit (with-repository directory repository
+                       (oid->string
+                        (reference-name->oid repository "HEAD"))))
+             (drv (derivation %store "git-download"
+                              "builtin:git-download" '()
+                              #:env-vars
+                              `(("url"
+                                 . ,(object->string
+                                     (string-append "file://" directory)))
+                                ("commit" . ,commit))
+                              #:hash-algo 'sha256
+                              #:hash (file-hash* directory
+                                                 #:algorithm
+                                                 (gcrypt:hash-algorithm
+                                                  gcrypt:sha256)
+                                                 #:recursive? #t)
+                              #:recursive? #t)))
+        (build-derivations %store (list drv))
+        (directory-contents (derivation->output-path drv) get-string-all)))))
+
+(unless (which (git-command)) (test-skip 1))
+(test-assert "'git-download' built-in builder, invalid hash"
+  (with-temporary-git-repository directory
+      `((add "a.txt" "AAA")
+        (add "b.scm" "#t")
+        (commit "Commit!"))
+    (let* ((commit (with-repository directory repository
+                     (oid->string
+                      (reference-name->oid repository "HEAD"))))
+           (drv (derivation %store "git-download"
+                            "builtin:git-download" '()
+                            #:env-vars
+                            `(("url"
+                               . ,(object->string
+                                   (string-append "file://" directory)))
+                              ("commit" . ,commit))
+                            #:hash-algo 'sha256
+                            #:hash (gcrypt:sha256 #vu8())
+                            #:recursive? #t)))
+      (guard (c ((store-protocol-error? c)
+                 (string-contains (store-protocol-error-message c) "failed")))
+        (build-derivations %store (list drv))
+        #f))))
+
+(unless (which (git-command)) (test-skip 1))
+(test-assert "'git-download' built-in builder, invalid commit"
+  (with-temporary-git-repository directory
+      `((add "a.txt" "AAA")
+        (add "b.scm" "#t")
+        (commit "Commit!"))
+    (let* ((drv (derivation %store "git-download"
+                            "builtin:git-download" '()
+                            #:env-vars
+                            `(("url"
+                               . ,(object->string
+                                   (string-append "file://" directory)))
+                              ("commit"
+                               . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"))
+                            #:hash-algo 'sha256
+                            #:hash (gcrypt:sha256 #vu8())
+                            #:recursive? #t)))
+      (guard (c ((store-protocol-error? c)
+                 (string-contains (store-protocol-error-message c) "failed")))
+        (build-derivations %store (list drv))
+        #f))))
+
+(test-assert "'git-download' built-in builder, not found"
+  (let* ((drv (derivation %store "git-download"
+                          "builtin:git-download" '()
+                          #:env-vars
+                          `(("url" . "file:///does-not-exist.git")
+                            ("commit"
+                             . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"))
+                          #:hash-algo 'sha256
+                          #:hash (gcrypt:sha256 #vu8())
+                          #:recursive? #t)))
+    (guard (c ((store-protocol-error? c)
+               (string-contains (store-protocol-error-message c) "failed")))
+      (build-derivations %store (list drv))
+      #f)))
+
 (test-equal "derivation-name"
   "foo-0.0"
   (let ((drv (derivation %store "foo-0.0" %bash '())))
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:25:58 2023
Received: from localhost ([127.0.0.1]:54274 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrQ-0006gA-43
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:58 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54994)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrK-0006fA-Tt
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:51 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrB-0002zP-HR; Mon, 11 Sep 2023 10:25:41 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=8VpxIxGrW5Ejr0OFvwQSmvFszi43poboHugffNEPfik=; b=fcnKcHe20tnhWkXLT8d1
 vWW/aQb7DryvAfqlcykaLxvQloLxyOiTPLN5Qk4eOT6k4AjmKEJIFi8+Zqq/AtYZKkuL75XefTYWo
 ptCLGHGSJoRItbC17+padWbtZ89McznXKRIyCzbPpORdAi5EKKczgt60eL1ErX7endI1J6K9X84dY
 NdTntlPsf3X/WoK4x5DwoHWP8KOIqPwU6ktxRJOf1MmfBTqf8mZ/4rgHzCDXeDIvNvYe91IUjvrqV
 VxAqOydcni5yp8ZgdqzO3NDEnTs9jA0TASqkr5GWVkqwJPXAkzoSUsjRXilmPXpFxhka4HhpfG4Q9
 oPYwuQNvLW7C6g==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 3/8] perform-download: Remove unused one-argument clause.
Date: Mon, 11 Sep 2023 16:25:21 +0200
Message-ID: <cd7c4b170a3063778a5c65b3b63e19b22037db07.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Code in ‘builtins.cc’ only ever invokes ‘guix perform-download’ with two
arguments.

* guix/scripts/perform-download.scm (guix-perform-download): Remove
unused one-argument clause.
---
 guix/scripts/perform-download.scm | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm
index 6889bcef79..c8f044e82e 100644
--- a/guix/scripts/perform-download.scm
+++ b/guix/scripts/perform-download.scm
@@ -120,13 +120,8 @@ (define-command (guix-perform-download . args)
     (match args
       (((? derivation-path? drv) (? store-path? output))
        (assert-low-privileges)
-       (perform-download (read-derivation-from-file drv)
-                         output
-                         #:print-build-trace? print-build-trace?))
-      (((? derivation-path? drv))                 ;backward compatibility
-       (assert-low-privileges)
-       (perform-download (read-derivation-from-file drv)
-                         #:print-build-trace? print-build-trace?))
+       (let ((drv (read-derivation-from-file drv)))
+         (perform-download drv output #:print-build-trace? print-build-trace?)))
       (("--version")
        (show-version-and-exit))
       (x
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:52 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:25:52 2023
Received: from localhost ([127.0.0.1]:54262 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrL-0006fe-OL
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:52 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54982)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrK-0006f9-1B
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:50 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhrA-0002yL-Ke; Mon, 11 Sep 2023 10:25:40 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=lbJIuSS34XKou1KAAp0q0w8yQNp1j9kF2Ik0svE2RAw=; b=kZJ47piQeRVOU/HkuP+U
 zzJ4xJEQlFz/qec3U8lI/jslJ+ZLqBvUqz9gqHdz2LrE1k+bOdG83e5zVYQbDFgWh4Adb7Pz/CywA
 ek/L7cE6aSavlkJUime6vaXeB0GhqBybDge2w4Smk6/K7d24dHYl4tC5DN3jdr3nxBPL0syQaj1Df
 sNwqUi4c1ZAbXxZgW4X5nZvcidTnEmlgNkfdaf26Q6qyDeWEZTC+jjmB1IF08Etw33sYVFw4b2na5
 ptWmOJ+kC8l5KXubtOo5Ck5F05yxUwXy1fv2o6vplknXASqX0UEZviyLhgL7ITdKPCVhNwpPUP/cZ
 0fss5/F6Ujymnw==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 2/8] =?UTF-8?q?git-download:=20Honor=20the=20=E2=80=98GUIX?=
 =?UTF-8?q?=5FDOWNLOAD=5FFALLBACK=5FTEST=E2=80=99=20environment=20variable?=
 =?UTF-8?q?.?=
Date: Mon, 11 Sep 2023 16:25:20 +0200
Message-ID: <2cc7a7d7d2d1c801ec4529b18f2a526d8d2b07ef.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* guix/git-download.scm (git-fetch): Honor ‘%download-fallback-test’.
---
 guix/git-download.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/guix/git-download.scm b/guix/git-download.scm
index 8989b1b463..f1f19397c6 100644
--- a/guix/git-download.scm
+++ b/guix/git-download.scm
@@ -28,6 +28,7 @@ (define-module (guix git-download)
   #:use-module (guix packages)
   #:use-module (guix modules)
   #:autoload   (guix build-system gnu) (standard-packages)
+  #:autoload   (guix download) (%download-fallback-test)
   #:autoload   (git bindings)   (libgit2-init!)
   #:autoload   (git repository) (repository-open
                                  repository-close!
@@ -161,7 +162,11 @@ (define* (git-fetch ref hash-algo hash
                       ;; downloads.
                       #:script-name "git-download"
                       #:env-vars
-                      `(("git url" . ,(git-reference-url ref))
+                      `(("git url" . ,(match (%download-fallback-test)
+                                        ('content-addressed-mirrors
+                                         "https://example.org/does-not-exist")
+                                        (_
+                                         (git-reference-url ref))))
                         ("git commit" . ,(git-reference-commit ref))
                         ("git recursive?" . ,(object->string
                                               (git-reference-recursive? ref))))
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at 65866 <at> debbugs.gnu.org:


Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:25:51 2023
Received: from localhost ([127.0.0.1]:54260 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhrL-0006fW-1i
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:51 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54974)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhrJ-0006f7-4k
 for 65866 <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:25:49 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhr9-0002xL-Lx; Mon, 11 Sep 2023 10:25:39 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To:
 From; bh=NhkD4LZUlsDDA0ql2tBJERZrS5hCR8P2YozzIJfod8I=; b=BlO+gHNPGdjPFbkWr6nz
 pzBLctux9qRsmOWlilsAMK5arxWUvitoXbjuhHAxm7dkL+dAT6DfFh+zTSu6ZMmXc5O6+Uxnp6ddl
 qjsMKIigHK5CR30NghNL8zKHBLlyXWjFl0Bcrb+A7sbW1DACjrolJq5Xsmg4mXHZDWftP7ELl0N2y
 bHAaDkfTiNH5OxQSRuZ/8XlQ7mshdXx2COVOhXsveAwHUqp/LJAD0XLQF5KLS73ZpIqEigW41S/oE
 jWsTQhkCIoSwx95S7xzWhbsW88b3jjzHx2KpqYH0riz/xK2s2ZjahCHolH7tFPpdRc89045Mscvqm
 UKi/45n7RK1oWw==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: 65866 <at> debbugs.gnu.org
Subject: [PATCH 1/8] git-download: Move fallback code to (guix build git).
Date: Mon, 11 Sep 2023 16:25:19 +0200
Message-ID: <b25aa2dd644ac85ed72dabf3cb2098fd8c6358d0.1694441831.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <cover.1694441830.git.ludo@HIDDEN>
References: <cover.1694441830.git.ludo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65866
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

* guix/build/git.scm (git-fetch-with-fallback): New procedure, with code
taken from…
* guix/git-download.scm (git-fetch): … here.
[modules]: Remove modules that are no longer directly used in ‘build’.
[build]: Use ‘git-fetch-with-fallback’.
---
 guix/build/git.scm    | 44 ++++++++++++++++++++++++++++++++++++++--
 guix/git-download.scm | 47 ++++++++-----------------------------------
 2 files changed, 50 insertions(+), 41 deletions(-)

diff --git a/guix/build/git.scm b/guix/build/git.scm
index deda10fee8..0ff263c81b 100644
--- a/guix/build/git.scm
+++ b/guix/build/git.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2016, 2019 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2014, 2016, 2019, 2023 Ludovic Courtès <ludo@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -18,9 +18,12 @@
 
 (define-module (guix build git)
   #:use-module (guix build utils)
+  #:autoload   (guix build download-nar) (download-nar)
+  #:autoload   (guix swh) (%verify-swh-certificate? swh-download)
   #:use-module (srfi srfi-34)
   #:use-module (ice-9 format)
-  #:export (git-fetch))
+  #:export (git-fetch
+            git-fetch-with-fallback))
 
 ;;; Commentary:
 ;;;
@@ -76,4 +79,41 @@ (define* (git-fetch url commit directory
       (delete-file-recursively ".git")
       #t)))
 
+
+(define* (git-fetch-with-fallback url commit directory
+                                  #:key (git-command "git") recursive?)
+  "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall back to
+alternative methods when fetching from URL fails: attempt to download a nar,
+and if that also fails, download from the Software Heritage archive."
+  (or (git-fetch url commit directory
+                 #:recursive? recursive?
+                 #:git-command git-command)
+      (download-nar directory)
+
+      ;; As a last resort, attempt to download from Software Heritage.
+      ;; Disable X.509 certificate verification to avoid depending
+      ;; on nss-certs--we're authenticating the checkout anyway.
+      ;; XXX: Currently recursive checkouts are not supported.
+      (and (not recursive?)
+           (parameterize ((%verify-swh-certificate? #f))
+             (format (current-error-port)
+                     "Trying to download from Software Heritage...~%")
+
+             (swh-download url commit directory)
+             (when (file-exists?
+                    (string-append directory "/.gitattributes"))
+               ;; Perform CR/LF conversion and other changes
+               ;; specificied by '.gitattributes'.
+               (invoke git-command "-C" directory "init")
+               (invoke git-command "-C" directory "config" "--local"
+                       "user.email" "you@HIDDEN")
+               (invoke git-command "-C" directory "config" "--local"
+                       "user.name" "Your Name")
+               (invoke git-command "-C" directory "add" ".")
+               (invoke git-command "-C" directory "commit" "-am" "init")
+               (invoke git-command "-C" directory "read-tree" "--empty")
+               (invoke git-command "-C" directory "reset" "--hard")
+               (delete-file-recursively
+                (string-append directory "/.git")))))))
+
 ;;; git.scm ends here
diff --git a/guix/git-download.scm b/guix/git-download.scm
index d88f4c40ee..8989b1b463 100644
--- a/guix/git-download.scm
+++ b/guix/git-download.scm
@@ -116,19 +116,16 @@ (define* (git-fetch ref hash-algo hash
   (define modules
     (delete '(guix config)
             (source-module-closure '((guix build git)
-                                     (guix build utils)
-                                     (guix build download-nar)
-                                     (guix swh)))))
+                                     (guix build utils)))))
 
   (define build
     (with-imported-modules modules
-      (with-extensions (list guile-json gnutls   ;for (guix swh)
+      (with-extensions (list guile-json gnutls    ;for (guix swh)
                              guile-lzlib)
         #~(begin
             (use-modules (guix build git)
-                         (guix build utils)
-                         (guix build download-nar)
-                         (guix swh)
+                         ((guix build utils)
+                          #:select (set-path-environment-variable))
                          (ice-9 match))
 
             (define recursive?
@@ -151,38 +148,10 @@ (define* (git-fetch ref hash-algo hash
             (setvbuf (current-output-port) 'line)
             (setvbuf (current-error-port) 'line)
 
-            (or (git-fetch (getenv "git url") (getenv "git commit")
-                           #$output
-                           #:recursive? recursive?
-                           #:git-command "git")
-                (download-nar #$output)
-
-                ;; As a last resort, attempt to download from Software Heritage.
-                ;; Disable X.509 certificate verification to avoid depending
-                ;; on nss-certs--we're authenticating the checkout anyway.
-                ;; XXX: Currently recursive checkouts are not supported.
-                (and (not recursive?)
-                     (parameterize ((%verify-swh-certificate? #f))
-                       (format (current-error-port)
-                               "Trying to download from Software Heritage...~%")
-
-                       (swh-download (getenv "git url") (getenv "git commit")
-                                     #$output)
-                       (when (file-exists?
-                              (string-append #$output "/.gitattributes"))
-                         ;; Perform CR/LF conversion and other changes
-                         ;; specificied by '.gitattributes'.
-                         (invoke "git" "-C" #$output "init")
-                         (invoke "git" "-C" #$output "config" "--local"
-                                 "user.email" "you@HIDDEN")
-                         (invoke "git" "-C" #$output "config" "--local"
-                                 "user.name" "Your Name")
-                         (invoke "git" "-C" #$output "add" ".")
-                         (invoke "git" "-C" #$output "commit" "-am" "init")
-                         (invoke "git" "-C" #$output "read-tree" "--empty")
-                         (invoke "git" "-C" #$output "reset" "--hard")
-                         (delete-file-recursively
-                          (string-append #$output "/.git"))))))))))
+            (git-fetch-with-fallback (getenv "git url") (getenv "git commit")
+                                     #$output
+                                     #:recursive? recursive?
+                                     #:git-command "git")))))
 
   (mlet %store-monad ((guile (package->derivation guile system)))
     (gexp->derivation (or name "git-checkout") build
-- 
2.41.0





Information forwarded to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 11 Sep 2023 14:24:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 10:24:21 2023
Received: from localhost ([127.0.0.1]:54249 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qfhpq-0006cX-17
	for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:24:21 -0400
Received: from lists.gnu.org ([2001:470:142::17]:32830)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qfhpl-0006cE-CH
 for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 10:24:17 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qfhpX-0002l0-9K
 for guix-patches@HIDDEN; Mon, 11 Sep 2023 10:23:59 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1qfhpW-0002cM-Ac; Mon, 11 Sep 2023 10:23:58 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to:
 references; bh=lgd1VrghGcFnkob0HT8OwrhqRwdtNyRBFmXiiUyfdSU=; b=ry2Fq4eZIW4m+u
 ckRi/NvUvzqtUTLztFlwsmEDUZ8pI4er9qiJLcyDGFjFwKB5KtRFtOffBrsjaZ/nT+dE5PAH/qTCf
 RGlpWmsERgGTbJUDTJu69ylZdsUiTQG8s0E3Ts44E1nZiOBTJ0FqhWJFPJHy92TcfWlxIp6c53v54
 vZirMioyN5rEzYjwkgBosJGzGl0UxV8NmHSkiXuBB/OjUmTndBiBTB3OhuTxBmETqRYNb3w1321MW
 bm8U08xz8WOIpSWA8jjwICMHQDPJoRZbzoJF9/2IacBLb2s8T3s+UKmOH+l//KLtMjQ1sxL3vTQd4
 OEC7ok7hPC/w7VrwCQGw==;
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH 0/8] Add built-in builder for Git checkouts
Date: Mon, 11 Sep 2023 16:23:42 +0200
Message-ID: <cover.1694441830.git.ludo@HIDDEN>
X-Mailer: git-send-email 2.41.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: submit
Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello Guix!

This patch series is a first step towards getting Git out of
derivation graphs when it’s only used to fetch source code
(origins with ‘git-fetch’), with the goal of fixing:

  https://issues.guix.gnu.org/63331

The is similar to how we solved the problem for regular file
downloads: we add a new “builtin:git-download” builder for
derivations, which is implemented on the daemon size by the
‘guix perform-download’ helper.  That command uses the same
code that is currently used by ‘git-fetch’.

Eventually, when users are all running recent versions of
‘guix-daemon’ with support for “builtin:git-download” (2–4
years from now?), we’ll be able to use “builtin:git-download”
unconditionally and thus be sure there are no risks of
derivation cycles.

Note that the patch series adds a hard dependency on Git.
This is because the existing ‘git-fetch’ code depends on Git,
which is itself motivated by the fact that Git supports
shallow clones and libgit2/Guile-Git doesn’t.

As a side effect, this dependency will prove useful to
address <https://issues.guix.gnu.org/65720>.

Thoughts?

Ludo’.

Ludovic Courtès (8):
  git-download: Move fallback code to (guix build git).
  git-download: Honor the ‘GUIX_DOWNLOAD_FALLBACK_TEST’ environment
    variable.
  perform-download: Remove unused one-argument clause.
  daemon: Add “git-download” built-in builder.
  build: Add dependency on Git.
  perform-git-download: Use the ‘git’ command captured at configure
    time.
  git-download: Use “builtin:git-download” when available.
  tests: Assume ‘git’ is always available.

 configure.ac                      |   7 ++
 doc/guix.texi                     |   1 +
 guix/build/git.scm                |  44 ++++++++++-
 guix/config.scm.in                |   6 +-
 guix/git-download.scm             | 122 ++++++++++++++++++------------
 guix/scripts/perform-download.scm |  59 +++++++++++----
 guix/self.scm                     |  10 ++-
 nix/libstore/builtins.cc          |   5 +-
 tests/builders.scm                |  29 ++++++-
 tests/channels.scm                |   7 +-
 tests/derivations.scm             |  94 ++++++++++++++++++++++-
 tests/git-authenticate.scm        |   1 -
 tests/git.scm                     |  10 ---
 tests/import-git.scm              |  18 -----
 14 files changed, 308 insertions(+), 105 deletions(-)


base-commit: a4c35c607cfd7d6b0bad90cfcc46188d489e1754
-- 
2.41.0





Acknowledgement sent to Ludovic Courtès <ludo@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#65866; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Sep 2023 14:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.