GNU bug report logs - #22990
Grafts leads to inefficient substitute info retrieval

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: important; Reported by: ludo@HIDDEN (Ludovic Courtès); merged with #25137; dated Fri, 11 Mar 2016 16:54:01 UTC; Maintainer for guix is bug-guix@HIDDEN.
Severity set to 'important' from 'normal' Request was from ludo@HIDDEN (Ludovic Courtès) to control <at> debbugs.gnu.org. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 11 Jan 2017 21:19:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 16:19:33 2017
Received: from localhost ([127.0.0.1]:50226 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cRQIv-00047s-Dd
	for submit <at> debbugs.gnu.org; Wed, 11 Jan 2017 16:19:33 -0500
Received: from eggs.gnu.org ([208.118.235.92]:49280)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1cRQIt-00047e-2K
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 16:19:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1cRQIk-0000ft-2Y
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 16:19:26 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56705)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1cRQIj-0000fo-Up; Wed, 11 Jan 2017 16:19:21 -0500
Received: from reverse-83.fdn.fr ([80.67.176.83]:38774 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1cRQIj-0003BI-Cg; Wed, 11 Jan 2017 16:19:21 -0500
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: David Craven <david@HIDDEN>
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN> <87k2a5rzue.fsf@HIDDEN>
 <877f633kt1.fsf@HIDDEN> <874m15yl99.fsf@HIDDEN>
 <CAL1_imnWoGFQ=NBjghV1wracigF676j4zomgCsfCTPO+JiPbKQ@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 22 =?utf-8?Q?Niv=C3=B4se?= an 225 de la =?utf-8?Q?R?=
 =?utf-8?Q?=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-unknown-linux-gnu
Date: Wed, 11 Jan 2017 22:19:19 +0100
In-Reply-To: <CAL1_imnWoGFQ=NBjghV1wracigF676j4zomgCsfCTPO+JiPbKQ@HIDDEN>
 (David Craven's message of "Wed, 11 Jan 2017 11:51:29 +0100")
Message-ID: <87k2a1wazs.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -8.2 (--------)
X-Debbugs-Envelope-To: 22990
Cc: 22990 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -8.2 (--------)

David Craven <david@HIDDEN> skribis:

>> On second thought, the whole idea of applying grafts on the final result
>> (instead of applying grafts at each step like we do now) doesn=E2=80=99t=
 fly.
>> It works well for things like a profile or the system derivation, but
>> breaks for less trivial things.
>
>> For example, if you=E2=80=99re building a VM image or a binary tarball, =
you
>> really need to graft packages early on; trying to graft the VM image or
>> binary tarball wouldn=E2=80=99t have the desired effect.
>
> Isn't a system derivation or a profile derivation an intermediate step
> to these derivations?

Yes, you=E2=80=99re right.  However the implementation I had come up with r=
elied
on =E2=80=9Cbuild continuations=E2=80=9D, which only worked for the =E2=80=
=9Ctop-level=E2=80=9D
derivations (those you pass to =E2=80=98build-derivations=E2=80=99.)

> Can't there be a flag or something called #:already-grafted? #t?

Yeah, we need something like that.  I need to chew a bit more on this to
find a nice way to achieve that.

Thanks for your feedback, I feel less lonely now.  :-)

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 11 Jan 2017 10:51:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 05:51:36 2017
Received: from localhost ([127.0.0.1]:49435 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cRGVE-0002lG-Ai
	for submit <at> debbugs.gnu.org; Wed, 11 Jan 2017 05:51:36 -0500
Received: from mail-qt0-f169.google.com ([209.85.216.169]:34444)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <david@HIDDEN>) id 1cRGVD-0002l1-5t
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 05:51:35 -0500
Received: by mail-qt0-f169.google.com with SMTP id l7so142575291qtd.1
 for <22990 <at> debbugs.gnu.org>; Wed, 11 Jan 2017 02:51:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=craven.ch; s=google;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc:content-transfer-encoding;
 bh=fnR3BzWXg9zH+oz7DlIxF0nDKjqa9biKzR3J9MGA87c=;
 b=F/mMGKwa1YZR+mKT5PPquYNLky3fMTI4cfR4VFJ5zBWNbtzEPfCpugYMOkk1jKn2AW
 lWY7uscaNjbQhiOJfORBeRCFMMeN5e1NwzseEJ8/o57D54isbXz5W1iYIOwKCzMU12Ko
 qIFkbXQbBX2YMPqZd5AKF5fouWegQ/gya1Zk8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc:content-transfer-encoding;
 bh=fnR3BzWXg9zH+oz7DlIxF0nDKjqa9biKzR3J9MGA87c=;
 b=gjxboqGPvXK5PvP3UHmSaN6VpyPZA4mQ8CawBe0c8UcXcAbH2frjGMNnbciVY+HsG4
 4b/YCxD76/pI2SDSLk8V1oU0/o4PQAuKLFKB10huxqiisabuq6JWndriqwCqPKSEki2q
 mzxiP+alZS7XqHU0ZBO1h9SEs5UpMhU1moRVeN4wB1KcYsuNHPVEY5TliGXt43+D9knO
 jRtN5j9RSWCT9N8xz2JjKR3JGzYIxcBhJIif+dGEEbMH4mIh3kR46nSdyXAIKK0/iIK8
 NfzcO7Zk9+RhMvzXSpCdYHq8zfSFBbODJ4zGWVjm7FXbqG9ZRPQ4K+kdjXxC5aGz9t/F
 HZjg==
X-Gm-Message-State: AIkVDXJ5PziZOzyq9ZC7JW8VRkzGpFu2otCCokl7Bnm5wmZwX+ekaPHmRPk39lUnh4ruu/1LKGt/v6ePY+BU1g==
X-Received: by 10.237.59.186 with SMTP id r55mr6862998qte.22.1484131889543;
 Wed, 11 Jan 2017 02:51:29 -0800 (PST)
MIME-Version: 1.0
Received: by 10.12.133.38 with HTTP; Wed, 11 Jan 2017 02:51:29 -0800 (PST)
X-Originating-IP: [151.248.146.13]
In-Reply-To: <874m15yl99.fsf@HIDDEN>
References: <8737rxx8gk.fsf@HIDDEN> <87k2a5rzue.fsf@HIDDEN>
 <877f633kt1.fsf@HIDDEN> <874m15yl99.fsf@HIDDEN>
From: David Craven <david@HIDDEN>
Date: Wed, 11 Jan 2017 11:51:29 +0100
Message-ID: <CAL1_imnWoGFQ=NBjghV1wracigF676j4zomgCsfCTPO+JiPbKQ@HIDDEN>
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 22990
Cc: 22990 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)

> On second thought, the whole idea of applying grafts on the final result
> (instead of applying grafts at each step like we do now) doesn=E2=80=99t =
fly.
> It works well for things like a profile or the system derivation, but
> breaks for less trivial things.

> For example, if you=E2=80=99re building a VM image or a binary tarball, y=
ou
> really need to graft packages early on; trying to graft the VM image or
> binary tarball wouldn=E2=80=99t have the desired effect.

Isn't a system derivation or a profile derivation an intermediate step
to these derivations? Can't there be a flag or something called
#:already-grafted? #t?




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 11 Jan 2017 09:54:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 04:54:58 2017
Received: from localhost ([127.0.0.1]:49391 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cRFcP-0001OL-Us
	for submit <at> debbugs.gnu.org; Wed, 11 Jan 2017 04:54:58 -0500
Received: from eggs.gnu.org ([208.118.235.92]:60301)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1cRFcO-0001O6-Qv
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 04:54:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1cRFcF-0003Rf-Sp
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 04:54:51 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47715)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1cRFcF-0003Ra-Pw
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 04:54:47 -0500
Received: from reverse-83.fdn.fr ([80.67.176.83]:37620 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1cRFcF-0003tS-56
 for 22990 <at> debbugs.gnu.org; Wed, 11 Jan 2017 04:54:47 -0500
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: 22990 <at> debbugs.gnu.org
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN> <87k2a5rzue.fsf@HIDDEN>
 <877f633kt1.fsf@HIDDEN>
Date: Wed, 11 Jan 2017 10:54:42 +0100
In-Reply-To: <877f633kt1.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Mon, 09 Jan 2017 23:55:38 +0100")
Message-ID: <874m15yl99.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -8.2 (--------)
X-Debbugs-Envelope-To: 22990
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -8.2 (--------)

ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:

> ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:
>
>> ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:
>
> [...]
>
>>> To achieve this, I=E2=80=99m thinking of extending gexp code such that =
gexp
>>> compilers can return a list of applicable grafts.  The =E2=80=98package=
=E2=80=99
>>> compiler would do #:graft? #f and instead let =E2=80=98gexp->derivation=
=E2=80=99 call
>>> =E2=80=98graft-derivation=E2=80=99.
>>
>> The =E2=80=98wip-gexp-grafts=E2=80=99 branch does that.  Namely, it=E2=
=80=99s possible to know
>> what grafts would apply to a gexp derivation build, so that one can
>> first build ungrafted, and then apply the grafts to the results.
>> So for a profile, we=E2=80=99d first build the profile as is, and only t=
hen
>> would we graft it.
>>
>> Right now the tip of this branch is a hack such that =E2=80=98guix packa=
ge=E2=80=99:
>>
>>   1. Builds the original (ungrafted) derivation of the profile;
>>
>>   2. Manually calls =E2=80=98graft-derivation=E2=80=99 on that, passing =
it the list of
>>      applicable grafts.
>>
>> Conceptually it=E2=80=99s what we want to do, but the drawback is that t=
he
>> caller (here =E2=80=98guix package=E2=80=99) goes through a lot of hops =
to get the list
>> of grafts and to apply it.
>>
>> I think we should instead have a way to annotate a derivation with a
>> list of grafts, as well as a procedure to build that derivations in two
>> phases (first the original derivation, then the grafts).
>
> The current iteration introduces =E2=80=9Cbuild continuation=E2=80=9D: a =
derivation can
> be annotated with a continuation, and the =E2=80=98build-things=E2=80=99 =
procedure will
> loop over continuations and return the final results (that only works
> for derivations directly passed as an argument to =E2=80=98build-things=
=E2=80=99.)

On second thought, the whole idea of applying grafts on the final result
(instead of applying grafts at each step like we do now) doesn=E2=80=99t fl=
y.
It works well for things like a profile or the system derivation, but
breaks for less trivial things.

For example, if you=E2=80=99re building a VM image or a binary tarball, you
really need to graft packages early on; trying to graft the VM image or
binary tarball wouldn=E2=80=99t have the desired effect.

I=E2=80=99ve pushed =E2=80=98wip-gexp-grafts=E2=80=99 again but it seems to=
 be a dead end.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 9 Jan 2017 22:55:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 09 17:55:51 2017
Received: from localhost ([127.0.0.1]:48249 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cQir1-0002Pd-IO
	for submit <at> debbugs.gnu.org; Mon, 09 Jan 2017 17:55:51 -0500
Received: from eggs.gnu.org ([208.118.235.92]:59576)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1cQiqz-0002PP-8l
 for 22990 <at> debbugs.gnu.org; Mon, 09 Jan 2017 17:55:49 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1cQiqq-0004p0-Sf
 for 22990 <at> debbugs.gnu.org; Mon, 09 Jan 2017 17:55:44 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50048)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1cQiqq-0004oq-PZ
 for 22990 <at> debbugs.gnu.org; Mon, 09 Jan 2017 17:55:40 -0500
Received: from reverse-83.fdn.fr ([80.67.176.83]:34442 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1cQiqq-0005El-3O
 for 22990 <at> debbugs.gnu.org; Mon, 09 Jan 2017 17:55:40 -0500
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: 22990 <at> debbugs.gnu.org
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN> <87k2a5rzue.fsf@HIDDEN>
Date: Mon, 09 Jan 2017 23:55:38 +0100
In-Reply-To: <87k2a5rzue.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Sun, 08 Jan 2017 22:44:41 +0100")
Message-ID: <877f633kt1.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -8.2 (--------)
X-Debbugs-Envelope-To: 22990
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -8.2 (--------)

ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:

> ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:

[...]

>> To achieve this, I=E2=80=99m thinking of extending gexp code such that g=
exp
>> compilers can return a list of applicable grafts.  The =E2=80=98package=
=E2=80=99
>> compiler would do #:graft? #f and instead let =E2=80=98gexp->derivation=
=E2=80=99 call
>> =E2=80=98graft-derivation=E2=80=99.
>
> The =E2=80=98wip-gexp-grafts=E2=80=99 branch does that.  Namely, it=E2=80=
=99s possible to know
> what grafts would apply to a gexp derivation build, so that one can
> first build ungrafted, and then apply the grafts to the results.
> So for a profile, we=E2=80=99d first build the profile as is, and only th=
en
> would we graft it.
>
> Right now the tip of this branch is a hack such that =E2=80=98guix packag=
e=E2=80=99:
>
>   1. Builds the original (ungrafted) derivation of the profile;
>
>   2. Manually calls =E2=80=98graft-derivation=E2=80=99 on that, passing i=
t the list of
>      applicable grafts.
>
> Conceptually it=E2=80=99s what we want to do, but the drawback is that the
> caller (here =E2=80=98guix package=E2=80=99) goes through a lot of hops t=
o get the list
> of grafts and to apply it.
>
> I think we should instead have a way to annotate a derivation with a
> list of grafts, as well as a procedure to build that derivations in two
> phases (first the original derivation, then the grafts).

The current iteration introduces =E2=80=9Cbuild continuation=E2=80=9D: a de=
rivation can
be annotated with a continuation, and the =E2=80=98build-things=E2=80=99 pr=
ocedure will
loop over continuations and return the final results (that only works
for derivations directly passed as an argument to =E2=80=98build-things=E2=
=80=99.)

Example (where Guile 2.0.12 replaced by 2.0.13):

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env  guix package -p foo -i guile-json guile-ssh gdb
The following packages will be installed:
   guile-json	0.5.0	/gnu/store/sd8jm2rw7cp3bnrk421kr97ki6sqxnhz-guile-json-=
0.5.0
   guile-ssh	0.10.2	/gnu/store/w90isin9pnm9ri8w9njxby8h98lfnkzq-guile-ssh-0=
.10.2
   gdb	7.12	/gnu/store/d2a4lmycc13ssdf47a9h410knlfqqq41-gdb-7.12

applying 6 grafts to /gnu/store/s7w6r9ih52kdzcl7kprf2rq48s69zh98-profile
3 packages in profile
The following environment variable definitions may be needed:
   export PATH=3D"foo/bin${PATH:+:}$PATH"
$ guix gc -R foo | grep guile
/gnu/store/7fisf4frrgsjzmknjbab1dal23wxrp8d-guile-2.0.13
/gnu/store/w90isin9pnm9ri8w9njxby8h98lfnkzq-guile-ssh-0.10.2
/gnu/store/sd8jm2rw7cp3bnrk421kr97ki6sqxnhz-guile-json-0.5.0
--8<---------------cut here---------------end--------------->8---

That seems to be a good model.

Now we must make sure that it works also for =E2=80=98guix build foo=E2=80=
=99 and =E2=80=98guix
system build=E2=80=99.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 8 Jan 2017 21:44:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 08 16:44:54 2017
Received: from localhost ([127.0.0.1]:47176 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cQLGo-0005QX-Ij
	for submit <at> debbugs.gnu.org; Sun, 08 Jan 2017 16:44:54 -0500
Received: from eggs.gnu.org ([208.118.235.92]:54108)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1cQLGn-0005QL-BP
 for 22990 <at> debbugs.gnu.org; Sun, 08 Jan 2017 16:44:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1cQLGe-0007mm-9i
 for 22990 <at> debbugs.gnu.org; Sun, 08 Jan 2017 16:44:48 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33368)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1cQLGe-0007mY-65
 for 22990 <at> debbugs.gnu.org; Sun, 08 Jan 2017 16:44:44 -0500
Received: from reverse-83.fdn.fr ([80.67.176.83]:47630 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1cQLGd-0003OK-HJ
 for 22990 <at> debbugs.gnu.org; Sun, 08 Jan 2017 16:44:43 -0500
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: 22990 <at> debbugs.gnu.org
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN>
Date: Sun, 08 Jan 2017 22:44:41 +0100
In-Reply-To: <8737rxx8gk.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Fri, 11 Mar 2016 17:52:59 +0100")
Message-ID: <87k2a5rzue.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -8.2 (--------)
X-Debbugs-Envelope-To: 22990
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -8.2 (--------)

ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:

> Concretely, what this means is this:
>
>   $ guix package -u
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   [=E2=80=A6]
>   The following files would be downloaded:
>
> Each of the initial =E2=80=9Cupdating list=E2=80=9D message corresponds t=
o an HTTP
> request for a single narinfo file, which can take around 1 second.
>
> Instead, the ideal thing would be to fetch the narinfo files for all the
> relevant packages at once; that way, we=E2=80=99d spawn =E2=80=98guix sub=
stitute=E2=80=99 only
> once, and it would benefit from HTTP pipelining (one round-trip instead
> of N.)
>
> To achieve this, I=E2=80=99m thinking of extending gexp code such that ge=
xp
> compilers can return a list of applicable grafts.  The =E2=80=98package=
=E2=80=99
> compiler would do #:graft? #f and instead let =E2=80=98gexp->derivation=
=E2=80=99 call
> =E2=80=98graft-derivation=E2=80=99.

The =E2=80=98wip-gexp-grafts=E2=80=99 branch does that.  Namely, it=E2=80=
=99s possible to know
what grafts would apply to a gexp derivation build, so that one can
first build ungrafted, and then apply the grafts to the results.
So for a profile, we=E2=80=99d first build the profile as is, and only then
would we graft it.

Right now the tip of this branch is a hack such that =E2=80=98guix package=
=E2=80=99:

  1. Builds the original (ungrafted) derivation of the profile;

  2. Manually calls =E2=80=98graft-derivation=E2=80=99 on that, passing it =
the list of
     applicable grafts.

Conceptually it=E2=80=99s what we want to do, but the drawback is that the
caller (here =E2=80=98guix package=E2=80=99) goes through a lot of hops to =
get the list
of grafts and to apply it.

I think we should instead have a way to annotate a derivation with a
list of grafts, as well as a procedure to build that derivations in two
phases (first the original derivation, then the grafts).

To be continued=E2=80=A6

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.
Merged 22990 25137. Request was from ludo@HIDDEN (Ludovic Courtès) to control <at> debbugs.gnu.org. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 15 Mar 2016 21:50:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 15 17:50:19 2016
Received: from localhost ([127.0.0.1]:49490 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1afwr5-0006QK-7y
	for submit <at> debbugs.gnu.org; Tue, 15 Mar 2016 17:50:19 -0400
Received: from eggs.gnu.org ([208.118.235.92]:48155)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1afwr3-0006Q7-L9
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 17:50:17 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1afwqv-0002Qd-Cz
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 17:50:12 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:44015)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1afwqv-0002QZ-9c; Tue, 15 Mar 2016 17:50:09 -0400
Received: from reverse-83.fdn.fr ([80.67.176.83]:39208 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1afwqu-0003s7-IN; Tue, 15 Mar 2016 17:50:09 -0400
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Mark H Weaver <mhw@HIDDEN>
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN> <87a8m4123s.fsf@HIDDEN>
 <874mcazifb.fsf@HIDDEN> <8737rrmv8s.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 26 =?utf-8?Q?Vent=C3=B4se?= an 224 de la =?utf-8?Q?R?=
 =?utf-8?Q?=C3=A9volution?=
X-PGP-Key-ID: 0x3D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-unknown-linux-gnu
Date: Tue, 15 Mar 2016 22:50:05 +0100
In-Reply-To: <8737rrmv8s.fsf@HIDDEN> (Mark H. Weaver's message of "Tue, 15
 Mar 2016 14:49:55 -0400")
Message-ID: <87twk7qulu.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 22990
Cc: Alex Kost <alezost@HIDDEN>, 22990 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Mark H Weaver <mhw@HIDDEN> skribis:

> ludo@HIDDEN (Ludovic Court=C3=A8s) writes:
>
>> Alex Kost <alezost@HIDDEN> skribis:
>>
>>> Ludovic Court=C3=A8s (2016-03-11 19:52 +0300) wrote:
>>>
>>>> As of right now (v0.9.0-2007-g66a30a3), =E2=80=98graft-derivation=E2=
=80=99 works either by:
>>>>
>>>>   1. Fetching substitute info about the things being built so that it
>>>>      can determine its references, which in turns allows it to determi=
ne
>>>>      whether they need to be grafted.
>>>>
>>>>   2. Building stuff, as a last resort, so that it can determine its
>>>>      references.
>>>
>>> I noticed that #1 is happening even with --no-substitutes option.  Is it
>>> intended?
>>
>> Not really, but I see this is because =E2=80=98substitutable-path-info=
=E2=80=99 (called
>> from =E2=80=98references/substitutes=E2=80=99, called from =E2=80=98graf=
t-derivation=E2=80=99) works
>> regardless of whether substitutes are enabled:
>>
>> scheme@(guile-user)> ,use(guix)
>> scheme@(guile-user)> (define s (open-connection))
>> scheme@(guile-user)> (set-build-options s #:use-substitutes? #f)
>> $2 =3D #t
>> scheme@(guile-user)> (valid-path? s "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1=
fazv6smn-mutt-1.5.24")
>> $3 =3D #f
>> scheme@(guile-user)> (substitutable-path-info s '("/gnu/store/qf2lm7jpii=
yygxz8zq0r1ca1fazv6smn-mutt-1.5.24"))
>> $4 =3D (#<<substitutable> path: "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1fazv=
6smn-mutt-1.5.24" deriver: "/gnu/store/jcl9c3w463xa2g963q5a60rrd97y1g28-mut=
t-1.5.24.drv" refs: ("/gnu/store/3gmzl5jpk700hqyr8p3kfg0vgcnw8d97-libassuan=
-2.4.2" "/gnu/store/b02lmk67jq1vcflk2m2bwzc8gmwmndqp-ncurses-6.0" "/gnu/sto=
re/d3xdc2w87yw3raafwb9q34gxx4xqci8k-cyrus-sasl-2.1.26" "/gnu/store/pkasxags=
a4z4viscfpl6sjszmdmwncl1-gcc-4.9.3-lib" "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca=
1fazv6smn-mutt-1.5.24" "/gnu/store/qvx4q6lbwi4s3cwr8wqaa7kcva0a5c4b-openssl=
-1.0.2f" "/gnu/store/sb40mddkia0brc814xkbnhxccfm32q3a-gpgme-1.6.0" "/gnu/st=
ore/sgzfawy95pfn7nsw3xvmca58llm5zzbc-glibc-2.22" "/gnu/store/x2p2biyybcb2wa=
c77qz9468asc5fm48i-perl-5.22.1" "/gnu/store/x8dmdlrn5qn0wrbcnngj55y3ab73h0p=
p-bash-4.3.42" "/gnu/store/zpxg45dq67psrn4wmfk4l635h0si8q63-libgpg-error-1.=
21") dl-size: 0 nar-size: 6661016>)
>
> Is the information from the substitute server authenticated by checking
> hydra's signature against the list of keys in /etc/guix/acls?

Yes.

>> However, substitutes are not downloaded, so in this regard
>> --no-substitutes is honored.
>
> It depends on the intent of --no-substitutes.  If the intent is to avoid
> trusting the substitute server, then by relying on the accuracy of the
> runtime dependency data from Hydra, we are failing to honor that intent.
>
> That said, I think it's okay to document that --no-substitutes alone is
> not sufficient to avoid trusting a substitute server, and that the
> proper way to accomplish that is to make sure its key is not in
> /etc/guix/acls.

The sysadmin gets to choose which principals are trusted; unprivileged
users can only shrink this set.

The weird thing is that in this case, passing --substitute-urls=3D'' on
the client side would effectively disable substitutes entirely.

> What do you think?

We could augment the doc for --no-substitutes, I guess we should first
document that grafting relies on server-provided info.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 15 Mar 2016 18:50:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 15 14:50:08 2016
Received: from localhost ([127.0.0.1]:49372 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1afu2i-00020X-I7
	for submit <at> debbugs.gnu.org; Tue, 15 Mar 2016 14:50:08 -0400
Received: from world.peace.net ([50.252.239.5]:39761 ident=hope0)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mhw@HIDDEN>) id 1afu2g-00020P-Rk
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 14:50:07 -0400
Received: from [10.1.10.78] (helo=jojen)
 by world.peace.net with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.72) (envelope-from <mhw@HIDDEN>)
 id 1afu2Z-0004re-0v; Tue, 15 Mar 2016 14:49:59 -0400
From: Mark H Weaver <mhw@HIDDEN>
To: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN> <87a8m4123s.fsf@HIDDEN>
 <874mcazifb.fsf@HIDDEN>
Date: Tue, 15 Mar 2016 14:49:55 -0400
In-Reply-To: <874mcazifb.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Sun, 13 Mar 2016 13:11:36 +0100")
Message-ID: <8737rrmv8s.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.92 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 22990
Cc: Alex Kost <alezost@HIDDEN>, 22990 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

ludo@HIDDEN (Ludovic Court=C3=A8s) writes:

> Alex Kost <alezost@HIDDEN> skribis:
>
>> Ludovic Court=C3=A8s (2016-03-11 19:52 +0300) wrote:
>>
>>> As of right now (v0.9.0-2007-g66a30a3), =E2=80=98graft-derivation=E2=80=
=99 works either by:
>>>
>>>   1. Fetching substitute info about the things being built so that it
>>>      can determine its references, which in turns allows it to determine
>>>      whether they need to be grafted.
>>>
>>>   2. Building stuff, as a last resort, so that it can determine its
>>>      references.
>>
>> I noticed that #1 is happening even with --no-substitutes option.  Is it
>> intended?
>
> Not really, but I see this is because =E2=80=98substitutable-path-info=E2=
=80=99 (called
> from =E2=80=98references/substitutes=E2=80=99, called from =E2=80=98graft=
-derivation=E2=80=99) works
> regardless of whether substitutes are enabled:
>
> scheme@(guile-user)> ,use(guix)
> scheme@(guile-user)> (define s (open-connection))
> scheme@(guile-user)> (set-build-options s #:use-substitutes? #f)
> $2 =3D #t
> scheme@(guile-user)> (valid-path? s "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1f=
azv6smn-mutt-1.5.24")
> $3 =3D #f
> scheme@(guile-user)> (substitutable-path-info s '("/gnu/store/qf2lm7jpiiy=
ygxz8zq0r1ca1fazv6smn-mutt-1.5.24"))
> $4 =3D (#<<substitutable> path: "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1fazv6=
smn-mutt-1.5.24" deriver: "/gnu/store/jcl9c3w463xa2g963q5a60rrd97y1g28-mutt=
-1.5.24.drv" refs: ("/gnu/store/3gmzl5jpk700hqyr8p3kfg0vgcnw8d97-libassuan-=
2.4.2" "/gnu/store/b02lmk67jq1vcflk2m2bwzc8gmwmndqp-ncurses-6.0" "/gnu/stor=
e/d3xdc2w87yw3raafwb9q34gxx4xqci8k-cyrus-sasl-2.1.26" "/gnu/store/pkasxagsa=
4z4viscfpl6sjszmdmwncl1-gcc-4.9.3-lib" "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1=
fazv6smn-mutt-1.5.24" "/gnu/store/qvx4q6lbwi4s3cwr8wqaa7kcva0a5c4b-openssl-=
1.0.2f" "/gnu/store/sb40mddkia0brc814xkbnhxccfm32q3a-gpgme-1.6.0" "/gnu/sto=
re/sgzfawy95pfn7nsw3xvmca58llm5zzbc-glibc-2.22" "/gnu/store/x2p2biyybcb2wac=
77qz9468asc5fm48i-perl-5.22.1" "/gnu/store/x8dmdlrn5qn0wrbcnngj55y3ab73h0pp=
-bash-4.3.42" "/gnu/store/zpxg45dq67psrn4wmfk4l635h0si8q63-libgpg-error-1.2=
1") dl-size: 0 nar-size: 6661016>)

Is the information from the substitute server authenticated by checking
hydra's signature against the list of keys in /etc/guix/acls?

The reason I ask is that if the set of runtime dependencies received is
incomplete, it could lead to incorrect grafting, namely that references
to compromised libraries could be retained.

> However, substitutes are not downloaded, so in this regard
> --no-substitutes is honored.

It depends on the intent of --no-substitutes.  If the intent is to avoid
trusting the substitute server, then by relying on the accuracy of the
runtime dependency data from Hydra, we are failing to honor that intent.

That said, I think it's okay to document that --no-substitutes alone is
not sufficient to avoid trusting a substitute server, and that the
proper way to accomplish that is to make sure its key is not in
/etc/guix/acls.

What do you think?

    Thanks,
      Mark




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 15 Mar 2016 08:24:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 15 04:24:55 2016
Received: from localhost ([127.0.0.1]:48481 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1afkHf-00023R-Js
	for submit <at> debbugs.gnu.org; Tue, 15 Mar 2016 04:24:55 -0400
Received: from eggs.gnu.org ([208.118.235.92]:34251)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1afkHd-00023E-Qp
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 04:24:54 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1afkHV-0005NB-Fe
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 04:24:48 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57703)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1afkHV-0005N7-C6
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 04:24:45 -0400
Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:56398 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1afkHU-0005Ix-Qi
 for 22990 <at> debbugs.gnu.org; Tue, 15 Mar 2016 04:24:45 -0400
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: 22990 <at> debbugs.gnu.org
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN>
Date: Tue, 15 Mar 2016 09:24:42 +0100
In-Reply-To: <8737rxx8gk.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Fri, 11 Mar 2016 17:52:59 +0100")
Message-ID: <87mvq0tagl.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 22990
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

ludo@HIDDEN (Ludovic Court=C3=A8s) skribis:

> Concretely, what this means is this:
>
>   $ guix package -u
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   substitute: updating list of substitutes from 'http://mirror.guixsd.org=
'... 100.0%
>   [=E2=80=A6]
>   The following files would be downloaded:

This is somewhat mitigated by commits
f09aea1b58b3ef961d3cc712f116fe4617bc8f90 and
264fdedb408ba3620d1e361de6c77e7925025301.

In addition, 026ca50fa4c46a8e280cd51621bbec76b12c0757 slightly improves
=E2=80=98guix substitute=E2=80=99 performance.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 13 Mar 2016 12:11:52 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 13 08:11:52 2016
Received: from localhost ([127.0.0.1]:46660 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1af4sC-0001ie-0V
	for submit <at> debbugs.gnu.org; Sun, 13 Mar 2016 08:11:52 -0400
Received: from eggs.gnu.org ([208.118.235.92]:53736)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <ludo@HIDDEN>) id 1af4sA-0001iS-3d
 for 22990 <at> debbugs.gnu.org; Sun, 13 Mar 2016 08:11:50 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1af4s0-0001kE-0E
 for 22990 <at> debbugs.gnu.org; Sun, 13 Mar 2016 08:11:44 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36331)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1af4rz-0001kA-Td; Sun, 13 Mar 2016 08:11:39 -0400
Received: from reverse-83.fdn.fr ([80.67.176.83]:60896 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1af4ry-0003dP-TK; Sun, 13 Mar 2016 08:11:39 -0400
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Alex Kost <alezost@HIDDEN>
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN> <87a8m4123s.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 24 =?utf-8?Q?Vent=C3=B4se?= an 224 de la =?utf-8?Q?R?=
 =?utf-8?Q?=C3=A9volution?=
X-PGP-Key-ID: 0x3D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-unknown-linux-gnu
Date: Sun, 13 Mar 2016 13:11:36 +0100
In-Reply-To: <87a8m4123s.fsf@HIDDEN> (Alex Kost's message of "Sat, 12 Mar
 2016 12:23:35 +0300")
Message-ID: <874mcazifb.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 22990
Cc: 22990 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Alex Kost <alezost@HIDDEN> skribis:

> Ludovic Court=C3=A8s (2016-03-11 19:52 +0300) wrote:
>
>> As of right now (v0.9.0-2007-g66a30a3), =E2=80=98graft-derivation=E2=80=
=99 works either by:
>>
>>   1. Fetching substitute info about the things being built so that it
>>      can determine its references, which in turns allows it to determine
>>      whether they need to be grafted.
>>
>>   2. Building stuff, as a last resort, so that it can determine its
>>      references.
>
> I noticed that #1 is happening even with --no-substitutes option.  Is it
> intended?

Not really, but I see this is because =E2=80=98substitutable-path-info=E2=
=80=99 (called
from =E2=80=98references/substitutes=E2=80=99, called from =E2=80=98graft-d=
erivation=E2=80=99) works
regardless of whether substitutes are enabled:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix)
scheme@(guile-user)> (define s (open-connection))
scheme@(guile-user)> (set-build-options s #:use-substitutes? #f)
$2 =3D #t
scheme@(guile-user)> (valid-path? s "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1faz=
v6smn-mutt-1.5.24")
$3 =3D #f
scheme@(guile-user)> (substitutable-path-info s '("/gnu/store/qf2lm7jpiiyyg=
xz8zq0r1ca1fazv6smn-mutt-1.5.24"))
$4 =3D (#<<substitutable> path: "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1fazv6sm=
n-mutt-1.5.24" deriver: "/gnu/store/jcl9c3w463xa2g963q5a60rrd97y1g28-mutt-1=
.5.24.drv" refs: ("/gnu/store/3gmzl5jpk700hqyr8p3kfg0vgcnw8d97-libassuan-2.=
4.2" "/gnu/store/b02lmk67jq1vcflk2m2bwzc8gmwmndqp-ncurses-6.0" "/gnu/store/=
d3xdc2w87yw3raafwb9q34gxx4xqci8k-cyrus-sasl-2.1.26" "/gnu/store/pkasxagsa4z=
4viscfpl6sjszmdmwncl1-gcc-4.9.3-lib" "/gnu/store/qf2lm7jpiiyygxz8zq0r1ca1fa=
zv6smn-mutt-1.5.24" "/gnu/store/qvx4q6lbwi4s3cwr8wqaa7kcva0a5c4b-openssl-1.=
0.2f" "/gnu/store/sb40mddkia0brc814xkbnhxccfm32q3a-gpgme-1.6.0" "/gnu/store=
/sgzfawy95pfn7nsw3xvmca58llm5zzbc-glibc-2.22" "/gnu/store/x2p2biyybcb2wac77=
qz9468asc5fm48i-perl-5.22.1" "/gnu/store/x8dmdlrn5qn0wrbcnngj55y3ab73h0pp-b=
ash-4.3.42" "/gnu/store/zpxg45dq67psrn4wmfk4l635h0si8q63-libgpg-error-1.21"=
) dl-size: 0 nar-size: 6661016>)
--8<---------------cut here---------------end--------------->8---

However, substitutes are not downloaded, so in this regard
--no-substitutes is honored.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at 22990 <at> debbugs.gnu.org:


Received: (at 22990) by debbugs.gnu.org; 12 Mar 2016 09:23:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 12 04:23:25 2016
Received: from localhost ([127.0.0.1]:45716 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aefld-0000Lw-Iv
	for submit <at> debbugs.gnu.org; Sat, 12 Mar 2016 04:23:25 -0500
Received: from mail-lb0-f172.google.com ([209.85.217.172]:34464)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <alezost@HIDDEN>) id 1aeflb-0000Ld-Mt
 for 22990 <at> debbugs.gnu.org; Sat, 12 Mar 2016 04:23:24 -0500
Received: by mail-lb0-f172.google.com with SMTP id xr8so179102683lbb.1
 for <22990 <at> debbugs.gnu.org>; Sat, 12 Mar 2016 01:23:23 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=4WuO3BqpwsW0S32YaYjqP8MLksyEX1YyUIDb5fQNzXE=;
 b=hzCONfhQtsN6FUnaH1eBEol0cGpcUAYYEcyEbtKKrmwICwo5Y3WWXa7dp+ir4jhkJe
 gY+CPEgKebmG6dMhXWU//9e6hHy0RoNLrRFYm6BuMNJNOzoJB2o/Z5cy0YxUfMLwWyHr
 syIN8Y7GJGKsXJvtpnVinOJa0+F4GMwc4egYN7Zs5b9lUBJzy4e2T27BKzJrrkne+JQP
 3bXCYMzz9NpBuTBQufoD5Esw061cLxsAHNjW3kbgzU76QIAGJmmAl3dLiXQgJF9iSdqi
 qMWP/Pg4TZHPYyWzI15m6oE1PnTxvySGoD0ZXBjX8kYcDQ6l0oSuMfeGNeqFuFzDMyvg
 n0QQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=4WuO3BqpwsW0S32YaYjqP8MLksyEX1YyUIDb5fQNzXE=;
 b=Y4mX+H8XBdtHMhYdbK+odlcWMx8Ukl7BtEhVuBp+BeoAwtDL9Sg2kqICYL0h/jBbpT
 7ZwbtmdmmiXjZDUC+95Gwz/wmJBXontl0d6xPdfqAmYWRJE85fype548jlwX7imKsUQs
 Gl8mHyK0CuAxGrpVMnsLeUdNl+enpVYTC24xrFK8O1qKTOmfUKWHC8F871elUa4qcoVg
 i3RKNXdvciA+YbXRxGZrVH1ZsPyU5fUfBoCZMrGCMdSNf1vhoSjmAHhTiGvZHyglcWng
 +/NsQ/KOgRvVT2H1zWxFl3wMgJH+BhIk8prL/lxPC0oNM7WLzzi49OpAWBaLSeUf/byA
 WhGQ==
X-Gm-Message-State: AD7BkJLpKw5/dQyguHeeKt2VEXbsynRJnuxRlub2k55OC/prSBju8PImpWqaycNpP7qg6g==
X-Received: by 10.25.88.69 with SMTP id m66mr3615553lfb.137.1457774597995;
 Sat, 12 Mar 2016 01:23:17 -0800 (PST)
Received: from leviafan ([217.107.192.146])
 by smtp.gmail.com with ESMTPSA id f6sm1965935lbd.4.2016.03.12.01.23.17
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 12 Mar 2016 01:23:17 -0800 (PST)
From: Alex Kost <alezost@HIDDEN>
To: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
Subject: Re: bug#22990: Grafts leads to inefficient substitute info retrieval
References: <8737rxx8gk.fsf@HIDDEN>
Date: Sat, 12 Mar 2016 12:23:35 +0300
In-Reply-To: <8737rxx8gk.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Fri, 11 Mar 2016 17:52:59 +0100")
Message-ID: <87a8m4123s.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 22990
Cc: 22990 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Ludovic Court=C3=A8s (2016-03-11 19:52 +0300) wrote:

> As of right now (v0.9.0-2007-g66a30a3), =E2=80=98graft-derivation=E2=80=
=99 works either by:
>
>   1. Fetching substitute info about the things being built so that it
>      can determine its references, which in turns allows it to determine
>      whether they need to be grafted.
>
>   2. Building stuff, as a last resort, so that it can determine its
>      references.

I noticed that #1 is happening even with --no-substitutes option.  Is it
intended?

I've tried this:

$ guix build --dry-run --no-substitutes mutt
substitute: updating list of substitutes from 'http://hydra.gnu.org'... 100=
.0%
substitute: updating list of substitutes from 'http://hydra.gnu.org'... 100=
.0%
substitute: updating list of substitutes from 'http://hydra.gnu.org'... 100=
.0%
substitute: updating list of substitutes from 'http://hydra.gnu.org'... 100=
.0%
substitute: updating list of substitutes from 'http://hydra.gnu.org'... 100=
.0%
The following derivations would be built:
   /gnu/store/a2w22xlmfwkgwx4vw11dxc6zrdmww435-mutt-1.5.24.drv
   ...

--=20
Alex




Information forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 11 Mar 2016 16:53:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 11 11:53:21 2016
Received: from localhost ([127.0.0.1]:45348 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aeQJV-0001gI-17
	for submit <at> debbugs.gnu.org; Fri, 11 Mar 2016 11:53:21 -0500
Received: from eggs.gnu.org ([208.118.235.92]:38696)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <ludo@HIDDEN>) id 1aeQJS-0001g6-GT
 for submit <at> debbugs.gnu.org; Fri, 11 Mar 2016 11:53:18 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1aeQJK-0001xV-Lf
 for submit <at> debbugs.gnu.org; Fri, 11 Mar 2016 11:53:13 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:43762)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1aeQJK-0001xJ-IS
 for submit <at> debbugs.gnu.org; Fri, 11 Mar 2016 11:53:10 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:57899)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1aeQJE-0003BY-PZ
 for bug-guix@HIDDEN; Fri, 11 Mar 2016 11:53:10 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1aeQJC-0001sH-3V
 for bug-guix@HIDDEN; Fri, 11 Mar 2016 11:53:04 -0500
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48889)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1aeQJC-0001sD-0d
 for bug-guix@HIDDEN; Fri, 11 Mar 2016 11:53:02 -0500
Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:49892 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1aeQJB-0005Gm-Eo
 for bug-guix@HIDDEN; Fri, 11 Mar 2016 11:53:01 -0500
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: bug-guix@HIDDEN
Subject: Grafts leads to inefficient substitute info retrieval
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 22 =?utf-8?Q?Vent=C3=B4se?= an 224 de la =?utf-8?Q?R?=
 =?utf-8?Q?=C3=A9volution?=
X-PGP-Key-ID: 0x3D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-unknown-linux-gnu
Date: Fri, 11 Mar 2016 17:52:59 +0100
Message-ID: <8737rxx8gk.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

As of right now (v0.9.0-2007-g66a30a3), =E2=80=98graft-derivation=E2=80=99 =
works either by:

  1. Fetching substitute info about the things being built so that it
     can determine its references, which in turns allows it to determine
     whether they need to be grafted.

  2. Building stuff, as a last resort, so that it can determine its
     references.

Case #1 is hopefully going to be the most common.

The problem with #1 is that when building a profile, we do one
=E2=80=98package-derivation=E2=80=99 call for each package in the profile, =
which
translates in one =E2=80=98graft-derivation=E2=80=99 call for each relevant=
 package=C2=B9,
which translates into one =E2=80=98references/substitutes=E2=80=99 call for=
 each.

Concretely, what this means is this:

  $ guix package -u
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'.=
.. 100.0%
  [=E2=80=A6]
  The following files would be downloaded:

Each of the initial =E2=80=9Cupdating list=E2=80=9D message corresponds to =
an HTTP
request for a single narinfo file, which can take around 1 second.

Instead, the ideal thing would be to fetch the narinfo files for all the
relevant packages at once; that way, we=E2=80=99d spawn =E2=80=98guix subst=
itute=E2=80=99 only
once, and it would benefit from HTTP pipelining (one round-trip instead
of N.)

To achieve this, I=E2=80=99m thinking of extending gexp code such that gexp
compilers can return a list of applicable grafts.  The =E2=80=98package=E2=
=80=99
compiler would do #:graft? #f and instead let =E2=80=98gexp->derivation=E2=
=80=99 call
=E2=80=98graft-derivation=E2=80=99.

I=E2=80=99ll give it a try and report back.

Ludo=E2=80=99.

=C2=B9 A package is =E2=80=9Crelevant=E2=80=9D if =E2=80=98package-grafts=
=E2=80=99 returns a non-empty list.




Acknowledgement sent to ludo@HIDDEN (Ludovic Courtès):
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#22990; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sat, 27 May 2017 10:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.