Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Debbugs Internal Request <help-debbugs@HIDDEN>
to internal_control <at> debbugs.gnu.org.
Full text available.
Received: (at 41702-done) by debbugs.gnu.org; 23 Oct 2020 14:51:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 23 10:51:43 2020
Received: from localhost ([127.0.0.1]:57963 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1kVyPv-0000Mg-KF
for submit <at> debbugs.gnu.org; Fri, 23 Oct 2020 10:51:43 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48214)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1kVyPu-0000MR-1m
for 41702-done <at> debbugs.gnu.org; Fri, 23 Oct 2020 10:51:42 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:50976)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <ludo@HIDDEN>)
id 1kVyPo-0004Qm-Ib; Fri, 23 Oct 2020 10:51:36 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56962 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1kVyPo-0005FE-1h; Fri, 23 Oct 2020 10:51:36 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Lars-Dominik Braun <ldb@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36>
<87eepuyk99.fsf@HIDDEN> <20200702070000.GA4532@zpidnp36>
<20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@HIDDEN>
<20200706125800.GA3176@zpidnp36> <20200720095059.GB3143@zpidnp36>
<87mu3tq2hh.fsf@HIDDEN> <87r1pqc6sy.fsf@HIDDEN>
<20201023072621.GA3115@zpidnp36>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 2 Brumaire an 229 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Fri, 23 Oct 2020 16:51:34 +0200
In-Reply-To: <20201023072621.GA3115@zpidnp36> (Lars-Dominik Braun's message of
"Fri, 23 Oct 2020 09:26:21 +0200")
Message-ID: <873625ngg9.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41702-done
Cc: Maxim Cournoyer <maxim.cournoyer@HIDDEN>, 41702-done <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi,
Lars-Dominik Braun <ldb@HIDDEN> skribis:
> grafting is still a performance issue imo. Compare for example:
Agreed. The fix in 58bb833365db4e8934a386497d5b00a063cfd27d is
incomplete: we=E2=80=99re still potentially doing things several times.
> $ time guix environment --ad-hoc --search-paths r-learnr
> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s syste=
m 210% cpu 2,844 total
> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts
> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user=
0,08s system 164% cpu 1,277 total
Here=E2=80=99s what I get over a local socket:
--8<---------------cut here---------------start------------->8---
ludo@ribbon ~/src/guix$ time guix environment --ad-hoc --search-paths r-le=
arnr --no-grafts
export PATH=3D"/gnu/store/wkhd1fnmg1q30g21bfn5jprz24m041w6-profile/bin${PAT=
H:+:}$PATH"
real 0m1.790s
user 0m1.928s
sys 0m0.102s
ludo@ribbon ~/src/guix$ time guix environment --ad-hoc --search-paths r-le=
arnr
export PATH=3D"/gnu/store/pwzrpzdbli2561f2yq70kljywn93h5w6-profile/bin${PAT=
H:+:}$PATH"
real 0m3.820s
user 0m4.954s
sys 0m0.105s
--8<---------------cut here---------------end--------------->8---
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Received: (at 41702-done) by debbugs.gnu.org; 23 Oct 2020 07:26:31 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 23 03:26:31 2020 Received: from localhost ([127.0.0.1]:54699 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1kVrT5-0006pF-2g for submit <at> debbugs.gnu.org; Fri, 23 Oct 2020 03:26:31 -0400 Received: from mail-wm1-f49.google.com ([209.85.128.49]:37692) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ldb@HIDDEN>) id 1kVrT3-0006p2-3F for 41702-done <at> debbugs.gnu.org; Fri, 23 Oct 2020 03:26:29 -0400 Received: by mail-wm1-f49.google.com with SMTP id c16so394036wmd.2 for <41702-done <at> debbugs.gnu.org>; Fri, 23 Oct 2020 00:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=9tPOu+m/+Dbo8/5z4YTf2cdbkh8hbDIT2Y0dZIFFWR0=; b=BTRk+/OrmpXoxo8wGodWFBFCNmWY0nnhVFnhz61hV0deX0zgD20hHYum0glS0EU27i 7oo2x7mw6/39PCsxOG023aIXdSy5PzLU2ajxCiFiETep+ftaGsYF5ao6v39df9W8MqN1 uPF/fHD9jfEXhaoXKU54m2Z8QV6iuaRrU8wuVUxnYV9q/5wgowrTDbzxTExKy9TMGrSK hqIgvo8RsnzOUoM9V2ettP/GPvObCKn/+iWg5qCr3KJoSJRT3cEpxtBKDUyrDtnPY5KB qZR1zTnhgasFTvnWyHWUOJCHH/IIuvZWmnieqrI3MjaZHTMkkVGKkO29thDfxs75qs7t WOEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=9tPOu+m/+Dbo8/5z4YTf2cdbkh8hbDIT2Y0dZIFFWR0=; b=p0vBRu8Vm6whvu4s3naOBRMf8xg2gmIRlmhDPyWR4yFo1hIrKepE5tQbO0L6+9OTpT OPoUrgH7bf0tRugXjbxOHfiyfaQfll+l+5X6YWpP6Ybm9wNpy5GSVM4LR8HJnpri3Yx+ 8wK0fRIvWZUB3kR3+rUiKYLeQAssRV0BBFpJRBImY4hsrRG4bpDVzHEZQ3sJccNZXQiG 0mljnTgHb/qCpX79idAsvTkyQtYxiKtQzFhlG0gnkLNjp8IxGX78MX+/gKbWn+2xWHja Oq/HCm4B47zJDhtBOLPh3ws24jjNQDURIzK/6Cmlv+q09t6y0ePVYDRZ01ZOIp5EBij2 Rg4Q== X-Gm-Message-State: AOAM533R1Ggz4StqaIu2KLMdjnZi7XwxhwqxJqdezP0vsCJil1vlOuSC mhl0AtlJZ2999SeKzRcaQJhQhFEHWWHcJkhJ9f7Ih19ItrVWu4EUYL3/OlWYt+tWCCKc0QUJp92 3CVOzVOhfiDxeUwJbr4ecZLja6bX4LIH/WGsJMXsfdBhumUBRK0ax1kW7Xpxh0YdmE31V3/g9sP QGRi4MYtQV0Q== X-Google-Smtp-Source: ABdhPJzbCjZtFnN/xazilSRQ3xBQy6BEgX4jHl182q7Rs3pRotCCWwX81dyAD36XlifIclKkZNkoCg== X-Received: by 2002:a1c:6743:: with SMTP id b64mr813432wmc.157.1603437983193; Fri, 23 Oct 2020 00:26:23 -0700 (PDT) Received: from localhost (dynamic-2a01-0c23-8068-f000-9df9-e3dd-fa3d-1b09.c23.pool.telefonica.de. [2a01:c23:8068:f000:9df9:e3dd:fa3d:1b09]) by smtp.gmail.com with ESMTPSA id g14sm1482603wrx.22.2020.10.23.00.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 00:26:22 -0700 (PDT) Date: Fri, 23 Oct 2020 09:26:21 +0200 From: Lars-Dominik Braun <ldb@HIDDEN> To: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues Message-ID: <20201023072621.GA3115@zpidnp36> References: <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@HIDDEN> <20200706125800.GA3176@zpidnp36> <20200720095059.GB3143@zpidnp36> <87mu3tq2hh.fsf@HIDDEN> <87r1pqc6sy.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2fHTh5uZTiUOsy+g" Content-Disposition: inline In-Reply-To: <87r1pqc6sy.fsf@HIDDEN> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41702-done Cc: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN>, 41702-done <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --2fHTh5uZTiUOsy+g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi Maxim, > Judging from the above, it seems this issue has been resolved. grafting is still a performance issue imo. Compare for example: $ time guix environment --ad-hoc --search-paths r-learnr guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s system 210% cpu 2,844 total $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user 0,08s system 164% cpu 1,277 total Cheers, Lars --2fHTh5uZTiUOsy+g Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl+ShZkACgkQQhN3ARo3 hEZzLgv/eXYwosDaui61yH06wGUcj41RHTGO/fkq56vI2oqXXBcwxEXvIqoixdj9 MMbU5YWIDg8W2GCFQ3Nf0Nul2XyyEtt7z+DeQmbBLEeqnasuPdbkesLuWpE/1vlm qShuckvvSvhz4DsYPsOFQuQQDXGDSUP6+vWIcP5WVGIUl+eXlmYmxCMOFPPT//wS ItJbiOICE68ghSaUf5X8U4sgIM2XipK+sf1lLLlmQX8Ws8PCOBWYQsKdMq5pq3Bt YvDKqD40c4RLlZ4mw642cf6YB7jYjCZnzIOIc7DkgcD0mM/fFE28LEy5TTKQxE7R WLTDFJ10K1glr/OOpXDZsUg7yR7ERDTEGg6iVy5Rr80gBx+D9tG7Ku2XCfE3Aoob q7hcYwMOVwkS5agNlr5tY4+0GhCicOgDcYeos8qWVAkhbP+TOl5NnMRDpvT8XAFE yNDcD2rfDfPL9KEpkXd8U215cLPQ+hVljmPJbVQZG4HvmtxzWtCcOM7cSFLgX82K 2nzwqE3m =Wdtz -----END PGP SIGNATURE----- --2fHTh5uZTiUOsy+g--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702-done) by debbugs.gnu.org; 22 Oct 2020 21:03:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 22 17:03:18 2020
Received: from localhost ([127.0.0.1]:54047 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1kVhjy-0003UQ-1t
for submit <at> debbugs.gnu.org; Thu, 22 Oct 2020 17:03:18 -0400
Received: from mail-qk1-f175.google.com ([209.85.222.175]:39380)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <maxim.cournoyer@HIDDEN>) id 1kVhjw-0003UD-RA
for 41702-done <at> debbugs.gnu.org; Thu, 22 Oct 2020 17:03:17 -0400
Received: by mail-qk1-f175.google.com with SMTP id k9so3298004qki.6
for <41702-done <at> debbugs.gnu.org>; Thu, 22 Oct 2020 14:03:16 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version:content-transfer-encoding;
bh=70zN2uyP7jXFz6Qa86ZC64h5ZKVAoWvpoDi3IGMXTQc=;
b=Tan1yna8WNuMg2/URcTbH96Vi1p7ZlKfBQwflpxgfL2zGDSACqfiYYRL9ludF06qkQ
PX4l45a+fg3tmkcZRURL5WIlkM2F24n/wNHAGfHgHVy2FKEhPUPCr99HN/oMMj1gUBc+
RSkksOnAWZRu6bOws1TtF/eF91b7s1usHHo1z41Ut5PNb+8yztijl2526aMuipomUKfb
e52Zq9Mrv1vqdZMW06reQVgTgmjrsNCUNskEJBqTSOIXcy/0IWWjNGD64NqCV7fLoS7m
jrGfW/jZIlgaDDz7Pj19FannsP1MF3HTv3F1vSE3ENHpUiR8vn7IN+HkUTJ4KIzd/bUJ
SGZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version:content-transfer-encoding;
bh=70zN2uyP7jXFz6Qa86ZC64h5ZKVAoWvpoDi3IGMXTQc=;
b=HJkW02pqCaOjcPfGOSGcyIFdXhbBZIc2QTa3JePocTUvXWE+6fy+2f0DbpDIbypmjp
bId7gr4U6GW+K2p4S63vvZmJ+fPHXSG82gotAFYXRelVcmQ4A0cXinIuSO6HbJhnhmjb
enVYNYgn9V+mavQhMGBgiNqxrBdrGGYDIINgsXO2xJlMLm6OmBGNdKJXXdhdEdsKM0uY
DdVD9fX+RrnxkaL++4XalhCKHSMmMa3gLEHEg6P4nVh4a6zJRGaOWoOhUJio2bVm7zfW
vHx3x4VRnJpSrZRh1F/fAP5lZpZ7yglUheV6i9yam2FIImk0yJuTlCg+pOmjfJ22mR9W
hAeA==
X-Gm-Message-State: AOAM533+JKl82mIUa6KVUPKE1LLBstFNzRfla1frovd0idmnqxfhEMaf
3rY9DWiBc4ndsZerGm73aWMEZ045a64=
X-Google-Smtp-Source: ABdhPJyOMYQ/Ocm6BDQ+2gWFlTMifYt0pVpbmG7XmaafWfLX0US8BzCUBCDQG3yuyAvWMVcb+StWFA==
X-Received: by 2002:a05:620a:19:: with SMTP id
j25mr2979514qki.498.1603400591165;
Thu, 22 Oct 2020 14:03:11 -0700 (PDT)
Received: from hurd (dsl-159-119.b2b2c.ca. [66.158.159.119])
by smtp.gmail.com with ESMTPSA id 69sm1886257qko.48.2020.10.22.14.03.10
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Thu, 22 Oct 2020 14:03:10 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36>
<87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN>
<20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN>
<20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36>
<87k0zhkoyf.fsf@HIDDEN> <20200706125800.GA3176@zpidnp36>
<20200720095059.GB3143@zpidnp36> <87mu3tq2hh.fsf@HIDDEN>
Date: Thu, 22 Oct 2020 17:03:09 -0400
In-Reply-To: <87mu3tq2hh.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Mon, 20 Jul 2020 23:51:38 +0200")
Message-ID: <87r1pqc6sy.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702-done
Cc: 41702-done <at> debbugs.gnu.org, Lars-Dominik Braun <ldb@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello,
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Hi Lars,
>
> Lars-Dominik Braun <ldb@HIDDEN> skribis:
>
>> From 42ff75c9d4da4eb2aeece96119ae5a65a9ccac82 Mon Sep 17 00:00:00 2001
>> From: Lars-Dominik Braun <ldb@HIDDEN>
>> Date: Mon, 20 Jul 2020 11:27:35 +0200
>> Subject: [PATCH 1/2] gnu: guile-ssh: Update to 0.13.0.
>>
>> * gnu/packages/ssh.scm (guile-ssh): Update to 0.13.0.
>
> [...]
>
>> From 6a18c9a4533075d1ee9795777a998fde3e64030f Mon Sep 17 00:00:00 2001
>> From: Lars-Dominik Braun <ldb@HIDDEN>
>> Date: Mon, 20 Jul 2020 11:28:51 +0200
>> Subject: [PATCH 2/2] guix: Speed up RPC via SSH connections.
>>
>> See issue #41702
>>
>> * guix/ssh.scm (open-ssh-session): Enable #:nodelay.
>> * m4/guix.m4 (GUIX_CHECK_GUILE_SSH): Add feature check for this new para=
meter.
>
> I adjusted the commit log and pushed.
>
> I think it=E2=80=99s okay to use the new Guile-SSH feature right away bec=
ause
> it=E2=80=99s not central, and =E2=80=98guix pull=E2=80=99 gives people th=
e ability to get the
> new Guile-SSH version.
Judging from the above, it seems this issue has been resolved.
Closing.
Thank you,
Maxim
Lars-Dominik Braun <ldb@HIDDEN>:Maxim Cournoyer <maxim.cournoyer@HIDDEN>:Received: (at 41702) by debbugs.gnu.org; 20 Jul 2020 21:51:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 20 17:51:47 2020 Received: from localhost ([127.0.0.1]:36567 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jxdhL-0005Us-39 for submit <at> debbugs.gnu.org; Mon, 20 Jul 2020 17:51:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40424) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1jxdhJ-0005Uf-Tc for 41702 <at> debbugs.gnu.org; Mon, 20 Jul 2020 17:51:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42097) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1jxdhE-0006ek-GA; Mon, 20 Jul 2020 17:51:40 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51128 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1jxdhD-0006hi-SC; Mon, 20 Jul 2020 17:51:40 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Lars-Dominik Braun <ldb@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues References: <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@HIDDEN> <20200706125800.GA3176@zpidnp36> <20200720095059.GB3143@zpidnp36> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 3 Thermidor an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 20 Jul 2020 23:51:38 +0200 In-Reply-To: <20200720095059.GB3143@zpidnp36> (Lars-Dominik Braun's message of "Mon, 20 Jul 2020 11:50:59 +0200") Message-ID: <87mu3tq2hh.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41702 Cc: 41702 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi Lars, Lars-Dominik Braun <ldb@HIDDEN> skribis: > From 42ff75c9d4da4eb2aeece96119ae5a65a9ccac82 Mon Sep 17 00:00:00 2001 > From: Lars-Dominik Braun <ldb@HIDDEN> > Date: Mon, 20 Jul 2020 11:27:35 +0200 > Subject: [PATCH 1/2] gnu: guile-ssh: Update to 0.13.0. > > * gnu/packages/ssh.scm (guile-ssh): Update to 0.13.0. [...] > From 6a18c9a4533075d1ee9795777a998fde3e64030f Mon Sep 17 00:00:00 2001 > From: Lars-Dominik Braun <ldb@HIDDEN> > Date: Mon, 20 Jul 2020 11:28:51 +0200 > Subject: [PATCH 2/2] guix: Speed up RPC via SSH connections. > > See issue #41702 > > * guix/ssh.scm (open-ssh-session): Enable #:nodelay. > * m4/guix.m4 (GUIX_CHECK_GUILE_SSH): Add feature check for this new param= eter. I adjusted the commit log and pushed. I think it=E2=80=99s okay to use the new Guile-SSH feature right away becau= se it=E2=80=99s not central, and =E2=80=98guix pull=E2=80=99 gives people the = ability to get the new Guile-SSH version. Thanks! Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 20 Jul 2020 09:51:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 20 05:51:10 2020
Received: from localhost ([127.0.0.1]:33864 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jxSRy-0003Z9-4r
for submit <at> debbugs.gnu.org; Mon, 20 Jul 2020 05:51:10 -0400
Received: from mail-wm1-f51.google.com ([209.85.128.51]:35382)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ldb@HIDDEN>) id 1jxSRv-0003Yi-4X
for 41702 <at> debbugs.gnu.org; Mon, 20 Jul 2020 05:51:08 -0400
Received: by mail-wm1-f51.google.com with SMTP id l2so24554276wmf.0
for <41702 <at> debbugs.gnu.org>; Mon, 20 Jul 2020 02:51:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=ob4xu2FSoalsz6iiOGLUWCFQEW2nGyZf/aFqadauDw4=;
b=F++IhODkPCX5glsN1QLZBLKLB89Mv1JWedl1vbJ8E6aRwOX2AQ9WHkqMbeIjPd6lwL
jXHM10559IRExgUyIDHMgsddAjA0ijiv6cIXjA256lbm35ZqNQh7OdpTvulKPAn3IBkK
UlPvlwr/XEmheaTwqMYrVts9XQoWEfosipxrHp86Lwhfg1QPR5En/yPRNIXHQOe/NL2N
sIYHkXmIM+IfPoM8mVW/YoJsu0Rlts3ABKVHCsaCygWJPyuRLHucV9Ds2djMM8BbNJOp
H/iLUrUZ+YEgoG1wU8EIqpu9sK1MtAUpnncHb+lcJAq7tIGYvctt2fL4SS2OVf7+VZck
JDwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:cc:subject:message-id:references
:mime-version:content-disposition:in-reply-to:user-agent;
bh=ob4xu2FSoalsz6iiOGLUWCFQEW2nGyZf/aFqadauDw4=;
b=Q0NUnJyzNeTM1niuAoFy+hGV/yWayCaQmN7f4Bh7MCWatkuTo7JfdbefqMEOoJylQb
kP9WyyKJ0dxm6HQC9xInktlnlh5A5g0dKYfBnKoj8CmyNEYN6L/eW3/DQWHJmw2fF/4i
9nRsyC3eIp1IJcJWz62Vs0ul1DWKsqXhUPikLkl9o3x5mAaEbGgBlj6pTUtH/O/+qrN6
pBgQxd4TyHt4O4mEO4wWMDxJcMKjww/Aze/x2xs26WAh9zzkcso8vOI41suk8ElOpyG7
ED1IVequDQl4gxdj5Ya3KQZkdnjbkYphNPQlXggzhA7K9Sku7xYqcgRmDE3v9Co9ZLl6
90Ew==
X-Gm-Message-State: AOAM533bU9vf6Bsje5nry7tJ0tlcsGbSM3F/PVGnXK/0xvCUXdcS6woy
oijflgF7xDUjyAl+KCmqtHM/uD7zzfrsfDvW8Clu13oWhFXoXPzwMvdW3x12ph4FE9jYlwUADiH
bfUGlpW2MeBEsB5nHEQEF3R2AJ0lze50nilOiQgZ6QjH4TlcTXF/plNTGnvgdiGBwjX0W5tNOv5
uVM5Me873P
X-Google-Smtp-Source: ABdhPJwtWIaPBZBgi2I7ov9rMPdo01Jnf7LMtTRR8V+16cHfQgdDPpBEWg2LduYf+OvPDtgQbNT0BA==
X-Received: by 2002:a7b:c94a:: with SMTP id i10mr21837654wml.183.1595238661035;
Mon, 20 Jul 2020 02:51:01 -0700 (PDT)
Received: from localhost
(dynamic-2a01-0c22-c82f-7100-d8d0-ddb1-e2de-8de9.c22.pool.telefonica.de.
[2a01:c22:c82f:7100:d8d0:ddb1:e2de:8de9])
by smtp.gmail.com with ESMTPSA id n3sm21077077wre.29.2020.07.20.02.51.00
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 20 Jul 2020 02:51:00 -0700 (PDT)
Date: Mon, 20 Jul 2020 11:50:59 +0200
From: Lars-Dominik Braun <ldb@HIDDEN>
To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
Message-ID: <20200720095059.GB3143@zpidnp36>
References: <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36>
<87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN>
<20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN>
<20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36>
<87k0zhkoyf.fsf@HIDDEN> <20200706125800.GA3176@zpidnp36>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="hYooF8G/hrfVAmum"
Content-Disposition: inline
In-Reply-To: <20200706125800.GA3176@zpidnp36>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--hYooF8G/hrfVAmum
Content-Type: multipart/mixed; boundary="JYK4vJDZwFMowpUq"
Content-Disposition: inline
--JYK4vJDZwFMowpUq
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi Ludo,
a new version has been released, so here is the updated patchset. I=E2=80=
=99m not sure
if it=E2=80=99s possible to make the #:nodelay optional somehow, i.e. retai=
ning
compatibility with guile-ssh<0.13.0. For now there=E2=80=99s a hard feature=
check.
Lars
--JYK4vJDZwFMowpUq
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0001-gnu-guile-ssh-Update-to-0.13.0.patch"
Content-Transfer-Encoding: quoted-printable
=46rom 42ff75c9d4da4eb2aeece96119ae5a65a9ccac82 Mon Sep 17 00:00:00 2001
=46rom: Lars-Dominik Braun <ldb@HIDDEN>
Date: Mon, 20 Jul 2020 11:27:35 +0200
Subject: [PATCH 1/2] gnu: guile-ssh: Update to 0.13.0.
* gnu/packages/ssh.scm (guile-ssh): Update to 0.13.0.
---
gnu/packages/ssh.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/ssh.scm b/gnu/packages/ssh.scm
index a612b9dd15..50316ea1be 100644
--- a/gnu/packages/ssh.scm
+++ b/gnu/packages/ssh.scm
@@ -297,7 +297,7 @@ Additionally, various channel-specific options can be n=
egotiated.")
(define-public guile-ssh
(package
(name "guile-ssh")
- (version "0.12.0")
+ (version "0.13.0")
(home-page "https://github.com/artyom-poptsov/guile-ssh")
(source (origin
(method git-fetch)
@@ -307,7 +307,7 @@ Additionally, various channel-specific options can be n=
egotiated.")
(file-name (string-append name "-" version ".tar.gz"))
(sha256
(base32
- "054hd9rzfhb48gc1hw3rphhp0cnnd4bs5qmidy5ygsyvy9ravlad"))
+ "1q96h98p6x7ah6nc0d2wfx503fmsj36riv9ka9s79z3lzwaf0k26"))
(modules '((guix build utils)))))
(build-system gnu-build-system)
(outputs '("out" "debug"))
--=20
2.20.1
--JYK4vJDZwFMowpUq
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0002-guix-Speed-up-RPC-via-SSH-connections.patch"
Content-Transfer-Encoding: quoted-printable
=46rom 6a18c9a4533075d1ee9795777a998fde3e64030f Mon Sep 17 00:00:00 2001
=46rom: Lars-Dominik Braun <ldb@HIDDEN>
Date: Mon, 20 Jul 2020 11:28:51 +0200
Subject: [PATCH 2/2] guix: Speed up RPC via SSH connections.
See issue #41702
* guix/ssh.scm (open-ssh-session): Enable #:nodelay.
* m4/guix.m4 (GUIX_CHECK_GUILE_SSH): Add feature check for this new paramet=
er.
---
guix/ssh.scm | 5 ++++-
m4/guix.m4 | 6 ++++--
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/guix/ssh.scm b/guix/ssh.scm
index 2d7ca7d01d..3bffb0b525 100644
--- a/guix/ssh.scm
+++ b/guix/ssh.scm
@@ -129,7 +129,10 @@ Throw an error on failure."
;; We need lightweight compression when
;; exchanging full archives.
#:compression compression
- #:compression-level 3)))
+ #:compression-level 3
+
+ ;; Speed up RPC.
+ #:nodelay #t)))
=20
;; Honor ~/.ssh/config.
(session-parse-config! session)
diff --git a/m4/guix.m4 b/m4/guix.m4
index 7c27ae74df..cce03045db 100644
--- a/m4/guix.m4
+++ b/m4/guix.m4
@@ -142,14 +142,16 @@ dnl GUIX_CHECK_GUILE_SSH
dnl
dnl Check whether a recent-enough Guile-SSH is available.
AC_DEFUN([GUIX_CHECK_GUILE_SSH], [
- dnl Check whether 'userauth-gssapi!' (introduced in 0.12.0) is present.
+ dnl Check whether '#:nodelay' paramater to 'make-session' (introduced in
+ dnl 0.13.0) is present.
AC_CACHE_CHECK([whether Guile-SSH is available and recent enough],
[guix_cv_have_recent_guile_ssh],
[GUILE_CHECK([retval],
[(and (@ (ssh channel) channel-send-eof)
(@ (ssh popen) open-remote-pipe)
(@ (ssh dist node) node-eval)
- (@ (ssh auth) userauth-gssapi!))])
+ (@ (ssh auth) userauth-gssapi!)
+ ((@ (ssh session) make-session) #:nodelay #t))])
if test "$retval" =3D 0; then
guix_cv_have_recent_guile_ssh=3D"yes"
else
--=20
2.20.1
--JYK4vJDZwFMowpUq--
--hYooF8G/hrfVAmum
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl8VaQAACgkQQhN3ARo3
hEakuQwA0kCZuyAU1nBpZSQcTuvhuHwbgqIkX2JBGe9oasSkakBZ5XM5dVtARCer
k7lZuyd7Z6QjgUU0+yvLQ2yY05++xGx7Bh+7Cazp7LtbNjKbfTxzVUUKKbqE+qnc
5XYWSHpB7XmuP6RfbBNUtBvw6SUgkNO3mCU1FgSRgVAzFpI8uNS4nTZ8Ise8/X3d
zJRimFqs5qh7gGHI1zRju346c60MhUTexJwkFgxW4UfwlIuu7wZ2WSTTNbXfehrQ
w9zqDCGG8MKftYPLDE9KG1CcB0XB00ssgI7NTf/mlpQOmAizpkLK3PsyV5Cmu4NV
PBAIzn12XQBUCU0brelhfiLTE0t/ueJc3oen2D18KA+Etsjy6hzhFEpfNzLoWgqU
r2niVSUBGvTrwSH2iAdbWL0c2g8PbBQtx6KYoaTORH0xAc7PJcR+pq0QzbM4PSD4
V7StAoQ3hAQOPd3FFXYv2XSZ/yv8VXkJ26y7pT9tFz9e+NsOgnD6JyYy3wK8LRWj
CPdtRngb
=eeqN
-----END PGP SIGNATURE-----
--hYooF8G/hrfVAmum--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Received: (at 41702) by debbugs.gnu.org; 6 Jul 2020 12:58:10 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 06 08:58:10 2020 Received: from localhost ([127.0.0.1]:33774 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jsQhF-0005g1-OW for submit <at> debbugs.gnu.org; Mon, 06 Jul 2020 08:58:09 -0400 Received: from mail-wr1-f53.google.com ([209.85.221.53]:45901) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ldb@HIDDEN>) id 1jsQhD-0005fo-WA for 41702 <at> debbugs.gnu.org; Mon, 06 Jul 2020 08:58:08 -0400 Received: by mail-wr1-f53.google.com with SMTP id s10so40735642wrw.12 for <41702 <at> debbugs.gnu.org>; Mon, 06 Jul 2020 05:58:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=Fruag3AffXbfLibiVmvXQTg9XndnCuVKKhK/OVBeu8s=; b=x6PCkarMXsf8wbFm2sKndpu8+Y24njvIR+4EJbFDA/r6k78e/rrJH347mC2CSwcF2g RiZO5B9Ua/anRl9p/cos7u/f2Vv7BsN8lxPIOkrOlbazC4YKHqc+BiUUZHh3eSBMJEUt AUjN3LNHX2edpXrDWsP3zy+c/69XE4EYTHChxYTGehuQ6bi0Z5wnPXEJ8JE4QcH7La0b fAk63Z+yVtBhphrzDogUM5VHredJWsQGqygdJQwlnsMUW680RG+re4BbRK+X0F6OZq90 YBk9WICVyz+5ixnYGQm5Gv36f8/gNB0PdMyhVeEOuyr/SJtLgGepAxpG/Bu7NgEklyp9 KwSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=Fruag3AffXbfLibiVmvXQTg9XndnCuVKKhK/OVBeu8s=; b=fCwL7D+WIHwiEJV6I7txfM/zVUSyQaAfEftK2uzLEh0HwpR+ixplT7QxA4v7fGBblX XUKEkeXPMx3TtQ995LD51oVvCkfW71mPbNjM9AxXtdskwKvQvvdYUZbDatq0oweWnijS YhIM7hxo0BYPl/0hy/PEyQN+KUuI0TvF0fimsw9dfkW3/JnO7P30NJUAAqNaEHFQ9Lu4 h0D39FTsyRhcHBmFFG3otEBsjQIk0WUVGU6iBWfVIeGVAPdcymdPHhIyYYBF+XLSGzeM bv8dfK9m9LZdh9K3+eiDBhKnCjNej2wrNMGI3g3B/IhWgUmPZ3vvxV+xnIHuMY4I/PFn e1HA== X-Gm-Message-State: AOAM530oF3eEUrLjd67FNhNdBt3EXWZfPfr3peEoluHQuQeKqXYvzAG9 c2Sgmy/nb7GIxSpIGzAfhb0GZVXZuMB9vcSjBKUaqO6sOJnqJW/n3DDni9OrDowsg4zPYpTNWa9 8Zfmkr3oZKz1sivxC5PE/tFNAciltTY/VeLiGQFMQT/oxjDt1XtLbKyLQhP9zzjT3WEh2+2wcqg cj0eY= X-Google-Smtp-Source: ABdhPJz7924LGPAPPxAWRRT3PCL+P0d6/JLrD5JQ/C2xWfilw2K/2b6ipanZfp0a7uZnicNWWA9a2A== X-Received: by 2002:adf:e6c8:: with SMTP id y8mr51623159wrm.40.1594040281897; Mon, 06 Jul 2020 05:58:01 -0700 (PDT) Received: from localhost (dynamic-2a01-0c22-cc82-3000-c581-47a9-adab-bbc8.c22.pool.telefonica.de. [2a01:c22:cc82:3000:c581:47a9:adab:bbc8]) by smtp.gmail.com with ESMTPSA id q1sm23670454wro.82.2020.07.06.05.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jul 2020 05:58:01 -0700 (PDT) Date: Mon, 6 Jul 2020 14:58:00 +0200 From: Lars-Dominik Braun <ldb@HIDDEN> To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues Message-ID: <20200706125800.GA3176@zpidnp36> References: <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="pf9I7BMVVzbSWLtt" Content-Disposition: inline In-Reply-To: <87k0zhkoyf.fsf@HIDDEN> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41702 Cc: 41702 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --pf9I7BMVVzbSWLtt Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Ludo, > I=E2=80=99d rather wait; perhaps you can ask Artyom whether they=E2=80=99= re planning for > a new release soonish? =E2=80=9CI'll see if I can fix some random test failures with Guile 2.2 tha= t sometimes occur, and then I'll prepare a new release.=E2=80=9D > If there=E2=80=99s no plan for a release within two weeks, we can go ahea= d with > the patch you propose. Alright, I=E2=80=99ll prepare a new patch if either of the two events happe= n. Cheers, Lars --pf9I7BMVVzbSWLtt Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl8DH9QACgkQQhN3ARo3 hEYWtAv+OuMtQymFw/zfJqmiftlasJUPsEQzhxmZEyIy31YkPIoE/LUVEBwf1Vmi it+3N/GSAex4OboX3xb1zjLdDrju+joLCGMLqJhQvRnp7tJOTmZC0daYlIwlF/uR xXTx2JWbCWlkuiN0X6UCuG7wT0fIVe12ngpwLsZH2umcclUYH+4PZzH9nlwlPCGE mnq9VZTYwXzcxMCX4Dy2CmF9v5ZrB02NJlAKHRSlUOmk5aJGkX5iPPNFDydq7gow zs72yQQXOn+NfZs6+CZnHzmApjV60eJtcGGX7UALA7IWkRzcdgBsNTj7gAsaLrN8 i62fdT41fJd59QdRPGrnzFEWwy8Ke4kgobUVaDD/kv8FJg6r22KoiDUKEMDDRUjZ SptSBE7q2je4pV+bem2c+rDxyl0SledUst148O/rYzceM4CRn6s2ubBaHz7WlEza g5pu2uMkh0QjHL75hWrzGmz92ugdMwoqZwJqbgDlLK4pHEsdlpI2xlHETnZwJLFA 9DWnFPkp =xtF5 -----END PGP SIGNATURE----- --pf9I7BMVVzbSWLtt--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Received: (at 41702) by debbugs.gnu.org; 6 Jul 2020 08:50:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 06 04:50:01 2020 Received: from localhost ([127.0.0.1]:33588 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jsMp3-0005yl-9v for submit <at> debbugs.gnu.org; Mon, 06 Jul 2020 04:50:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52492) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1jsMox-0005yP-RW for 41702 <at> debbugs.gnu.org; Mon, 06 Jul 2020 04:49:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50422) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1jsMos-0001Do-G5; Mon, 06 Jul 2020 04:49:46 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50332 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1jsMor-0001GE-VN; Mon, 06 Jul 2020 04:49:46 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Lars-Dominik Braun <ldb@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN> <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 06 Jul 2020 10:49:44 +0200 In-Reply-To: <20200702120333.GB4532@zpidnp36> (Lars-Dominik Braun's message of "Thu, 2 Jul 2020 14:03:33 +0200") Message-ID: <87k0zhkoyf.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41702 Cc: 41702 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Lars-Dominik Braun <ldb@HIDDEN> skribis: > and the attached patch uses the new nodelay option, if we don=E2=80=99t w= ant to wait > for another guile-ssh release. I=E2=80=99d rather wait; perhaps you can ask Artyom whether they=E2=80=99re= planning for a new release soonish? If there=E2=80=99s no plan for a release within two weeks, we can go ahead = with the patch you propose. > From 2892f79f819dd2dd9420f7e74bcb6e293d377452 Mon Sep 17 00:00:00 2001 > From: Lars-Dominik Braun <ldb@HIDDEN> > Date: Thu, 2 Jul 2020 13:59:51 +0200 > Subject: [PATCH] guix: Add nodelay to SSH store connection > > * gnu/packages/ssh.scm (guile-ssh)[patches]: Add patch. > * gnu/packages/patches/guile-ssh-0.12.0-nodelay.patch: New file. > * gnu/local.mk (dist_patch_DATA): Add it. > * guix/ssh.scm (open-ssh-session): Use new nodelay option. If we take this route, the two patches (guile-ssh changes, then (guix ssh) changes) should be separate. Also #:nodelay will cause a hard failure for people using an older Guile-SSH. I wonder how to guard against that; configure check, or run-time Guile-SSH version check? > +++ b/gnu/packages/patches/guile-ssh-0.12.0-nodelay.patch It would be good to include the URL of the upstream issue Thanks! Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 2 Jul 2020 12:03:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 02 08:03:51 2020
Received: from localhost ([127.0.0.1]:54500 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jqxwQ-0005CF-SK
for submit <at> debbugs.gnu.org; Thu, 02 Jul 2020 08:03:51 -0400
Received: from mail-wr1-f42.google.com ([209.85.221.42]:36967)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ldb@HIDDEN>) id 1jqxwK-0005Bw-LT
for 41702 <at> debbugs.gnu.org; Thu, 02 Jul 2020 08:03:45 -0400
Received: by mail-wr1-f42.google.com with SMTP id a6so27917088wrm.4
for <41702 <at> debbugs.gnu.org>; Thu, 02 Jul 2020 05:03:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=K/5na8FuwM55UNbi/xMntH2GcPMJVqqa7qFMjXMAHy4=;
b=P/x41HSieEdYQmwksqY5uKBFIg8+P7GLsGcPQBV3f1/vYGs5fNBhKgl5Fhpz9JoZjn
wFgVxnCxqcqvFJUI08BJlu9C0zOIU+k+PAs5aHsLOi7uMPiEhRrhnk09DzxR1nuKMqWT
3PzDsgilDTsXu0vjcAEx2XowtWIIJOU2oUD6Jb8J4zug756qt6mNzRpp5ASnbHxZogTD
Qb8IagcPF2AWImFI3MphwEQa9PkWZupfeaeVjmHZcA19CDmcrKroz/MzGneqpgbZL6ez
E5dUyrJjmM682mGkPRpD90Mjnt9gZjzGWOgWrtxd70q4lRKy38R5FTXyzRBP8g6vvclC
AETw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:cc:subject:message-id:references
:mime-version:content-disposition:in-reply-to:user-agent;
bh=K/5na8FuwM55UNbi/xMntH2GcPMJVqqa7qFMjXMAHy4=;
b=fenRobRaabPRHEWdvuL87YzLdAG/sP+ecOYD4o+bYfne5pJ6qC2gOj98Er49Q0xxx/
WY3sUgAypWArLUQh6SqQK2kafjRbjBDSgwtr+oSGB47khmPr88keHcDFiMth/wvLQKpE
fyDc7LW4jqGgWqb6v6ICGUbK68/sxWdqcEDk9dMTBc0TOw+sdp49byNCLPW55cniJfuI
Gly1Ut1gRafctyUJHOQsgCXVNQX89QiGgt+WDPvG3Rl5an2tHX2f+b3TOZvD5Kkt8t7+
H35E+M15i4O6tJjgW9DwKSiL7yR6YlAZ3+uhh89t++qcg+RKVMRC2GNvvqHmUeRZW5vP
j9qQ==
X-Gm-Message-State: AOAM532C40GZe+YleK4bfKE4mxy8xfIeqxAm+aBh+bJcFOfFcvidj7O5
+5Cnn8UBJeEHWyIV8HG1+2E3uZttYntIHchVPRkNCbZo7TmHIbGjRJgsQwtQKJmAexK7JoY563/
l4jJkvvOD4bIdTlzc969qp1IE3ZINZRdpTqQgfNWOSQr8vqh5CXSUpILYA3O6vrYViXnbKF20Bh
/m1vY=
X-Google-Smtp-Source: ABdhPJyNW3HtLj8dKo1cK4GDDYqFqxC9qO7ekQOpJWwLPsfQAPSVF1vi4XuWjUMExfbWIaZFSdiQqg==
X-Received: by 2002:adf:dd83:: with SMTP id x3mr33969406wrl.292.1593691414730;
Thu, 02 Jul 2020 05:03:34 -0700 (PDT)
Received: from localhost
(dynamic-2a01-0c23-8021-ca00-a9a3-ebbc-f46e-d012.c23.pool.telefonica.de.
[2a01:c23:8021:ca00:a9a3:ebbc:f46e:d012])
by smtp.gmail.com with ESMTPSA id z10sm5408231wrm.21.2020.07.02.05.03.33
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Thu, 02 Jul 2020 05:03:33 -0700 (PDT)
Date: Thu, 2 Jul 2020 14:03:33 +0200
From: Lars-Dominik Braun <ldb@HIDDEN>
To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
Message-ID: <20200702120333.GB4532@zpidnp36>
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
<20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN>
<20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN>
<87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36>
<87eepuyk99.fsf@HIDDEN> <20200702070000.GA4532@zpidnp36>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="sHrvAb52M6C8blB9"
Content-Disposition: inline
In-Reply-To: <20200702070000.GA4532@zpidnp36>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--sHrvAb52M6C8blB9
Content-Type: multipart/mixed; boundary="UHN/qo2QbUvPLonB"
Content-Disposition: inline
--UHN/qo2QbUvPLonB
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi (again),
and the attached patch uses the new nodelay option, if we don=E2=80=99t wan=
t to wait
for another guile-ssh release.
Lars
--UHN/qo2QbUvPLonB
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment; filename="0001-guix-Add-nodelay-to-SSH-store-connection.patch"
Content-Transfer-Encoding: quoted-printable
=46rom 2892f79f819dd2dd9420f7e74bcb6e293d377452 Mon Sep 17 00:00:00 2001
=46rom: Lars-Dominik Braun <ldb@HIDDEN>
Date: Thu, 2 Jul 2020 13:59:51 +0200
Subject: [PATCH] guix: Add nodelay to SSH store connection
* gnu/packages/ssh.scm (guile-ssh)[patches]: Add patch.
* gnu/packages/patches/guile-ssh-0.12.0-nodelay.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* guix/ssh.scm (open-ssh-session): Use new nodelay option.
---
gnu/local.mk | 1 +
.../patches/guile-ssh-0.12.0-nodelay.patch | 81 +++++++++++++++++++
gnu/packages/ssh.scm | 1 +
guix/ssh.scm | 4 +-
4 files changed, 86 insertions(+), 1 deletion(-)
create mode 100644 gnu/packages/patches/guile-ssh-0.12.0-nodelay.patch
diff --git a/gnu/local.mk b/gnu/local.mk
index 8bb56010c2..26725cbca1 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1068,6 +1068,7 @@ dist_patch_DATA =3D \
%D%/packages/patches/guile-present-coding.patch \
%D%/packages/patches/guile-rsvg-pkgconfig.patch \
%D%/packages/patches/guile-emacs-fix-configure.patch \
+ %D%/packages/patches/guile-ssh-0.12.0-nodelay.patch \
%D%/packages/patches/gtk2-respect-GUIX_GTK2_PATH.patch \
%D%/packages/patches/gtk2-respect-GUIX_GTK2_IM_MODULE_FILE.patch \
%D%/packages/patches/gtk2-theme-paths.patch \
diff --git a/gnu/packages/patches/guile-ssh-0.12.0-nodelay.patch b/gnu/pack=
ages/patches/guile-ssh-0.12.0-nodelay.patch
new file mode 100644
index 0000000000..bc649b7469
--- /dev/null
+++ b/gnu/packages/patches/guile-ssh-0.12.0-nodelay.patch
@@ -0,0 +1,81 @@
+commit f697c603ef196eb88c1d7d629e456bf5054e35a4
+Author: Lars-Dominik Braun <ldb@HIDDEN>
+Date: Thu Jul 2 13:26:21 2020 +0200
+
+ modules/ssh/session.scm: Add nodelay to make-session
+ =20
+ * modules/ssh/session.scm (make-session): Add key nodelay
+
+diff --git a/modules/ssh/session.scm b/modules/ssh/session.scm
+index d7c0831..042bd5f 100644
+--- a/modules/ssh/session.scm
++++ b/modules/ssh/session.scm
+@@ -74,7 +74,7 @@
+ knownhosts timeout timeout-usec ssh1 ssh2 log-verb=
osity
+ ciphers-c-s ciphers-s-c compression-c-s compressio=
n-s-c
+ proxycommand stricthostkeycheck compression
+- compression-level callbacks config)
++ compression-level nodelay callbacks config)
+ "Make a new SSH session with specified configuration.\n
+ Return a new SSH session."
+ (let ((session (%make-session)))
+@@ -98,6 +98,7 @@ Return a new SSH session."
+ (session-set-if-specified! stricthostkeycheck)
+ (session-set-if-specified! compression)
+ (session-set-if-specified! compression-level)
++ (session-set-if-specified! nodelay)
+ (session-set-if-specified! callbacks)
+=20
+ (when config
+
+commit b80121ae3e851a676f25fd2b7e33371b94a6e030
+Author: Lars-Dominik Braun <ldb@HIDDEN>
+Date: Thu Jul 2 08:52:13 2020 +0200
+
+ libguile-ssh/session-func.c: Add NODELAY option
+ =20
+ * libguile-ssh/session-func.c (session_options, set_option): Add libss=
h=E2=80=99s
+ nodelay option.
+ * tests/session.scm ("session-set!, valid values", "session-set!, inva=
lid
+ values"): Add unit-tests.
+
+diff --git a/libguile-ssh/session-func.c b/libguile-ssh/session-func.c
+index 48db779..9240734 100644
+--- a/libguile-ssh/session-func.c
++++ b/libguile-ssh/session-func.c
+@@ -71,6 +71,7 @@ static struct symbol_mapping session_options[] =3D {
+ { "stricthostkeycheck", SSH_OPTIONS_STRICTHOSTKEYCHECK },
+ { "compression", SSH_OPTIONS_COMPRESSION },
+ { "compression-level", SSH_OPTIONS_COMPRESSION_LEVEL },
++ { "nodelay", SSH_OPTIONS_NODELAY },
+ { "callbacks", GSSH_OPTIONS_CALLBACKS },
+ { NULL, -1 }
+ };
+@@ -366,6 +367,7 @@ set_option (SCM scm_session, struct session_data* sd, =
int type, SCM value)
+ case SSH_OPTIONS_SSH1:
+ case SSH_OPTIONS_SSH2:
+ case SSH_OPTIONS_STRICTHOSTKEYCHECK:
++ case SSH_OPTIONS_NODELAY:
+ return set_bool_opt (session, type, value);
+=20
+ case SSH_OPTIONS_FD:
+diff --git a/tests/session.scm b/tests/session.scm
+index 2eb1df7..83a864d 100644
+--- a/tests/session.scm
++++ b/tests/session.scm
+@@ -70,6 +70,7 @@
+ nolog)
+ (compression "yes" "no")
+ (compression-level 1 2 3 4 5 6 7 8 9)
++ (nodelay #f #t)
+ (callbacks ((user-data . "hello")
+ (global-request-callback . ,(const #f)=
)))))
+ (res #t))
+@@ -95,6 +96,7 @@
+ (log-verbosity "string" -1 0 1 2 3 4 5)
+ (compression 12345)
+ (compression-level -1 0 10)
++ (nodelay 12345 "string")
+ (callbacks "not a list"
+ ((global-request-callback . #f)))))
+ (res #t))
diff --git a/gnu/packages/ssh.scm b/gnu/packages/ssh.scm
index ea42e72de2..0d9fa5fd54 100644
--- a/gnu/packages/ssh.scm
+++ b/gnu/packages/ssh.scm
@@ -259,6 +259,7 @@ Additionally, various channel-specific options can be n=
egotiated.")
(sha256
(base32
"054hd9rzfhb48gc1hw3rphhp0cnnd4bs5qmidy5ygsyvy9ravlad"))
+ (patches (search-patches "guile-ssh-0.12.0-nodelay.patch"))
(modules '((guix build utils)))))
(build-system gnu-build-system)
(outputs '("out" "debug"))
diff --git a/guix/ssh.scm b/guix/ssh.scm
index 2d7ca7d01d..df7a9bed4a 100644
--- a/guix/ssh.scm
+++ b/guix/ssh.scm
@@ -129,7 +129,9 @@ Throw an error on failure."
;; We need lightweight compression when
;; exchanging full archives.
#:compression compression
- #:compression-level 3)))
+ #:compression-level 3
+
+ #:nodelay #t)))
=20
;; Honor ~/.ssh/config.
(session-parse-config! session)
--=20
2.20.1
--UHN/qo2QbUvPLonB--
--sHrvAb52M6C8blB9
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl79zQ8ACgkQQhN3ARo3
hEbEKAwAuHGzi/EBcAbji98iXuwjFX5pKKTvVGnb+EusvCFDwgQzOfKUBnFmDcb3
+C3R0/Y4rsDuvMfmpeNSdiZB4GPxiX7bOJ+uyzJLUDi/1CPvyGDsAHvF0oAGF+vp
+316dHsPryS8mYUgkYOb9aLzbzTaCm/qikohd19S8jkpcb9rbxvLVxlvEsAz7s22
fxDfuYhJOGxprK3XWRK0xga8aG58x48O8uHSh93TXkLr77829Mu4zoKKzfA6Dds6
ChJWpOEbGGBnl2txafXulLTDpywbGs6Ir4kDvfYVBENAyOXsws/VVtZwfNWbgZ6B
PuSDP9dXaOa0hRU9hcpzNma9h3qwYEvad1TG62V9la6aBtO3/f6P49uXUW9Azo3q
3zj/npLqKcrn7fr+vk29p62KUqze5rvJ80IYbPEyhWtOhMD3CS5dkRiW2Q8igKGS
DxVVz0R/AFgFPJc/RTpF1KvyPBvSeQgv48OamXBOYEF5C18qZO/yOE0lCmO4JVwi
vrjtLqdk
=tkfu
-----END PGP SIGNATURE-----
--sHrvAb52M6C8blB9--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Received: (at 41702) by debbugs.gnu.org; 2 Jul 2020 07:00:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 02 03:00:12 2020 Received: from localhost ([127.0.0.1]:54215 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jqtCe-0003tw-Dr for submit <at> debbugs.gnu.org; Thu, 02 Jul 2020 03:00:12 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:53782) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ldb@HIDDEN>) id 1jqtCa-0003sg-MI for 41702 <at> debbugs.gnu.org; Thu, 02 Jul 2020 03:00:11 -0400 Received: by mail-wm1-f54.google.com with SMTP id j18so25468919wmi.3 for <41702 <at> debbugs.gnu.org>; Thu, 02 Jul 2020 00:00:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=UPGPJnLlu8Wg40azJu+SWRRx/4W3UdotZrOsAJJ+GAY=; b=Vbd4paqSr8yrwwOG0Y4yNknfy2NO7fdqaZD5mUhCWUKq0h63+1Ii6gecxxMuLRGcEF RGwlDxBYl1oX33wr0aTt9PkAsnOx1WYE3fI7ZyBt7IDlwh2bIy4A3ckTMGZJL5oBLZuh n1eQYgCScb9iPJIlZvkF3n7IarrnGlsTgDdwij27eKmd5NR63mknG5RyPmcu+tP6sBI7 IKvugF6jtIWE0j8+WYFITtDM+0wrSMlr/ddC+IN4XyuAfNXmh4vBp2n0/Tei7iUL9XF9 +SenXYuYY2FIdnNZbWp0pC3yH0D02SP+jx0Ayfyr12e2OffYj/G7Cv3UERg3cX86c67H 5Zkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=UPGPJnLlu8Wg40azJu+SWRRx/4W3UdotZrOsAJJ+GAY=; b=ApQZZ920F2IW/SJv9nrCpabno1xNTKY8iUWAysx7heIJM6tzeMnpHvDYAK5uJTkeS/ bBOljK+qt3mr4aYslK8n5Cg2r9rJqm/r3yvwkXN9Uc28Cf5AS0A/mxT/29n62d739YQh N8mc368TVB6K3qVgAmczxbt2ZdIr86C5kokyVDh64gMDbgUxu4R9u0nj/hTOm1Bw0EtD 8xhReq7VvelSdLrDXH9Tu+szV25WiI2Jp3rBNRk1TNTT0KkXxG/tK6hNMmScSbO29Rkh CyvTeFoxVsDO/8H9ZsP0A1G1XGtWQs1djdBmXp+YzA8L9iPQB4y0MMrHml9Ii9u5INd5 FCfg== X-Gm-Message-State: AOAM530BZ7B/ghHvjYXl65dTeT0+zODXJg8BmRAovuZn34NiNjBNg00A K3AyT0KFQ20YkSzGB/IyIGbGFu02qFIIdgc+qx4N20yhfTuSVM+Lj3Yy2d43GIoxeFK2QsfiRTN CagMT56qEttzL+/AqUolGFUFFWlu+vngli8gWNF0m1YVLUkzWw7RBJZF9QVL1AVSKPaszlISaGN lB1wbWlXTy X-Google-Smtp-Source: ABdhPJz6IHR9NRQW7e0m7WR6jrWiCvS3DaHEUIFjooeseTjyHAqLbOAxqNCn1jFe8JH0k52hAV3Omg== X-Received: by 2002:a1c:f702:: with SMTP id v2mr29306670wmh.71.1593673202318; Thu, 02 Jul 2020 00:00:02 -0700 (PDT) Received: from localhost (dynamic-2a01-0c23-8021-ca00-a9a3-ebbc-f46e-d012.c23.pool.telefonica.de. [2a01:c23:8021:ca00:a9a3:ebbc:f46e:d012]) by smtp.gmail.com with ESMTPSA id 12sm9346872wmg.6.2020.07.02.00.00.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jul 2020 00:00:01 -0700 (PDT) Date: Thu, 2 Jul 2020 09:00:00 +0200 From: Lars-Dominik Braun <ldb@HIDDEN> To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues Message-ID: <20200702070000.GA4532@zpidnp36> References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN> <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="envbJBWh7q8WU6mo" Content-Disposition: inline In-Reply-To: <87eepuyk99.fsf@HIDDEN> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41702 Cc: 41702 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --envbJBWh7q8WU6mo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, > Would you like to send them a patch? done: https://github.com/artyom-poptsov/guile-ssh/pull/21 Cheers, Lars --envbJBWh7q8WU6mo Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl79hewACgkQQhN3ARo3 hEakxwwA1QUBS0VlTTNH5L49/5dGCmvkc4pTJRv9BtneXSMJTUryIIpSEv/mPYEe yn5Lr02uyZPTqlp75sOndJmhuRnSCz6F9HTHXgTAls7fsU3Ka0kiUO4WHhM8gOH4 sfyYUEq654ZuYU54sjAE3x/aMCfhxECXPmGECiz6/R5J0BcaFvr5h+lTqjWVtg6g KFKPc8C/UQb4DxLjJaEC5KPzA70T/t6zoFk4ZYTa4bPArEHbiqfvbXNpifee/8np e8KD6IL4rBEepblPdAAtdJ+x0EuFm40Ac4fGJv9b+QIe+d2go2Ixz5m2Y5x6U/Hr kn9v8ZNKFUZukQj23RY6mke0Nek+WKgTAvV2FfYZnJ2OEhhJDW7oO4U4jxbOaomc 26WTErFEOvVls/ZXPIhBf4dsqcMUNmo6w2l59LnMNeWQkD19TE7LsOUGMQ6+aWQo yBf4sl1wihnG1+ljSmf088COqNtsKNHEBxWYL317BoqIiD0Rk3V6OwDjrIuis7J9 Ou2EYXVE =jsIk -----END PGP SIGNATURE----- --envbJBWh7q8WU6mo--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Received: (at 41702) by debbugs.gnu.org; 1 Jul 2020 21:54:04 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 01 17:54:04 2020 Received: from localhost ([127.0.0.1]:53856 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jqkg8-0005NC-Mc for submit <at> debbugs.gnu.org; Wed, 01 Jul 2020 17:54:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34336) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1jqkg6-0005Mi-FR for 41702 <at> debbugs.gnu.org; Wed, 01 Jul 2020 17:54:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47296) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1jqkg1-00006e-4B; Wed, 01 Jul 2020 17:53:57 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51352 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1jqkg0-0000av-4u; Wed, 01 Jul 2020 17:53:56 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Lars-Dominik Braun <ldb@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN> <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN> <20200701112453.GB25179@zpidnp36> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 14 Messidor an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 01 Jul 2020 23:53:54 +0200 In-Reply-To: <20200701112453.GB25179@zpidnp36> (Lars-Dominik Braun's message of "Wed, 1 Jul 2020 13:24:53 +0200") Message-ID: <87eepuyk99.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41702 Cc: 41702 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Lars-Dominik Braun <ldb@HIDDEN> skribis: >> What we need is Guile-SSH bindings for =E2=80=98ssh_get_fd=E2=80=99, whi= ch would allow >> us to get at the actual file descriptor after the connection has been >> opened, and to set TCP_NODELAY there: >>=20 >> http://api.libssh.org/stable/group__libssh__session.html#gafe509fcea47= 714b5cd277d1e35e83276 >>=20 >> It=E2=80=99s not 100% clear though if it returns that =E2=80=9Cright=E2= =80=9D FD when there=E2=80=99s a >> proxy command. > wouldn=E2=80=99t it be easier to add the SSH_OPTIONS_NODELAY option (see = [1]) to > guile-ssh (should be trivial[2]) and let guile-ssh handle all the connect= ion > stuff without touching the socket? Or do we need the raw socket elsewhere? Good idea, I had overlooked that possibility. Would you like to send them a patch? Thanks, Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Received: (at 41702) by debbugs.gnu.org; 1 Jul 2020 11:25:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 01 07:25:03 2020 Received: from localhost ([127.0.0.1]:52425 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jqarP-0004g2-GK for submit <at> debbugs.gnu.org; Wed, 01 Jul 2020 07:25:03 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:50795) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ldb@HIDDEN>) id 1jqarN-0004fS-4x for 41702 <at> debbugs.gnu.org; Wed, 01 Jul 2020 07:25:01 -0400 Received: by mail-wm1-f54.google.com with SMTP id l17so22006312wmj.0 for <41702 <at> debbugs.gnu.org>; Wed, 01 Jul 2020 04:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=dc8mwMnMK4IzsAhMmGxX0tiYzDPS0eRIX6zH8ljOznw=; b=A7g5Tvh7NXH3dMSb2m6Up7sKMGhO+iHWlSeUf0VFkRvoDjXBp47QTBDk1fYuaXiY7i byWoFMYYbo2St26u9O/So0Bc2NM/W+SnyC2zxd3eULiyhXtwgEWyKepIwW9g80EDfQa0 D7wB5d/ANL1fOlbc5EMj5gCpoSzCnB4rHZiLC+gvlXUyNxQ+MDLlfrGwhJsp0hJLaBT+ YumPTklP0fo1lh4UoV7iMn29zVSDEccGNMOln67QgBRBihFObvKa8CRoGJNSKXAHeEBL HVfFMHNwe1w8szT3NeReSRtRTKpl5PIYwsVr2FsxQXgY0PZWXEmphfxm1iEMvtR7j68i EctQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=dc8mwMnMK4IzsAhMmGxX0tiYzDPS0eRIX6zH8ljOznw=; b=b3Rv2FNwQktWB03qMT86zas7oq9TpT8zEIMOK/dZA4HB39IvRuPHXVMZ550+aGqJEz UJCOY0z0QUW+UfGcX5jncdrpT0YUcqAH+QDaZSwixk7KitVMk9EK0JKHDTEbqv6PCn+a o+qhTOXbnw85FlHKSJjtzbo5dLYzO5fWTAWzpbT650BhazNRBMITUMDkYQg69rPeGPwt FyfK6pDUgEB+kmzS/LeBeBiqwH7ZTQkYpIoR1pIEaII+BsP6Vk/zYiYp36Nu3KMyMdag xOKy8w2Frlzu0+PkLjDtbnWhV3oD1jarjjEZijZmAmuS7nHRMuIc3Xu0acL+NSxpVwbV De3g== X-Gm-Message-State: AOAM5308Q2nxgmhx3E4vSTibri9kh9GClA1B2jmwJG4j641CWOctlemV RTs51TtNzZV/iF6vZATOPg8CVl4t68rxqc/mNZEI8zyza0zStC2CLBBA7oUTQnLzfXs//EdAM1+ l0aVkphIqVfUubhOCy7yVBhKHLYHt6NivpmG4IttsCk6s4hlNbcK2mcEhDRvyBWZWMTzrT8og5h h2H0dQRhpy X-Google-Smtp-Source: ABdhPJyx82S2kD7BMAac4GSyrHqUxE0ON6lRHbtR1f1KE1nRaLG8ReHLaUayRrQ8dIsCsVFtg34uoA== X-Received: by 2002:a1c:f608:: with SMTP id w8mr25414929wmc.78.1593602695080; Wed, 01 Jul 2020 04:24:55 -0700 (PDT) Received: from localhost (dynamic-2a01-0c22-d42f-e700-35e4-ba94-86d1-9650.c22.pool.telefonica.de. [2a01:c22:d42f:e700:35e4:ba94:86d1:9650]) by smtp.gmail.com with ESMTPSA id j41sm7320271wre.12.2020.07.01.04.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jul 2020 04:24:54 -0700 (PDT) Date: Wed, 1 Jul 2020 13:24:53 +0200 From: Lars-Dominik Braun <ldb@HIDDEN> To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN> Subject: Re: bug#41702: `guix environment` performance issues Message-ID: <20200701112453.GB25179@zpidnp36> References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN> <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN> <20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN> <87y2o3zf00.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6sX45UoQRIJXqkqR" Content-Disposition: inline In-Reply-To: <87y2o3zf00.fsf@HIDDEN> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41702 Cc: 41702 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --6sX45UoQRIJXqkqR Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey, > What we need is Guile-SSH bindings for =E2=80=98ssh_get_fd=E2=80=99, whic= h would allow > us to get at the actual file descriptor after the connection has been > opened, and to set TCP_NODELAY there: >=20 > http://api.libssh.org/stable/group__libssh__session.html#gafe509fcea477= 14b5cd277d1e35e83276 >=20 > It=E2=80=99s not 100% clear though if it returns that =E2=80=9Cright=E2= =80=9D FD when there=E2=80=99s a > proxy command. wouldn=E2=80=99t it be easier to add the SSH_OPTIONS_NODELAY option (see [1= ]) to guile-ssh (should be trivial[2]) and let guile-ssh handle all the connection stuff without touching the socket? Or do we need the raw socket elsewhere? Cheers, Lars [1] http://api.libssh.org/stable/group__libssh__session.html#ga7a801b85800b= aa3f4e16f5b47db0a73d [2] https://github.com/artyom-poptsov/guile-ssh/blob/master/libguile-ssh/se= ssion-func.c#L52 --6sX45UoQRIJXqkqR Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl78coIACgkQQhN3ARo3 hEZJEQv9GJYnyKD23RlY9Et60BMB2YsJSjSqWxxjUneLfLnFDrIvusQZf+Lhtj3V o7aBVzoJ6GEPrSOTi2FcecSUGfuSaC1w17Zs2prjMuqq66Opkx9caeBS/dv/4iLZ HFf+Kci8+gaWQkn4BR7+tspfkjRxIulywpK4P7nP8KxUDhVDKXi6NLSJB5XCH4bg qwNSzeQnu9L0CDzlZPHq5h5bZ7Op4VyZ/oMa1koRYBBVA0KCDGHZUQGs/Mh7QaYe 5N5AMRFabXWP4hN0qDw3H7Dic/+4jYi+2QMicx1Ysh+6bVB0R666/Qwg0PGZuHvX sTwfcuHVHtHLubXEKCnzh7P5WkhzC8NpRL5iTaaadSQL/mrYe3y7ihPRtthfMkVD zv53GVPXUnxWUc1JcxMO+9OrHVdQY9HPiNbrFmJ+FjEFGuqvmpAoA8K54aolIepf D9dd9tC21M/yYcI7RCgJ5lSGbbWkHM5fz/P1b9UmKIbyQkk8WvqSt1N+IQQZM6DN QbXGmtBu =COgO -----END PGP SIGNATURE----- --6sX45UoQRIJXqkqR--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 1 Jul 2020 10:50:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 01 06:50:03 2020
Received: from localhost ([127.0.0.1]:52358 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jqaJW-0003iI-Pm
for submit <at> debbugs.gnu.org; Wed, 01 Jul 2020 06:50:02 -0400
Received: from eggs.gnu.org ([209.51.188.92]:57450)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1jqaJU-0003hk-Kz
for 41702 <at> debbugs.gnu.org; Wed, 01 Jul 2020 06:50:01 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:37461)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <ludo@HIDDEN>)
id 1jqaJP-0001mO-9I; Wed, 01 Jul 2020 06:49:55 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=53866 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1jqaJO-0008B6-JD; Wed, 01 Jul 2020 06:49:54 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Lars-Dominik Braun <ldb@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
<20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN>
<20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN>
Date: Wed, 01 Jul 2020 12:49:51 +0200
In-Reply-To: <87wo3sjjdw.fsf@HIDDEN> ("Ludovic
\=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
\=\?utf-8\?Q\?s\?\= message of "Sat, 27 Jun 2020 23:20:27 +0200")
Message-ID: <87y2o3zf00.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi,
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> + (when open-connection
> + (let* ((sock (open-connection host port)))
> + ;; Since 'session-set!' doesn't increase the revealed count of S=
OCK
> + ;; (as of Guile-SSH 0.12.0), do it ourselves.
> + (set-port-revealed! sock 1)
> + (session-set! session 'fd sock)))
The problem with this approach is that =E2=80=98open-connection=E2=80=99 is=
past the raw
host name; aliases, proxy commands, etc. from ~/.ssh/config are ignored.
So it cannot go in as is. :-/
What we need is Guile-SSH bindings for =E2=80=98ssh_get_fd=E2=80=99, which =
would allow
us to get at the actual file descriptor after the connection has been
opened, and to set TCP_NODELAY there:
http://api.libssh.org/stable/group__libssh__session.html#gafe509fcea47714=
b5cd277d1e35e83276
It=E2=80=99s not 100% clear though if it returns that =E2=80=9Cright=E2=80=
=9D FD when there=E2=80=99s a
proxy command.
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 30 Jun 2020 09:59:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 30 05:59:45 2020
Received: from localhost ([127.0.0.1]:50186 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jqD3J-0002NS-HZ
for submit <at> debbugs.gnu.org; Tue, 30 Jun 2020 05:59:45 -0400
Received: from mail-wm1-f48.google.com ([209.85.128.48]:35284)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ldb@HIDDEN>) id 1jqD3I-0002NG-45
for 41702 <at> debbugs.gnu.org; Tue, 30 Jun 2020 05:59:44 -0400
Received: by mail-wm1-f48.google.com with SMTP id l2so17495617wmf.0
for <41702 <at> debbugs.gnu.org>; Tue, 30 Jun 2020 02:59:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=33WJBfRDbKZpIgB/k/tjkyLp2VSOdL2CPuRmRNSBCcM=;
b=U8s3l679tv8tdFavRpMhId3CdDxKRrx1a/mb/NaY1f7SQr6PqmWbBIQ0V3ibr89Cbs
IOqeFcKSkfi22X/0U5oTPhSJ07ZslrDsV9aEsI5KP53omf+HMx8n6iN7926poZ7m5DP3
C4FdA2n1FaefmsiGKwuxbpe9bPhgjUtvKeUceG3guDZhLfWudqOgqcSQ9MVnXoymzlmW
CQ370T0aavNBc7mg8eVHSmMTQZRUrMjYVGWQ9sjGyT+v/VFze2GtYYdPfzC4vq+AkFjh
TWWzl5sMil2I+JvUpFjyeDKKIDA+53ep6YmNdw4XQeeoMLfgMdyOxmHUDiG6FvI+PYoP
TN7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:cc:subject:message-id:references
:mime-version:content-disposition:in-reply-to:user-agent;
bh=33WJBfRDbKZpIgB/k/tjkyLp2VSOdL2CPuRmRNSBCcM=;
b=Wa46A+0OP3uv2iqaUvoGB9+Na3aAfzdGgDBHS3RPQc3g0+kYWqUovYQ25cNdOys+4Y
Sn0DtKLZd28tQTG6HvZbZuaSxuYHEl6+ZaqJqkHz2TJbsc7e9aN9tS/VJDNuTMGY/450
XW8vTdV8RvwHNhNu+DyvdKvcSv579aE7AgYGvNN/TpUAtjySNpu7TBZi1diPxTK6e93o
/uYOlrw5fOM/Zkui1dOruJpkJ85+5160jjYG73ynmRsEH9GmyAX/EgLqqcjTbggW2wdb
R4yB1Y0pBXlp7Cp5b1am0/e6DCat4RyE/U3kGREnq7ik0ctjXkFEoxvAnODNxcPS9aQM
YveA==
X-Gm-Message-State: AOAM530R23XAEBYSDG2u9HTt9Rwsc3eMfQACZhO/f8Ps3Z0AxfTz9rIV
wRejwnj4PSGKkOqaJsNrfI/fhojjeqIosTPBLZ3vF1ESv0vOWZZqklo0Bo+po+3Q/LN1kHeE1OI
i390RrXOAapTJgxSaR3CxH+bYdBA3MQLbh5RaD+GC6LAZixSJhvS7fhDq27EkJSyoqzMb4SJ2tv
2r+fE=
X-Google-Smtp-Source: ABdhPJzlsXk2tczt+s0x51fhmd+pOVknWGrnBGjgKOGTcEjQ1v46yGf8k48TfyqGwg6ut7OmJh41uA==
X-Received: by 2002:a7b:c348:: with SMTP id l8mr22449071wmj.54.1593511178007;
Tue, 30 Jun 2020 02:59:38 -0700 (PDT)
Received: from localhost
(dynamic-2a01-0c23-8458-b600-ad60-edb5-3902-c635.c23.pool.telefonica.de.
[2a01:c23:8458:b600:ad60:edb5:3902:c635])
by smtp.gmail.com with ESMTPSA id i67sm877690wma.12.2020.06.30.02.59.37
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Tue, 30 Jun 2020 02:59:37 -0700 (PDT)
Date: Tue, 30 Jun 2020 11:59:36 +0200
From: Lars-Dominik Braun <ldb@HIDDEN>
To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
Message-ID: <20200630095936.GA3630@zpidnp36>
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
<20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN>
<20200609091522.GB8597@zpidnp36> <87wo3sjjdw.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="mP3DRpeJDSE+ciuQ"
Content-Disposition: inline
In-Reply-To: <87wo3sjjdw.fsf@HIDDEN>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--mP3DRpeJDSE+ciuQ
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi Ludo,
> The patch below is a noticeable improvement for me. On my laptop,
>=20
> GUIX_DAEMON_SOCKET=3Dssh://localhost ./pre-inst-env guix build libreoff=
ice -n=20
>=20
> goes from 5.8s to 3.3s. It just does the same thing as we do for
> guix://.
>=20
> Could you check what results it gives you?
looking pretty good, comparing master without/with patch:
---snip---
N Min Max Median Avg Stddev
x 10 16.87 20.77 17.44 17.802 1.1228515
+ 10 9.92 13.67 10.19 10.569 1.1254574
Difference at 99.0% confidence
-7.233 +/- 1.44688
-40.6303% +/- 6.68735%
---snap---
> There=E2=80=99s a project to rewrite the daemon in Scheme, started by Cal=
eb
> Ristvedt (reepa) as part of GSoC a few years ago. This could be an
> opportunity to add a new version of the protocol that would support
> pipelining.
Given how much improvement you=E2=80=99ve achieved so far I=E2=80=99d say w=
e can (for now)
probably get away with the current protocol. Still, what=E2=80=99s the stat=
us of this
daemon?
Lars
--mP3DRpeJDSE+ciuQ
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl77DQMACgkQQhN3ARo3
hEZuWAwAs5btXPUr3WH4g/FqdVHSqbn6QRe6Lo8WPqKZq2IHKpk+HR+VWwRT+uyu
o9S41JWMk00cbFr9I4xi2xyyld1J3NSDDOrGPP8sjxz7Z/y/h13Dh2vV3lHborEx
2ItAxK9+nGThJaFdFFdVOTd/1DZb7nLnaoQ6jrhVaQ2kMD3XmXd2dkNu/vEd8tat
DV2li5sWznnvmyglyGA7gRyPVdJITy8lhRPeOn1MVNWVRxAgHZMmB/pUKcL+7KB1
msmUuylIa0suKoFwqSOh9Mw6t8tGIaNEPaaJtjWKu5ew9tg0vnEsRPJU3cunuDtJ
RS/7tWk3nU+0wXCMErUgdfb9Tm3yebTopHCuyIFhC+z9M5X1owPmUNQdk5hdgbxE
+972C/fzlVABPtj2UEhfV9E48gEuON66NJOjdQBeGH0qohAadTQZmy4ln1ZMkYdw
+cKeLzWKj/w07oeZQGO9IE38QO26xHMye0Dx219gmQ/ar0pFgTA4QPs1bs8j5BM6
XvzW6T5G
=YULY
-----END PGP SIGNATURE-----
--mP3DRpeJDSE+ciuQ--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 27 Jun 2020 21:20:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 27 17:20:38 2020
Received: from localhost ([127.0.0.1]:44708 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jpIFZ-0005cz-TX
for submit <at> debbugs.gnu.org; Sat, 27 Jun 2020 17:20:38 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35264)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1jpIFX-0005cl-88
for 41702 <at> debbugs.gnu.org; Sat, 27 Jun 2020 17:20:36 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:33545)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <ludo@HIDDEN>)
id 1jpIFR-0008B6-Ss; Sat, 27 Jun 2020 17:20:29 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38494 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1jpIFR-0003Ps-Az; Sat, 27 Jun 2020 17:20:29 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Lars-Dominik Braun <ldb@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
<20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN>
<20200609091522.GB8597@zpidnp36>
Date: Sat, 27 Jun 2020 23:20:27 +0200
In-Reply-To: <20200609091522.GB8597@zpidnp36> (Lars-Dominik Braun's message of
"Tue, 9 Jun 2020 11:15:22 +0200")
Message-ID: <87wo3sjjdw.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,
Lars-Dominik Braun <ldb@HIDDEN> skribis:
> That would have been my best guess too, but it does not seem to be the bi=
ggest
> problem right now. Looking at the numbers again (both patches applied) wi=
th the
> attached manifest[1], I see that:
>
> ---snip---
> Local UNIX socket with and without --no-grafts:
> N Min Max Median Avg Stdd=
ev
> x 10 6.07 6.35 6.145 6.16 0.082327=
26
> + 10 17.47 17.89 17.545 17.602 0.143511=
52
> Difference at 99.0% confidence
> 11.442 +/- 0.150576
> 185.747% +/- 4.07133%
>
> Local UNIX socket vs. guix://localhost transport:
> N Min Max Median Avg Stdd=
ev
> x 10 17.47 17.89 17.545 17.602 0.143511=
52
> + 10 17.43 18.1 17.61 17.642 0.201317=
88
> No difference proven at 99.0% confidence
>
> Local UNIX socket vs ssh://localhost transport:
> N Min Max Median Avg Stdd=
ev
> x 10 17.47 17.89 17.545 17.602 0.143511=
52
> + 10 33.46 35.27 34.315 34.359 0.538732=
05
> Difference at 99.0% confidence
> 16.757 +/- 0.5074
> 95.1994% +/- 3.13957%
> ---snap---
>
> So I would conclude:
>
> 1) Grafting still takes a lot of time and needs more work
Yes, I noticed that there=E2=80=99s still redundant work being done, so we =
may
need global caching in addition to what
58bb833365db4e8934a386497d5b00a063cfd27d did.
> 2) Linux optimizes localhost networking pretty well
> 3) Our SSH transport is terribly slow
Yes.
The patch below is a noticeable improvement for me. On my laptop,
GUIX_DAEMON_SOCKET=3Dssh://localhost ./pre-inst-env guix build libreoffic=
e -n=20
goes from 5.8s to 3.3s. It just does the same thing as we do for
guix://.
Could you check what results it gives you?
> So, I guess it would make sense for me to look at the SSH transport itself
> again and see if there are any other low-hanging fruit. Not sure how much=
I can
> help with profiling guile/guix itself. A different/better RPC protocol is
> probably GSoC/v2.0-worthy?
There=E2=80=99s a project to rewrite the daemon in Scheme, started by Caleb
Ristvedt (reepa) as part of GSoC a few years ago. This could be an
opportunity to add a new version of the protocol that would support
pipelining.
Thanks,
Ludo=E2=80=99.
--=-=-=
Content-Type: text/x-patch; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
diff --git a/guix/ssh.scm b/guix/ssh.scm
index 2d7ca7d01d..a9312b7c8c 100644
--- a/guix/ssh.scm
+++ b/guix/ssh.scm
@@ -98,7 +98,7 @@ actual key does not match."
key type))))))))
=20
(define* (open-ssh-session host #:key user port identity
- host-key
+ host-key open-connection
(compression %compression)
(timeout 3600))
"Open an SSH session for HOST and return it. IDENTITY specifies the file
@@ -110,6 +110,10 @@ When HOST-KEY is true, it must be a string like \"ssh-=
ed25519 AAAAC3Nz=E2=80=A6
root@HIDDEN\"; the server is authenticated and an error is raised if =
its
host key is different from HOST-KEY.
=20
+When OPEN-CONNECTION is true, it must be a two-argument procedure; it is
+passed HOST and PORT and must return a socket (a file port). When
+OPEN-CONNECTION is false, Guile-SSH takes care of opening the connection.
+
Install TIMEOUT as the maximum time in seconds after which a read or write
operation on a channel of the returned session is considered as failing.
=20
@@ -134,6 +138,13 @@ Throw an error on failure."
;; Honor ~/.ssh/config.
(session-parse-config! session)
=20
+ (when open-connection
+ (let* ((sock (open-connection host port)))
+ ;; Since 'session-set!' doesn't increase the revealed count of SOCK
+ ;; (as of Guile-SSH 0.12.0), do it ourselves.
+ (set-port-revealed! sock 1)
+ (session-set! session 'fd sock)))
+
(match (connect! session)
('ok
(if host-key
diff --git a/guix/store/ssh.scm b/guix/store/ssh.scm
index 09c0832505..d07f0d7467 100644
--- a/guix/store/ssh.scm
+++ b/guix/store/ssh.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright =C2=A9 2017 Ludovic Court=C3=A8s <ludo@HIDDEN>
+;;; Copyright =C2=A9 2017, 2020 Ludovic Court=C3=A8s <ludo@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -19,6 +19,7 @@
(define-module (guix store ssh)
#:use-module (guix ssh)
#:use-module (web uri)
+ #:use-module (ssh session)
#:export (connect-to-daemon))
=20
;;; Commentary:
@@ -29,11 +30,27 @@
;;;
;;; End:
=20
+(define (open-connection host port)
+ "Open a connection to HOST and PORT. Use the standard SSH port if PORT =
is
+false."
+ (let* ((lst (getaddrinfo host
+ (if port (number->string port) "ssh")
+ (if port AI_NUMERICSERV 0)))
+ (addr (addrinfo:addr (car lst)))
+ (sock (socket (sockaddr:fam addr) SOCK_STREAM 0)))
+ ;; Setting this option makes a dramatic difference because it avoids t=
he
+ ;; "ACK delay" on our RPC messages.
+ (setsockopt sock IPPROTO_TCP TCP_NODELAY 1)
+
+ (connect sock addr)
+ sock))
+
(define (connect-to-daemon uri)
"Connect to the SSH daemon at URI, a URI object with the 'ssh' scheme."
(remote-daemon-channel
(open-ssh-session (uri-host uri)
#:port (or (uri-port uri) 22)
- #:user (uri-userinfo uri))))
+ #:user (uri-userinfo uri)
+ #:open-connection open-connection)))
=20
;;; ssh.scm ends here
--=-=-=--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 9 Jun 2020 09:15:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 09 05:15:32 2020
Received: from localhost ([127.0.0.1]:58937 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jiaM0-0000O6-6n
for submit <at> debbugs.gnu.org; Tue, 09 Jun 2020 05:15:32 -0400
Received: from mail-wm1-f49.google.com ([209.85.128.49]:53192)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ldb@HIDDEN>) id 1jiaLy-0000Ns-98
for 41702 <at> debbugs.gnu.org; Tue, 09 Jun 2020 05:15:30 -0400
Received: by mail-wm1-f49.google.com with SMTP id r9so2095910wmh.2
for <41702 <at> debbugs.gnu.org>; Tue, 09 Jun 2020 02:15:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=qz4Fu3nZaZG2CLuasY5u1szhQ0Dyk5hOv98/XWZ5f1I=;
b=jP3zCMyf+vrHZiMZ0et32GFF4IeY7qtzwFGaS9kzDVwzKpR4C4Pq5LBBZ1tIX1Ltq0
3/IGb9m26WjiOYTax6fw6rXH4bCF+Wv29imYKoV6bIWbqOe7daYNYLdgQgq7yi2VJKLR
Jgj6BUGa3+z6fMUlZ7AagJkFFVi4A0ujK++jyr73jnTLqgjWCepC4K3bDoth2OBMSZLc
BKPL68T+ZAgWyH5uG+nhwPtFZqVeXz13SblFnba+XAMmmelUGrhIz0a/TTDeAF2SoDpY
ruTRpOYZvAQfdPoX6ZukaPCx4Wol7ZE0KGwskRAP33YlUC/yi/etozTE4D7vPRMN+Kfz
h4hQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:cc:subject:message-id:references
:mime-version:content-disposition:in-reply-to:user-agent;
bh=qz4Fu3nZaZG2CLuasY5u1szhQ0Dyk5hOv98/XWZ5f1I=;
b=rzIzad18CtNbriiEpVqMr2comC4pEnNQXWoJ0WLb8LYX+8F905YeILSe/IxJ5cgv+t
k2fecfMbAyDuyjTV+q/bE3nkuV2Vn7zwYKMgmtQwJFjyi7HGS/oOBXQj5vM+yIZXdm5t
TxnIe54klIl04SxZnbavdQEWBu0ZJMkkCw92NB8xHWJ56P1L5mf6+IChxp7LaKcRYsbQ
TuzV1SE/SRHmnAzIBJxo6lvQ9gqZQUcoeGiiulog7gCxbWrCVRzcCkYiWTCiVC/sczXu
ZNzKqPa1aoDT6E0Wa4Wvb26l5uaM91Ff2CW3ADg7/zFAckl4ikDb/762ZD7Y9RLUFi5+
MrgA==
X-Gm-Message-State: AOAM530kEJIIvRddXlfoJgHjhNiknSDKlO50mjQ1vOkwoUyEJRSGjYFM
bK2kapAObljXTagm/AE2j5+nygtBZNeZLWHdzZ6MJ7Ak1OefOUgxfIzwzlRWfMADvC+xupUkQ6C
ZxLmP3fL/Hp9+VOwFqs68mwwpN+8J91r4QBBF20Ujj+icrrRDhbSVOp2t1fuAk3XLKOuZ9+BsE9
WGAbo=
X-Google-Smtp-Source: ABdhPJzlQ3huw0HzeLj0aBtVws0x62rRoHHH8RdbCo+jm94YcF1nhx9VjXOLOa8enzfdEKbhAMTIVw==
X-Received: by 2002:a1c:98cc:: with SMTP id a195mr2967897wme.89.1591694124024;
Tue, 09 Jun 2020 02:15:24 -0700 (PDT)
Received: from localhost
(dynamic-2a01-0c23-7c5e-9e00-3d61-c8ee-9cd9-ef49.c23.pool.telefonica.de.
[2a01:c23:7c5e:9e00:3d61:c8ee:9cd9:ef49])
by smtp.gmail.com with ESMTPSA id o18sm2276486wme.19.2020.06.09.02.15.22
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Tue, 09 Jun 2020 02:15:23 -0700 (PDT)
Date: Tue, 9 Jun 2020 11:15:22 +0200
From: Lars-Dominik Braun <ldb@HIDDEN>
To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
Message-ID: <20200609091522.GB8597@zpidnp36>
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
<20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="Pk6IbRAofICFmK5e"
Content-Disposition: inline
In-Reply-To: <87k10hp6ba.fsf@HIDDEN>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--Pk6IbRAofICFmK5e
Content-Type: multipart/mixed; boundary="XF85m9dhOBO43t/C"
Content-Disposition: inline
--XF85m9dhOBO43t/C
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hey,
> That=E2=80=99s over SSH, right?
correct, the worst possible case: Inside two VM=E2=80=99s on a Laptop, SSH =
transport
between them and /gnu+/var/guix on an NFS share (nfsd is in the same VM as
guix-daemon).
> Probably what=E2=80=99s killing us is the round-trip time for all these s=
mall
> RPCs. We would need pipelining but the RPC protocol is not designed to
> make that easy.
That would have been my best guess too, but it does not seem to be the bigg=
est
problem right now. Looking at the numbers again (both patches applied) with=
the
attached manifest[1], I see that:
---snip---
Local UNIX socket with and without --no-grafts:
N Min Max Median Avg Stddev
x 10 6.07 6.35 6.145 6.16 0.08232726
+ 10 17.47 17.89 17.545 17.602 0.14351152
Difference at 99.0% confidence
11.442 +/- 0.150576
185.747% +/- 4.07133%
Local UNIX socket vs. guix://localhost transport:
N Min Max Median Avg Stddev
x 10 17.47 17.89 17.545 17.602 0.14351152
+ 10 17.43 18.1 17.61 17.642 0.20131788
No difference proven at 99.0% confidence
Local UNIX socket vs ssh://localhost transport:
N Min Max Median Avg Stddev
x 10 17.47 17.89 17.545 17.602 0.14351152
+ 10 33.46 35.27 34.315 34.359 0.53873205
Difference at 99.0% confidence
16.757 +/- 0.5074
95.1994% +/- 3.13957%
---snap---
So I would conclude:
1) Grafting still takes a lot of time and needs more work
2) Linux optimizes localhost networking pretty well
3) Our SSH transport is terribly slow
Moving to non-localhost communication between two VM=E2=80=99s:
---snip---
guix://localhost vs. guix://remote-host transport:
N Min Max Median Avg Stddev
x 10 17.43 18.1 17.61 17.642 0.20131788
+ 10 20.88 22.58 21.095 21.222 0.49689704
Difference at 99.0% confidence
3.58 +/- 0.487934
20.2925% +/- 2.85159%
guix://remote-host vs. ssh://remote-host:
N Min Max Median Avg Stddev
x 10 20.88 22.58 21.095 21.222 0.49689704
+ 10 30.1 32.56 31.005 31.093 0.70740606
Difference at 99.0% confidence
9.871 +/- 0.786769
46.5131% +/- 4.35326%
---snap---
Conclusion here is the same: Not alot of impact of networking/NFS and SSH
transport is still terribly slow. (Confusingly faster than localhost though=
=2E)
> Perhaps you could =E2=80=9Cstrace -Tt=E2=80=9D the thing to check whether=
this
> hypothesis is correct by looking at the time we spend waiting for
> replies?
I=E2=80=99m not sure this will yield meaningful data for SSH, so I analyzet=
it for
guix://localhost vs. guix://remote-host. Takeaway is, yes, of course there =
is a
statistically significant difference and it=E2=80=99s about 40%=C2=B150%, w=
hich means this
method is pretty useless, because we can=E2=80=99t bin RPC=E2=80=99s by typ=
e.
So, I guess it would make sense for me to look at the SSH transport itself
again and see if there are any other low-hanging fruit. Not sure how much I=
can
help with profiling guile/guix itself. A different/better RPC protocol is
probably GSoC/v2.0-worthy?
Sorry for all the lengthy emails,
Lars
[1] You=E2=80=99ll need this channel: https://github.com/leibniz-psychology=
/guix-zpid
--XF85m9dhOBO43t/C
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="manifest.scm"
(specifications->manifest
'(
"coreutils"
"findutils"
"procps"
"strace"
"openssh"
"mit-krb5"
"bash"
"which"
"zip"
"geeqie"
"util-linux"
"grep"
"glibc"
;; for locales
"glibc-locales"
;; front-end software
"jupyter-zpid"
"python-jupyterlab"
;; available notebook kernel
; provided by jupyter-zpid
;"python-ipykernel"
"r-irkernel"
"rstudio-server-zpid"
"r"
;; for RMarkdown
"r-knitr"
"r-yaml"
"r-markdown"
"r-rmarkdown"
"texlive"
;; commonly used r packages
"r-psych"
"r-ggplot2"
"r-lattice"
"r-foreign"
"r-readr"
"r-haven"
"r-dplyr"
"r-tidyr"
"r-stringr"
"r-forecast"
"r-lme4"
"r-nlme"
"r-nnet"
"r-glmnet"
"r-caret"
"r-xmisc"
"r-splitstackshape"
"r-tm"
"r-quanteda"
"r-topicmodels"
"r-stm"
;;"r-parallel"
"r-dt"
"r-nlp"
"r-data-table"
"r-hmisc"
"r-learnr"
"r-metafor"
;; for rmarkdown
"ghc-pandoc"
))
--XF85m9dhOBO43t/C--
--Pk6IbRAofICFmK5e
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl7fUycACgkQQhN3ARo3
hEbGwgv9G/Jy0viE4XGqrmhCfmnQSKMvxgGZg7WPUBNYaOFcUr3fjpR13egOlcqS
mvebo/JZ8cs+nRNQB7/v+FTYFPTmk/xFIN3jLNdHnqUmHHBmhLsirbyQRdMXYr6F
sD7eho57lQ/nlnTsEU6NZQwO3WA3kd4aFjvNxSBpaVeO1HPaxyWTPEQVqpWGN7u3
nFzw4866BBBNP1R1nSdbUKsAzilHKTWMJ0B6O34dNK1/fdaVSJ/yNaAzehN+cykF
SgYCzl1eUYquNUtZvyHjNptYm9BX54YQPA7HrMmpsd8qOaeTe6R4HHrV6+JPy/oK
rrr2RPsVnI1bY5o58mRKzk8up75FsTlN6tWBwuempr00dIsjny/gjBHuL6J5v3vC
GqmHOMhjF66PNmiwheqCGHNae8tq7Li4diFmSo9uV+kFpwgmGQ13ywZeKex2l6xV
g0vpX0bGkr1gxK/XWy2dYRtgkNYf5ycAhUBBLOhTlN6/+nWI6QLEgKiAmZR16Ju5
AVtbp5Gm
=zWIs
-----END PGP SIGNATURE-----
--Pk6IbRAofICFmK5e--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 8 Jun 2020 21:59:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 08 17:59:51 2020
Received: from localhost ([127.0.0.1]:58122 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jiPo7-0006dy-Ex
for submit <at> debbugs.gnu.org; Mon, 08 Jun 2020 17:59:51 -0400
Received: from eggs.gnu.org ([209.51.188.92]:42706)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1jiPo5-0006di-Lx
for 41702 <at> debbugs.gnu.org; Mon, 08 Jun 2020 17:59:50 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:57740)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <ludo@HIDDEN>)
id 1jiPo0-0000Xt-9M; Mon, 08 Jun 2020 17:59:44 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56824 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1jiPnu-0007mO-Cg; Mon, 08 Jun 2020 17:59:40 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Lars-Dominik Braun <ldb@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
<20200608090453.GC3166@zpidnp36>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 21 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 08 Jun 2020 23:59:37 +0200
In-Reply-To: <20200608090453.GC3166@zpidnp36> (Lars-Dominik Braun's message of
"Mon, 8 Jun 2020 11:04:53 +0200")
Message-ID: <87k10hp6ba.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi,
Lars-Dominik Braun <ldb@HIDDEN> skribis:
>> --8<---------------cut here---------------start------------->8---
>> $ time guix environment --ad-hoc r-learnr --search-paths
>> export PATH=3D"/gnu/store/n4wxbmqpafjfyawrla8xymzzdm5hxwph-profile/bin${=
PATH:+:}$PATH"
>>=20
>> real 0m11.328s
>> user 0m20.155s
>> sys 0m0.172s
>> $ time ./pre-inst-env guix environment --ad-hoc r-learnr --search-paths
>> export PATH=3D"/gnu/store/if6z77la3mx0qdzvcyl4qv9i5cyp48i0-profile/bin${=
PATH:+:}$PATH"
>>=20
>> real 0m4.602s
>> user 0m6.189s
>> sys 0m0.136s
>> --8<---------------cut here---------------end--------------->8---
> that=E2=80=99s awesome and brings me much closer to my goal of running al=
l applications
> inside a `guix environment` container for reproducibility. Including the
> protocol fixes from #41720 I=E2=80=99m now down to ~30s from ~50s, which =
may be called
> somewhat usable. Obviously I=E2=80=99d be very interested in further spee=
dups.
That=E2=80=99s over SSH, right?
Probably what=E2=80=99s killing us is the round-trip time for all these sma=
ll
RPCs. We would need pipelining but the RPC protocol is not designed to
make that easy.
Perhaps you could =E2=80=9Cstrace -Tt=E2=80=9D the thing to check whether t=
his
hypothesis is correct by looking at the time we spend waiting for
replies?
As for the CPU cost (i.e., going below the 4.6s above), we should keep
profiling just like you did.
Thanks,
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 8 Jun 2020 09:05:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 08 05:05:03 2020
Received: from localhost ([127.0.0.1]:55909 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jiDiJ-0001vo-Az
for submit <at> debbugs.gnu.org; Mon, 08 Jun 2020 05:05:03 -0400
Received: from mail-wr1-f54.google.com ([209.85.221.54]:43113)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ldb@HIDDEN>) id 1jiDiG-0001vF-SP
for 41702 <at> debbugs.gnu.org; Mon, 08 Jun 2020 05:05:01 -0400
Received: by mail-wr1-f54.google.com with SMTP id l10so16482913wrr.10
for <41702 <at> debbugs.gnu.org>; Mon, 08 Jun 2020 02:05:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=GW5p1cLtkLJAdMUYkRVfcZ0FyNzwHqKrU4i4Uimt6sE=;
b=xhHvlDg4wzwUBGPuelUztI3BtEYMILpT7dz72SHzOgRDweabseWaNtgVUIn3TMqIXU
V6MX38AOtsnOofjw/9s92rC6/MtDCJ+d0wHzNHWODXYLUafy2Bg62Zb5qjriTUtdRIU5
f/frqueexetliNWh2cS3o3uLZhm5j+p6y1Ugo86KS2xEuLZfs7pacAx9KCROnF+rGoS7
Eb0URgH+VfVnkKdXxkomhLpnNYXU0w3CxbpBBPhtYmoxuMcWnjsiEbjyE5BqRseoE8oq
iLgjqTxdhIcY/ysWo7eTswvlAjBKRyMHxfa6x5e37MwBGfAKIZKBJoSCIHqDAczBOr91
jwng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:cc:subject:message-id:references
:mime-version:content-disposition:in-reply-to:user-agent;
bh=GW5p1cLtkLJAdMUYkRVfcZ0FyNzwHqKrU4i4Uimt6sE=;
b=XYeIPzVmPKHUiTLEgCAI0XM9gM7RCJdcy27JRpd2E1PCa7qnoMGYK9UxjX7IogL+aS
32mCaVPaaaS98iz8JJPngmtT2RJXRvNLy/kMUJxOc1WD5agioWhwXUMjxb3RUXmToMgo
OkY99qO5M8l+jD6clxzsR1GwU1gbBTVSQwqsWOvt75jBfR1ZDChn39mX+1sgfsRfOGkl
Inqhmye+tAPje++OVK/K7u5+uNQH14FShpbzph5Qo+0+rJLbhUfxbDo2IXI6984rdn4V
LQTMRrBQcDsNrQxuquiunRmdCTTIRIqiI2aX4IRroyGxvX5SWExAg4yrhEtp++9E1aLh
7INQ==
X-Gm-Message-State: AOAM531lS4tCwkY2ka9pSXi0h/PTOgcVIOcNZCUyr/Jzy7Gq4EcAlfWF
yOPc/NH1gp5eKQIqRQrH84AWSn3vYj9zxz8R3WMQwIQ+yZfvEgMlM3wew6MsI/Mgf+NitfQGd2l
W8FoNM0HUwMfhTbNFJocJ32u/Wny/Vq87+yAiiUtkzYQVkMJiKqD1GrjnVYJ9xG1v25PZeTIxhL
fm6OA=
X-Google-Smtp-Source: ABdhPJxLr/jCEuBDj1TZ0X812FBfPjdUXFpxN0ZOnLYhq7O5xvhlGJ9HQtDQ69+s0Yt/q//6apxFZA==
X-Received: by 2002:a5d:6283:: with SMTP id k3mr21788910wru.422.1591607094849;
Mon, 08 Jun 2020 02:04:54 -0700 (PDT)
Received: from localhost
(dynamic-2a01-0c22-d48d-f900-f58e-6c44-3fc3-d82c.c22.pool.telefonica.de.
[2a01:c22:d48d:f900:f58e:6c44:3fc3:d82c])
by smtp.gmail.com with ESMTPSA id d24sm21105373wmb.45.2020.06.08.02.04.53
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 08 Jun 2020 02:04:53 -0700 (PDT)
Date: Mon, 8 Jun 2020 11:04:53 +0200
From: Lars-Dominik Braun <ldb@HIDDEN>
To: Ludovic =?iso-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
Message-ID: <20200608090453.GC3166@zpidnp36>
References: <20200604082316.GA3146@zpidnp36>
<87mu5gtbwg.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="Pk6IbRAofICFmK5e"
Content-Disposition: inline
In-Reply-To: <87mu5gtbwg.fsf@HIDDEN>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--Pk6IbRAofICFmK5e
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi Ludo,
> --8<---------------cut here---------------start------------->8---
> $ time guix environment --ad-hoc r-learnr --search-paths
> export PATH=3D"/gnu/store/n4wxbmqpafjfyawrla8xymzzdm5hxwph-profile/bin${P=
ATH:+:}$PATH"
>=20
> real 0m11.328s
> user 0m20.155s
> sys 0m0.172s
> $ time ./pre-inst-env guix environment --ad-hoc r-learnr --search-paths
> export PATH=3D"/gnu/store/if6z77la3mx0qdzvcyl4qv9i5cyp48i0-profile/bin${P=
ATH:+:}$PATH"
>=20
> real 0m4.602s
> user 0m6.189s
> sys 0m0.136s
> --8<---------------cut here---------------end--------------->8---
that=E2=80=99s awesome and brings me much closer to my goal of running all =
applications
inside a `guix environment` container for reproducibility. Including the
protocol fixes from #41720 I=E2=80=99m now down to ~30s from ~50s, which ma=
y be called
somewhat usable. Obviously I=E2=80=99d be very interested in further speedu=
ps.
Thanks,
Lars
--Pk6IbRAofICFmK5e
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl7d/zEACgkQQhN3ARo3
hEYcoQwAtb945F+lwytXrepW+8lYfu/S2Pdi8uwyyMfFFjx8oP+t9CmQbVByt224
4N7YLTJReP1BFEN3lnOiV8R1bA9stKCWaSCcUR2jI4LgtOlIDFfr6YwMRSf2krbo
ylpAX6x9dHxdEl32M++ODzBGy5BgxDBKga6gk7OPoGUzld9nFYQX6s7VQGB6IcJg
IyjP9zFhCTEO9eTFoUyUynIdx/LCT6l0vRobX0hy+sStXoMa7weaoBwEMbQ08Hed
ltxv35iBAYcOJoTM68r/Ph9G8ipvL6KQMBIPfTRxGm/UXH2z1OH4/sDYST0wNbZ8
mG4JGt/oiewQ/ssZRM8JzrRAd5s0pOK5c0B9olojRerfUvF8ubfVvZIqpvfyKPX5
ny8SN75MyEDhvodqQbjCAXlAPFvlW+0vWxlMTC+SufZceQ4cBaGCDNeuV2K3KtzW
ywW/nwkpHm5y5d3jePyBZreB8WO9ml7UTfRX7AawlmKSMx6OGWM7m3tYJh2fghJ5
rMerWnCh
=SDlJ
-----END PGP SIGNATURE-----
--Pk6IbRAofICFmK5e--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 6 Jun 2020 21:40:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 06 17:40:59 2020
Received: from localhost ([127.0.0.1]:52612 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jhgYl-0005b7-AB
for submit <at> debbugs.gnu.org; Sat, 06 Jun 2020 17:40:59 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48508)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1jhgYk-0005av-C0
for 41702 <at> debbugs.gnu.org; Sat, 06 Jun 2020 17:40:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:39201)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <ludo@HIDDEN>)
id 1jhgYe-0005ux-TA; Sat, 06 Jun 2020 17:40:52 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38998 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1jhgYe-0008GV-Dc; Sat, 06 Jun 2020 17:40:52 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Lars-Dominik Braun <ldb@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@HIDDEN>
Date: Sat, 06 Jun 2020 23:40:51 +0200
In-Reply-To: <87mu5gtbwg.fsf@HIDDEN> ("Ludovic
\=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
\=\?utf-8\?Q\?s\?\= message of "Sat, 06 Jun 2020 18:08:31 +0200")
Message-ID: <87d06bub30.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> The patch below arranges so that =E2=80=98cumulative-grafts=E2=80=99 proc=
esses
> dependencies in a batch, such that the derivation=E2=80=99s dependency gr=
aph is
> traversed once for all, which makes a difference for derivations with
> lots of inputs.
Pushed:
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3D58bb833365db4e893=
4a386497d5b00a063cfd27d
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
Received: (at 41702) by debbugs.gnu.org; 6 Jun 2020 17:51:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 06 13:51:40 2020
Received: from localhost ([127.0.0.1]:52388 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jhcyp-0006KB-UR
for submit <at> debbugs.gnu.org; Sat, 06 Jun 2020 13:51:40 -0400
Received: from eggs.gnu.org ([209.51.188.92]:43682)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1jhcyn-0006Jr-F0
for 41702 <at> debbugs.gnu.org; Sat, 06 Jun 2020 13:51:37 -0400
Received: from [2001:470:142:3::e] (port=34123 helo=fencepost.gnu.org)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <ludo@HIDDEN>)
id 1jhbN9-0000lM-Mq; Sat, 06 Jun 2020 12:08:39 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52514 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1jhbN3-0006Ob-AW; Sat, 06 Jun 2020 12:08:33 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Lars-Dominik Braun <ldb@HIDDEN>
Subject: Re: bug#41702: `guix environment` performance issues
References: <20200604082316.GA3146@zpidnp36>
Date: Sat, 06 Jun 2020 18:08:31 +0200
In-Reply-To: <20200604082316.GA3146@zpidnp36> (Lars-Dominik Braun's message of
"Thu, 4 Jun 2020 10:23:16 +0200")
Message-ID: <87mu5gtbwg.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41702
Cc: 41702 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,
Lars-Dominik Braun <ldb@HIDDEN> skribis:
> Total time: 24.672604202 seconds (19.431122691 seconds in GC)
> ./pre-inst-env guix environment --ad-hoc r-learnr -- true 25,18s user 0,=
24s system 308% cpu 8,248 total
>
> More specifically in an anonymous function and reap-pipes, which is a gc =
hook,
> I believe:
>
> % cumulative self
> time seconds seconds calls procedure
> 33.41 14.49 8.24 anon #xbb8480
> 27.95 6.90 6.90 ice-9/popen.scm:145=
:0:reap-pipes
> 4.37 1.08 1.08 anon #xbbdcd8
> 3.28 0.86 0.81 ice-9/vlist.scm:539=
:0:vhash-assq
> 2.40 2.37 0.59 guix/grafts.scm:202=
:22
Guile master has a fix for statprof that yields more useful info:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix scripts environment)
scheme@(guile-user)> ,pr (guix-environment "--ad-hoc" "r-learnr" "--" "true=
")
% cumulative self=20=20=20=20=20=20=20=20=20=20=20=20=20
time seconds seconds procedure
29.84 9.87 6.16 append
19.56 4.04 4.04 %after-gc-thunk
6.85 1.87 1.42 ice-9/vlist.scm:539:0:vhash-assq
5.44 1.17 1.12 write
3.23 0.67 0.67 guix/derivations.scm:665:0:derivation->output-p=
aths
2.82 0.58 0.58 string=3D?
2.42 2.37 0.50 guix/grafts.scm:202:22
2.42 0.50 0.50 list?
2.22 0.46 0.46 hashq
2.02 0.42 0.42 display
1.61 15.82 0.33 guix/grafts.scm:186:0:reference-origin
1.61 0.87 0.33 guix/grafts.scm:204:31
1.21 0.33 0.25 guix/derivations.scm:667:7
1.21 0.29 0.25 srfi/srfi-1.scm:817:0:any
1.01 1232.14 0.21 srfi/srfi-1.scm:584:5:map1
0.81 0.83 0.17 guix/derivations.scm:697:0:derivation/masked-in=
puts
0.81 0.75 0.17 srfi/srfi-1.scm:580:2:map
0.81 0.17 0.17 guix/derivations.scm:158:0:%derivation-input-de=
rivation-procedure
0.60 0.17 0.12 reverse
0.60 0.12 0.12 hashq-ref
0.60 0.12 0.12 get-bytevector-n
0.60 0.12 0.12 procedure?
0.40 0.67 0.08 guix/packages.scm:1232:0:fold-bag-dependencies
0.40 0.12 0.08 string->utf8
0.40 0.12 0.08 ice-9/vlist.scm:534:0:vhash-assoc
0.40 0.12 0.08 ice-9/vlist.scm:449:0:vhash-cons
0.40 0.12 0.08 delete-duplicates
0.40 0.08 0.08 ice-9/boot-9.scm:1389:0:->bool
0.40 0.08 0.08 ice-9/boot-9.scm:2201:0:%load-announce
0.40 0.08 0.08 hash
0.40 0.08 0.08 guix/derivations.scm:665:0:derivation->output-p=
aths
0.20 20.73 0.04 guix/gexp.scm:1061:2
--8<---------------cut here---------------end--------------->8---
Notice that the same command with =E2=80=98--no-grafts=E2=80=99 takes 2s in=
stead of 11s.
The patch below arranges so that =E2=80=98cumulative-grafts=E2=80=99 proces=
ses
dependencies in a batch, such that the derivation=E2=80=99s dependency grap=
h is
traversed once for all, which makes a difference for derivations with
lots of inputs.
Here=E2=80=99s the before/after comparison:
--8<---------------cut here---------------start------------->8---
$ time guix environment --ad-hoc r-learnr --search-paths
export PATH=3D"/gnu/store/n4wxbmqpafjfyawrla8xymzzdm5hxwph-profile/bin${PAT=
H:+:}$PATH"
real 0m11.328s
user 0m20.155s
sys 0m0.172s
$ time ./pre-inst-env guix environment --ad-hoc r-learnr --search-paths
export PATH=3D"/gnu/store/if6z77la3mx0qdzvcyl4qv9i5cyp48i0-profile/bin${PAT=
H:+:}$PATH"
real 0m4.602s
user 0m6.189s
sys 0m0.136s
--8<---------------cut here---------------end--------------->8---
There=E2=80=99s still room for improvement, but it=E2=80=99s much better.
Ludo=E2=80=99.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline
diff --git a/guix/grafts.scm b/guix/grafts.scm
index 69d6fe4469..910dcadc8a 100644
--- a/guix/grafts.scm
+++ b/guix/grafts.scm
@@ -20,10 +20,12 @@
#:use-module (guix store)
#:use-module (guix monads)
#:use-module (guix records)
+ #:use-module (guix combinators)
#:use-module (guix derivations)
#:use-module ((guix utils) #:select (%current-system))
#:use-module (guix sets)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-9 gnu)
#:use-module (srfi srfi-26)
#:use-module (srfi srfi-34)
@@ -183,32 +185,47 @@ references."
(set-current-state (vhash-cons key result cache))
(return result)))))))
-(define (reference-origin drv item)
- "Return the derivation/output pair among the inputs of DRV, recursively,
-that produces ITEM. Return #f if ITEM is not produced by a derivation (i.e.,
-it's a content-addressed \"source\"), or if it's not produced by a dependency
-of DRV."
+(define (reference-origins drv items)
+ "Return the derivation/output pairs among the inputs of DRV, recursively,
+that produce ITEMS. Elements of ITEMS not produced by a derivation (i.e.,
+it's a content-addressed \"source\"), or not produced by a dependency of DRV,
+have no corresponding element in the resulting list."
+ (define (lookup-derivers drv result items)
+ ;; Return RESULT augmented by all the drv/output pairs producing one of
+ ;; ITEMS, and ITEMS stripped of matching items.
+ (fold2 (match-lambda*
+ (((output . file) result items)
+ (if (member file items)
+ (values (alist-cons drv output result)
+ (delete file items))
+ (values result items))))
+ result items
+ (derivation->output-paths drv)))
+
;; Perform a breadth-first traversal of the dependency graph of DRV in
- ;; search of the derivation that produces ITEM.
+ ;; search of the derivations that produce ITEMS.
(let loop ((drv (list drv))
+ (items items)
+ (result '())
(visited (setq)))
(match drv
(()
- #f)
+ result)
((drv . rest)
- (if (set-contains? visited drv)
- (loop rest visited)
- (let ((inputs (derivation-inputs drv)))
- (or (any (lambda (input)
- (let ((drv (derivation-input-derivation input)))
- (any (match-lambda
- ((output . file)
- (and (string=? file item)
- (cons drv output))))
- (derivation->output-paths drv))))
- inputs)
- (loop (append rest (map derivation-input-derivation inputs))
- (set-insert drv visited)))))))))
+ (cond ((null? items)
+ result)
+ ((set-contains? visited drv)
+ (loop rest items result visited))
+ (else
+ (let*-values (((inputs)
+ (map derivation-input-derivation
+ (derivation-inputs drv)))
+ ((result items)
+ (fold2 lookup-derivers
+ result items inputs)))
+ (loop (append rest inputs)
+ items result
+ (set-insert drv visited)))))))))
(define* (cumulative-grafts store drv grafts
#:key
@@ -233,25 +250,27 @@ derivations to the corresponding set of grafts."
(_
#f)))
- (define (dependency-grafts item)
- (match (reference-origin drv item)
- ((drv . output)
- ;; If GRAFTS already contains a graft from DRV, do not override it.
- (if (find (cut graft-origin? drv <>) grafts)
- (state-return grafts)
- (cumulative-grafts store drv grafts
- #:outputs (list output)
- #:guile guile
- #:system system)))
- (#f
- (state-return grafts))))
+ (define (dependency-grafts items)
+ (mapm %store-monad
+ (lambda (drv+output)
+ (match drv+output
+ ((drv . output)
+ ;; If GRAFTS already contains a graft from DRV, do not
+ ;; override it.
+ (if (find (cut graft-origin? drv <>) grafts)
+ (state-return grafts)
+ (cumulative-grafts store drv grafts
+ #:outputs (list output)
+ #:guile guile
+ #:system system)))))
+ (reference-origins drv items)))
(with-cache (cons (derivation-file-name drv) outputs)
(match (non-self-references store drv outputs)
(() ;no dependencies
(return grafts))
(deps ;one or more dependencies
- (mlet %state-monad ((grafts (mapm %state-monad dependency-grafts deps)))
+ (mlet %state-monad ((grafts (dependency-grafts deps)))
(let ((grafts (delete-duplicates (concatenate grafts) equal?)))
(match (filter (lambda (graft)
(member (graft-origin-file-name graft) deps))
--=-=-=--
bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at submit) by debbugs.gnu.org; 4 Jun 2020 08:23:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 04 04:23:25 2020
Received: from localhost ([127.0.0.1]:45057 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jgl9o-0001rh-TY
for submit <at> debbugs.gnu.org; Thu, 04 Jun 2020 04:23:25 -0400
Received: from lists.gnu.org ([209.51.188.17]:35836)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ldb@HIDDEN>) id 1jgl9m-0001rY-TE
for submit <at> debbugs.gnu.org; Thu, 04 Jun 2020 04:23:23 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:48448)
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ldb@HIDDEN>)
id 1jgl9m-0001xr-N6
for bug-guix@HIDDEN; Thu, 04 Jun 2020 04:23:22 -0400
Received: from mail-ed1-x532.google.com ([2a00:1450:4864:20::532]:41745)
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.90_1) (envelope-from <ldb@HIDDEN>)
id 1jgl9k-0003uP-Mo
for bug-guix@HIDDEN; Thu, 04 Jun 2020 04:23:22 -0400
Received: by mail-ed1-x532.google.com with SMTP id ce8so3952525edb.8
for <bug-guix@HIDDEN>; Thu, 04 Jun 2020 01:23:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:subject:message-id:mime-version:content-disposition
:user-agent; bh=GUsCJnysXURzM+btb8b0onoT10tehoeNWU2ITasfoqo=;
b=F0ioEYnIrxb2P6DUvkTx5mjIHZmeaSfhuZZpxVV+WXLHmcg7zdsl35OvXazTvREdDQ
h8qi9pnGsC77d/A1Ftm5bcCYJ7Yl5RoS/fZOmjAD6rPwpJndVZBEFeL5OLNu3NWNGyoj
z+uvUNiK/KBI7c35dM/L5FTizzo9mGq7Qk+O/akSmovt6fTblQN8chVceJumA6B2hdxF
WgWF6zZmT5EAVxJ29H+C1fYSES4dNLcOqxypFFfvh5HR2vvvLQSVdDtn3uyo2N385lb/
UZ/aK1u2yc0l+Wi0yt2+/46Zup4uOGFPvmIcL/0NVQ9zoXcuNIzFD42z12D+mQa1du2o
ulIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:subject:message-id:mime-version
:content-disposition:user-agent;
bh=GUsCJnysXURzM+btb8b0onoT10tehoeNWU2ITasfoqo=;
b=GXXbcDTL/5s43ExdeNp8ZxO1UxnhrZdI0NfNU3JFuAy0vagLfCQ5YFIl6PORxkUMjb
ZQiiB34tRld2rB9BkSa7E14C4U8iHMg37x0VYbxQbCtS22muUi0OnWewzPiqMIq8HhUP
OPl/FQ8UqemTo+pM52rHUJq2gAQK+5cf0ygdMNu79tXbBiUvZ9/rAIInwZ5EFMKtCJxV
3Bvbn6U65p5HWLQyf2aZDl0kzjjXKhJWx7am//ikth8eIlEOmZuvcCutxwh9H0vaDEFK
jYPAJukqVPOyZ7s37FglCc1kW84KvQYXDkvXXhHVlUdSR+KEeCOjD97VWVPNtCCvkm8x
hFdg==
X-Gm-Message-State: AOAM530c6T6z+gu6dTEOUpRwEeeO1cLqZW2J98GFOM3YbAqd1Ez5/Gme
x0p21YRfL2V47kC476u8pU1aAETxfkbDSx8BTRF4HVNQW8PE9yeMxoLOzIMKVTWdcr8nhlEwLMQ
X9Jp/ZD8k5acUtiec84pL+d/kiQSTMHegKDvunPNILi0roG3YkPPFH372feW8rsyfcoWC2LRQ/I
oCKw==
X-Google-Smtp-Source: ABdhPJx985fJ7VBpf5/94I81LK81huj7+wGLOedoy4RDvfx8qH+XfFSfQdRQqlJ9NK8tgL7SSTwQBw==
X-Received: by 2002:a05:6402:3ca:: with SMTP id
t10mr3339066edw.128.1591258998105;
Thu, 04 Jun 2020 01:23:18 -0700 (PDT)
Received: from localhost
(dynamic-2a01-0c23-8458-f500-08f2-7cdd-6690-abd3.c23.pool.telefonica.de.
[2a01:c23:8458:f500:8f2:7cdd:6690:abd3])
by smtp.gmail.com with ESMTPSA id v12sm1687474eda.39.2020.06.04.01.23.17
for <bug-guix@HIDDEN>
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Thu, 04 Jun 2020 01:23:17 -0700 (PDT)
Date: Thu, 4 Jun 2020 10:23:16 +0200
From: Lars-Dominik Braun <ldb@HIDDEN>
To: bug-guix@HIDDEN
Subject: `guix environment` performance issues
Message-ID: <20200604082316.GA3146@zpidnp36>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="UugvWAfsgieZRqgk"
Content-Disposition: inline
User-Agent: Mutt/1.10.1 (2018-07-13)
Received-SPF: none client-ip=2a00:1450:4864:20::532;
envelope-from=ldb@HIDDEN; helo=mail-ed1-x532.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
That's all we know.
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001,
URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--UugvWAfsgieZRqgk
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi,
I=E2=80=99ve noticed that `guix environment` can be very very slow for some=
packages.
Whereas usually a call like
time guix environment --ad-hoc <package> -- true
takes 600ms to 1.5s, it takes 8.4s for the package r-learnr on my Ryzen 5 3=
600
with NVMe SSD, 32G of RAM and a warm cache. This seems to accumulate when
trying to install multiple unrelated =E2=80=9Cslow=E2=80=9D packages into t=
he same environment,
i.e. jupyter (2.9s) + r-rlearnr (8.4s) + r-stm (2.9s) =3D 12.1s. What these
packages seem to have in common is a deep chain of propagated inputs.
And in fact, if I create artificial test packages where test-999 propagates
test-998, test-998=E2=86=92test-997 and so on the command above takes 1m 9s=
=2E A single
package propagating 999 packages takes 4.5s.
Instrumenting guix-environment=E2=80=99s with-store call with statprof I se=
e a lot of
time is spent in the garbage collector:
Total time: 24.672604202 seconds (19.431122691 seconds in GC)
=2E/pre-inst-env guix environment --ad-hoc r-learnr -- true 25,18s user 0,=
24s system 308% cpu 8,248 total
More specifically in an anonymous function and reap-pipes, which is a gc ho=
ok,
I believe:
% cumulative self
time seconds seconds calls procedure
33.41 14.49 8.24 anon #xbb8480
27.95 6.90 6.90 ice-9/popen.scm:145:0=
:reap-pipes
4.37 1.08 1.08 anon #xbbdcd8
3.28 0.86 0.81 ice-9/vlist.scm:539:0=
:vhash-assq
2.40 2.37 0.59 guix/grafts.scm:202:22
Not sure if that helps tracking the issue down though.
Cheers,
Lars
PS: Here=E2=80=99s `guix describe`:
Generation 1 Jun 03 2020 10:38:17 (current)
guix 9ab2790
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 9ab2790e2acf872c0f2cf41ec180de80d1eee3f3
zpid bab33c1
repository URL: https://github.com/leibniz-psychology/guix-zpid.git
branch: master
commit: bab33c19e0f0304d7aa82751653eb85d8cad8c0a
--UugvWAfsgieZRqgk
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl7Yr3EACgkQQhN3ARo3
hEZvXgwAortYSBFszMEb4QyWF4EtfFQ8GMqhj9Av2r72HXGKtqnu+ZoY4VLK3eZx
Ntt0HIn50+BLQ/0TkWRCy5znN5t4NPVg7Qdn46DlCM9rOL6kkr61xrVGmmi5yzva
rir2cUIgLFHEwHQPP3AoKZAVrkzIoKxHhHPzFGyLZPglf0MER82X30pj5Yq14h3r
WlA+u6ktDsnHGYc9dD25mTSvuhITT8NAp4viNBC328FEZ+b/ct2/lpeln/khZYmU
HSG+bYI9R+fwg6isCPtKuZB+B7T5703KOyzvwbiasnCToMr1/PCV91z3Arba8MsC
eYZUmv8FfsrENz7iFs7le9J3csH39bhrZn7DNADwSt3HTILGHrrbEikgD3R3Lyvq
RcYRA9NZLZHSRdYntPbktFvK4i4H7UIBHx/b2MmiGAgrhqYvAPPFgTU8D0Y/w4Rd
rq6uXckgl3WlGnbmsVO7CEefanSv3RwWg7+m2LJ4sziXpT5fmyp+qru/l58yCUsZ
MiyfjkrB
=lFqp
-----END PGP SIGNATURE-----
--UugvWAfsgieZRqgk--
Lars-Dominik Braun <ldb@HIDDEN>:bug-guix@HIDDEN.
Full text available.bug-guix@HIDDEN:bug#41702; Package guix.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.