GNU bug report logs - #39258
Faster guix search using an sqlite cache

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: Arun Isaac <arunisaac@HIDDEN>; dated Thu, 23 Jan 2020 19:53:02 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 20:20:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 11 15:20:56 2020
Received: from localhost ([127.0.0.1]:57623 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j1c1g-0005dF-HE
	for submit <at> debbugs.gnu.org; Tue, 11 Feb 2020 15:20:56 -0500
Received: from mail-qk1-f195.google.com ([209.85.222.195]:39360)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1j1c1e-0005d2-D2
 for 39258 <at> debbugs.gnu.org; Tue, 11 Feb 2020 15:20:54 -0500
Received: by mail-qk1-f195.google.com with SMTP id w15so11491345qkf.6
 for <39258 <at> debbugs.gnu.org>; Tue, 11 Feb 2020 12:20:54 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=o6e6xjO5pBKI6J43h1ASM3POuFGUcPM015oYlxJw9xI=;
 b=KnTSeZs8OwpPZzrgjksQOEgXfS3bhJRoQoinCnRaq6v6R9LDrwXJfieA23Yi2Om5AR
 yUfD3hf3yDHhV3CtFWX8t9pGwQ6qvvpv/b6Lmqs1Ffq6RRlyKAX+FHTDLC+l7kQsTKuD
 tBJpeCtu+d05ytW17fx53uqo/80pTfNrkEDkPxs5E2/6jFAn6IoMt90WnEKwfE7xS3y3
 ZSfqJOzHWOmqk7XKVJODgEroK6tN0b0r6ZmfoYeWgxxxogfRMsSdXbdCRg9IFZR+iVul
 yStz6qNZ+KZtehFR/jKh35wM21VxPmkkB+meKyF2ZWqryfCrywWU4Uwp4EWdCrwn35z8
 mCgg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=o6e6xjO5pBKI6J43h1ASM3POuFGUcPM015oYlxJw9xI=;
 b=lo5q7R2wki1yDgsRl7rU/VF1GVAr8w/Vlq+G6UvOaPcGKqb6w8lfsb0PH9IJB/HKgj
 zrrYWJA4GFRbQTHFKfBtKlcXTCawfMd+uwaIU/zax+rAowxuoXsztv9BWUZEmTR5h4zH
 hpuISjzTq5i2fGaoIDbjZ0NTSlJ9YmovoWE38QChlPI6eaGDJlqultzwADr2iakT5u56
 rb5QNrg0SBS5M5Ijoqamczl5uRZUTYjudqFLZ+oTuU9oSCySExKCI6zDbeRyXU0nbZJc
 J6t0Vpg3qEuUKtouTfsUWTRomM97S0UZjVlR6HoXvrl530q9VMDCXvisQ4Z1AKc4+MuJ
 uHuQ==
X-Gm-Message-State: APjAAAW29F4KqyXmJXsMQ+rS6Cb8gTSM582o2/oH006/EtVrSCgMRAxD
 cmLCW+NZv++y7B/8P6IXK1n36RcbaXCivyxYzIEv2djH
X-Google-Smtp-Source: APXvYqxZNbub2wRlSXX/06gF7KIyA9RwNOfpHV/hP/vOeiXwDKk/Zxs7jonEpRDBN/JI3rcbMRpVk9eDpTSH3lCKIBs=
X-Received: by 2002:a05:620a:124f:: with SMTP id
 a15mr7211654qkl.201.1581452448834; 
 Tue, 11 Feb 2020 12:20:48 -0800 (PST)
MIME-Version: 1.0
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
 <cu7v9oka341.fsf@HIDDEN> <8736bhytn9.fsf@HIDDEN>
 <CAJ3okZ2EF5244EN3XcO-U9Uz8me42byuPMeUGeGQ_mWi6r864A@HIDDEN>
 <87sgjhx92g.fsf@HIDDEN> <cu7zhdplz84.fsf@HIDDEN>
In-Reply-To: <cu7zhdplz84.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Tue, 11 Feb 2020 21:20:37 +0100
Message-ID: <CAJ3okZ3nTmv+dD_2m7mtcHXQRsoVcxcqW1bJyRzOZfKWaq0Opw@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
To: Arun Isaac <arunisaac@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>, 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Arun,

On Tue, 11 Feb 2020 at 20:07, Arun Isaac <arunisaac@HIDDEN> wrote:

> > Arun started the discussion on guix-devel with the idea of an inverted
> > index, and I thought this would become a second index (possibly
> > implemented using SQLite).  Perhaps I misunderstood the discussion all
> > along though, let me know!  :-)
>
> No, you didn't misunderstand. That's where it began. But, while
> implementing it, I thought I might as well replace the existing cache.

An inverted index backed by Guile as you did on guix-devel or backed
by SQLite seems a good improvement for "guix search".

> Also, I've started working on guile-xapian bindings. With that, it seems
> simpler to keep the current package cache and add a xapian index only to
> speed up package search.

Xapian would be cool!
And an SQLite based index seems easier to index locally the packages
and their history. The Guix Data Service is already doing such thing
but AFAIK using PostgreSQL and some magic. :-)

http://data.guix.gnu.org/repository/1/branch/master/package/git


All the best,
simon




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 20:13:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 11 15:13:54 2020
Received: from localhost ([127.0.0.1]:57613 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j1bus-0003TN-Ij
	for submit <at> debbugs.gnu.org; Tue, 11 Feb 2020 15:13:54 -0500
Received: from mail-qt1-f172.google.com ([209.85.160.172]:44009)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1j1buq-0003TA-LG
 for 39258 <at> debbugs.gnu.org; Tue, 11 Feb 2020 15:13:53 -0500
Received: by mail-qt1-f172.google.com with SMTP id d18so8978066qtj.10
 for <39258 <at> debbugs.gnu.org>; Tue, 11 Feb 2020 12:13:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc:content-transfer-encoding;
 bh=Si1Km8WMwVjV002T5tmwksWftaiHa+fzxlF8ZpTj1+4=;
 b=umbl1g03luGy9yUh9mZxdJAOFaSHPcb0ZI+FxcS3EmA6yYG353CQhwWsQoTJ1H4zCQ
 RKFWvblFtn6OFvcnAoI9tYoSKzDseWOe+ER06yhB0RqCaY3P70VRoSoJQLDUODyceCSm
 YUh/eiKwjOqxQ4N5krQKqKLN6UH2F+oKzEG1eN4su0ZULCNR7oySVu96mNEm9X+x4sDK
 bMvWXykcGb2gjtgQCt8o3SEU9h0o1VLJBvZZPMoS4CME1pypTEo0Upl3e1ZSHWpybygt
 OaAP4xtXGn9/u1DIz5QvpKSwxXg7izjdGGa6oftrqTNRFv126ccJLrIalMrAj2UUWs+h
 UvNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc:content-transfer-encoding;
 bh=Si1Km8WMwVjV002T5tmwksWftaiHa+fzxlF8ZpTj1+4=;
 b=aNoAxEQLpvEwv1SGWE1MYt/FUf22V6gShssC3nRnPTeBH4Sm4/iyMF4B9tzM3wZtto
 xRnZKanLRfOyH2jKmFMlC/W68l396jW7qdUcYwDVs8hUl4BkTbmxfcqKEs/3/T5m62LQ
 qIPNngzEKy9bApaK5003YCnFUV314AYPDPBBnWfHMepkBigBJhqCqxZEuBAbbFskGJwc
 O6iuMGs9hkFg5O0OY3NeGD6VR4513j0raO+x3hCRPZWj9sWaR+wkJ6EQ2dBY/gSaB8Ju
 3AJYKbAxBUHwP6uwicjx0He77y9EOkuH6nk/Zjk96hkt8TrsmLwVfk1fS734c0X3eTZY
 WINw==
X-Gm-Message-State: APjAAAVaLTpha4P5xBkalgGM1d+Rms32Sw2vPUq+xquk5bz8SdL4N/DZ
 Z/IR36bf9NAjViXSty8PhSjIfXbCLaeH7dQ2adU=
X-Google-Smtp-Source: APXvYqxiBNnEvthzkE7C0NA95KeLh5JU77B+/hfSKnbYcbn4XheYphHgeYYwdFqqrS9UZxo8vKugV02PGFUHfZGWxXE=
X-Received: by 2002:ac8:740f:: with SMTP id p15mr3938408qtq.211.1581452027124; 
 Tue, 11 Feb 2020 12:13:47 -0800 (PST)
MIME-Version: 1.0
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
 <cu7v9oka341.fsf@HIDDEN> <8736bhytn9.fsf@HIDDEN>
 <CAJ3okZ2EF5244EN3XcO-U9Uz8me42byuPMeUGeGQ_mWi6r864A@HIDDEN>
 <87sgjhx92g.fsf@HIDDEN>
In-Reply-To: <87sgjhx92g.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Tue, 11 Feb 2020 21:13:35 +0100
Message-ID: <CAJ3okZ0sXg7xqFLahk+Ej_aE8bR8GmUMX9DFaRjgNV417xvAsw@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: Arun Isaac <arunisaac@HIDDEN>, 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Tue, 11 Feb 2020 at 19:39, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> > About performance, the idea was to first implement something with
> > sqlite and then see if it makes the difference. I mean I have
> > understood that.
>
> Yes.  But keep in mind that this package cache is used exclusively for
> package lookups by name.  Namely, the goal is to speed package lookup in

I agree that some confusion happens here. And this cache cannot be improved=
.


> >> However, using sqlite for keyword search as you initially proposed on
> >> guix-devel does sound like a great idea to me.
> >
> > If I understand correctly, you are proposing 2 caches, right?
> > Or are you proposing an inverted index (VHash/VList table) based on
> > trigrams to speed up the lookup?
>
> Arun started the discussion on guix-devel with the idea of an inverted
> index, and I thought this would become a second index (possibly
> implemented using SQLite).  Perhaps I misunderstood the discussion all
> along though, let me know!  :-)

Well, your suggestion is very welcome and 2 caches are required: one
for the lookup by name, as it is already (and does a good job);
another one for "guix search" speeds up, SQLite or whatever (based on
inverted index or whatever).


Thanks,
simon




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 19:07:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 11 14:07:49 2020
Received: from localhost ([127.0.0.1]:57553 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j1asu-0001tB-Vq
	for submit <at> debbugs.gnu.org; Tue, 11 Feb 2020 14:07:49 -0500
Received: from mugam.systemreboot.net ([139.59.75.54]:39568)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <arunisaac@HIDDEN>) id 1j1asr-0001t0-G8
 for 39258 <at> debbugs.gnu.org; Tue, 11 Feb 2020 14:07:47 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
 d=systemreboot.net; s=default; h=Content-Type:MIME-Version:Message-ID:Date:
 References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:
 Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:
 Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:
 List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=dWwvXq2h0nPLqZkCN248aPTXqYXwC6j1nkR+IzH3O5w=; b=LPRG7y9LDnNhbAefgUW5f6ivK
 1hPKfV7y7CG15l5GnJVMFIxyQ2IswOyCaCPWhXob9btazOnXUnI+PtyS91MiS1ZjkuXFBcMlWqlU5
 mfUbHU2Ioi45k+w36M6zQ3tCq+ag6CsW6I0ehaSDP+4wVUT8ZqbLwzmRZ8s85i+DZ9Juw=;
Received: from [192.168.2.1] (helo=steel)
 by systemreboot.net with esmtpsa  (TLS1.3) tls TLS_AES_256_GCM_SHA384
 (Exim 4.93) (envelope-from <arunisaac@HIDDEN>)
 id 1j1asb-001VB3-VN; Wed, 12 Feb 2020 00:37:30 +0530
From: Arun Isaac <arunisaac@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, zimoun
 <zimon.toutoune@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
In-Reply-To: <87sgjhx92g.fsf@HIDDEN>
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
 <cu7v9oka341.fsf@HIDDEN> <8736bhytn9.fsf@HIDDEN>
 <CAJ3okZ2EF5244EN3XcO-U9Uz8me42byuPMeUGeGQ_mWi6r864A@HIDDEN>
 <87sgjhx92g.fsf@HIDDEN>
Date: Wed, 12 Feb 2020 00:37:23 +0530
Message-ID: <cu7zhdplz84.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


>>> I would rather keep the current package cache as-is instead of inserting
>>> sqlite in here.  I don=E2=80=99t expect it to bring much compared
>>> performance-wise to the current simple cache (especially if we look at
>>> load time), and it does increase complexity quite a bit.
>>
>> Complexity is about taste. ;-)
>
> It=E2=80=99s measurable to some extent (lines of code, cyclomatic complex=
ity,
> etc.)

I agree with Ludo here. I think it does increase the complexity, and
probably unnecessarily so.

> Arun started the discussion on guix-devel with the idea of an inverted
> index, and I thought this would become a second index (possibly
> implemented using SQLite).  Perhaps I misunderstood the discussion all
> along though, let me know!  :-)

No, you didn't misunderstand. That's where it began. But, while
implementing it, I thought I might as well replace the existing cache.

Also, I've started working on guile-xapian bindings. With that, it seems
simpler to keep the current package cache and add a xapian index only to
speed up package search.

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl5C+2sACgkQLiXui2GA
K7O3DQgAnz3GCW1FVXQowVB/RqkTb4meDytN8r/LXnA06VVUHhjPuMMSV6d2xPj8
B9p0lagCeCYN8ivFvBq0X5qFwT4gBtmdXWkn2VmU4nBvUAw+uFd9d5S8bFjNeXe4
IGnmWOBuokF6ttYRkGH9Z4wStz49a/9CJcVW4jpCKyjEyodM/PJ9bVOiOTFgqpFA
TTQhMSQM1GuJO8OKhctsyZ8xpNejTW/LFPe2CgoIEQavdlMnzgL7Nl3d5NOa2rS8
VyIamqbofxI9Uj9QBXjn8HxIV5FaA2On/XmWXF/uCr44Fw1xGLyy/AFlReE9KgJE
jP/TUwhXMr/s9/e8k0bEDQMESnEfFw==
=L3bY
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 18:39:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 11 13:39:30 2020
Received: from localhost ([127.0.0.1]:57542 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j1aRW-0001BU-K8
	for submit <at> debbugs.gnu.org; Tue, 11 Feb 2020 13:39:30 -0500
Received: from eggs.gnu.org ([209.51.188.92]:51554)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1j1aRV-0001BI-0c
 for 39258 <at> debbugs.gnu.org; Tue, 11 Feb 2020 13:39:29 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:49111)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1j1aRO-0008Cx-Vw; Tue, 11 Feb 2020 13:39:23 -0500
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34260 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1j1aRO-0003u6-G6; Tue, 11 Feb 2020 13:39:22 -0500
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: zimoun <zimon.toutoune@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
 <cu7v9oka341.fsf@HIDDEN> <8736bhytn9.fsf@HIDDEN>
 <CAJ3okZ2EF5244EN3XcO-U9Uz8me42byuPMeUGeGQ_mWi6r864A@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 23 =?utf-8?Q?Pluvi=C3=B4se?= an 228 de la
 =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Tue, 11 Feb 2020 19:39:19 +0100
In-Reply-To: <CAJ3okZ2EF5244EN3XcO-U9Uz8me42byuPMeUGeGQ_mWi6r864A@HIDDEN>
 (zimoun's message of "Tue, 11 Feb 2020 19:21:55 +0100")
Message-ID: <87sgjhx92g.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 39258
Cc: Arun Isaac <arunisaac@HIDDEN>, 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

zimoun <zimon.toutoune@HIDDEN> skribis:

> On Tue, 11 Feb 2020 at 17:29, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:
>
>> I would rather keep the current package cache as-is instead of inserting
>> sqlite in here.  I don=E2=80=99t expect it to bring much compared
>> performance-wise to the current simple cache (especially if we look at
>> load time), and it does increase complexity quite a bit.
>
> Complexity is about taste. ;-)

It=E2=80=99s measurable to some extent (lines of code, cyclomatic complexit=
y,
etc.)

> About performance, the idea was to first implement something with
> sqlite and then see if it makes the difference. I mean I have
> understood that.

Yes.  But keep in mind that this package cache is used exclusively for
package lookups by name.  Namely, the goal is to speed package lookup in
operations like =E2=80=9Cguix install foo=E2=80=9D (mapping =E2=80=9Cfoo=E2=
=80=9D to the right <package>
in the right module without walking through all the modules) and =E2=80=9Cg=
uix
package -A=E2=80=9D (which is what the shell completion hooks use).

Currently =E2=80=9Cguix package -A=E2=80=9D runs in .5s on my laptop, and I=
 suspect it=E2=80=99s
going to be hard to do better just by touching the cache.

>> However, using sqlite for keyword search as you initially proposed on
>> guix-devel does sound like a great idea to me.
>
> If I understand correctly, you are proposing 2 caches, right?
> Or are you proposing an inverted index (VHash/VList table) based on
> trigrams to speed up the lookup?

Arun started the discussion on guix-devel with the idea of an inverted
index, and I thought this would become a second index (possibly
implemented using SQLite).  Perhaps I misunderstood the discussion all
along though, let me know!  :-)

Thanks,
Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 18:22:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 11 13:22:14 2020
Received: from localhost ([127.0.0.1]:57506 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j1aAo-0000iG-1s
	for submit <at> debbugs.gnu.org; Tue, 11 Feb 2020 13:22:14 -0500
Received: from mail-qt1-f171.google.com ([209.85.160.171]:35008)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1j1aAl-0000hz-Gv
 for 39258 <at> debbugs.gnu.org; Tue, 11 Feb 2020 13:22:11 -0500
Received: by mail-qt1-f171.google.com with SMTP id n17so8713605qtv.2
 for <39258 <at> debbugs.gnu.org>; Tue, 11 Feb 2020 10:22:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc:content-transfer-encoding;
 bh=BWhBwj73c9JxCfGkWaSjk77vwHXygt/320xHkfIIULE=;
 b=u9WWgDmNL2RVToEicjOHinUXd/FNdOUuhBz05RsoO5VWwSQAuRI2q9E0BUHOAw8Md/
 09jxgalfDJI6DJ+uoCUIXHVNoc300SecKkggSh/2iFEHgyTNeBaVzLVP9VGmweClnfJf
 dVe6gjaX9kQR+O03gZhNGpEOc8tan9KRtTr0ym3D6NyQymJrntXriBcaTWj8i2RGRxMA
 xPKSZx5Qd/FckJ5yJsQBYqI0cz2wnOl+hmwcQDmJ/fa65FD/HJeOL0M0SEwVb6kKqsZM
 BuBRs11R2XJqz8REb55OawKz90FxH+GB9OWpyC3svGpWoWeAx99MVldBfVGKU5ojOe88
 /kxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc:content-transfer-encoding;
 bh=BWhBwj73c9JxCfGkWaSjk77vwHXygt/320xHkfIIULE=;
 b=eaiE2WJnDFbgQnZmk+av5elqpsCChFDyn3XpbbfE3bdGiYjagI4MB4etz35SFeQGdv
 C1o0JWysxfBaTpzGAuZlc3QUicykJhtJpag1qBWLYmIF/8Eg+s0buGdv97bsinoj0mAn
 fh4aR+HLciXaQKmTP/bFEufG/YZifpcsGUUxBCQZ8ky5WWVFIIw3Iatu0fn73hIojEVd
 nqOg1leNT8OpPXPhx95ut9Z2gqDa0In/7q3H8d/0rIKisRBe5LNcOod5QpHgdvhT46k3
 LDczSH+VAkROVFYL1Yjx9zMHywdobkABW3Kta77WYYU4N1Vx8ihrKEpzVEnJ5dEkREeq
 KdEw==
X-Gm-Message-State: APjAAAXDJgd5bT7GvQVb6O30W8KjxR+zVJNPJrH6qT5H91vZ6haMY+LI
 /D3TtkJbCdbcHBEOwpnNnO6SsInwGY1KUdVZJmnVjQ==
X-Google-Smtp-Source: APXvYqxBTFgcfBi0JAIqY4Fo9W3y/JGyDyUm3jkNaohC04J8F08dZh/9IhZIdKZCCDB4fqS3VfDen8FW8xkNpuy8J3s=
X-Received: by 2002:ac8:319c:: with SMTP id h28mr16315257qte.186.1581445326078; 
 Tue, 11 Feb 2020 10:22:06 -0800 (PST)
MIME-Version: 1.0
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
 <cu7v9oka341.fsf@HIDDEN> <8736bhytn9.fsf@HIDDEN>
In-Reply-To: <8736bhytn9.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Tue, 11 Feb 2020 19:21:55 +0100
Message-ID: <CAJ3okZ2EF5244EN3XcO-U9Uz8me42byuPMeUGeGQ_mWi6r864A@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: Arun Isaac <arunisaac@HIDDEN>, 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludo,

On Tue, 11 Feb 2020 at 17:29, Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> I would rather keep the current package cache as-is instead of inserting
> sqlite in here.  I don=E2=80=99t expect it to bring much compared
> performance-wise to the current simple cache (especially if we look at
> load time), and it does increase complexity quite a bit.

Complexity is about taste. ;-)
About performance, the idea was to first implement something with
sqlite and then see if it makes the difference. I mean I have
understood that.

> However, using sqlite for keyword search as you initially proposed on
> guix-devel does sound like a great idea to me.

If I understand correctly, you are proposing 2 caches, right?
Or are you proposing an inverted index (VHash/VList table) based on
trigrams to speed up the lookup?

Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 16:29:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 11 11:29:42 2020
Received: from localhost ([127.0.0.1]:57411 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j1YPu-0004RI-Ij
	for submit <at> debbugs.gnu.org; Tue, 11 Feb 2020 11:29:42 -0500
Received: from eggs.gnu.org ([209.51.188.92]:40322)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1j1YPs-0004R6-FH
 for 39258 <at> debbugs.gnu.org; Tue, 11 Feb 2020 11:29:41 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:46770)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1j1YPm-0002am-Gq; Tue, 11 Feb 2020 11:29:34 -0500
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=34126 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1j1YPl-000741-De; Tue, 11 Feb 2020 11:29:34 -0500
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Arun Isaac <arunisaac@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
 <cu7v9oka341.fsf@HIDDEN>
Date: Tue, 11 Feb 2020 17:29:30 +0100
In-Reply-To: <cu7v9oka341.fsf@HIDDEN> (Arun Isaac's message of "Thu, 
 06 Feb 2020 07:28:22 +0530")
Message-ID: <8736bhytn9.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org, zimoun <zimon.toutoune@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Hello Arun!

Arun Isaac <arunisaac@HIDDEN> skribis:

> From 4c883fcff1f44339b28df6ccdb2b10c906439e3d Mon Sep 17 00:00:00 2001
> From: Arun Isaac <arunisaac@HIDDEN>
> Date: Tue, 21 Jan 2020 20:45:43 +0530
> Subject: [PATCH] fast search

[...]

> --- a/gnu/packages.scm
> +++ b/gnu/packages.scm
> @@ -43,6 +43,7 @@
>    #:use-module (srfi srfi-34)
>    #:use-module (srfi srfi-35)
>    #:use-module (srfi srfi-39)
> +  #:use-module (sqlite3)
>    #:export (search-patch
>              search-patches
>              search-auxiliary-file
> @@ -204,10 +205,8 @@ PROC is called along these lines:
>  PROC can use #:allow-other-keys to ignore the bits it's not interested i=
n.
>  When a package cache is available, this procedure does not actually load=
 any
>  package module."
> -  (define cache
> -    (load-package-cache (current-profile)))
> -
> -  (if (and cache (cache-is-authoritative?))
> +  (if (and (cache-is-authoritative?)
> +           (current-profile))
>        (vhash-fold (lambda (name vector result)
>                      (match vector
>                        (#(name version module symbol outputs
> @@ -220,7 +219,7 @@ package module."
>                               #:supported? supported?
>                               #:deprecated? deprecated?))))
>                    init
> -                  cache)
> +                  (cache-lookup (current-profile)))
>        (fold-packages (lambda (package result)
>                         (proc (package-name package)
>                               (package-version package)
> @@ -252,31 +251,7 @@ is guaranteed to never traverse the same package twi=
ce."
>=20=20
>  (define %package-cache-file
>    ;; Location of the package cache.
> -  "/lib/guix/package.cache")
> -
> -(define load-package-cache

[...]

> +(define* (cache-lookup profile #:optional name)
>    "Lookup package NAME in CACHE.  Return a list sorted in increasing ver=
sion
>  order."
>    (define (package-version<? v1 v2)
>      (version>? (vector-ref v2 1) (vector-ref v1 1)))
>=20=20
> -  (sort (vhash-fold* cons '() name cache)
> -        package-version<?))
> +  (define (int->boolean n)
> +    (case n
> +      ((0) #f)
> +      ((1) #t)))
> +
> +  (define (string->list str)
> +    (call-with-input-string str read))
> +
> +  (define select-statement
> +    (string-append
> +     "SELECT name, version, module, symbol, outputs, supported, supersed=
ed, locationFile, locationLine, locationColumn from packages"
> +     (if name " WHERE name =3D :name" "")))

I would rather keep the current package cache as-is instead of inserting
sqlite in here.  I don=E2=80=99t expect it to bring much compared
performance-wise to the current simple cache (especially if we look at
load time), and it does increase complexity quite a bit.

However, using sqlite for keyword search as you initially proposed on
guix-devel does sound like a great idea to me.

WDYT?

Thanks,
Ludo=E2=80=99.




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 6 Feb 2020 01:58:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 05 20:58:49 2020
Received: from localhost ([127.0.0.1]:46966 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1izWRN-0006k1-89
	for submit <at> debbugs.gnu.org; Wed, 05 Feb 2020 20:58:49 -0500
Received: from mugam.systemreboot.net ([139.59.75.54]:35458)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <arunisaac@HIDDEN>) id 1izWRI-0006jp-O3
 for 39258 <at> debbugs.gnu.org; Wed, 05 Feb 2020 20:58:47 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
 d=systemreboot.net; s=default; h=Content-Type:MIME-Version:Message-ID:Date:
 References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:
 Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:
 Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:
 List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=x5WRND8ZKSjnqBSnMgncM63QAmvrxCWp/A8/EJmx3OM=; b=LH/Q70zv8XIRoRMAowYy+i66e
 D3QS04ZxdKQuP+Hi59m1BKIwKbBTIutoMVolctvUBS9yWuPG3uMb9+m6yeQL3YZOOEKU15kKZDOm0
 r/o1z+Q07axTi7eSIz2O3YncvURqqOL6tvtBP6xrYgAB5voAH7eNO0Q5apyIylXp7f90M=;
Received: from [192.168.2.1] (helo=steel)
 by systemreboot.net with esmtpsa  (TLS1.3) tls TLS_AES_256_GCM_SHA384
 (Exim 4.93) (envelope-from <arunisaac@HIDDEN>)
 id 1izWR8-000zOQ-NP; Thu, 06 Feb 2020 07:28:34 +0530
From: Arun Isaac <arunisaac@HIDDEN>
To: zimoun <zimon.toutoune@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
In-Reply-To: <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
 <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
Date: Thu, 06 Feb 2020 07:28:22 +0530
Message-ID: <cu7v9oka341.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="

--=-=-=
Content-Type: text/plain


>> Thank you, this was useful. I was able to catch and report the error. I
>
> Where have you reported the error?

I reported the error to the derivation log. For example, if the
derivation for the guix-package-cache derivation is
/gnu/store/cyf2h3frcjxm147dii5qic8d6kpm39nq-guix-package-cache.drv, the
log file will be at
/var/log/guix/drvs/cy/f2h3frcjxm147dii5qic8d6kpm39nq-guix-package-cache.drv.bz2. Notice
that the directory name under drvs is the first two letters of the hash,
and the file name under that directory is the remaining letters.

Also please find attached a dump of my code so far.

>> This could be a permission error, or something to do with the existence
>> or lack thereof of certain directories (such as /var) in the chroot of
>> the build daemon. I'm still figuring it out.
>
> Hum? And this should explain why it is working with the REPL and not
> with the CLI, right?

This could expalin it, but I am not sure if this is the correct
explanation.

>> I'm also in half a mind to get some guile xapian bindings ready so we
>> can just do that instead of messing with sqlite here. But, let's
>> see. :-P
>
> Cool!
> Let me know if you push something somewhere.

Sure, will let you know.


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=0001-fast-search.patch
Content-Transfer-Encoding: quoted-printable

From=204c883fcff1f44339b28df6ccdb2b10c906439e3d Mon Sep 17 00:00:00 2001
From: Arun Isaac <arunisaac@HIDDEN>
Date: Tue, 21 Jan 2020 20:45:43 +0530
Subject: [PATCH] fast search

=2D--
 build-aux/build-self.scm |   5 +
 gnu/packages.scm         | 234 +++++++++++++++++++++++++--------------
 2 files changed, 155 insertions(+), 84 deletions(-)

diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index fc13032b73..c123ad3b11 100644
=2D-- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -264,6 +264,9 @@ interface (FFI) of Guile.")
   (define fake-git
     (scheme-file "git.scm" #~(define-module (git))))
=20
+  (define fake-sqlite3
+    (scheme-file "sqlite3.scm" #~(define-module (sqlite3))))
+
   (with-imported-modules `(((guix config)
                             =3D> ,(make-config.scm))
=20
@@ -278,6 +281,8 @@ interface (FFI) of Guile.")
                            ;; (git) to placate it.
                            ((git) =3D> ,fake-git)
=20
+                           ((sqlite3) =3D> ,fake-sqlite3)
+
                            ,@(source-module-closure `((guix store)
                                                       (guix self)
                                                       (guix derivations)
diff --git a/gnu/packages.scm b/gnu/packages.scm
index d22c992bb1..0ae5b84284 100644
=2D-- a/gnu/packages.scm
+++ b/gnu/packages.scm
@@ -43,6 +43,7 @@
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
   #:use-module (srfi srfi-39)
+  #:use-module (sqlite3)
   #:export (search-patch
             search-patches
             search-auxiliary-file
@@ -204,10 +205,8 @@ PROC is called along these lines:
 PROC can use #:allow-other-keys to ignore the bits it's not interested in.
 When a package cache is available, this procedure does not actually load a=
ny
 package module."
=2D  (define cache
=2D    (load-package-cache (current-profile)))
=2D
=2D  (if (and cache (cache-is-authoritative?))
+  (if (and (cache-is-authoritative?)
+           (current-profile))
       (vhash-fold (lambda (name vector result)
                     (match vector
                       (#(name version module symbol outputs
@@ -220,7 +219,7 @@ package module."
                              #:supported? supported?
                              #:deprecated? deprecated?))))
                   init
=2D                  cache)
+                  (cache-lookup (current-profile)))
       (fold-packages (lambda (package result)
                        (proc (package-name package)
                              (package-version package)
@@ -252,31 +251,7 @@ is guaranteed to never traverse the same package twice=
."
=20
 (define %package-cache-file
   ;; Location of the package cache.
=2D  "/lib/guix/package.cache")
=2D
=2D(define load-package-cache
=2D  (mlambda (profile)
=2D    "Attempt to load the package cache.  On success return a vhash keyed=
 by
=2Dpackage names.  Return #f on failure."
=2D    (match profile
=2D      (#f #f)
=2D      (profile
=2D       (catch 'system-error
=2D         (lambda ()
=2D           (define lst
=2D             (load-compiled (string-append profile %package-cache-file)))
=2D           (fold (lambda (item vhash)
=2D                   (match item
=2D                     (#(name version module symbol outputs
=2D                             supported? deprecated?
=2D                             file line column)
=2D                      (vhash-cons name item vhash))))
=2D                 vlist-null
=2D                 lst))
=2D         (lambda args
=2D           (if (=3D ENOENT (system-error-errno args))
=2D               #f
=2D               (apply throw args))))))))
+  "/lib/guix/package-cache.sqlite")
=20
 (define find-packages-by-name/direct              ;bypass the cache
   (let ((packages (delay
@@ -297,25 +272,57 @@ decreasing version order."
                     matching)
             matching)))))
=20
=2D(define (cache-lookup cache name)
+(define* (cache-lookup profile #:optional name)
   "Lookup package NAME in CACHE.  Return a list sorted in increasing versi=
on
 order."
   (define (package-version<? v1 v2)
     (version>? (vector-ref v2 1) (vector-ref v1 1)))
=20
=2D  (sort (vhash-fold* cons '() name cache)
=2D        package-version<?))
+  (define (int->boolean n)
+    (case n
+      ((0) #f)
+      ((1) #t)))
+
+  (define (string->list str)
+    (call-with-input-string str read))
+
+  (define select-statement
+    (string-append
+     "SELECT name, version, module, symbol, outputs, supported, superseded=
, locationFile, locationLine, locationColumn from packages"
+     (if name " WHERE name =3D :name" "")))
+
+  (define cache-file
+    (string-append profile %package-cache-file))
+
+  (let* ((db (sqlite-open cache-file SQLITE_OPEN_READONLY))
+         (statement (sqlite-prepare db select-statement)))
+    (when name
+      (sqlite-bind-arguments statement #:name name))
+    (let ((result (sqlite-fold (lambda (v result)
+                                 (match v
+                                   (#(name version module symbol outputs s=
upported superseded file line column)
+                                    (cons
+                                     (vector name
+                                             version
+                                             (string->list module)
+                                             (string->symbol symbol)
+                                             (string->list outputs)
+                                             (int->boolean supported)
+                                             (int->boolean superseded)
+                                             (list file line column))
+                                     result))))
+                               '() statement)))
+      (sqlite-finalize statement)
+      (sqlite-close db)
+      (sort result package-version<?))))
=20
 (define* (find-packages-by-name name #:optional version)
   "Return the list of packages with the given NAME.  If VERSION is not #f,
 then only return packages whose version is prefixed by VERSION, sorted in
 decreasing version order."
=2D  (define cache
=2D    (load-package-cache (current-profile)))
=2D
=2D  (if (and (cache-is-authoritative?) cache)
=2D      (match (cache-lookup cache name)
=2D        (#f #f)
+  (if (and (cache-is-authoritative?)
+           (current-profile))
+      (match (cache-lookup (current-profile) name)
         ((#(_ versions modules symbols _ _ _ _ _ _) ...)
          (fold (lambda (version* module symbol result)
                  (if (or (not version)
@@ -331,12 +338,9 @@ decreasing version order."
 (define* (find-package-locations name #:optional version)
   "Return a list of version/location pairs corresponding to each package
 matching NAME and VERSION."
=2D  (define cache
=2D    (load-package-cache (current-profile)))
=2D
=2D  (if (and cache (cache-is-authoritative?))
=2D      (match (cache-lookup cache name)
=2D        (#f '())
+  (if (and (cache-is-authoritative?)
+           (current-profile))
+      (match (cache-lookup (current-profile) name)
         ((#(name versions modules symbols outputs
                  supported? deprecated?
                  files lines columns) ...)
@@ -372,6 +376,33 @@ VERSION."
 ;; Prevent Guile 3 from inlining this procedure so we can mock it in tests.
 (set! find-best-packages-by-name find-best-packages-by-name)
=20
+;; (generate-package-cache "/tmp/test")
+
+;; XXX: missing in guile-sqlite3@HIDDEN
+(define SQLITE_BUSY 5)
+
+(define (call-with-transaction db proc)
+  "Start a transaction with DB (make as many attempts as necessary) and run
+PROC.  If PROC exits abnormally, abort the transaction, otherwise commit t=
he
+transaction after it finishes."
+  (catch 'sqlite-error
+    (lambda ()
+      ;; We use begin immediate here so that if we need to retry, we
+      ;; figure that out immediately rather than because some SQLITE_BUSY
+      ;; exception gets thrown partway through PROC - in which case the
+      ;; part already executed (which may contain side-effects!) would be
+      ;; executed again for every retry.
+      (sqlite-exec db "begin immediate;")
+      (let ((result (proc)))
+        (sqlite-exec db "commit;")
+        result))
+    (lambda (key who error description)
+      (if (=3D error SQLITE_BUSY)
+          (call-with-transaction db proc)
+          (begin
+            (sqlite-exec db "rollback;")
+            (throw 'sqlite-error who error description))))))
+
 (define (generate-package-cache directory)
   "Generate under DIRECTORY a cache of all the available packages.
=20
@@ -381,49 +412,84 @@ reducing the memory footprint."
   (define cache-file
     (string-append directory %package-cache-file))
=20
=2D  (define (expand-cache module symbol variable result+seen)
+  (define schema
+    "CREATE TABLE packages (name text,
+version text,
+module text,
+symbol text,
+outputs text,
+supported int,
+superseded int,
+locationFile text,
+locationLine int,
+locationColumn int);
+CREATE VIRTUAL TABLE packageSearch USING fts5(name, searchText);")
+
+  (define insert-statement
+    "INSERT INTO packages(name, version, module, symbol, outputs, supporte=
d, superseded, locationFile, locationLine, locationColumn)
+VALUES(:name, :version, :module, :symbol, :outputs, :supported, :supersede=
d, :locationfile, :locationline, :locationcolumn)")
+
+  (define insert-package-search-statement
+    "INSERT INTO packageSearch(name, searchText) VALUES(:name, :searchtext=
)")
+
+  (define (boolean->int x)
+    (if x 1 0))
+
+  (define (list->string x)
+    (call-with-output-string (cut write x <>)))
+
+  (define (insert-package db module symbol variable seen)
     (match (false-if-exception (variable-ref variable))
       ((? package? package)
=2D       (match result+seen
=2D         ((result . seen)
=2D          (if (or (vhash-assq package seen)
=2D                  (hidden-package? package))
=2D              (cons result seen)
=2D              (cons (cons `#(,(package-name package)
=2D                             ,(package-version package)
=2D                             ,(module-name module)
=2D                             ,symbol
=2D                             ,(package-outputs package)
=2D                             ,(->bool (supported-package? package))
=2D                             ,(->bool (package-superseded package))
=2D                             ,@(let ((loc (package-location package)))
=2D                                 (if loc
=2D                                     `(,(location-file loc)
=2D                                       ,(location-line loc)
=2D                                       ,(location-column loc))
=2D                                     '(#f #f #f))))
=2D                          result)
=2D                    (vhash-consq package #t seen))))))
=2D      (_
=2D       result+seen)))
=2D
=2D  (define exp
=2D    (first
=2D     (fold-module-public-variables* expand-cache
=2D                                    (cons '() vlist-null)
=2D                                    (all-modules (%package-module-path)
=2D                                                 #:warn
=2D                                                 warn-about-load-error))=
))
+       (cond
+        ((or (vhash-assq package seen)
+             (hidden-package? package))
+         seen)
+        (else
+         (let ((statement (sqlite-prepare db insert-statement)))
+           (sqlite-bind-arguments statement
+                                  #:name (package-name package)
+                                  #:version (package-version package)
+                                  #:module (list->string (module-name modu=
le))
+                                  #:symbol (symbol->string symbol)
+                                  #:outputs (list->string (package-outputs=
 package))
+                                  #:supported (boolean->int (supported-pac=
kage? package))
+                                  #:superseded (boolean->int (package-supe=
rseded package))
+                                  #:locationfile (cond
+                                                  ((package-location packa=
ge) =3D> location-file)
+                                                  (else #f))
+                                  #:locationline (cond
+                                                  ((package-location packa=
ge) =3D> location-line)
+                                                  (else #f))
+                                  #:locationcolumn (cond
+                                                    ((package-location pac=
kage) =3D> location-column)
+                                                    (else #f)))
+           (sqlite-fold cons '() statement)
+           (sqlite-finalize statement))
+         (let ((statement (sqlite-prepare db insert-package-search-stateme=
nt)))
+           (sqlite-bind-arguments statement
+                                  #:name (package-name package)
+                                  #:searchtext (package-description packag=
e))
+           (sqlite-fold cons '() statement)
+           (sqlite-finalize statement))
+         (vhash-consq package #t seen))))
+      (_ seen)))
=20
   (mkdir-p (dirname cache-file))
=2D  (call-with-output-file cache-file
=2D    (lambda (port)
=2D      ;; Store the cache as a '.go' file.  This makes loading fast and r=
educes
=2D      ;; heap usage since some of the static data is directly mmapped.
=2D      (put-bytevector port
=2D                      (compile `'(,@exp)
=2D                               #:to 'bytecode
=2D                               #:opts '(#:to-file? #t)))))
+  (let ((tmp (string-append (dirname cache-file) "/tmp")))
+    (mkdir-p tmp)
+    (setenv "SQLITE_TMPDIR" tmp))
+  (let ((db (sqlite-open cache-file)))
+    (sqlite-exec db schema)
+    (call-with-transaction db
+        (lambda ()
+          (fold-module-public-variables* (cut insert-package db <> <> <> <=
>)
+                                         vlist-null
+                                         (all-modules (%package-module-pat=
h)
+                                                      #:warn
+                                                      warn-about-load-erro=
r))))
+    (sqlite-close db))
+
   cache-file)
=20
 
=2D-=20
2.23.0


--=-=-=--

--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl47cr4ACgkQLiXui2GA
K7OLPwf9FgQ/N8AazOSXRwuXn8N+1QZ0b0G1bq2ZpQYppHpBl0e8SgmTyYXpE4za
U2tBfJEMTUXW7YUZD/36DKNpN5JUD1c4Hnsn/BDqRQFFXt4agzTE3PhtOV2gFLjC
wYH/i9xtLoRM+7g36/ZYAmTfAZoiFro4A3kq1ndy8nQCcf++v/v3tQvFVWn3ywKT
Fig63/wm8hXz7kP0+fi6MVKlhKe05VDBXxZCNfM7tSjmEw2qFNmwALUncWUyN9Sz
uF0aU+/JzPUF6kvXMZR9cQxBI5ISZ+gxsdg7rikmwso9jvyxRbEDFAck4Etv9j32
drcNfNH4ZSUaZY/NcxIpzu/yI9dKKA==
=M4NZ
-----END PGP SIGNATURE-----
--==-=-=--




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 4 Feb 2020 10:19:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 04 05:19:31 2020
Received: from localhost ([127.0.0.1]:42635 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iyvIp-0000tR-JW
	for submit <at> debbugs.gnu.org; Tue, 04 Feb 2020 05:19:31 -0500
Received: from mail-qk1-f169.google.com ([209.85.222.169]:44536)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1iyvIo-0000t9-5R
 for 39258 <at> debbugs.gnu.org; Tue, 04 Feb 2020 05:19:30 -0500
Received: by mail-qk1-f169.google.com with SMTP id v195so17284398qkb.11
 for <39258 <at> debbugs.gnu.org>; Tue, 04 Feb 2020 02:19:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=NuzaTIrcWuV1Td2S53i/wfjU4uvdID+cSbqGU8HW9Ls=;
 b=eoX040qBxEr23eWFOQm6KYCq45W1h0wPf+MhW5jyIWfyP0EXdnyD9mVLYr7tqKvttJ
 RguoAwYWA9QjOakSA7smbNZix2kmkertldHvVGXDELjI3N3UIEDD1QcfOGpcp4YBMxBt
 ClCZLosxMxtIGpsxzOCkOidlyB32emJeVtrQVamUsrHDmezJp5hb5boP8um3eJIwCKgk
 QY756DTJpCBFyhNgJ9Le9GFdPp+tKwCmeSoqtF+/Nw/6vg8UTKIyujhNh76bE7KhJIYO
 UDjY0dwkrCwrFQifjiwd5m5FjH17yAZS2f016RtNHK9bNsESQQh9t1HS/58oWEQ8/wUz
 FD2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=NuzaTIrcWuV1Td2S53i/wfjU4uvdID+cSbqGU8HW9Ls=;
 b=sStLv7LkVy4fN53+VnRLdF7RvuqP0N9xLf/LIeWRgqHv8SZuyesb/IqqXaj4BlKxDi
 vjANOx9rgeBrckgFxPRX3fFqOb3tUVTGBPLADZNQYkCjdo5vdjiEIZrOQgLPpbTUEILz
 4CjpTYPhhnB/P0a71yHdW2s1RyNVQ9Mph0O23zkZ1NihHHwQAHsvmkPSbSclfJXbmp7f
 JYUuXNLoUnIELinjUj3xPm56bJxh2D8Abli3Svu96XZcFhJaxUDePjEnPKdlw/kiytwf
 LbjUMi9eZ45UpwRXObwV36ONyIPDmzZvDiFRtSBwSq540J33AP/VvR+U6hsfym/EGaHy
 LZbA==
X-Gm-Message-State: APjAAAWWG78SjCeebKU8tYEkvsVCHJQC/3Qh+Cj787q348jiOwt6G8i2
 aBHbtX12NIoakPZ/huFomDMGA7qLDJabpgj0gx89lQ==
X-Google-Smtp-Source: APXvYqzqdiAP3XECLyP2fPtX9EjgBUFbpvIaWuPlEzwuwC6FAYfaC+f+8FZy8vHJlReEghxcNjIZ4qmfAMv0sjCmliE=
X-Received: by 2002:a37:63c7:: with SMTP id
 x190mr27760819qkb.232.1580811564425; 
 Tue, 04 Feb 2020 02:19:24 -0800 (PST)
MIME-Version: 1.0
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
 <cu7y2tkadwi.fsf@HIDDEN>
In-Reply-To: <cu7y2tkadwi.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Tue, 4 Feb 2020 11:19:13 +0100
Message-ID: <CAJ3okZ37rFh=SB-f6nqc0kMpCai=pw2D0-GFiakXY1p4FHHftA@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
To: Arun Isaac <arunisaac@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On Sun, 2 Feb 2020 at 22:16, Arun Isaac <arunisaac@HIDDEN> wrote:

> Thank you, this was useful. I was able to catch and report the error. I

Where have you reported the error?


> also found the log file for the guix-package-cache profile hook. It says
>
> (repl-version 0 0)
> Generating package cache for '/gnu/store/b6f9b5qbcn4r932whrr6m15rdimbgrhs-profile'...
> (exception sqlite-error (value sqlite-open) (value 14) (value "Unable to open the database file"))
>
> This could be a permission error, or something to do with the existence
> or lack thereof of certain directories (such as /var) in the chroot of
> the build daemon. I'm still figuring it out.

Hum? And this should explain why it is working with the REPL and not
with the CLI, right?


> I'm also in half a mind to get some guile xapian bindings ready so we
> can just do that instead of messing with sqlite here. But, let's
> see. :-P

Cool!
Let me know if you push something somewhere.


Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 2 Feb 2020 21:16:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 02 16:16:40 2020
Received: from localhost ([127.0.0.1]:40646 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iyMbf-0002zK-Vo
	for submit <at> debbugs.gnu.org; Sun, 02 Feb 2020 16:16:40 -0500
Received: from mugam.systemreboot.net ([139.59.75.54]:47868)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <arunisaac@HIDDEN>) id 1iyMbd-0002z9-Cg
 for 39258 <at> debbugs.gnu.org; Sun, 02 Feb 2020 16:16:39 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
 d=systemreboot.net; s=default; h=Content-Type:MIME-Version:Message-ID:Date:
 References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:
 Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:
 Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:
 List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=fTtfIfuOCVGXh2uLN340Z4DGMr/Nno1Tk/gevwJV00E=; b=mXRVK0ouzN6JWvA7W3DClT0tI
 Ka9AHo6YT4e+9xGOfFQgvBh5GMpBT3RZVY76/ZkMH+/cmUcnhtrZoRvPy5KKkQfMYP9D4zoeWra6P
 dSrfYF7MCRJKY0BgwV8qJ43ZLGqIWwe1rbnTGZk29NlhFOgeIRiNhijEYai57EOT+uzbY=;
Received: from [192.168.2.1] (helo=steel)
 by systemreboot.net with esmtpsa  (TLS1.3) tls TLS_AES_256_GCM_SHA384
 (Exim 4.93) (envelope-from <arunisaac@HIDDEN>)
 id 1iyMbT-000lGl-8p; Mon, 03 Feb 2020 02:46:27 +0530
From: Arun Isaac <arunisaac@HIDDEN>
To: zimoun <zimon.toutoune@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
In-Reply-To: <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
 <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
Date: Mon, 03 Feb 2020 02:46:13 +0530
Message-ID: <cu7y2tkadwi.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain


> Danny told me something like:
>
>   (catch (sqlite-error
>
> I have not tried yet.

Thank you, this was useful. I was able to catch and report the error. I
also found the log file for the guix-package-cache profile hook. It says

(repl-version 0 0)
Generating package cache for '/gnu/store/b6f9b5qbcn4r932whrr6m15rdimbgrhs-profile'...
(exception sqlite-error (value sqlite-open) (value 14) (value "Unable to open the database file"))

This could be a permission error, or something to do with the existence
or lack thereof of certain directories (such as /var) in the chroot of
the build daemon. I'm still figuring it out.

I'm also in half a mind to get some guile xapian bindings ready so we
can just do that instead of messing with sqlite here. But, let's
see. :-P

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl43PB4ACgkQLiXui2GA
K7O0Hwf/bpqqdDy9FynrZNWUqne4w/IQ5nqJLIxZJ30IcbBu8TxWZp31+UjrBE5M
lJI9GQglb+K1atySjm1ZPhQ/uzsiheXF+ENULTP6HeC1bHgZaWQpPhzmknMEfeoG
d1NU3ePQHLiE5IjPm6PptdYnbzCRmcmGupD23VHCIPbr1LVFSS7ajsFiKofehDyt
LgpANYLVDokhe0w99Oa6Htf4C3WwkICMPCVJA01LborzdVs65Kso+HpQkBh3egjw
+m8sdF+xoPBYKSR+GlhZjXTWFKv8l/btMu6UCJmjRFflN+jqHT01HCiWmeXtESV1
zowEz7TtWQOjPhfJeW7igm73ECKkGA==
=62li
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 31 Jan 2020 15:15:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 31 10:15:48 2020
Received: from localhost ([127.0.0.1]:37856 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ixY1M-0003Rx-Hg
	for submit <at> debbugs.gnu.org; Fri, 31 Jan 2020 10:15:48 -0500
Received: from mail-qk1-f177.google.com ([209.85.222.177]:43958)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1ixY1K-0003Rh-RT
 for 39258 <at> debbugs.gnu.org; Fri, 31 Jan 2020 10:15:47 -0500
Received: by mail-qk1-f177.google.com with SMTP id j20so6823389qka.10
 for <39258 <at> debbugs.gnu.org>; Fri, 31 Jan 2020 07:15:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=uLjKv6khMsD8lK7a1k/F7GVdxZN0Kl+Rk/DrW/HFIuE=;
 b=l9egGbJoTXnY3x2FpjS3uT960Y/To8ZM0rNnuz8NWdLFTmv8p49OY5C3zzRjCeS0wt
 nTtO+hi18b/XivDU6k1XE2nc2QUtiIcotOIr3+z1/Idi2F0o3ZTmQe5sXC17OOzwV0nM
 VVLwsggka4dWfplUdkR3uvrSZmvfprTiVYm5JmVDPbX5KVhgv29Y7aKTNIwfr8Rxnc61
 sxklW0N90BtpB3/Dzj6aAZtqaL64s7mCsoVQAkGK39lsvqLWHc1TRslPpHPg7QO/NVgJ
 UDq8Yo5jTbhX3pnDZvob/HWpWjQbMTTh7CQfBS9Oj1f+45p7odWefbV7NPUP7i/T5izo
 GN+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=uLjKv6khMsD8lK7a1k/F7GVdxZN0Kl+Rk/DrW/HFIuE=;
 b=YrhNembiAhbYO2GDu5C7Kpu4FUSm9h3lH73vt9sqSYP39SeM3jqHs0wr6la9IPPW69
 9uMPzR2nd4+Y/RLkfn9IyoRvNCj6v+EIaEs5Sx99qv2HUdz0ttQTH2gmMXf533DIPzq/
 yb8ZA80i4C2F/7IAAQrox6XaoqP1xyJo+WCncCf0nXPE4EuqQ/r9Yh/5YFuohR5U67G9
 73JH0AJT6+o0iABn8UQuSOL2WhBMnENBeuTPnReINqQsoS8/6KL0IsSIdpSS+D2vMe7M
 NcZQg/mDQSOvn3SOGafGjOXhan7KrbZscUQnSMZK7673MI+vQwTwPdioQQUA1n8AM0Ug
 /e6g==
X-Gm-Message-State: APjAAAW0QDF2IkkG0JKLYpT3hslmOaIX8Gi3UwXHMD2QSDphWhB3a8yM
 8i/0fcpj4Ln8orvwE3MqkHn63ZBL8AWkA52tnty5sQ==
X-Google-Smtp-Source: APXvYqyehJXpMxoJQ7/aFAj6F0UOgcZtfkAmkZ4wFW/9w+fuEoEXRCkfVxFbyRqYLHLhLfV+kzbx13CxuEmt3r4CXVk=
X-Received: by 2002:a37:710:: with SMTP id 16mr10250950qkh.304.1580483741286; 
 Fri, 31 Jan 2020 07:15:41 -0800 (PST)
MIME-Version: 1.0
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
 <cu74kwdawc7.fsf@HIDDEN>
In-Reply-To: <cu74kwdawc7.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Fri, 31 Jan 2020 13:48:16 +0100
Message-ID: <CAJ3okZ0GXS9ibh__1DFsuCCjO82R6-YXtPAOF3GoquN=sSsaWA@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
To: Arun Isaac <arunisaac@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On Thu, 30 Jan 2020 at 14:49, Arun Isaac <arunisaac@HIDDEN> wrote:

> >> Any ideas what's going on?
[...]
> Do you know of any way sqlite can create an error log to report what's
> going on? That might really help debug this issue.

Danny told me something like:

  (catch (sqlite-error

I have not tried yet.



> > The docstring of 'cache-lookup' is not coherent anymore. :-)
>
> Yes, I haven't gotten around to fixing up all those yet. I thought I'll
> get the code working first.

Yes, I imagine. Just to notice. :-)



All the best,
simon




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 30 Jan 2020 13:49:26 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 30 08:49:26 2020
Received: from localhost ([127.0.0.1]:35060 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ixACE-0004jZ-2k
	for submit <at> debbugs.gnu.org; Thu, 30 Jan 2020 08:49:26 -0500
Received: from mugam.systemreboot.net ([139.59.75.54]:55156)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <arunisaac@HIDDEN>) id 1ixACA-0004jL-Fw
 for 39258 <at> debbugs.gnu.org; Thu, 30 Jan 2020 08:49:24 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
 d=systemreboot.net; s=default; h=Content-Type:MIME-Version:Message-ID:Date:
 References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:
 Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:
 Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:
 List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=po6WSlvm76W+cyGEAyCt0xLGxVCdJ4XqVkHzxhbvwms=; b=b6/ogKEfMeZiLTZlTcqX8goIO
 e7LR9+siJ21xW/bpjup5wdEI6vG7m9IqLXeDg3XSYfmqaY6cq/AfyDAgIMD36EfZRS6CK8b9asRci
 2dDEymoSSndhamPODvbTDPt8N19e9a0hKt83HH5Ntodvr78+1FpMGLO2bGV7RriraLaXA=;
Received: from [192.168.2.1] (helo=steel)
 by systemreboot.net with esmtpsa  (TLS1.3) tls TLS_AES_256_GCM_SHA384
 (Exim 4.93) (envelope-from <arunisaac@HIDDEN>)
 id 1ixABz-000WUq-Mj; Thu, 30 Jan 2020 19:19:11 +0530
From: Arun Isaac <arunisaac@HIDDEN>
To: zimoun <zimon.toutoune@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
In-Reply-To: <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
References: <cu7pnfaar36.fsf@HIDDEN>
 <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
Date: Thu, 30 Jan 2020 19:18:56 +0530
Message-ID: <cu74kwdawc7.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain


> Hum? weird...
> Is it possible that a module is loaded when Guile repl is used and not
> with Guix pull?

It could be. But I don't know how to confirm this theory.

> What about "guix repl"?

I just tried 'guix repl'. It worked correctly, just like guile repl.

>> If I try to populate the database with each package record being
>> inserted in its own transaction, at least some of the insertions
>
> You mean 'commit' the database after each insertion, right?

Yes, that is what I mean.

>> work. But the journal file still lingers. My unverified guess is that
>> everything except the last transaction was successful.
>
> And this does not happen with the repl, right?

No, this does not happen with the repl.

>> Any ideas what's going on?
>
> I have no idea.
> Weird.

Do you know of any way sqlite can create an error log to report what's
going on? That might really help debug this issue.

> What about adding 'last-insert-row-id' from 'guix/store/database.scm'?
> I mean without really understanding and just grepping
> 'sqlite-finalize' spots that 'last-insert-row-id' seems often around.
> :-)

I tried this just now, but still the journal lingers.

> Otherwise, 'list->string' is defined twice. And the first one is not
> necessary, I guess.

Ah, thanks for catching this!

> The docstring of 'cache-lookup' is not coherent anymore. :-)

Yes, I haven't gotten around to fixing up all those yet. I thought I'll
get the code working first.

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl4y3sgACgkQLiXui2GA
K7O5mQf/QRSZdYPR4DAVXlfg04OEoCKgTi4m9SLL3ByJjMazxfuR9gXS6o97Sr/p
5IH2tchH2WwXM/ZN1R8a5utlrWR49azy9Z9uW+b2Bk48aQkp2l5uLz+uQDekBZ+z
o7gIYA+dnuGtJMlLG90kB2P6TP1LoaOm0rZyOCRPb4GoNnIjv3v9yxEU2Z7vMcbQ
6AOd/lj2Owr4Ur3BrXn5+XNfZO34uXMHHleZq/R7cwxPqmK8xj7xsArPG0/5o2XN
vJYVe9AC8J2DMEcyU5wafxs9DebxnuZZAB3umSQeUuFiLIT34fkBSQmW2vEIvaC2
R5zEZ9E6PqcpRz8K7rp18z8hcX+jVw==
=Akh8
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at 39258 <at> debbugs.gnu.org:


Received: (at 39258) by debbugs.gnu.org; 29 Jan 2020 23:34:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 29 18:34:15 2020
Received: from localhost ([127.0.0.1]:34709 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iwwqc-0003Yo-So
	for submit <at> debbugs.gnu.org; Wed, 29 Jan 2020 18:34:15 -0500
Received: from mail-qt1-f172.google.com ([209.85.160.172]:46296)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1iwwqb-0003Yb-IF
 for 39258 <at> debbugs.gnu.org; Wed, 29 Jan 2020 18:34:14 -0500
Received: by mail-qt1-f172.google.com with SMTP id e25so944644qtr.13
 for <39258 <at> debbugs.gnu.org>; Wed, 29 Jan 2020 15:34:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=LoCPANSQcRYKL7WvsA1UPP8c/yFpkPntg+GlDHw5LfQ=;
 b=nOUNoINq2LgFQVkvghW+wmN65dXoTfG1RDsbw8g7PjauqVkLS7p9IWwrzalMmYRar7
 OtNPHgQXtBJtLcSovSsPJH099NyJUegPvYCMqqjFDCm89XO0WcFHtDNONMgGwNDtRFle
 uqE9G0S7u2TnQ/x8eu8O/9JF384W5pPMlpLdqXJiGsPtxO+MGJloZWtIeJYo+tEQNmtk
 Fp4ka7Dx9+XjA560MD7DSdc3SZmtTeJ7a3uLaqSBF2Yz8DYlwelxMm7v5KC9gUguAdi8
 y9IPzjedL2nDLgiU14/ojUNsBaEpJon1ESUV6IKUna3Uu/2P7Phy22bV/57QKPzN+YUM
 oL/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=LoCPANSQcRYKL7WvsA1UPP8c/yFpkPntg+GlDHw5LfQ=;
 b=DP2QMenHuANFHPaLyyYUFhBJ6GezBr7LyW/pLw0d6RhnaU8WRafcTfBufS6CTw05Gk
 PoQ4olKqYZo2Fo6PR97s6SbOnewE8gfjASNjAsxHCale4jlC2GRaf/sJ9BlKV0Kgbfii
 wFOLbd8OTEP0fP/DBxPv8nJulafFhdEqQTjmtD0ho2xOY5RLjGGq8JrpHeUYZeyfH0fA
 xfcDPUJgxVisimDAPCRU6muua4RHJHA/TOwEi5/XWGQ6WIxVshkyohek+/XYQuLMDFJl
 js9tmxT4VRLumNVJS9dvUFashX08tHH39whk4j/mdi5O2KAVB85H7Fo+1Lbrg2bsQHq4
 1LMw==
X-Gm-Message-State: APjAAAUbFmYBaK9yF2sWO1W3E8DUMLZL0fToJWM9m3aha4D2HhSHl7Wu
 rNIZ3JieuYD4lcnO/HlprbtTryWyP7umMXtT6McRLfF8eSk=
X-Google-Smtp-Source: APXvYqxvZjie7g6Bak2iifeJhhT+gOSKffcNZJUzUXfyzaz3AIMKTDA+MP1eofrQA9rGFDBNoVWY1XlIIVUcIV43QkU=
X-Received: by 2002:ac8:2e53:: with SMTP id s19mr1967828qta.354.1580340847857; 
 Wed, 29 Jan 2020 15:34:07 -0800 (PST)
MIME-Version: 1.0
References: <cu7pnfaar36.fsf@HIDDEN>
In-Reply-To: <cu7pnfaar36.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Thu, 30 Jan 2020 00:33:56 +0100
Message-ID: <CAJ3okZ05JXVgbKJd_WeJKERJd6rc4SwxLRrRR0AyW48QDyX8eQ@HIDDEN>
Subject: Re: [bug#39258] Faster guix search using an sqlite cache
To: Arun Isaac <arunisaac@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: 39258 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Arun,

Thank you for the patch!
Cool! :-)

I have not tested it yet. Sorry.


On Thu, 23 Jan 2020 at 20:53, Arun Isaac <arunisaac@HIDDEN> wrote:

> At the moment, I am having some difficulty populating the sqlite
> database. generate-package-cache populates the database correctly when
> invoked from a normal guile REPL using geiser, but fails to do so when
> run by the guix daemon during guix pull.

[...]

> On examining package-cache.sqlite, I find that no records have been
> written. And, there is a lingering journal file that shouldn't be
> there. For some reason, populating the sqlite database does not work
> with guix pull. sqlite probably crashes and leaves the journal file.

Hum? weird...
Is it possible that a module is loaded when Guile repl is used and not
with Guix pull?
What about "guix repl"?


> If I try to populate the database with each package record being
> inserted in its own transaction, at least some of the insertions

You mean 'commit' the database after each insertion, right?


> work. But the journal file still lingers. My unverified guess is that
> everything except the last transaction was successful.

And this does not happen with the repl, right?


> Any ideas what's going on?

I have no idea.
Weird.

What about adding 'last-insert-row-id' from 'guix/store/database.scm'?
I mean without really understanding and just grepping
'sqlite-finalize' spots that 'last-insert-row-id' seems often around.
:-)


> Also, inserting each package in its own transaction is ridiculously slow
> and so that is out of the question. See https://www.sqlite.org/faq.html#q19

Agree that it is not an option. :-)


Otherwise, 'list->string' is defined twice. And the first one is not
necessary, I guess.
The docstring of 'cache-lookup' is not coherent anymore. :-)


Cheers,
simon




Information forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 23 Jan 2020 19:52:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 23 14:52:33 2020
Received: from localhost ([127.0.0.1]:53208 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iuiWn-00019K-8s
	for submit <at> debbugs.gnu.org; Thu, 23 Jan 2020 14:52:33 -0500
Received: from lists.gnu.org ([209.51.188.17]:55538)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <arunisaac@HIDDEN>) id 1iuiWk-00019A-Ju
 for submit <at> debbugs.gnu.org; Thu, 23 Jan 2020 14:52:31 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:38921)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <arunisaac@HIDDEN>) id 1iuiWh-0008EY-Ap
 for guix-patches@HIDDEN; Thu, 23 Jan 2020 14:52:30 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED
 autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <arunisaac@HIDDEN>) id 1iuiWe-00087M-FS
 for guix-patches@HIDDEN; Thu, 23 Jan 2020 14:52:26 -0500
Received: from mugam.systemreboot.net ([139.59.75.54]:60578)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <arunisaac@HIDDEN>)
 id 1iuiWd-00082X-4H
 for guix-patches@HIDDEN; Thu, 23 Jan 2020 14:52:24 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
 d=systemreboot.net; s=default; h=Content-Type:MIME-Version:Message-ID:Date:
 Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=m58VqhXMlYNbWwgkqnBfSm2SHXEBcN2pNnULcRktOt0=; b=lxKPiLfEtbVbt5FKBuTu0mWTDC
 vVyWwl+Ps0w4MfGSzWCpqt36Yvby5jcnx5o4BkhoKEii3opfHxnin65h/IynHQxTP3DjU8DMP72y9
 tsh9x6HEU/Y7q3hJHhg882OczgmkYenh6z5R/zKtLzShLalFSabpDgQoHLBIpqA3DUFY=;
Received: from [192.168.2.1] (helo=steel)
 by systemreboot.net with esmtpsa  (TLS1.3) tls TLS_AES_256_GCM_SHA384
 (Exim 4.93) (envelope-from <arunisaac@HIDDEN>)
 id 1iuiWX-0009z3-Pm
 for guix-patches@HIDDEN; Fri, 24 Jan 2020 01:22:17 +0530
From: Arun Isaac <arunisaac@HIDDEN>
To: guix-patches@HIDDEN
Subject: Faster guix search using an sqlite cache
Date: Fri, 24 Jan 2020 01:21:57 +0530
Message-ID: <cu7pnfaar36.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 139.59.75.54
X-Spam-Score: 0.2 (/)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Hi,

As discussed on guix-devel at
https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00310.html , I
am working on an sqlite cache to improve guix search performance. I have
attached a highly incomplete WIP patch. The patch attempts to
reimplement the package-cache-file hook in guix/channels.scm using a
sqlite database. To this end, it rewrites most of the
generate-package-cache and cache-lookup functions in gnu/packages.scm. I
am yet to hook this up to guix search.

At the moment, I am having some difficulty populating the sqlite
database. generate-package-cache populates the database correctly when
invoked from a normal guile REPL using geiser, but fails to do so when
run by the guix daemon during guix pull.

I ran guix pull using

$ ./pre-inst-env guix pull --url=3D$PWD --branch=3Dsearch -p /tmp/test

where search is the branch I am working on.

Running

$ ls /tmp/test/lib/guix -lh

shows

total 2.1M
=2Dr--r--r-- 2 root root 2.1M =E0=AE=9C=E0=AE=A9.   1  1970 package-cache.s=
qlite
=2Dr--r--r-- 2 root root  26K =E0=AE=9C=E0=AE=A9.   1  1970 package-cache.s=
qlite-journal

On examining package-cache.sqlite, I find that no records have been
written. And, there is a lingering journal file that shouldn't be
there. For some reason, populating the sqlite database does not work
with guix pull. sqlite probably crashes and leaves the journal file.

If I try to populate the database with each package record being
inserted in its own transaction, at least some of the insertions
work. But the journal file still lingers. My unverified guess is that
everything except the last transaction was successful.

Any ideas what's going on?

Also, inserting each package in its own transaction is ridiculously slow
and so that is out of the question. See https://www.sqlite.org/faq.html#q19


--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline; filename=0001-fast-search.patch
Content-Transfer-Encoding: quoted-printable

From=20d1305351a90a84eb75e4769284d5e06927eade3e Mon Sep 17 00:00:00 2001
From: Arun Isaac <arunisaac@HIDDEN>
Date: Tue, 21 Jan 2020 20:45:43 +0530
Subject: [PATCH] fast search

=2D--
 build-aux/build-self.scm |   5 +
 gnu/packages.scm         | 207 +++++++++++++++++++++++----------------
 2 files changed, 128 insertions(+), 84 deletions(-)

diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index fc13032b73..c123ad3b11 100644
=2D-- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -264,6 +264,9 @@ interface (FFI) of Guile.")
   (define fake-git
     (scheme-file "git.scm" #~(define-module (git))))
=20
+  (define fake-sqlite3
+    (scheme-file "sqlite3.scm" #~(define-module (sqlite3))))
+
   (with-imported-modules `(((guix config)
                             =3D> ,(make-config.scm))
=20
@@ -278,6 +281,8 @@ interface (FFI) of Guile.")
                            ;; (git) to placate it.
                            ((git) =3D> ,fake-git)
=20
+                           ((sqlite3) =3D> ,fake-sqlite3)
+
                            ,@(source-module-closure `((guix store)
                                                       (guix self)
                                                       (guix derivations)
diff --git a/gnu/packages.scm b/gnu/packages.scm
index d22c992bb1..4e2c52e62d 100644
=2D-- a/gnu/packages.scm
+++ b/gnu/packages.scm
@@ -43,6 +43,7 @@
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
   #:use-module (srfi srfi-39)
+  #:use-module (sqlite3)
   #:export (search-patch
             search-patches
             search-auxiliary-file
@@ -204,10 +205,8 @@ PROC is called along these lines:
 PROC can use #:allow-other-keys to ignore the bits it's not interested in.
 When a package cache is available, this procedure does not actually load a=
ny
 package module."
=2D  (define cache
=2D    (load-package-cache (current-profile)))
=2D
=2D  (if (and cache (cache-is-authoritative?))
+  (if (and (cache-is-authoritative?)
+           (current-profile))
       (vhash-fold (lambda (name vector result)
                     (match vector
                       (#(name version module symbol outputs
@@ -220,7 +219,7 @@ package module."
                              #:supported? supported?
                              #:deprecated? deprecated?))))
                   init
=2D                  cache)
+                  (cache-lookup (current-profile)))
       (fold-packages (lambda (package result)
                        (proc (package-name package)
                              (package-version package)
@@ -252,31 +251,7 @@ is guaranteed to never traverse the same package twice=
."
=20
 (define %package-cache-file
   ;; Location of the package cache.
=2D  "/lib/guix/package.cache")
=2D
=2D(define load-package-cache
=2D  (mlambda (profile)
=2D    "Attempt to load the package cache.  On success return a vhash keyed=
 by
=2Dpackage names.  Return #f on failure."
=2D    (match profile
=2D      (#f #f)
=2D      (profile
=2D       (catch 'system-error
=2D         (lambda ()
=2D           (define lst
=2D             (load-compiled (string-append profile %package-cache-file)))
=2D           (fold (lambda (item vhash)
=2D                   (match item
=2D                     (#(name version module symbol outputs
=2D                             supported? deprecated?
=2D                             file line column)
=2D                      (vhash-cons name item vhash))))
=2D                 vlist-null
=2D                 lst))
=2D         (lambda args
=2D           (if (=3D ENOENT (system-error-errno args))
=2D               #f
=2D               (apply throw args))))))))
+  "/lib/guix/package-cache.sqlite")
=20
 (define find-packages-by-name/direct              ;bypass the cache
   (let ((packages (delay
@@ -297,25 +272,57 @@ decreasing version order."
                     matching)
             matching)))))
=20
=2D(define (cache-lookup cache name)
+(define* (cache-lookup profile #:optional name)
   "Lookup package NAME in CACHE.  Return a list sorted in increasing versi=
on
 order."
   (define (package-version<? v1 v2)
     (version>? (vector-ref v2 1) (vector-ref v1 1)))
=20
=2D  (sort (vhash-fold* cons '() name cache)
=2D        package-version<?))
+  (define (int->boolean n)
+    (case n
+      ((0) #f)
+      ((1) #t)))
+
+  (define (string->list str)
+    (call-with-input-string str read))
+
+  (define select-statement
+    (string-append
+     "SELECT name, version, module, symbol, outputs, supported, superseded=
, locationFile, locationLine, locationColumn from packages"
+     (if name " WHERE name =3D :name" "")))
+
+  (define cache-file
+    (string-append profile %package-cache-file))
+
+  (let* ((db (sqlite-open cache-file SQLITE_OPEN_READONLY))
+         (statement (sqlite-prepare db select-statement)))
+    (when name
+      (sqlite-bind-arguments statement #:name name))
+    (let ((result (sqlite-fold (lambda (v result)
+                                 (match v
+                                   (#(name version module symbol outputs s=
upported superseded file line column)
+                                    (cons
+                                     (vector name
+                                             version
+                                             (string->list module)
+                                             (string->symbol symbol)
+                                             (string->list outputs)
+                                             (int->boolean supported)
+                                             (int->boolean superseded)
+                                             (list file line column))
+                                     result))))
+                               '() statement)))
+      (sqlite-finalize statement)
+      (sqlite-close db)
+      (sort result package-version<?))))
=20
 (define* (find-packages-by-name name #:optional version)
   "Return the list of packages with the given NAME.  If VERSION is not #f,
 then only return packages whose version is prefixed by VERSION, sorted in
 decreasing version order."
=2D  (define cache
=2D    (load-package-cache (current-profile)))
=2D
=2D  (if (and (cache-is-authoritative?) cache)
=2D      (match (cache-lookup cache name)
=2D        (#f #f)
+  (if (and (cache-is-authoritative?)
+           (current-profile))
+      (match (cache-lookup (current-profile) name)
         ((#(_ versions modules symbols _ _ _ _ _ _) ...)
          (fold (lambda (version* module symbol result)
                  (if (or (not version)
@@ -331,12 +338,9 @@ decreasing version order."
 (define* (find-package-locations name #:optional version)
   "Return a list of version/location pairs corresponding to each package
 matching NAME and VERSION."
=2D  (define cache
=2D    (load-package-cache (current-profile)))
=2D
=2D  (if (and cache (cache-is-authoritative?))
=2D      (match (cache-lookup cache name)
=2D        (#f '())
+  (if (and (cache-is-authoritative?)
+           (current-profile))
+      (match (cache-lookup (current-profile) name)
         ((#(name versions modules symbols outputs
                  supported? deprecated?
                  files lines columns) ...)
@@ -372,6 +376,9 @@ VERSION."
 ;; Prevent Guile 3 from inlining this procedure so we can mock it in tests.
 (set! find-best-packages-by-name find-best-packages-by-name)
=20
+(define (list->string x)
+  (call-with-output-string (cut write x <>)))
+
 (define (generate-package-cache directory)
   "Generate under DIRECTORY a cache of all the available packages.
=20
@@ -381,49 +388,81 @@ reducing the memory footprint."
   (define cache-file
     (string-append directory %package-cache-file))
=20
=2D  (define (expand-cache module symbol variable result+seen)
+  (define schema
+    "CREATE TABLE packages (name text,
+version text,
+module text,
+symbol text,
+outputs text,
+supported int,
+superseded int,
+locationFile text,
+locationLine int,
+locationColumn int);
+CREATE VIRTUAL TABLE packageSearch USING fts5(name, searchText);")
+
+  (define insert-statement
+    "INSERT INTO packages(name, version, module, symbol, outputs, supporte=
d, superseded, locationFile, locationLine, locationColumn)
+VALUES(:name, :version, :module, :symbol, :outputs, :supported, :supersede=
d, :locationfile, :locationline, :locationcolumn)")
+
+  (define insert-package-search-statement
+    "INSERT INTO packageSearch(name, searchText) VALUES(:name, :searchtext=
)")
+
+  (define (boolean->int x)
+    (if x 1 0))
+
+  (define (list->string x)
+    (call-with-output-string (cut write x <>)))
+
+  (define (insert-package db module symbol variable seen)
     (match (false-if-exception (variable-ref variable))
       ((? package? package)
=2D       (match result+seen
=2D         ((result . seen)
=2D          (if (or (vhash-assq package seen)
=2D                  (hidden-package? package))
=2D              (cons result seen)
=2D              (cons (cons `#(,(package-name package)
=2D                             ,(package-version package)
=2D                             ,(module-name module)
=2D                             ,symbol
=2D                             ,(package-outputs package)
=2D                             ,(->bool (supported-package? package))
=2D                             ,(->bool (package-superseded package))
=2D                             ,@(let ((loc (package-location package)))
=2D                                 (if loc
=2D                                     `(,(location-file loc)
=2D                                       ,(location-line loc)
=2D                                       ,(location-column loc))
=2D                                     '(#f #f #f))))
=2D                          result)
=2D                    (vhash-consq package #t seen))))))
=2D      (_
=2D       result+seen)))
=2D
=2D  (define exp
=2D    (first
=2D     (fold-module-public-variables* expand-cache
=2D                                    (cons '() vlist-null)
=2D                                    (all-modules (%package-module-path)
=2D                                                 #:warn
=2D                                                 warn-about-load-error))=
))
+       (cond
+        ((or (vhash-assq package seen)
+             (hidden-package? package))
+         seen)
+        (else
+         (let ((statement (sqlite-prepare db insert-statement)))
+           (sqlite-bind-arguments statement
+                                  #:name (package-name package)
+                                  #:version (package-version package)
+                                  #:module (list->string (module-name modu=
le))
+                                  #:symbol (symbol->string symbol)
+                                  #:outputs (list->string (package-outputs=
 package))
+                                  #:supported (boolean->int (supported-pac=
kage? package))
+                                  #:superseded (boolean->int (package-supe=
rseded package))
+                                  #:locationfile (cond
+                                                  ((package-location packa=
ge) =3D> location-file)
+                                                  (else #f))
+                                  #:locationline (cond
+                                                  ((package-location packa=
ge) =3D> location-line)
+                                                  (else #f))
+                                  #:locationcolumn (cond
+                                                    ((package-location pac=
kage) =3D> location-column)
+                                                    (else #f)))
+           (sqlite-fold cons '() statement)
+           (sqlite-finalize statement))
+         (let ((statement (sqlite-prepare db insert-package-search-stateme=
nt)))
+           (sqlite-bind-arguments statement
+                                  #:name (package-name package)
+                                  #:searchtext (package-description packag=
e))
+           (sqlite-fold cons '() statement)
+           (sqlite-finalize statement))
+         (vhash-consq package #t seen))))
+      (_ seen)))
=20
   (mkdir-p (dirname cache-file))
=2D  (call-with-output-file cache-file
=2D    (lambda (port)
=2D      ;; Store the cache as a '.go' file.  This makes loading fast and r=
educes
=2D      ;; heap usage since some of the static data is directly mmapped.
=2D      (put-bytevector port
=2D                      (compile `'(,@exp)
=2D                               #:to 'bytecode
=2D                               #:opts '(#:to-file? #t)))))
+  (let ((db (sqlite-open cache-file)))
+    (sqlite-exec db schema)
+    (sqlite-exec db "BEGIN")
+    (fold-module-public-variables* (cut insert-package db <> <> <> <>)
+                                   vlist-null
+                                   (all-modules (%package-module-path)
+                                                #:warn
+                                                warn-about-load-error))
+    (sqlite-exec db "COMMIT;")
+    (sqlite-close db))
+
   cache-file)
=20
 
=2D-=20
2.23.0


--=-=-=--

--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl4p+V0ACgkQLiXui2GA
K7Oe/wf/VUd/Gcd+KFyJxXTfML9vGIxR7xUXl6M92mevuADXCxI4JECqMmjIuj9f
ZbC6o/D+XcnJz7XyttQZi/iyjbwZIA0DwUbdAg5BRP8cK6ZkCflPfjamWNQ2RVYu
2S+oITgatidZTLDTFGP6RYeXN27I+fkK5P28XSJHa69aE34bVor0R3bb7Ki57OVS
+cEYu6nlGbADqpFpLT6VjB7ewgr9wt0tQyq721JevZzi3PNb+WVq6Pi2N69nHDdO
kslYfUVi2kWnN9i3gtBnEwVo2cj2uaD2eSYd5YoA2c5kxHVomo5/CiuDUqdCyxd+
/Xbjbug+6C7SeCcBbfVW0RgRo2Dvng==
=HLpI
-----END PGP SIGNATURE-----
--==-=-=--




Acknowledgement sent to Arun Isaac <arunisaac@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#39258; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 11 Feb 2020 20:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.