GNU bug report logs - #43528
Download all issues so I can search?

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43528 in the body.
You can then email your comments to 43528 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Yasuaki Kudo <yasu <at> yasuaki.com>
To: bug-guix <at> gnu.org
Subject: Download all issues so I can search?
Date: Sun, 20 Sep 2020 16:30:14 +0900

Hi,

If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally.   Is it possible?

Cheers,
Yasu

Message #8 received at 43528 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Yasuaki Kudo <yasu <at> yasuaki.com>, 43528 <at> debbugs.gnu.org
Subject: Re: bug#43528: Download all issues so I can search?
Date: Sun, 20 Sep 2020 19:33:49 +0200

Hi!

On 2020-09-20 9:30, Yasuaki Kudo wrote:
> Hi,
> 
> If possible, I would like to download all guix issues in the tracker
> (in other words, database dump?) so I can just use full text search
> locally.   Is it possible?

Probably!  This is basically what Mumi, the software behind 
issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail from 
debbugs.gnu.org and index it locally using Mu.

I don't *think* it uses any secret API/authentication/whitelisted IPs 
etc. but am not very familiar with its innards.

If you can read Scheme, you can probably figure it out from 
<https://git.elephly.net/gitweb.cgi?p=software/mumi.git>.

Hoping to've been of some help,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: Yasuaki Kudo <yasu <at> yasuaki.com>, 43528 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#43528: Download all issues so I can search?
Date: Sun, 20 Sep 2020 21:32:33 +0200

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org> writes:

> Hi!
>
> On 2020-09-20 9:30, Yasuaki Kudo wrote:
>> Hi,
>> If possible, I would like to download all guix issues in the tracker
>> (in other words, database dump?) so I can just use full text search
>> locally.   Is it possible?
>
> Probably!  This is basically what Mumi, the software behind
> issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail
> from debbugs.gnu.org and index it locally using Mu.
>
> I don't *think* it uses any secret API/authentication/whitelisted IPs
> etc. but am not very familiar with its innards.

Actually it does exactly that (whitelisted IP doing rsync), ever since
The Indicident.  I used to just fetch all mbox files for all Guix bugs,
but this put the Debbugs servers under excessive load.  After the
server’s IP was blocked from accessing Debbugs we negotiated rsync
access.

We do have full text search on issues.guix.gnu.org.  It was brought to
my attention that it’s not yielding expected results since a while,
though, so this would have to be investigated.

-- 
Ricardo

Message #17 received at 43528 <at> debbugs.gnu.org (full text, mbox):

From: Michael Rohleder <mike <at> rohleder.de>
To: Yasuaki Kudo <yasu <at> yasuaki.com>
Cc: 43528 <at> debbugs.gnu.org
Subject: Re: bug#43528: Download all issues so I can search?
Date: Tue, 22 Sep 2020 00:27:12 +0200

[Message part 1 (text/plain, inline)]

Here's [1] an archive of the ML in mbox format.
Maybe this helps if you want to grep for things.


[1] https://lists.gnu.org/archive/mbox/bug-guix/

-- 
Das Leben kann nur rückwärts verstanden werden,
aber es muss vorwärts gelebt werden.
  – Søren Kierkegaard

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 43528 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Yasuaki Kudo <yasu <at> yasuaki.com>
Cc: 43528 <at> debbugs.gnu.org
Subject: Re: bug#43528: Download all issues so I can search?
Date: Tue, 22 Sep 2020 00:54:56 +0200

Dear,

On Sun, 20 Sep 2020 at 11:17, Yasuaki Kudo <yasu <at> yasuaki.com> wrote:

> If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally.   Is it possible?

Here [1], unofficial public-inbox mirrors so you can "git clone" all
the issues and search offline.  Otherwise, please report the searching
issue you could hit with the searching facilities of the official
<issues.guix.gnu.org>.

[1] https://yhetil.org/


All the best,
simon

Message #27 received at 43528-done <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Michael Rohleder <mike <at> rohleder.de>
Cc: Yasuaki Kudo <yasu <at> yasuaki.com>, 43528-done <at> debbugs.gnu.org
Subject: Re: bug#43528: Download all issues so I can search?
Date: Thu, 08 Oct 2020 21:41:14 -0400

Michael Rohleder <mike <at> rohleder.de> writes:

> Here's [1] an archive of the ML in mbox format.
> Maybe this helps if you want to grep for things.
>
>
> [1] https://lists.gnu.org/archive/mbox/bug-guix/

That is indeed a good suggestion.

Closing.

Thanks,

Maxim

Message #30 received at 43528 <at> debbugs.gnu.org (full text, mbox):

From: Bengt Richter <bokr <at> bokr.com>
To: 43528 <at> debbugs.gnu.org, maxim.cournoyer <at> gmail.com, yasu <at> yasuaki.com
Subject: Re: bug#43528: Download all issues so I can search?
Date: Fri, 9 Oct 2020 05:37:53 +0200

On +2020-10-08 21:41:14 -0400, Maxim Cournoyer wrote:
> Michael Rohleder <mike <at> rohleder.de> writes:
> 
> > Here's [1] an archive of the ML in mbox format.
> > Maybe this helps if you want to grep for things.
> >
> >
> > [1] https://lists.gnu.org/archive/mbox/bug-guix/
> 
> That is indeed a good suggestion.
> 
> Closing.
> 
> Thanks,
> 
> Maxim
> 
> 
>

I discovered that the server supports incremental download using the '-c'
option of wget (so it only downloads what was added to the archive
since you last did it), e.g.,
    wget -c https://lists.gnu.org/archive/mbox/bug-guix/2020-10
then
    mutt -f 2020-10 # (or gnus equivalent I guess) looks normal.

Probably not news to most devs, but I thought it nice that it works.

(For me anyway :)

-- 
Regards,
Bengt Richter

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.