GNU bug report logs -
#33653
27.0.50; Change Gnus obarrays-as-hash-tables into real hash tables
Previous Next
Reported by: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Date: Thu, 6 Dec 2018 22:40:02 UTC
Severity: wishlist
Tags: fixed
Found in version 27.0.50
Fixed in version 27.1
Done: Lars Ingebrigtsen <larsi <at> gnus.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 33653 in the body.
You can then email your comments to 33653 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Thu, 06 Dec 2018 22:40:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Eric Abrahamsen <eric <at> ericabrahamsen.net>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Thu, 06 Dec 2018 22:40:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Here's the next thing: turning Gnus' obarrays-as-hash-tables into real
hash tables. Gnus currently stores information about groups by coercing
group names to unibyte, interning them in custom obarrays, and then
setting their symbol-value to whatever value needs to be stored. I think
all this was written before Emacs had actual hash tables.
I think real hash tables are better for a few reasons:
1. Hash table lookups seem to be marginally faster than obarray lookups,
2. It's "less weird" for contributors and bug hunters,
3. It allows us to reduces the amount of encoding/decoding going on:
group names can stay strings instead of being forced into symbols.
I've pushed a branch, scratch/gnus-hashtables, with two commits in it.
The first changes all the obarrays to hash tables. Apart from simply
replacing function calls, there were a few bumps.
1. Gnus uses `text-property-any' to find and compare group-name text
properties; I made a new `gnus-text-property-search' which behaves
similarly, but compares with equal.
2. Some of the "hash tables" were simply storing the value t, ie just
used for membership. I left these as hash tables, but I think in many
cases simple string-in-list membership tests would suffice.
3. The hash table in gnus-async.el didn't appear to be doing anything --
it seemed to be redundant with `gnus-async-article-alist'. I've
removed it, but it might need to be put back if I'm missing something.
4. The creation of `gnus-newsgroup-dependencies' was the most fiddly,
and I added tests for that. I'm not entirely convinced that
`gnus-thread-loop-p' behaves the way it's meant to, it appears to
only check for direct loops between parent and child, not parent and
descendant.
5. The old return value of (gnus-gethash <group-name>
gnus-newsrc-hashtb) was kind of a slice into `gnus-newsrc-alist': it
behaved a bit like `member', but also included the group *before* the
group you wanted, as well as all those after, so you could traverse
the list in either direction. It now no longer preserves the order of
`gnus-newsrc-alist' (this ordering wasn't actually used in many
places), and instead there's a new variable `gnus-group-list' which
records the proper sort order of the groups.
I feel fairly confident that all this is working okay. The second commit
I do *not* feel very confident about, and it's more of a "let's see how
things break" attempt. In theory, we should now be able to limit group
name encoding/decoding to the boundaries of Gnus -- reading/writing
active files, and talking to servers. Within Gnus, the group names can
remain decoded strings.
In the second commit I've just gone in and pulled out all the decoding I
can find, changed all the 'raw-text encoding options to 'undecided, and
deleted the `mm-disable-multibyte's. I have no confidence that I've
covered all the bases, though I have been using this branch for a couple
of weeks, with nnml, nnmaildir, and nnimap groups with multibyte names,
and so far nothing has broken. I'm less confident about nntp.
I'll continue working on this, and I hope I can get some feedback here,
particularly on the second commit.
Thanks, Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Thu, 06 Dec 2018 22:47:02 GMT)
Full text and
rfc822 format available.
Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
> Here's the next thing: turning Gnus' obarrays-as-hash-tables into real
> hash tables. Gnus currently stores information about groups by coercing
> group names to unibyte, interning them in custom obarrays, and then
> setting their symbol-value to whatever value needs to be stored. I think
> all this was written before Emacs had actual hash tables.
Also hounding Lars for his opinion...
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 11 Dec 2018 23:31:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 12/11/18 20:23 PM, Lars Ingebrigtsen wrote:
> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>>> Here's the next thing: turning Gnus' obarrays-as-hash-tables into real
>>> hash tables. Gnus currently stores information about groups by coercing
>>> group names to unibyte, interning them in custom obarrays, and then
>>> setting their symbol-value to whatever value needs to be stored. I think
>>> all this was written before Emacs had actual hash tables.
>>
>> Also hounding Lars for his opinion...
>
> Using real hash tables sounds nice, if there's no negative performance
> impact.
Cool! In fact it seems to be a hair faster.
If you have a moment for this, would you look at the second commit in
the scratch/gnus-hashtables branch? The first commit seems fairly
stable, but the second has to do with encoding -- if group names never
need to be unibyte, it should be possible to leave them decoded inside
Gnus itself.
Besides active files and server communication, are there any other
places where group names would need to be encoded?
Thanks,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 05 Feb 2019 02:06:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 12/11/18 20:23 PM, Lars Ingebrigtsen wrote:
> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>>> Here's the next thing: turning Gnus' obarrays-as-hash-tables into real
>>> hash tables. Gnus currently stores information about groups by coercing
>>> group names to unibyte, interning them in custom obarrays, and then
>>> setting their symbol-value to whatever value needs to be stored. I think
>>> all this was written before Emacs had actual hash tables.
>>
>> Also hounding Lars for his opinion...
>
> Using real hash tables sounds nice, if there's no negative performance
> impact.
I've pushed another commit to scratch/gnus-hashtables, fixing a small
bug in gnus-bklg.el. Otherwise, I've been using this branch for a couple
months now, with no ill effects.
I guess what I'd like to do is squash the first commit with the third
(the changes that actually implement the hash tables), but leave the
second aside for now (the one that stops doing internal
encoding/decoding).
That ought to be done eventually, but I'll need more time to figure it
out, and would like to get more exposure for this refactoring in the
meantime.
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 00:10:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 02/04/19 18:05 PM, Eric Abrahamsen wrote:
> On 12/11/18 20:23 PM, Lars Ingebrigtsen wrote:
>> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>>
>>>> Here's the next thing: turning Gnus' obarrays-as-hash-tables into real
>>>> hash tables. Gnus currently stores information about groups by coercing
>>>> group names to unibyte, interning them in custom obarrays, and then
>>>> setting their symbol-value to whatever value needs to be stored. I think
>>>> all this was written before Emacs had actual hash tables.
>>>
>>> Also hounding Lars for his opinion...
>>
>> Using real hash tables sounds nice, if there's no negative performance
>> impact.
>
> I've pushed another commit to scratch/gnus-hashtables, fixing a small
> bug in gnus-bklg.el. Otherwise, I've been using this branch for a couple
> months now, with no ill effects.
>
> I guess what I'd like to do is squash the first commit with the third
> (the changes that actually implement the hash tables), but leave the
> second aside for now (the one that stops doing internal
> encoding/decoding).
Weeeeelll... I'd really like to push this. I promise to be on hand with
quick fixes, and a revert if necessary.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 09:21:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 33653 <at> debbugs.gnu.org (full text, mbox):
>>>>> On Thu, 21 Mar 2019 17:09:07 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:
Eric> Weeeeelll... I'd really like to push this. I promise to be
Eric> on hand with quick fixes, and a revert if necessary.
This is for emacs-27, right? Iʼd say go for it :-)
Robert
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 17:23:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Robert Pluim <rpluim <at> gmail.com> writes:
>>>>>> On Thu, 21 Mar 2019 17:09:07 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:
>
> Eric> Weeeeelll... I'd really like to push this. I promise to be
> Eric> on hand with quick fixes, and a revert if necessary.
>
> This is for emacs-27, right?
Most definitely.
> Iʼd say go for it :-)
Okay, if it explodes I'll say you said I could do it! :)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 19:56:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
> Robert Pluim <rpluim <at> gmail.com> writes:
>
>>>>>>> On Thu, 21 Mar 2019 17:09:07 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:
>>
>> Eric> Weeeeelll... I'd really like to push this. I promise to be
>> Eric> on hand with quick fixes, and a revert if necessary.
>>
>> This is for emacs-27, right?
>
> Most definitely.
>
>> Iʼd say go for it :-)
>
> Okay, if it explodes I'll say you said I could do it! :)
And... it exploded!
Well, not really, but on my second machine there were a pile of stale
*.elc files that aren't rebuilt with the new macro definitions, and even
"make distclean" doesn't get rid of them (?), I had to go into the
lisp/gnus directory, manually delete the elc files, and run "make"
again. (Or maybe this is what "make bootstrap" does?)
This must happen from time to time -- is there anything to be done
besides warning people on the mailing lists?
The bummer is that you don't get an error until you run Gnus, so it just
looks like I broke Gnus :(
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 21:15:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 33653 <at> debbugs.gnu.org (full text, mbox):
> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Date: Fri, 22 Mar 2019 12:54:54 -0700
>
> is there anything to be done besides warning people on the mailing
> lists?
A warning would be nice, yes.
> "make distclean" doesn't get rid of them (?), I had to go into the
> lisp/gnus directory, manually delete the elc files, and run "make"
> again. (Or maybe this is what "make bootstrap" does?)
Yes, that's what a bootstrap does (if you don't mind doing a bootstrap
in such situations).
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 22:26:02 GMT)
Full text and
rfc822 format available.
Message #32 received at submit <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>> Date: Fri, 22 Mar 2019 12:54:54 -0700
>>
>> is there anything to be done besides warning people on the mailing
>> lists?
>
> A warning would be nice, yes.
Okay, I'll do that. I've already gotten a bug report :(
>> "make distclean" doesn't get rid of them (?), I had to go into the
>> lisp/gnus directory, manually delete the elc files, and run "make"
>> again. (Or maybe this is what "make bootstrap" does?)
>
> Yes, that's what a bootstrap does (if you don't mind doing a bootstrap
> in such situations).
Got it, thank you.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 22:42:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Eric Abrahamsen wrote:
> even "make distclean" doesn't get rid of them (?),
Because .elc files are distributed in release tarfiles.
> This must happen from time to time -- is there anything to be done
> besides warning people on the mailing lists?
1a) make bootstrap the default make target.
1b) rename the current default make target as
faster-but-sometimes-will-go-wrong-for-obscure-reasons
CPU time is cheaper than human time spent figuring out problems.
2) Make load-prefer-newer non-nil by default.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 22 Mar 2019 22:52:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 03/22/19 18:40 PM, Glenn Morris wrote:
> Eric Abrahamsen wrote:
>
>
>> even "make distclean" doesn't get rid of them (?),
>
> Because .elc files are distributed in release tarfiles.
Right, that makes sense.
>> This must happen from time to time -- is there anything to be done
>> besides warning people on the mailing lists?
>
> 1a) make bootstrap the default make target.
> 1b) rename the current default make target as
> faster-but-sometimes-will-go-wrong-for-obscure-reasons
Obviously this is the best solution!
> CPU time is cheaper than human time spent figuring out problems.
>
> 2) Make load-prefer-newer non-nil by default.
I didn't know about this, and really like it. Recently Emacs seems to
save up all the "foo.el is newer than compiled file" warnings and feed
them to me at random intervals, usually while I'm trying to use the
minibuffer. Maybe this option will fix that.
Thanks,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sat, 23 Mar 2019 06:47:02 GMT)
Full text and
rfc822 format available.
Message #41 received at 33653 <at> debbugs.gnu.org (full text, mbox):
> From: Glenn Morris <rgm <at> gnu.org>
> Date: Fri, 22 Mar 2019 18:40:45 -0400
> Cc: 33653 <at> debbugs.gnu.org
>
> 1a) make bootstrap the default make target.
Don't know about the others, but almost I never bootstrap, except in a
fresh checkout. The reasons are (a) it's slow, and I don't want to
wait for 10 or so minutes just because I updated from upstream; and
(b) it deletes all previous executables of Emacs, which I use for
fast-bisecting reported problems.
When these rare situations happen, it is not hard to guess the file
whose .elc should be removed and regenerated; if worse comes to worst,
deleting all the *.elc files in the relevant subdirectory (lisp/gnus/
in this case) and typing "make" solves the problem much faster for me,
and without the unwanted side effects.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sat, 23 Mar 2019 15:01:01 GMT)
Full text and
rfc822 format available.
Message #44 received at submit <at> debbugs.gnu.org (full text, mbox):
On Fri 22 Mar 2019, Eric Abrahamsen wrote:
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
>>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>>> Date: Fri, 22 Mar 2019 12:54:54 -0700
>>>
>>> is there anything to be done besides warning people on the mailing
>>> lists?
>>
>> A warning would be nice, yes.
>
> Okay, I'll do that. I've already gotten a bug report :(
After bootstrapping, I observe that the summary buffer does not update
the faces correctly for nntp articles that have been read: they still
have an unread face.
AndyM
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sat, 23 Mar 2019 16:15:01 GMT)
Full text and
rfc822 format available.
Message #47 received at submit <at> debbugs.gnu.org (full text, mbox):
Andy Moreton <andrewjmoreton <at> gmail.com> writes:
> On Fri 22 Mar 2019, Eric Abrahamsen wrote:
>
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>
>>>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>>>> Date: Fri, 22 Mar 2019 12:54:54 -0700
>>>>
>>>> is there anything to be done besides warning people on the mailing
>>>> lists?
>>>
>>> A warning would be nice, yes.
>>
>> Okay, I'll do that. I've already gotten a bug report :(
>
> After bootstrapping, I observe that the summary buffer does not update
> the faces correctly for nntp articles that have been read: they still
> have an unread face.
Thanks for testing!
I saw that problem while working on this patch, and thought it was
solved. I'll revisit the issue, and come back to you if I have more
questions.
Thanks again,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sun, 24 Mar 2019 22:30:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Hi Eric,
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
> The bummer is that you don't get an error until you run Gnus, so it just
> looks like I broke Gnus :(
I hope not! :)
I have not followed this change.
Can it corrupt the .newsrc.eld file?
--
Bastien
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sun, 24 Mar 2019 23:41:01 GMT)
Full text and
rfc822 format available.
Message #53 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 03/24/19 23:29 PM, Bastien wrote:
> Hi Eric,
>
> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>> The bummer is that you don't get an error until you run Gnus, so it just
>> looks like I broke Gnus :(
>
> I hope not! :)
>
> I have not followed this change.
> Can it corrupt the .newsrc.eld file?
Worst come to worst it will write non-ascii group names as unibyte when
they should be utf-8, or else the other way around. I thought I could
avoid dealing with this question for now, and just preserve Gnus'
original method of saving group names, but it looks like I may have
forced the issue instead...
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 02:16:02 GMT)
Full text and
rfc822 format available.
Message #56 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Hi,
Gnus got not to work for groups of which the group name contains
non-ASCII letters. For instance, I got this error when trying
to update the "nnml:テスト" group using `M-g'[1]:
nnml:\343\203\206\343\202\271\343\203\210 error: No such group: テスト
When trying to enter the group using `0 RET'[2] I got:
Group nnml:\343\203\206\343\202\271\343\203\210 couldn't be activated
Those raw bytes are utf-8 encoded "テスト", that is also used in
the group entry in gnus-newsrc-alist saved in the ~/.newsrc.eld
file as follows:
("nnml:\343\203\206\343\202\271\343\203\210" 1 nil ((unexist) (seen (1 . 5))) "nnml:" ((timestamp 23704 11958)))
Such groups I have are only a few, mainly for testing, so not so
damaged though, I'm afraid there may be those who using them as
normal. Info says that nntp, nnml, and nnrss back ends support
non-ASCII group names[3].
Regards,
[1] gnus-group-get-new-news-this-group
[2] gnus-group-select-group with the prefix arg 0
[3] (info "(gnus) Non-ASCII Group Names")
;; I did `make bootstrap' to recompile all the Gnus modules.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 02:36:02 GMT)
Full text and
rfc822 format available.
Message #59 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> Hi,
>
> Gnus got not to work for groups of which the group name contains
> non-ASCII letters. For instance, I got this error when trying
> to update the "nnml:テスト" group using `M-g'[1]:
>
> nnml:\343\203\206\343\202\271\343\203\210 error: No such group: テスト
>
> When trying to enter the group using `0 RET'[2] I got:
>
> Group nnml:\343\203\206\343\202\271\343\203\210 couldn't be activated
>
> Those raw bytes are utf-8 encoded "テスト", that is also used in
> the group entry in gnus-newsrc-alist saved in the ~/.newsrc.eld
> file as follows:
>
> ("nnml:\343\203\206\343\202\271\343\203\210" 1 nil ((unexist) (seen (1
> . 5))) "nnml:" ((timestamp 23704 11958)))
Yes, this is something I screwed up in c1b63af445. Gnus has always
stored group names as raw bytes in.newsrc.eld (at least I believe it
has, you probably know better than I do, it does in my experiments with
Emacs 26, anyway), and only encodes during display. But obviously I've
messed something up between file persistence and display, and I'm
working on sorting it out.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 14:46:02 GMT)
Full text and
rfc822 format available.
Message #62 received at submit <at> debbugs.gnu.org (full text, mbox):
On Sun 24 Mar 2019, Eric Abrahamsen wrote:
> Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
>
>> Hi,
>>
>> Gnus got not to work for groups of which the group name contains
>> non-ASCII letters. For instance, I got this error when trying
>> to update the "nnml:テスト" group using `M-g'[1]:
>>
>> nnml:\343\203\206\343\202\271\343\203\210 error: No such group: テスト
>>
>> When trying to enter the group using `0 RET'[2] I got:
>>
>> Group nnml:\343\203\206\343\202\271\343\203\210 couldn't be activated
>>
>> Those raw bytes are utf-8 encoded "テスト", that is also used in
>> the group entry in gnus-newsrc-alist saved in the ~/.newsrc.eld
>> file as follows:
>>
>> ("nnml:\343\203\206\343\202\271\343\203\210" 1 nil ((unexist) (seen (1
>> . 5))) "nnml:" ((timestamp 23704 11958)))
>
> Yes, this is something I screwed up in c1b63af445. Gnus has always
> stored group names as raw bytes in.newsrc.eld (at least I believe it
> has, you probably know better than I do, it does in my experiments with
> Emacs 26, anyway), and only encodes during display. But obviously I've
> messed something up between file persistence and display, and I'm
> working on sorting it out.
Perhaps it would be better to revert and reintroduce your changes after
further testing ? Taking time over this is better than causing data loss
for gnus users.
Other notes from reading the code:
1) In `gnus-gnus-to-quick-newsrc-format' you ignore the contents of
`gnus-newsrc-alist' when saving "newsrc.eld", and replace it with the
details from `gnus-newsrc-hashtb'. Why ? The rest of the gnus code
appears to treat `gnus-newsrc-alist' as the single source of truth,
with the hash tables being used only for faster access to it.
2) In `gnus-gnus-to-quick-newsrc-format' you dropped the code to remove
the dummy group from `gnus-newsrc-alist'. Why ? This internal dummy
group is now saved in "newsrc.eld", which is not needed.
3) The format of the entries in `gnus-newsrc-hashtb' has changed,
removing the second element. Why ?
4) You changed several hash tale sizesfrom 4096 to 4000, and 1024 to
1000. Why ?
Your patch contains several logical changes that would be easier to
understand (and bisect) as a series of patches with one logical change
in each patch:
- code layout changes
- add missing doc strings and code comments
- change hash table implementation
- change format of `gnus-newsrc-hashtb' entries
- change usage of `gnus-group-change-level'
- change coding of group names
While it can take extra work to split things up, the end result is much
easier to understand.
Thanks for working on gnus,
AndyM
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 17:36:02 GMT)
Full text and
rfc822 format available.
Message #65 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Andy Moreton <andrewjmoreton <at> gmail.com> writes:
> On Sun 24 Mar 2019, Eric Abrahamsen wrote:
>
>> Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
>>
>>> Hi,
>>>
>>> Gnus got not to work for groups of which the group name contains
>>> non-ASCII letters. For instance, I got this error when trying
>>> to update the "nnml:テスト" group using `M-g'[1]:
>>>
>>> nnml:\343\203\206\343\202\271\343\203\210 error: No such group: テスト
>>>
>>> When trying to enter the group using `0 RET'[2] I got:
>>>
>>> Group nnml:\343\203\206\343\202\271\343\203\210 couldn't be activated
>>>
>>> Those raw bytes are utf-8 encoded "テスト", that is also used in
>>> the group entry in gnus-newsrc-alist saved in the ~/.newsrc.eld
>>> file as follows:
>>>
>>> ("nnml:\343\203\206\343\202\271\343\203\210" 1 nil ((unexist) (seen (1
>>> . 5))) "nnml:" ((timestamp 23704 11958)))
>>
>> Yes, this is something I screwed up in c1b63af445. Gnus has always
>> stored group names as raw bytes in.newsrc.eld (at least I believe it
>> has, you probably know better than I do, it does in my experiments with
>> Emacs 26, anyway), and only encodes during display. But obviously I've
>> messed something up between file persistence and display, and I'm
>> working on sorting it out.
>
> Perhaps it would be better to revert and reintroduce your changes after
> further testing ? Taking time over this is better than causing data loss
> for gnus users.
That's the conclusion I'm coming to, yes.
> Other notes from reading the code:
>
> 1) In `gnus-gnus-to-quick-newsrc-format' you ignore the contents of
> `gnus-newsrc-alist' when saving "newsrc.eld", and replace it with the
> details from `gnus-newsrc-hashtb'. Why ? The rest of the gnus code
> appears to treat `gnus-newsrc-alist' as the single source of truth,
> with the hash tables being used only for faster access to it.
Eventually I would like to reduce the number of data structures so that
groups are held in `gnus-newsrc-hashtb', and ordering is kept in
`gnus-group-list', and that's it. `gnus-newsrc-alist' would only be used
when persisting to disk. My next proposed change (once I've recovered my
confidence) is to turn groups into actual objects, in which case the
alist would really just be a kind of serialization format.
The hash table ought to be in sync with the rest of the data structure
-- if it isn't, that's another bug.
> 2) In `gnus-gnus-to-quick-newsrc-format' you dropped the code to remove
> the dummy group from `gnus-newsrc-alist'. Why ? This internal dummy
> group is now saved in "newsrc.eld", which is not needed.
This was an error. (Though in my case, I've had the dummy group in my
newsrc.eld for months, and it hasn't done any harm. I don't know why
it's necessary.)
> 3) The format of the entries in `gnus-newsrc-hashtb' has changed,
> removing the second element. Why ?
Because the old `gnus-gethash' call returned a slice of
`gnus-newsrc-alist', where the second element was actually the group
*before* the group you wanted, and the third element was the cdr of
`gnus-newsrc-alist', starting with the group you wanted. This was
undocumented, and took a bit to figure out. Now, the gethash call just
gives you the group. Ideally, in the next set of changes, it will give
you an object.
> 4) You changed several hash tale sizesfrom 4096 to 4000, and 1024 to
> 1000. Why ?
My understanding is that using a prime number is significant when it
comes to vector access, but that the hash table implementation is
higher-level, where a prime number is no longer significant. If that's
incorrect I would like to know!
> Your patch contains several logical changes that would be easier to
> understand (and bisect) as a series of patches with one logical change
> in each patch:
> - code layout changes
> - add missing doc strings and code comments
> - change hash table implementation
> - change format of `gnus-newsrc-hashtb' entries
> - change usage of `gnus-group-change-level'
> - change coding of group names
> While it can take extra work to split things up, the end result is much
> easier to understand.
In principle I agree with this completely. In practice I found it
extraordinarily difficult to touch one part of Gnus without running into
knock-on repercussions.
The ultimate goal of the changes I have in mind for Gnus is to address
exactly this: to make it more modular, to improve isolation of code
paths, and to reduce the number of semi-redundant data structures. But
the process is evidently even messier than I thought. I held back
another commit to group name encoding in an attempt to keep things
simple, but that seems to have made things even worse.
But yes, if I end up backing this change out, I'll try to break it up
into smaller commits.
> Thanks for working on gnus,
Thanks for the code review! I wish I'd gotten this to begin with.
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 17:52:02 GMT)
Full text and
rfc822 format available.
Message #68 received at 33653 <at> debbugs.gnu.org (full text, mbox):
>>>>> On Mon, 25 Mar 2019 10:35:32 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:
>> Perhaps it would be better to revert and reintroduce your
>> changes after further testing ? Taking time over this is better
>> than causing data loss for gnus users.
Eric> That's the conclusion I'm coming to, yes.
FWIW, gnus on master has been working flawlessly for me since you
pushed the hash table changes. People running master should expect
occasional breakage, so I wouldnʼt give up too soon.
Robert
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 18:18:02 GMT)
Full text and
rfc822 format available.
Message #71 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Robert Pluim <rpluim <at> gmail.com> writes:
>>>>>> On Mon, 25 Mar 2019 10:35:32 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:
> >> Perhaps it would be better to revert and reintroduce your
> >> changes after further testing ? Taking time over this is better
> >> than causing data loss for gnus users.
>
> Eric> That's the conclusion I'm coming to, yes.
>
> FWIW, gnus on master has been working flawlessly for me since you
> pushed the hash table changes. People running master should expect
> occasional breakage, so I wouldnʼt give up too soon.
Seconded.
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 19:05:02 GMT)
Full text and
rfc822 format available.
Message #74 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Robert Pluim <rpluim <at> gmail.com> writes:
> FWIW, gnus on master has been working flawlessly for me since you
> pushed the hash table changes. People running master should expect
> occasional breakage, so I wouldnʼt give up too soon.
For something as critical as Gnus and for which many various configs
exist, I would recommend that a patch can be confirmed by *at least*
three persons on top of the contributor himself.
This is perhaps the case for this patch, I don't know.
If things are okay on your side, perhaps you don't feel the pain, but
I can tell you it's here :)
Also, the fact that Gnus has been crazily stable on the master branch
from... well, from as far as I can remember, makes the situation even
worse: you really don't expect anything to break in this area.
Not a big deal, I'll wait, and thanks again Eric for both the patches
and your patience in finding possible issues!
--
Bastien
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 25 Mar 2019 20:32:02 GMT)
Full text and
rfc822 format available.
Message #77 received at submit <at> debbugs.gnu.org (full text, mbox):
On Mon 25 Mar 2019, Eric Abrahamsen wrote:
> Andy Moreton <andrewjmoreton <at> gmail.com> writes:
>> Other notes from reading the code:
>>
>> 1) In `gnus-gnus-to-quick-newsrc-format' you ignore the contents of
>> `gnus-newsrc-alist' when saving "newsrc.eld", and replace it with the
>> details from `gnus-newsrc-hashtb'. Why ? The rest of the gnus code
>> appears to treat `gnus-newsrc-alist' as the single source of truth,
>> with the hash tables being used only for faster access to it.
>
> Eventually I would like to reduce the number of data structures so that
> groups are held in `gnus-newsrc-hashtb', and ordering is kept in
> `gnus-group-list', and that's it. `gnus-newsrc-alist' would only be used
> when persisting to disk. My next proposed change (once I've recovered my
> confidence) is to turn groups into actual objects, in which case the
> alist would really just be a kind of serialization format.
ok, but if the hash table and the alist reference the same lisp objects
then the existing setup is not so bad.
> The hash table ought to be in sync with the rest of the data structure
> -- if it isn't, that's another bug.
>
>> 2) In `gnus-gnus-to-quick-newsrc-format' you dropped the code to remove
>> the dummy group from `gnus-newsrc-alist'. Why ? This internal dummy
>> group is now saved in "newsrc.eld", which is not needed.
>
> This was an error. (Though in my case, I've had the dummy group in my
> newsrc.eld for months, and it hasn't done any harm. I don't know why
> it's necessary.)
I agree that it's harmless, it just seemed to be an unnecessary change.
I expect it is there to ensure that the alist is never empty to avoid
the need to check that everywhere.
>> 3) The format of the entries in `gnus-newsrc-hashtb' has changed,
>> removing the second element. Why ?
>
> Because the old `gnus-gethash' call returned a slice of
> `gnus-newsrc-alist', where the second element was actually the group
> *before* the group you wanted, and the third element was the cdr of
> `gnus-newsrc-alist', starting with the group you wanted. This was
> undocumented, and took a bit to figure out. Now, the gethash call just
> gives you the group. Ideally, in the next set of changes, it will give
> you an object.
ok, so it sounds like the old code was trying hard to use the same lisp
objects in both the hash table and the alist. That avoids alloacting
twice the storage, and ensures that the hash table and the alist stay in
sync.
>> 4) You changed several hash tale sizesfrom 4096 to 4000, and 1024 to
>> 1000. Why ?
>
> My understanding is that using a prime number is significant when it
> comes to vector access, but that the hash table implementation is
> higher-level, where a prime number is no longer significant. If that's
> incorrect I would like to know!
None of these numbers are prime. The old numbers are powers of two,
which are reasonable for allocation sizes on a binary machine. Again,
not a terribly important change, but not really needed either.
>> Your patch contains several logical changes that would be easier to
>> understand (and bisect) as a series of patches with one logical change
>> in each patch:
>> - code layout changes
>> - add missing doc strings and code comments
>> - change hash table implementation
>> - change format of `gnus-newsrc-hashtb' entries
>> - change usage of `gnus-group-change-level'
>> - change coding of group names
>> While it can take extra work to split things up, the end result is much
>> easier to understand.
>
> In principle I agree with this completely. In practice I found it
> extraordinarily difficult to touch one part of Gnus without running into
> knock-on repercussions.
Agreed. I find that this sort of work usually goes in two phases: some
exploratory programming to decide on the path forward and the eventual
goal, followed by rearranging the changes into a logical set of
(bisectable) patches that get to that goal in small, self-contained
steps. The second half is extra work, but worth it to make it easier for
other people to review the patches (and to simplify any archaeology
needed by a later maintainer).
If changing gnus is hard, then perhaps writing new tests to document
what you have discovered about the code will help to ensure that later
changes do not change the observed behaviour.
> The ultimate goal of the changes I have in mind for Gnus is to address
> exactly this: to make it more modular, to improve isolation of code
> paths, and to reduce the number of semi-redundant data structures. But
> the process is evidently even messier than I thought. I held back
> another commit to group name encoding in an attempt to keep things
> simple, but that seems to have made things even worse.
A worthwhile goal - please do not get discouraged.
AndyM
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 26 Mar 2019 18:42:02 GMT)
Full text and
rfc822 format available.
Message #80 received at submit <at> debbugs.gnu.org (full text, mbox):
On Sat 23 Mar 2019, Andy Moreton wrote:
> On Fri 22 Mar 2019, Eric Abrahamsen wrote:
>
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>
>>>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>>>> Date: Fri, 22 Mar 2019 12:54:54 -0700
>>>>
>>>> is there anything to be done besides warning people on the mailing
>>>> lists?
>>>
>>> A warning would be nice, yes.
>>
>> Okay, I'll do that. I've already gotten a bug report :(
>
> After bootstrapping, I observe that the summary buffer does not update
> the faces correctly for nntp articles that have been read: they still
> have an unread face.
After more debugging, I have found the problem.
Updating summary lines is handled in `gnus-summary-insert-line':
(condition-case ()
(put-text-property
(point)
(progn (eval gnus-summary-line-format-spec) (point))
'gnus-number gnus-tmp-number)
(error (gnus-message 5 "Error updating the summary line")))
In my setup, `gnus-summary-line-format' used starts with "%U%R%u&score;"
to use a user format function for printing score info. That results in
calls to my user function to display summary lines:
(defun gnus-user-format-function-score (header)
(let* ((article (mail-header-number header))
(thread (gnus-id-to-thread (mail-header-id header)))
...
This now fails because `gnus-id-to-thread' sometimes throws an error,
because `gnus-newsgroup-dependencies' is nil when it is called. This
usage worked in emacs-26 and in master before your changes.
I've fixed this in my user format function with:
(defun gnus-user-format-function-score (header)
(let* ((article (mail-header-number header))
(thread (and gnus-newsgroup-dependencies
(gnus-id-to-thread (mail-header-id header))))
...
After that change, the marks are displayed correctly in the summary
buffer. I'm not sure if that is something that only worked by chance in
old code, or if it is a regression, but it is a change of behaviour.
AndyM
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 26 Mar 2019 19:50:02 GMT)
Full text and
rfc822 format available.
Message #83 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Andy Moreton <andrewjmoreton <at> gmail.com> writes:
> On Sat 23 Mar 2019, Andy Moreton wrote:
>
>> On Fri 22 Mar 2019, Eric Abrahamsen wrote:
>>
>>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>>
>>>>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>>>>> Date: Fri, 22 Mar 2019 12:54:54 -0700
>>>>>
>>>>> is there anything to be done besides warning people on the mailing
>>>>> lists?
>>>>
>>>> A warning would be nice, yes.
>>>
>>> Okay, I'll do that. I've already gotten a bug report :(
>>
>> After bootstrapping, I observe that the summary buffer does not update
>> the faces correctly for nntp articles that have been read: they still
>> have an unread face.
>
> After more debugging, I have found the problem.
>
> Updating summary lines is handled in `gnus-summary-insert-line':
> (condition-case ()
> (put-text-property
> (point)
> (progn (eval gnus-summary-line-format-spec) (point))
> 'gnus-number gnus-tmp-number)
> (error (gnus-message 5 "Error updating the summary line")))
>
> In my setup, `gnus-summary-line-format' used starts with "%U%R%u&score;"
> to use a user format function for printing score info. That results in
> calls to my user function to display summary lines:
>
> (defun gnus-user-format-function-score (header)
> (let* ((article (mail-header-number header))
> (thread (gnus-id-to-thread (mail-header-id header)))
> ...
>
> This now fails because `gnus-id-to-thread' sometimes throws an error,
> because `gnus-newsgroup-dependencies' is nil when it is called. This
> usage worked in emacs-26 and in master before your changes.
>
> I've fixed this in my user format function with:
>
> (defun gnus-user-format-function-score (header)
> (let* ((article (mail-header-number header))
> (thread (and gnus-newsgroup-dependencies
> (gnus-id-to-thread (mail-header-id header))))
> ...
>
> After that change, the marks are displayed correctly in the summary
> buffer. I'm not sure if that is something that only worked by chance in
> old code, or if it is a regression, but it is a change of behaviour.
Yes, the format spec is eval'ed twice (to set summary buffer positions)
before the threads are actually built, and `gnus-newsgroup-dependencies'
is nil at that point. Back when this was a call to `intern', it would
just silently return nil, now it errors. Your fix works, but I could
also have `gnus-id-to-thread' return nil if
`gnus-newsgroup-dependencies' is nil. That would mimic Gnus' previous
behavior.
Thanks,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 26 Mar 2019 20:02:01 GMT)
Full text and
rfc822 format available.
Message #86 received at submit <at> debbugs.gnu.org (full text, mbox):
Andy Moreton <andrewjmoreton <at> gmail.com> writes:
> On Mon 25 Mar 2019, Eric Abrahamsen wrote:
>
>> Andy Moreton <andrewjmoreton <at> gmail.com> writes:
>>> Other notes from reading the code:
>>>
>>> 1) In `gnus-gnus-to-quick-newsrc-format' you ignore the contents of
>>> `gnus-newsrc-alist' when saving "newsrc.eld", and replace it with the
>>> details from `gnus-newsrc-hashtb'. Why ? The rest of the gnus code
>>> appears to treat `gnus-newsrc-alist' as the single source of truth,
>>> with the hash tables being used only for faster access to it.
>>
>> Eventually I would like to reduce the number of data structures so that
>> groups are held in `gnus-newsrc-hashtb', and ordering is kept in
>> `gnus-group-list', and that's it. `gnus-newsrc-alist' would only be used
>> when persisting to disk. My next proposed change (once I've recovered my
>> confidence) is to turn groups into actual objects, in which case the
>> alist would really just be a kind of serialization format.
>
> ok, but if the hash table and the alist reference the same lisp objects
> then the existing setup is not so bad.
There's nothing wrong with the existing setup here! I'm just laying the
groundwork for the next round of changes.
>> The hash table ought to be in sync with the rest of the data structure
>> -- if it isn't, that's another bug.
>>
>>> 2) In `gnus-gnus-to-quick-newsrc-format' you dropped the code to remove
>>> the dummy group from `gnus-newsrc-alist'. Why ? This internal dummy
>>> group is now saved in "newsrc.eld", which is not needed.
>>
>> This was an error. (Though in my case, I've had the dummy group in my
>> newsrc.eld for months, and it hasn't done any harm. I don't know why
>> it's necessary.)
>
> I agree that it's harmless, it just seemed to be an unnecessary change.
> I expect it is there to ensure that the alist is never empty to avoid
> the need to check that everywhere.
Yes, for sure, and I didn't mean to leave it in there, that's just a bug.
>>> 3) The format of the entries in `gnus-newsrc-hashtb' has changed,
>>> removing the second element. Why?
>>
>> Because the old `gnus-gethash' call returned a slice of
>> `gnus-newsrc-alist', where the second element was actually the group
>> *before* the group you wanted, and the third element was the cdr of
>> `gnus-newsrc-alist', starting with the group you wanted. This was
>> undocumented, and took a bit to figure out. Now, the gethash call just
>> gives you the group. Ideally, in the next set of changes, it will give
>> you an object.
>
> ok, so it sounds like the old code was trying hard to use the same lisp
> objects in both the hash table and the alist. That avoids alloacting
> twice the storage, and ensures that the hash table and the alist stay in
> sync.
Yes, I'm looking forward to when the groups are actual objects, . But in
the meantime, the lisp objects are still the same, I haven't doubled
storage (at least `eq' returns t, so I assume that's what that means).
>>> 4) You changed several hash tale sizesfrom 4096 to 4000, and 1024 to
>>> 1000. Why?
>>
>> My understanding is that using a prime number is significant when it
>> comes to vector access, but that the hash table implementation is
>> higher-level, where a prime number is no longer significant. If that's
>> incorrect I would like to know!
>
> None of these numbers are prime. The old numbers are powers of two,
> which are reasonable for allocation sizes on a binary machine. Again,
> not a terribly important change, but not really needed either.
Yes, sorry, powers of two. It's true it's a do-nothing change, I guess
I've been erring on the side of trying to make the code more
self-explanatory, less "odd". Probably this change didn't need to be
made, but it seems about as un-risky as a code change could be.
>>> Your patch contains several logical changes that would be easier to
>>> understand (and bisect) as a series of patches with one logical change
>>> in each patch:
>>> - code layout changes
>>> - add missing doc strings and code comments
>>> - change hash table implementation
>>> - change format of `gnus-newsrc-hashtb' entries
>>> - change usage of `gnus-group-change-level'
>>> - change coding of group names
>>> While it can take extra work to split things up, the end result is much
>>> easier to understand.
>>
>> In principle I agree with this completely. In practice I found it
>> extraordinarily difficult to touch one part of Gnus without running into
>> knock-on repercussions.
>
> Agreed. I find that this sort of work usually goes in two phases: some
> exploratory programming to decide on the path forward and the eventual
> goal, followed by rearranging the changes into a logical set of
> (bisectable) patches that get to that goal in small, self-contained
> steps. The second half is extra work, but worth it to make it easier for
> other people to review the patches (and to simplify any archaeology
> needed by a later maintainer).
>
> If changing gnus is hard, then perhaps writing new tests to document
> what you have discovered about the code will help to ensure that later
> changes do not change the observed behaviour.
The previous commit did add some new Gnus tests -- perhaps the first!
I'm planning more, once Gnus group are actual objects. Everything's so
interconnected now (and data-reliant) that it's very hard to write
effective tests.
>> The ultimate goal of the changes I have in mind for Gnus is to address
>> exactly this: to make it more modular, to improve isolation of code
>> paths, and to reduce the number of semi-redundant data structures. But
>> the process is evidently even messier than I thought. I held back
>> another commit to group name encoding in an attempt to keep things
>> simple, but that seems to have made things even worse.
>
> A worthwhile goal - please do not get discouraged.
Thank you,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 26 Mar 2019 21:45:02 GMT)
Full text and
rfc822 format available.
Message #89 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> Hi,
>
> Gnus got not to work for groups of which the group name contains
> non-ASCII letters. For instance, I got this error when trying
> to update the "nnml:テスト" group using `M-g'[1]:
Thanks for this report. When you have a moment will you please try the
attached patch and see if it fixes the problem?
Yours,
Eric
[fix-nnml-assoc.diff (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Wed, 27 Mar 2019 04:55:01 GMT)
Full text and
rfc822 format available.
Message #92 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Tue, 26 Mar 2019 14:44:32 -0700, Eric Abrahamsen wrote:
> Thanks for this report. When you have a moment will you please try the
> attached patch and see if it fixes the problem?
Works great. Now I can enter and read the nnml:テスト group.
Thank you. But there seem to be some more fix needed.
The group level of nnml:テスト is 1 and there are some unread
articles in it, however the group is not listed in the Group
buffer when I launch Gnus by `C-u 1 M-x gnus RET' (it also
happened when I worked on non-ASCII group names years ago, but
I don't recall how I fixed it, sorry). Typing L helps it.
I tried changing `assoc' with `assoc-string' in nnrss.el, too.
It works for non-ASCII group names so far, though only I glanced
a few feeds. A diff is attached.
Regards,
[Message part 2 (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Wed, 27 Mar 2019 18:55:02 GMT)
Full text and
rfc822 format available.
Message #95 received at submit <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> On Tue, 26 Mar 2019 14:44:32 -0700, Eric Abrahamsen wrote:
>> Thanks for this report. When you have a moment will you please try the
>> attached patch and see if it fixes the problem?
>
> Works great. Now I can enter and read the nnml:テスト group.
> Thank you. But there seem to be some more fix needed.
>
> The group level of nnml:テスト is 1 and there are some unread
> articles in it, however the group is not listed in the Group
> buffer when I launch Gnus by `C-u 1 M-x gnus RET' (it also
> happened when I worked on non-ASCII group names years ago, but
> I don't recall how I fixed it, sorry). Typing L helps it.
Excellent -- I know where to go from here (I've also run into this
before). I hope I've nearly got this sorted.
> I tried changing `assoc' with `assoc-string' in nnrss.el, too.
> It works for non-ASCII group names so far, though only I glanced
> a few feeds. A diff is attached.
Thanks!
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Wed, 27 Mar 2019 21:28:02 GMT)
Full text and
rfc822 format available.
Message #98 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 03/27/19 13:54 PM, Katsumi Yamaoka wrote:
> On Tue, 26 Mar 2019 14:44:32 -0700, Eric Abrahamsen wrote:
>> Thanks for this report. When you have a moment will you please try the
>> attached patch and see if it fixes the problem?
>
> Works great. Now I can enter and read the nnml:テスト group.
> Thank you. But there seem to be some more fix needed.
>
> The group level of nnml:テスト is 1 and there are some unread
> articles in it, however the group is not listed in the Group
> buffer when I launch Gnus by `C-u 1 M-x gnus RET' (it also
> happened when I worked on non-ASCII group names years ago, but
> I don't recall how I fixed it, sorry). Typing L helps it.
I think I've found the source of the problem. The error you and others
are seeing comes when the group has an entry in gnus-newsrc-hashtb (ie,
you're subscribed to the group) but not in gnus-active-hashtb (Gnus
can't tell if the group has articles).
Gnus writes "nnml:テスト" to .newsrc.eld as
"nnml:\343\203\206\343\202\271\343\203\210", and that's used as the key
in gnus-newsrc-hashtb. But when it reads the active info from nnml, the
(unibyte) buffer contains:
nnml:\343\203\206\343\202\271\343\203\210
As a symbol, not a string. My current code then uses `symbol-name' to
turn it into a string, but it's crucially *not the same string* as the
one saved to.newsrc.eld, even though it looks the same in edebug, the
*scratch* buffer, etc.
Hence a mismatch between the two keys in the two hash-tables, and things
going haywire.
Obviously this arises from a misunderstanding on my part about how
bytes and strings interact. The code in question is at line 2145 in
gnus-start.el -- it uses `read' to pick pieces out of the buffer.
Right now my best idea is to swap out `read' for a some sort of regexp
crawl, but I am open to suggestions.
Thanks,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Wed, 27 Mar 2019 22:11:02 GMT)
Full text and
rfc822 format available.
Message #101 received at submit <at> debbugs.gnu.org (full text, mbox):
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
> On 03/27/19 13:54 PM, Katsumi Yamaoka wrote:
>> On Tue, 26 Mar 2019 14:44:32 -0700, Eric Abrahamsen wrote:
>>> Thanks for this report. When you have a moment will you please try the
>>> attached patch and see if it fixes the problem?
>>
>> Works great. Now I can enter and read the nnml:テスト group.
>> Thank you. But there seem to be some more fix needed.
>>
>> The group level of nnml:テスト is 1 and there are some unread
>> articles in it, however the group is not listed in the Group
>> buffer when I launch Gnus by `C-u 1 M-x gnus RET' (it also
>> happened when I worked on non-ASCII group names years ago, but
>> I don't recall how I fixed it, sorry). Typing L helps it.
>
> I think I've found the source of the problem. The error you and others
> are seeing comes when the group has an entry in gnus-newsrc-hashtb (ie,
> you're subscribed to the group) but not in gnus-active-hashtb (Gnus
> can't tell if the group has articles).
>
> Gnus writes "nnml:テスト" to .newsrc.eld as
> "nnml:\343\203\206\343\202\271\343\203\210", and that's used as the key
> in gnus-newsrc-hashtb. But when it reads the active info from nnml, the
> (unibyte) buffer contains:
>
> nnml:\343\203\206\343\202\271\343\203\210
>
> As a symbol, not a string.
Sorry, as raw bytes I guess. But `read' interprets it as a symbol.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sat, 30 Mar 2019 12:10:02 GMT)
Full text and
rfc822 format available.
Message #104 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Hi Eric,
It seems you are implementing some pretty drastic upgrades to Gnus.
Is there a write-up on the planned changes, benefits, what is going to be
affected and what to expect ?
I'm interested and would like to read it.
On Sun, Mar 24 2019, Eric Abrahamsen wrote:
> On 03/24/19 23:29 PM, Bastien wrote:
>> Hi Eric,
>>
>> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>>
>>> The bummer is that you don't get an error until you run Gnus, so it just
>>> looks like I broke Gnus :(
>>
>> I hope not! :)
>>
>> I have not followed this change.
>> Can it corrupt the .newsrc.eld file?
>
> Worst come to worst it will write non-ascii group names as unibyte when
> they should be utf-8, or else the other way around. I thought I could
> avoid dealing with this question for now, and just preserve Gnus'
> original method of saving group names, but it looks like I may have
> forced the issue instead...
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sun, 31 Mar 2019 23:30:02 GMT)
Full text and
rfc822 format available.
Message #107 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 03/27/19 14:27 PM, Eric Abrahamsen wrote:
> On 03/27/19 13:54 PM, Katsumi Yamaoka wrote:
>> On Tue, 26 Mar 2019 14:44:32 -0700, Eric Abrahamsen wrote:
>>> Thanks for this report. When you have a moment will you please try the
>>> attached patch and see if it fixes the problem?
>>
>> Works great. Now I can enter and read the nnml:テスト group.
>> Thank you. But there seem to be some more fix needed.
>>
>> The group level of nnml:テスト is 1 and there are some unread
>> articles in it, however the group is not listed in the Group
>> buffer when I launch Gnus by `C-u 1 M-x gnus RET' (it also
>> happened when I worked on non-ASCII group names years ago, but
>> I don't recall how I fixed it, sorry). Typing L helps it.
>
> I think I've found the source of the problem. The error you and others
> are seeing comes when the group has an entry in gnus-newsrc-hashtb (ie,
> you're subscribed to the group) but not in gnus-active-hashtb (Gnus
> can't tell if the group has articles).
Okay, it might be as simple as the attached patch, applied to current
master. Basically this forces nnmail to always write the group names as
strings, which are then read correctly later on.
Also cc'ing Bastien and Adam, who have been very helpful with
road-testing.
Thanks to all,
Eric
[0001-WIP-on-nnmail-group-names.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sun, 31 Mar 2019 23:30:02 GMT)
Full text and
rfc822 format available.
Message #110 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 03/30/19 14:09 PM, Deus Max wrote:
> Hi Eric,
>
> It seems you are implementing some pretty drastic upgrades to Gnus.
>
> Is there a write-up on the planned changes, benefits, what is going to be
> affected and what to expect ?
>
> I'm interested and would like to read it.
There is a writeup! Sort of -- I called it a roadmap. Originally I stuck
it in a branch called scratch/gnus-roadmap but soon realized it didn't
make any sense to keep track of something like this in a git branch.
I've attached the current version.
Obviously, this is _not_ meant as a list of "changes I will unilaterally
implement in Gnus", but as a document for record-keeping and discussion.
I didn't get much discussion, but I know that's to be expected!
Eric
[roadmap.org (application/vnd.lotus-organizer, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 01 Apr 2019 20:19:02 GMT)
Full text and
rfc822 format available.
Message #113 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Eric writes:
> Okay, it might be as simple as the attached patch, applied to current
> master. Basically this forces nnmail to always write the group names as
> strings, which are then read correctly later on.
>
> Also cc'ing Bastien and Adam, who have been very helpful with
> road-testing.
I'm running with this patch now.
I started by taking a copy of my .newsrc.eld, so I could return to it.
I swear the first time I started Gnus from master with the patch, my
groups with æøå showed up as previously. But then I stopped Emacs and
copied the .newsrc.eld back, and somehow they did not show up again.
It makes no sense to me, unless there is state somewhere besides
.newsrc.eld that I don't know of.
Since then Gnus from Emacs master with the patch has behaved as
expected.
Slighly confused, but happy that it works,
Adam
--
"Everything needs to change. Adam Sjøgren
And it has to start today." asjo <at> koldfront.dk
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 01 Apr 2019 20:58:01 GMT)
Full text and
rfc822 format available.
Message #116 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Adam Sjøgren <asjo <at> koldfront.dk> writes:
> Eric writes:
>
>> Okay, it might be as simple as the attached patch, applied to current
>> master. Basically this forces nnmail to always write the group names as
>> strings, which are then read correctly later on.
>>
>> Also cc'ing Bastien and Adam, who have been very helpful with
>> road-testing.
>
> I'm running with this patch now.
>
> I started by taking a copy of my .newsrc.eld, so I could return to it.
>
> I swear the first time I started Gnus from master with the patch, my
> groups with æøå showed up as previously. But then I stopped Emacs and
> copied the .newsrc.eld back, and somehow they did not show up again.
>
> It makes no sense to me, unless there is state somewhere besides
> .newsrc.eld that I don't know of.
>
> Since then Gnus from Emacs master with the patch has behaved as
> expected.
>
>
> Slighly confused, but happy that it works,
Thank you for testing. This whole process has made me suspect my sanity
(or the interference of cosmic rays), so I'm kind of glad you're
confused, too :)
I am even now able to reproduce bugs *once* (for instance the "j" bug
Lars reported yesterday) in my testing setup (the gnus-mock package),
but not again thereafter. If I restart gnus-mock, I can reproduce it
once again, but only once.
I think once I eventually figure out why, the problem will be solved!
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 01 Apr 2019 22:40:02 GMT)
Full text and
rfc822 format available.
Message #119 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Sun, Mar 31 2019, Eric Abrahamsen wrote:
> On 03/30/19 14:09 PM, Deus Max wrote:
>> Hi Eric,
>>
>> It seems you are implementing some pretty drastic upgrades to Gnus.
>>
>> Is there a write-up on the planned changes, benefits, what is going to be
>> affected and what to expect ?
>>
>> I'm interested and would like to read it.
>
> There is a writeup! Sort of -- I called it a roadmap. Originally I stuck
> it in a branch called scratch/gnus-roadmap but soon realized it didn't
> make any sense to keep track of something like this in a git branch.
> I've attached the current version.
>
> Obviously, this is _not_ meant as a list of "changes I will unilaterally
> implement in Gnus", but as a document for record-keeping and discussion.
> I didn't get much discussion, but I know that's to be expected!
>
> Eric
Hi Eric,
A roadmap, yes! that's the word I was looking for.
Thank you, very interesting.
I always wondered why there are multiple select method variables (as
~gnus-secondary-select-methods~), instead of one ~gnus-select-methods~.
I see there are plenty of things to simplify, such as group
reading/selection. Also i think, jumping to an article should be easier
and more stable.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 02 Apr 2019 05:24:02 GMT)
Full text and
rfc822 format available.
Message #122 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 04/02/19 01:39 AM, Deus Max wrote:
> On Sun, Mar 31 2019, Eric Abrahamsen wrote:
>
>> On 03/30/19 14:09 PM, Deus Max wrote:
>>> Hi Eric,
>>>
>>> It seems you are implementing some pretty drastic upgrades to Gnus.
>>>
>>> Is there a write-up on the planned changes, benefits, what is going to be
>>> affected and what to expect ?
>>>
>>> I'm interested and would like to read it.
>>
>> There is a writeup! Sort of -- I called it a roadmap. Originally I stuck
>> it in a branch called scratch/gnus-roadmap but soon realized it didn't
>> make any sense to keep track of something like this in a git branch.
>> I've attached the current version.
>>
>> Obviously, this is _not_ meant as a list of "changes I will unilaterally
>> implement in Gnus", but as a document for record-keeping and discussion.
>> I didn't get much discussion, but I know that's to be expected!
>>
>> Eric
>
> Hi Eric,
>
> A roadmap, yes! that's the word I was looking for.
>
> Thank you, very interesting.
> I always wondered why there are multiple select method variables (as
> ~gnus-secondary-select-methods~), instead of one ~gnus-select-methods~.
One thing that makes sense is distinguishing between servers you've
defined in your gnus.el file, versus servers you've added via the
*Server* buffer in Gnus. As things are set up currently, those do need
to be held in different variables. Perhaps that might change in the
future, but this much makes sense.
What makes less sense to me is the distinction between the primary
select method and the secondary select methods. So far as I can tell,
the only practical difference between the two is that groups on the
primary select method don't have the method name prefixed to the group
name. That's something that could be profitably simplified, I think
(there's a ton of code that just checks whether the current method is
primary or secondary).
> I see there are plenty of things to simplify, such as group
> reading/selection. Also i think, jumping to an article should be easier
> and more stable.
Noted! Please do let me know if you have any more ideas or responses to
that document.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 02 Apr 2019 16:44:02 GMT)
Full text and
rfc822 format available.
Message #125 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Eric writes:
> Thank you for testing. This whole process has made me suspect my sanity
> (or the interference of cosmic rays), so I'm kind of glad you're
> confused, too :)
;-)
> I am even now able to reproduce bugs *once* (for instance the "j" bug
> Lars reported yesterday) in my testing setup (the gnus-mock package),
> but not again thereafter. If I restart gnus-mock, I can reproduce it
> once again, but only once.
I was a little too quick to declare victory. While I don't see any
duplicates, nor any problems with the nnml-groups, I still see the
nntp-groups with ø and å in the name show up with a * in front.
If I enter the group, exit it again, the * changes to 0, and if I then
fetch new news, it doesn't show up. Until I start Gnus the next time.
Then it's back with *.
> I think once I eventually figure out why, the problem will be solved!
I think that is very likely!
Best regards,
Adam
--
"Everything needs to change. Adam Sjøgren
And it has to start today." asjo <at> koldfront.dk
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Wed, 03 Apr 2019 22:17:02 GMT)
Full text and
rfc822 format available.
Message #128 received at 33653 <at> debbugs.gnu.org (full text, mbox):
One more regression.
Performing `B <DEL>' or `B m' in the "nnml:inbox" group, i.e.,
that of an ASCII name, does not update the NOV database; in the
summary buffer the face for deleted or moved article lines get
changed so as to be absent at that time, however they revive
when re-entering to the group's summary.
`nnml-generate-nov-databases-directory' helps (marks survives).
I'd like to have time to fix it and others from the next week,
hopefully.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Wed, 03 Apr 2019 22:38:01 GMT)
Full text and
rfc822 format available.
Message #131 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> One more regression.
>
> Performing `B <DEL>' or `B m' in the "nnml:inbox" group, i.e.,
> that of an ASCII name, does not update the NOV database; in the
> summary buffer the face for deleted or moved article lines get
> changed so as to be absent at that time, however they revive
> when re-entering to the group's summary.
> `nnml-generate-nov-databases-directory' helps (marks survives).
>
> I'd like to have time to fix it and others from the next week,
> hopefully.
Thanks for your continued reports! Was this with the most recent WIP
commit I posted a day or so ago, or with origin/master? I'm working on
this right now, and hope to have an interim fix today.
If you do think you'll have a bit of time to put into this, what would
be _really_ awesome is if you'd help me check over the next step, which
will be keeping group names decoded everywhere inside Gnus. That ought
to simplify this whole process, and avoid similar types of bugs in the
future. That was the original point of this whole patch series; I think
my mistake was breaking the series up at the wrong point.
Anyway, I will solve as much as I can in the next week. It would be
really great to have experienced eyes check my work, though.
Thanks,
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 05 Apr 2019 04:26:02 GMT)
Full text and
rfc822 format available.
Message #134 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Wed, 03 Apr 2019 15:36:58 -0700, Eric Abrahamsen wrote:
> Thanks for your continued reports! Was this with the most recent WIP
> commit I posted a day or so ago, or with origin/master? I'm working on
> this right now, and hope to have an interim fix today.
It's origin/master, I've been installing it almost everyday of
weekday. ;)
I found a cause of the `move article' not to work and pushed a
fix to master. That is having changed `while loop' to `dolist'
in `gnus-summary-move-article'. Because of this, the 5th arg
`last' passed to `gnus-request-move-article' gets always nil,
and that function never saves NOV and also ACTIVE at least for
an nnml group.
This is a good lesson to me; changing `while loop' to `dolist'
should be done carefully.
Regards,
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 05 Apr 2019 06:45:01 GMT)
Full text and
rfc822 format available.
Message #137 received at 33653 <at> debbugs.gnu.org (full text, mbox):
One more regression. :)
In the group buffer the `j' command (`gnus-group-jump-to-group')
does not work for a group that is not activated yet. The point
goes to the top. Such a group will appear with the `*' mark by
the `l' or the `L' command like the followig:
*: nntp+gmane:gmane.emacs.devel
Similarly the "#" mark put in front of a marked such group will
not disappear after processing a certain job (normally
`gnus-group-get-new-news-this-group' on me) as follows:
# *: nntp+gmane:gmane.emacs.devel
↓↓↓
# 168: nntp+gmane:gmane.emacs.devel
The cause is the same; such non-activated groups are not listed
in `gnus-active-hashtb':
(gethash "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb)
=> nil
But in Gnus bundled with Emacs 26.1.92 it returns a group name
symbol, though its value (i.e., active) is nil:
(gnus-intern-safe "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb)
=> nntp+gmane:gmane\.emacs\.devel
(symbol-value
(gnus-intern-safe "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb))
=> nil
In my Gnus, there are many non-activated groups at just after
lauching Gnus with the prefix arg `1', because I set the group
level `1' to the groups normally used and larger level to the
others. In the starting time of work, my habit is to check
mails in the active groups for a while and then activate and
check groups of larger level.
Running `gnus-group-get-new-news-this-group' again on a group in
question helps (the "#" mark disappears) but I wish it could work
as before. How about making the value for an inactive group in
hashtb to `t' or something, that means the group exists but not
activated?
Regards,
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 05 Apr 2019 11:03:02 GMT)
Full text and
rfc822 format available.
Message #140 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> I found a cause of the `move article' not to work and pushed a
> fix to master. That is having changed `while loop' to `dolist'
> in `gnus-summary-move-article'. Because of this, the 5th arg
> `last' passed to `gnus-request-move-article' gets always nil,
> and that function never saves NOV and also ACTIVE at least for
> an nnml group.
Oops, mea culpa.
> This is a good lesson to me; changing `while loop' to `dolist'
> should be done carefully.
I tried to be careful, but clearly wasn't careful enough.
Thanks for fixing this, and sorry for the trouble.
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 05 Apr 2019 11:03:04 GMT)
Full text and
rfc822 format available.
Message #143 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> One more regression. :)
>
> In the group buffer the `j' command (`gnus-group-jump-to-group')
> does not work for a group that is not activated yet. The point
> goes to the top. Such a group will appear with the `*' mark by
> the `l' or the `L' command like the followig:
>
> *: nntp+gmane:gmane.emacs.devel
>
> Similarly the "#" mark put in front of a marked such group will
> not disappear after processing a certain job (normally
> `gnus-group-get-new-news-this-group' on me) as follows:
>
> # *: nntp+gmane:gmane.emacs.devel
> ↓↓↓
> # 168: nntp+gmane:gmane.emacs.devel
>
> The cause is the same; such non-activated groups are not listed
> in `gnus-active-hashtb':
>
> (gethash "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb)
> => nil
>
> But in Gnus bundled with Emacs 26.1.92 it returns a group name
> symbol, though its value (i.e., active) is nil:
>
> (gnus-intern-safe "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb)
> => nntp+gmane:gmane\.emacs\.devel
> (symbol-value
> (gnus-intern-safe "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb))
> => nil
>
> In my Gnus, there are many non-activated groups at just after
> lauching Gnus with the prefix arg `1', because I set the group
> level `1' to the groups normally used and larger level to the
> others. In the starting time of work, my habit is to check
> mails in the active groups for a while and then activate and
> check groups of larger level.
>
> Running `gnus-group-get-new-news-this-group' again on a group in
> question helps (the "#" mark disappears) but I wish it could work
> as before. How about making the value for an inactive group in
> hashtb to `t' or something, that means the group exists but not
> activated?
Would this fix bug#35070? I.e. are you describing the same bug?
Thanks,
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 05 Apr 2019 20:19:02 GMT)
Full text and
rfc822 format available.
Message #146 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 04/05/19 15:44 PM, Katsumi Yamaoka wrote:
> One more regression. :)
>
> In the group buffer the `j' command (`gnus-group-jump-to-group')
> does not work for a group that is not activated yet. The point
> goes to the top. Such a group will appear with the `*' mark by
> the `l' or the `L' command like the followig:
>
> *: nntp+gmane:gmane.emacs.devel
>
> Similarly the "#" mark put in front of a marked such group will
> not disappear after processing a certain job (normally
> `gnus-group-get-new-news-this-group' on me) as follows:
>
> # *: nntp+gmane:gmane.emacs.devel
> ↓↓↓
> # 168: nntp+gmane:gmane.emacs.devel
>
> The cause is the same; such non-activated groups are not listed
> in `gnus-active-hashtb':
>
> (gethash "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb)
> => nil
>
> But in Gnus bundled with Emacs 26.1.92 it returns a group name
> symbol, though its value (i.e., active) is nil:
>
> (gnus-intern-safe "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb)
> => nntp+gmane:gmane\.emacs\.devel
> (symbol-value
> (gnus-intern-safe "nntp+gmane:gmane.emacs.devel" gnus-active-hashtb))
> => nil
>
> In my Gnus, there are many non-activated groups at just after
> lauching Gnus with the prefix arg `1', because I set the group
> level `1' to the groups normally used and larger level to the
> others. In the starting time of work, my habit is to check
> mails in the active groups for a while and then activate and
> check groups of larger level.
>
> Running `gnus-group-get-new-news-this-group' again on a group in
> question helps (the "#" mark disappears) but I wish it could work
> as before. How about making the value for an inactive group in
> hashtb to `t' or something, that means the group exists but not
> activated?
I think the right solution might be to check
(member group-name (hash-table-keys gnus-active-hashtb))
Or else use the DFLT argument to `gethash' together with a
`make-symbol', to see if "nil" means "no such key", or if it means "key
with nil value".
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 08 Apr 2019 01:48:02 GMT)
Full text and
rfc822 format available.
Message #149 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Fri, 05 Apr 2019 12:02:32 +0100, Basil L. Contovounesios wrote:
> Would this fix bug#35070? I.e. are you describing the same bug?
Well, what I can reproduce doesn't seem to be exactly the same
with Lars' though, I guess the root cause would probably be:
(gethash "inactive-or-non-existent-group" gnus-active-hashtb)
returns nil. Maybe "undisplayed group" that Lars says means a
group that is inactive but in existence. Performing Lars'
recipe in my Gnus produces things as follows:
(gnus-group-jump-to-group "inactive-group")
↓
(gnus-group-goto-group "inactive-group")
↓
(let ((active (gethash "inactive-group" gnus-active-hashtb)))
↓
(gnus-text-property-search 'gnus-group active nil 'goto)
Where `active' is nil but `gnus-text-property-search' finds it
at (point-min) and goes to there. Though the "inactive-group"
does not appear there. But if forcing `active' to be an
inactive group name in the above flow, it appears at, as Lars
says, "either at the start of the buffer (before all topics) or
at the start of one of the topics it doesn't belong in" as
*: nnfoo:bar
and the point is positioned there.
Regards,
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 08 Apr 2019 01:59:01 GMT)
Full text and
rfc822 format available.
Message #152 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Fri, 05 Apr 2019 13:18:47 -0700, Eric Abrahamsen wrote:
> I think the right solution might be to check
> (member group-name (hash-table-keys gnus-active-hashtb))
It doesn't help because it returns nil for a non-activated (but
existing) group name. Probably the solution would be to make
hashtb list all the groups existing in `gnus-newsrc-alist' at
the startup. It will take time but is necessary, I think.
> Or else use the DFLT argument to `gethash' together with a
> `make-symbol', to see if "nil" means "no such key", or if it means "key
> with nil value".
Doesn't it make any group valid no matter what is really exists
or not?
Regards,
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 08 Apr 2019 04:32:02 GMT)
Full text and
rfc822 format available.
Message #155 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Mon, 08 Apr 2019 10:58:30 +0900, Katsumi Yamaoka wrote:
> Probably the solution would be to make hashtb list all
> the groups existing in `gnus-newsrc-alist' at the startup.
I found `gnus-newsrc-hashtb' has all the group names that
`gnus-newsrc-alist' holds. So, I'm not quite sure since I have
not tested it thoroughly yet though, the problem I brought up
--- j doesn't work for groups having not activated --- can be
fixed with:
--- gnus-group.el~ 2019-04-01 21:28:54.404887500 +0000
+++ gnus-group.el 2019-04-08 04:19:19.117882300 +0000
@@ -2562,3 +2562,3 @@
(let ((start (point))
- (active (and (gethash group gnus-active-hashtb)
+ (active (and (gethash group gnus-newsrc-hashtb)
group)))
;; The variable name `active' seems to be better to change to
;; `name' or something since it is a group name string, not
;; active data here.
This looks to fix Lars' bug#35070 as well. I'll check and check
it with investigating other issues...
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 08 Apr 2019 08:14:01 GMT)
Full text and
rfc822 format available.
Message #158 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Wed, 27 Mar 2019 13:54:42 +0900, Katsumi Yamaoka wrote:
> The group level of nnml:テスト is 1 and there are some unread
> articles in it, however the group is not listed in the Group
> buffer when I launch Gnus by `C-u 1 M-x gnus RET'...
I found a cause of it not to be listed in the group buffer just
after launching Gnus.
The group entry in the ~/Mail/active file is:
テスト 5 1 y
The file coding is utf-8-unix. So, the binary expression of the
group name is \343\203\206\343\202\271\343\203\210, where \343
is a single character, and this form is what the hash tables use
(in the ~/.newsrc.eld file, "\343" consists of four characters
"\", "3", "4", and "3", though).
`gnus-active-to-gnus-format' reads it in the following way:
・Load the active file to the " *nntpd*" buffer in the binary
mode. Copy it to another temp buffer. There \343 is a single
character.
・Read the group name using `(read (current-buffer))'.
Then it is read as a symbol; \343 is still a single character.
・Convert it to a string using `symbol-name'.
\343 is expanded into the one consists of four characters. :<
It should never match to the one in the hash tables.
Why the single-char to four-chars conversion happens is that
the buffer where those processes are done is in the multibyte
mode. So, the patch attached below solves the problem. I'm
going to test it for the other back ends...
[Message part 2 (text/x-patch, inline)]
--- gnus-start.el~ 2019-03-25 21:22:46.184139100 +0000
+++ gnus-start.el 2019-04-08 08:07:41.906976900 +0000
@@ -2139,3 +2139,3 @@
- (with-temp-buffer
+ (mm-with-unibyte-buffer
(insert-buffer-substring cur)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 08 Apr 2019 08:59:01 GMT)
Full text and
rfc822 format available.
Message #161 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Apr 08 2019, Katsumi Yamaoka <yamaoka <at> jpl.org> wrote:
> ・Load the active file to the " *nntpd*" buffer in the binary
> mode. Copy it to another temp buffer. There \343 is a single
> character.
> ・Read the group name using `(read (current-buffer))'.
> Then it is read as a symbol; \343 is still a single character.
> ・Convert it to a string using `symbol-name'.
> \343 is expanded into the one consists of four characters. :<
It's not expanded. That's just the print representation.
Andreas.
--
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Mon, 08 Apr 2019 18:33:02 GMT)
Full text and
rfc822 format available.
Message #164 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/08/19 17:13 PM, Katsumi Yamaoka wrote:
> On Wed, 27 Mar 2019 13:54:42 +0900, Katsumi Yamaoka wrote:
>> The group level of nnml:テスト is 1 and there are some unread
>> articles in it, however the group is not listed in the Group
>> buffer when I launch Gnus by `C-u 1 M-x gnus RET'...
>
> I found a cause of it not to be listed in the group buffer just
> after launching Gnus.
>
> The group entry in the ~/Mail/active file is:
>
> テスト 5 1 y
>
> The file coding is utf-8-unix. So, the binary expression of the
> group name is \343\203\206\343\202\271\343\203\210, where \343
> is a single character, and this form is what the hash tables use
> (in the ~/.newsrc.eld file, "\343" consists of four characters
> "\", "3", "4", and "3", though).
>
> `gnus-active-to-gnus-format' reads it in the following way:
>
> ・Load the active file to the " *nntpd*" buffer in the binary
> mode. Copy it to another temp buffer. There \343 is a single
> character.
> ・Read the group name using `(read (current-buffer))'.
> Then it is read as a symbol; \343 is still a single character.
> ・Convert it to a string using `symbol-name'.
> \343 is expanded into the one consists of four characters. :<
> It should never match to the one in the hash tables.
>
> Why the single-char to four-chars conversion happens is that
> the buffer where those processes are done is in the multibyte
> mode. So, the patch attached below solves the problem. I'm
> going to test it for the other back ends...
>
> --- gnus-start.el~ 2019-03-25 21:22:46.184139100 +0000
> +++ gnus-start.el 2019-04-08 08:07:41.906976900 +0000
> @@ -2139,3 +2139,3 @@
>
> - (with-temp-buffer
> + (mm-with-unibyte-buffer
> (insert-buffer-substring cur)
I was asking on emacs.devel about that, and with Andreas' help came up
with the attached patch. It's a terrible hack, but it seems to work, and
I think would be good as an intermediate step.
What you're doing -- changing the unibyte/multibyte status of the
buffers -- is I think part of the final, more correct solution to the
problem, that will leave group names decoded everywhere. But I would
like to Gnus back into an intermediate working state before tackling
that...
What do you think?
Eric
[0001-WIP-on-nnmail-group-names.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 09 Apr 2019 00:56:02 GMT)
Full text and
rfc822 format available.
Message #167 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Mon, 08 Apr 2019 10:57:57 +0200, Andreas Schwab wrote:
> On Apr 08 2019, Katsumi Yamaoka <yamaoka <at> jpl.org> wrote:
>> ・Convert it to a string using `symbol-name'.
>> \343 is expanded into the one consists of four characters. :<
> It's not expanded. That's just the print representation.
Indeed. I learned not only symbol-name, but also
eval-print-last-sexp, edebug-eval-expression, describe-variable,
etc. also just make it human readable one.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 09 Apr 2019 00:56:02 GMT)
Full text and
rfc822 format available.
Message #170 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Mon, 08 Apr 2019 11:31:57 -0700, Eric Abrahamsen wrote:
> On 04/08/19 17:13 PM, Katsumi Yamaoka wrote:
>> - (with-temp-buffer
>> + (mm-with-unibyte-buffer
>> (insert-buffer-substring cur)
> I was asking on emacs.devel about that, and with Andreas' help came up
> with the attached patch. It's a terrible hack, but it seems to work, and
> I think would be good as an intermediate step.
> What you're doing -- changing the unibyte/multibyte status of the
> buffers -- is I think part of the final, more correct solution to the
> problem, that will leave group names decoded everywhere. But I would
> like to Gnus back into an intermediate working state before tackling
> that...
> What do you think?
Whatever we should do finally, isn't it necessary to make Gnus on
Emacs master work now anyway even if it is immature? Those who
like the *latest* code by any means (me included) would be coming
to use it sooner or later. So, I'd like you to install to master
a fix whatever makes Gnus work.
> --- a/lisp/gnus/gnus-start.el
> +++ b/lisp/gnus/gnus-start.el
[...]
> (setq group (read cur)
> + group
> + (encode-coding-string
> + (cond ((numberp group)
> + (number-to-string group))
> + ((symbolp group)
> + (symbol-name group))
> + ((stringp group)
> + group))
> + 'latin-1))))
Makes sense to me. Though encoding the one having already encoded
looks redundant, I have no idea other than this and my patch, and
benchmark reports there is no notably difference in speed between
this and my patch.
Anyway I verified it works for listing the nnml:テスト group.
> --- a/lisp/gnus/nnmail.el
> +++ b/lisp/gnus/nnmail.el
> --- a/lisp/gnus/nnml.el
> +++ b/lisp/gnus/nnml.el
Verified that those patches enable updating groups by `M-g'.
> --- a/lisp/gnus/nnrss.el
> +++ b/lisp/gnus/nnrss.el
Verified that I can read some feeds.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 09 Apr 2019 02:02:02 GMT)
Full text and
rfc822 format available.
Message #173 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On 04/09/19 09:55 AM, Katsumi Yamaoka wrote:
> On Mon, 08 Apr 2019 11:31:57 -0700, Eric Abrahamsen wrote:
>> On 04/08/19 17:13 PM, Katsumi Yamaoka wrote:
>>> - (with-temp-buffer
>>> + (mm-with-unibyte-buffer
>>> (insert-buffer-substring cur)
>
>> I was asking on emacs.devel about that, and with Andreas' help came up
>> with the attached patch. It's a terrible hack, but it seems to work, and
>> I think would be good as an intermediate step.
>
>> What you're doing -- changing the unibyte/multibyte status of the
>> buffers -- is I think part of the final, more correct solution to the
>> problem, that will leave group names decoded everywhere. But I would
>> like to Gnus back into an intermediate working state before tackling
>> that...
>
>> What do you think?
>
> Whatever we should do finally, isn't it necessary to make Gnus on
> Emacs master work now anyway even if it is immature? Those who
> like the *latest* code by any means (me included) would be coming
> to use it sooner or later. So, I'd like you to install to master
> a fix whatever makes Gnus work.
Yes, that's what I was saying! Do the minimum needed now to get things
working again, then I'll be more careful putting a more thorough fix
later.
>> --- a/lisp/gnus/gnus-start.el
>> +++ b/lisp/gnus/gnus-start.el
> [...]
>> (setq group (read cur)
>> + group
>> + (encode-coding-string
>> + (cond ((numberp group)
>> + (number-to-string group))
>> + ((symbolp group)
>> + (symbol-name group))
>> + ((stringp group)
>> + group))
>> + 'latin-1))))
>
> Makes sense to me. Though encoding the one having already encoded
> looks redundant, I have no idea other than this and my patch, and
> benchmark reports there is no notably difference in speed between
> this and my patch.
Yes, it's ugly, but hopefully it's temporary. I am glad that it doesn't
affect speed, though.
> Anyway I verified it works for listing the nnml:テスト group.
>
>> --- a/lisp/gnus/nnmail.el
>> +++ b/lisp/gnus/nnmail.el
>
>> --- a/lisp/gnus/nnml.el
>> +++ b/lisp/gnus/nnml.el
>
> Verified that those patches enable updating groups by `M-g'.
>
>> --- a/lisp/gnus/nnrss.el
>> +++ b/lisp/gnus/nnrss.el
>
> Verified that I can read some feeds.
Thanks very much. I'll push this now, and then see what else needs
cleaning up.
Eric
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 09 Apr 2019 04:19:02 GMT)
Full text and
rfc822 format available.
Message #176 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Mon, 08 Apr 2019 19:01:08 -0700, Eric Abrahamsen wrote:
> Thanks very much. I'll push this now, and then see what else needs
> cleaning up.
Thanks! I added this one (cf. <b4mwok52k54.fsf <at> jpl.org>):
Make `jump to group' work even if it is not activated (bug#33653)
* lisp/gnus/gnus-group.el (gnus-group-goto-group):
Use gnus-newsrc-hashtb instead of gnus-newsrc-hashtb to check if
a group exists even if its server is not activated (bug#33653).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Tue, 09 Apr 2019 04:31:01 GMT)
Full text and
rfc822 format available.
Message #179 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
> On Mon, 08 Apr 2019 19:01:08 -0700, Eric Abrahamsen wrote:
>> Thanks very much. I'll push this now, and then see what else needs
>> cleaning up.
>
> Thanks! I added this one (cf. <b4mwok52k54.fsf <at> jpl.org>):
>
> Make `jump to group' work even if it is not activated (bug#33653)
>
> * lisp/gnus/gnus-group.el (gnus-group-goto-group):
> Use gnus-newsrc-hashtb instead of gnus-newsrc-hashtb to check if
> a group exists even if its server is not activated (bug#33653).
Yes, I think that's better behavior.
I'll wait to hear from one or two other reporters, then close this bug
(and reopen new ones as necessary).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Thu, 11 Apr 2019 21:30:02 GMT)
Full text and
rfc822 format available.
Message #182 received at 33653 <at> debbugs.gnu.org (full text, mbox):
[0001-Always-set-gnus-group-property-to-a-group-name.patch (text/x-diff, attachment)]
[Message part 2 (text/plain, inline)]
While looking at bug#35208 and bug#35233, I spotted one more place in
gnus-group.el which sets the gnus-group text property to the group's
active info instead of its name, namely in
gnus-group-prepare-flat-list-dead.
Is the attached patch, which includes some extra cleanup, okay to push?
Or have I misunderstood something?
Thanks,
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Thu, 11 Apr 2019 23:57:02 GMT)
Full text and
rfc822 format available.
Message #185 received at 33653 <at> debbugs.gnu.org (full text, mbox):
On Thu, 11 Apr 2019 22:29:23 +0100, Basil L. Contovounesios wrote:
> While looking at bug#35208 and bug#35233, I spotted one more place in
> gnus-group.el which sets the gnus-group text property to the group's
> active info instead of its name, namely in
> gnus-group-prepare-flat-list-dead.
I see this fix is necessary:
(add-text-properties
(the_beginning_of_each_group_line)
(list 'gnus-group (gethash group gnus-active-hashtb)
↓
(add-text-properties
(the_beginning_of_each_group_line)
(list 'gnus-group group
> Is the attached patch, which includes some extra cleanup, okay to push?
No problem I think. Please push the patch.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Fri, 12 Apr 2019 11:06:01 GMT)
Full text and
rfc822 format available.
Message #188 received at 33653 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
>> Is the attached patch, which includes some extra cleanup, okay to push?
>
> No problem I think. Please push the patch.
Done, thanks.
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#33653; Package
emacs.
(Sat, 22 Jun 2019 13:12:01 GMT)
Full text and
rfc822 format available.
Message #191 received at 33653 <at> debbugs.gnu.org (full text, mbox):
"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:
> Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
>
>>> Is the attached patch, which includes some extra cleanup, okay to push?
>>
>> No problem I think. Please push the patch.
>
> Done, thanks.
Browsing this thread, it seems like everything was resolved, so I'm
closing this bug report.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
Added tag(s) fixed.
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org.
(Sat, 22 Jun 2019 13:12:02 GMT)
Full text and
rfc822 format available.
bug marked as fixed in version 27.1, send any further explanations to
33653 <at> debbugs.gnu.org and Eric Abrahamsen <eric <at> ericabrahamsen.net>
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org.
(Sat, 22 Jun 2019 13:12:02 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 21 Jul 2019 11:24:05 GMT)
Full text and
rfc822 format available.
This bug report was last modified 4 years and 279 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.