GNU bug report logs - #35383
27.0.50; Complete process of decoding Gnus group names

Previous Next

Package: emacs;

Reported by: Eric Abrahamsen <eric <at> ericabrahamsen.net>

Date: Mon, 22 Apr 2019 18:42:02 UTC

Severity: normal

Tags: fixed

Found in version 27.0.50

Fixed in version 27.1

Done: Lars Ingebrigtsen <larsi <at> gnus.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35383 in the body.
You can then email your comments to 35383 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 22 Apr 2019 18:42:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eric Abrahamsen <eric <at> ericabrahamsen.net>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Mon, 22 Apr 2019 18:42:03 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 22 Apr 2019 11:39:40 -0700
This is part two and the completion of bug#33653, which changed Gnus's
obarrays into hash tables, and group names from symbols to (encoded)
strings. The commits in the recently-pushed "scratch/gnus-decoded"
branch change group names to decoded strings.

Bug 33653 was a mess for a few reasons: I partitioned the changes
poorly, didn't call for testers, and it turned out that I was locally
testing a mismash of that change plus a couple changes included in this
branch, which hid some bugs.

This time around I'll keep it cleaner: I'll locally test only this
change in isolation, I'm writing a semi-interactive test suite for Gnus,
and in a few weeks I'll rope in three or four users to road-test the
changes. The upside is that, once these changes are stabilized and put
in, it will eliminate a whole class of potential bugs.

In the meantime I'm hanging this here as a placeholder. If any brave
soul does decide to give it a test-run in the meantime, back up your
.newsrc.eld file first!

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 23 Apr 2019 08:13:02 GMT) Full text and rfc822 format available.

Message #8 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 23 Apr 2019 17:12:12 +0900
On Mon, 22 Apr 2019 11:39:40 -0700, Eric Abrahamsen wrote:
> This is part two and the completion of bug#33653, which changed Gnus's
> obarrays into hash tables, and group names from symbols to (encoded)
> strings. The commits in the recently-pushed "scratch/gnus-decoded"
> branch change group names to decoded strings.

> Bug 33653 was a mess for a few reasons: I partitioned the changes
> poorly, didn't call for testers, and it turned out that I was locally
> testing a mismash of that change plus a couple changes included in this
> branch, which hid some bugs.

> This time around I'll keep it cleaner: I'll locally test only this
> change in isolation, I'm writing a semi-interactive test suite for Gnus,
> and in a few weeks I'll rope in three or four users to road-test the
> changes. The upside is that, once these changes are stabilized and put
> in, it will eliminate a whole class of potential bugs.

> In the meantime I'm hanging this here as a placeholder. If any brave
> soul does decide to give it a test-run in the meantime, back up your
> .newsrc.eld file first!

No problem for hours.  Maybe it has been completed already?
I believe I can live with it anyway.  Though I'm not quite sure
yet, the new and the old newsrc.eld files (containing non-ASCII
nnml and nnrss groups) seem to be compatible with the new and
the old Gnus mutually.  That is, I didn't need to do something
special on the newsrc.eld file.

;; Where `new' means that of the scratch/gnus-decoded branch,
;; and `old' means that of Emacs 26.2.50.

Regards,




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 23 Apr 2019 15:43:01 GMT) Full text and rfc822 format available.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Moreton <andrewjmoreton <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 23 Apr 2019 16:42:05 +0100
On Mon 22 Apr 2019, Eric Abrahamsen wrote:

> This is part two and the completion of bug#33653, which changed Gnus's
> obarrays into hash tables, and group names from symbols to (encoded)
> strings. The commits in the recently-pushed "scratch/gnus-decoded"
> branch change group names to decoded strings.
>
> Bug 33653 was a mess for a few reasons: I partitioned the changes
> poorly, didn't call for testers, and it turned out that I was locally
> testing a mismash of that change plus a couple changes included in this
> branch, which hid some bugs.
>
> This time around I'll keep it cleaner: I'll locally test only this
> change in isolation, I'm writing a semi-interactive test suite for Gnus,
> and in a few weeks I'll rope in three or four users to road-test the
> changes. The upside is that, once these changes are stabilized and put
> in, it will eliminate a whole class of potential bugs.
>
> In the meantime I'm hanging this here as a placeholder. If any brave
> soul does decide to give it a test-run in the meantime, back up your
> .newsrc.eld file first!

There don;t seem to bnbe any problems with the .nersrc.eld, as I have
used the same file in emacs-26, master and scratch/gnus-decoded branches
without obvious problems.

There is a problem in the server buffer in the scratch/gnus-decoded buffer
with group names. In my setup:
1) Start gnus, and open the server buffer
2) Select the feedbase server. This is configured as:

  (gnus-secondary-select-methods
   '((nntp "feedbase"
           (nntp-open-connection-function nntp-open-tls-stream)
           (nntp-port-number 563)
           (nntp-address "feedbase.org"))
     ...

3) Note that some group names are not decoded correctly in the server
buffer. The group names are decoded properly in the article buffer.

For example, (showing only names with problems):

server buffer (master branch):

K    140: feedbase.blog.københavnsskiklub
K    208: feedbase.news.brk.høringer
K   2017: feedbase.news.dr.hovedstadsområdet


server buffer (scratch/gnus-decoded branch):

K    140: feedbase.blog.k\303\270benhavnsskiklub
K    208: feedbase.news.brk.h\303\270ringer
K   2017: feedbase.news.dr.hovedstadsomr\303\245det

The raw bytes (shown as escapes above) match the UTF-8 for the expected
characters, so the decoding is not quite right.

    AndyM





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 23 Apr 2019 19:56:02 GMT) Full text and rfc822 format available.

Message #14 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 23 Apr 2019 12:55:43 -0700
On 04/23/19 17:12 PM, Katsumi Yamaoka wrote:
> On Mon, 22 Apr 2019 11:39:40 -0700, Eric Abrahamsen wrote:
>> This is part two and the completion of bug#33653, which changed Gnus's
>> obarrays into hash tables, and group names from symbols to (encoded)
>> strings. The commits in the recently-pushed "scratch/gnus-decoded"
>> branch change group names to decoded strings.
>
>> Bug 33653 was a mess for a few reasons: I partitioned the changes
>> poorly, didn't call for testers, and it turned out that I was locally
>> testing a mismash of that change plus a couple changes included in this
>> branch, which hid some bugs.
>
>> This time around I'll keep it cleaner: I'll locally test only this
>> change in isolation, I'm writing a semi-interactive test suite for Gnus,
>> and in a few weeks I'll rope in three or four users to road-test the
>> changes. The upside is that, once these changes are stabilized and put
>> in, it will eliminate a whole class of potential bugs.
>
>> In the meantime I'm hanging this here as a placeholder. If any brave
>> soul does decide to give it a test-run in the meantime, back up your
>> .newsrc.eld file first!
>
> No problem for hours.  Maybe it has been completed already?
> I believe I can live with it anyway.  Though I'm not quite sure
> yet, the new and the old newsrc.eld files (containing non-ASCII
> nnml and nnrss groups) seem to be compatible with the new and
> the old Gnus mutually.  That is, I didn't need to do something
> special on the newsrc.eld file.

Yes, I also noticed that a fully-decoded .newsrc.eld actually seems to
work fine with both the new and the old code. But better to be cautious.

> ;; Where `new' means that of the scratch/gnus-decoded branch,
> ;; and `old' means that of Emacs 26.2.50.
>
> Regards,

Thanks for testing!




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 23 Apr 2019 21:43:02 GMT) Full text and rfc822 format available.

Message #17 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 23 Apr 2019 14:42:23 -0700
[Message part 1 (text/plain, inline)]
Andy Moreton <andrewjmoreton <at> gmail.com> writes:

> On Mon 22 Apr 2019, Eric Abrahamsen wrote:
>
>> This is part two and the completion of bug#33653, which changed Gnus's
>> obarrays into hash tables, and group names from symbols to (encoded)
>> strings. The commits in the recently-pushed "scratch/gnus-decoded"
>> branch change group names to decoded strings.
>>
>> Bug 33653 was a mess for a few reasons: I partitioned the changes
>> poorly, didn't call for testers, and it turned out that I was locally
>> testing a mismash of that change plus a couple changes included in this
>> branch, which hid some bugs.
>>
>> This time around I'll keep it cleaner: I'll locally test only this
>> change in isolation, I'm writing a semi-interactive test suite for Gnus,
>> and in a few weeks I'll rope in three or four users to road-test the
>> changes. The upside is that, once these changes are stabilized and put
>> in, it will eliminate a whole class of potential bugs.
>>
>> In the meantime I'm hanging this here as a placeholder. If any brave
>> soul does decide to give it a test-run in the meantime, back up your
>> .newsrc.eld file first!
>
> There don;t seem to bnbe any problems with the .nersrc.eld, as I have
> used the same file in emacs-26, master and scratch/gnus-decoded branches
> without obvious problems.
>
> There is a problem in the server buffer in the scratch/gnus-decoded buffer
> with group names. In my setup:
> 1) Start gnus, and open the server buffer
> 2) Select the feedbase server. This is configured as:
>
>   (gnus-secondary-select-methods
>    '((nntp "feedbase"
>            (nntp-open-connection-function nntp-open-tls-stream)
>            (nntp-port-number 563)
>            (nntp-address "feedbase.org"))
>      ...
>
> 3) Note that some group names are not decoded correctly in the server
> buffer. The group names are decoded properly in the article buffer.
>
> For example, (showing only names with problems):
>
> server buffer (master branch):
>
> K    140: feedbase.blog.københavnsskiklub
> K    208: feedbase.news.brk.høringer
> K   2017: feedbase.news.dr.hovedstadsområdet
>
>
> server buffer (scratch/gnus-decoded branch):
>
> K    140: feedbase.blog.k\303\270benhavnsskiklub
> K    208: feedbase.news.brk.h\303\270ringer
> K   2017: feedbase.news.dr.hovedstadsomr\303\245det
>
> The raw bytes (shown as escapes above) match the UTF-8 for the expected
> characters, so the decoding is not quite right.

Aha! Good catch. The attached patch fixes things for me. At first I was
a little hesitant to assume all group names are decodable as
'utf-8-emacs, but then I figured that's all that
`gnus-group-decoded-name' does, and that's worked for years, so I guess
this is safe.

And thanks for the nntp server with non-ascii group names! I had looked
for one of those with no success, this will be very useful for testing.

Eric


[gnus-browse-srv-decoded.diff (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 23 Apr 2019 23:01:01 GMT) Full text and rfc822 format available.

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Moreton <andrewjmoreton <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 23 Apr 2019 23:58:52 +0100
On Tue 23 Apr 2019, Eric Abrahamsen wrote:

> Aha! Good catch. The attached patch fixes things for me. At first I was
> a little hesitant to assume all group names are decodable as
> 'utf-8-emacs, but then I figured that's all that
> `gnus-group-decoded-name' does, and that's worked for years, so I guess
> this is safe.

Yes, the patch fixes things here too.

    AndyM





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 24 Apr 2019 08:07:02 GMT) Full text and rfc822 format available.

Message #23 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Wed, 24 Apr 2019 17:06:19 +0900
[Message part 1 (text/plain, inline)]
Hi,

I found that certain kinds[1] of Japanese mails (like the
attached text part) saved in a nnml group get corrupted when
viewing.  The root cause is this:

(defvar nnmail-file-coding-system 'undecided
  "Coding system used in nnmail.")

The default value used to be `raw-text'.
Because `nnml-file-coding-system' inherits it, when viewing
an article from a file `nnmail-find-file' reads and decodes
the contents, and then `gnus-display-mime'[1] decodes it again.
IIUC `gnus-display-mime' expects a raw article, so any back end
should not decode the first fetched data, I think.  This helps:

(setq nnml-file-coding-system 'raw-text)

Note that the backlog or the `gnus-original-article-buffer'
keeps an corrupted article, so (setq gnus-keep-backlog nil) and
each time killing the original buffer will make it easy to debug.

[1] a single text/plain part with format=flowed spec, or
    a multipart message containing a text/plain part, etc.
[2] The default value of `gnus-display-mime-function'.

Regards,
[Message part 2 (text/plain, inline)]
寿限無寿限無五劫の擦り切れ海砂利水魚の水行末雲来末風来末食う寝る
処に住む処やぶら小路の藪柑子パイポパイポパイポのシューリンガン
シューリンガンのグーリンダイグーリンダイのポンポコピーのポンポコ
ナーの長久命の長助

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 24 Apr 2019 17:05:02 GMT) Full text and rfc822 format available.

Message #26 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Wed, 24 Apr 2019 10:04:13 -0700
On 04/24/19 17:06 PM, Katsumi Yamaoka wrote:
> Hi,
>
> I found that certain kinds[1] of Japanese mails (like the
> attached text part) saved in a nnml group get corrupted when
> viewing.  The root cause is this:
>
> (defvar nnmail-file-coding-system 'undecided
>   "Coding system used in nnmail.")
>
> The default value used to be `raw-text'.
> Because `nnml-file-coding-system' inherits it, when viewing
> an article from a file `nnmail-find-file' reads and decodes
> the contents, and then `gnus-display-mime'[1] decodes it again.
> IIUC `gnus-display-mime' expects a raw article, so any back end
> should not decode the first fetched data, I think.  This helps:
>
> (setq nnml-file-coding-system 'raw-text)

Oh, that was a misunderstanding on my part, then -- I certainly didn't
mean to change anything about article encoding or display, or anything
related to MIME. I'll switch that default back again.

Thanks,
Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 24 Apr 2019 23:49:02 GMT) Full text and rfc822 format available.

Message #29 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 25 Apr 2019 08:48:19 +0900
[Message part 1 (text/plain, inline)]
On Wed, 24 Apr 2019 10:04:13 -0700, Eric Abrahamsen wrote:
> On 04/24/19 17:06 PM, Katsumi Yamaoka wrote:
>> The root cause is this:
>> (defvar nnmail-file-coding-system 'undecided
>>   "Coding system used in nnmail.")
>> The default value used to be `raw-text'.

> Oh, that was a misunderstanding on my part, then -- I certainly didn't
> mean to change anything about article encoding or display, or anything
> related to MIME. I'll switch that default back again.

Thanks.

I found one more issue, it is harmless though.  An active file
locally saved now has a coding cookie, so `(not (eobp))' at the
line 2150 in gnus-start.el is not enough to check if there is no
more active.  Because of this, when starting Gnus I got

Warning: Warning - invalid active:

for the nnnil method, that is my `gnus-select-method'.  Here are
the contents of ~/News/agent/nnnil/agent.lib/active:

--8<---------------cut here---------------start------------->8---
;; -*- encoding: utf-8-emacs; -*-

--8<---------------cut here---------------end--------------->8---

Why the warning is issued is to run (read (current-buffer)) at
the beginning of the contents.  This is actually an error but
`condition-case' conceals it.  A patch:

[Message part 2 (text/x-patch, inline)]
--- gnus-start.el~	2019-04-23 05:13:52.741025900 +0000
+++ gnus-start.el	2019-04-24 23:45:37.989239900 +0000
@@ -2149,2 +2149,3 @@
       (goto-char (point-min))
+      (re-search-forward "\\(?:^[\n ]*;.*[\n ]*\\)+" nil t)
       (while (not (eobp))
[Message part 3 (text/plain, inline)]
Regards,

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 25 Apr 2019 16:11:02 GMT) Full text and rfc822 format available.

Message #32 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 25 Apr 2019 09:10:13 -0700
On 04/25/19 08:48 AM, Katsumi Yamaoka wrote:
> On Wed, 24 Apr 2019 10:04:13 -0700, Eric Abrahamsen wrote:
>> On 04/24/19 17:06 PM, Katsumi Yamaoka wrote:
>>> The root cause is this:
>>> (defvar nnmail-file-coding-system 'undecided
>>>   "Coding system used in nnmail.")
>>> The default value used to be `raw-text'.
>
>> Oh, that was a misunderstanding on my part, then -- I certainly didn't
>> mean to change anything about article encoding or display, or anything
>> related to MIME. I'll switch that default back again.
>
> Thanks.
>
> I found one more issue, it is harmless though.  An active file
> locally saved now has a coding cookie, so `(not (eobp))' at the
> line 2150 in gnus-start.el is not enough to check if there is no
> more active.  Because of this, when starting Gnus I got
>
> Warning: Warning - invalid active:
>
> for the nnnil method, that is my `gnus-select-method'.  Here are
> the contents of ~/News/agent/nnnil/agent.lib/active:
>
> --8<---------------cut here---------------start------------->8---
> ;; -*- encoding: utf-8-emacs; -*-
>
> --8<---------------cut here---------------end--------------->8---
>
> Why the warning is issued is to run (read (current-buffer)) at
> the beginning of the contents.  This is actually an error but
> `condition-case' conceals it.  A patch:
>
> --- gnus-start.el~	2019-04-23 05:13:52.741025900 +0000
> +++ gnus-start.el	2019-04-24 23:45:37.989239900 +0000
> @@ -2149,2 +2149,3 @@
>        (goto-char (point-min))
> +      (re-search-forward "\\(?:^[\n ]*;.*[\n ]*\\)+" nil t)
>        (while (not (eobp))

Hmm, this is all done in a temp buffer, with
`insert-buffer-substring' -- I wonder if the encoding cookie will even
be honored in this case? I need to do a bit more testing here. Thanks
for flagging this up.

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Fri, 26 Apr 2019 05:22:01 GMT) Full text and rfc822 format available.

Message #35 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Fri, 26 Apr 2019 14:21:11 +0900
On Thu, 25 Apr 2019 09:10:13 -0700, Eric Abrahamsen wrote:
> On 04/25/19 08:48 AM, Katsumi Yamaoka wrote:

>> Warning: Warning - invalid active:

>> for the nnnil method, that is my `gnus-select-method'.  Here are
>> the contents of ~/News/agent/nnnil/agent.lib/active:

>> --8<---------------cut here---------------start------------->8---
>> ;; -*- encoding: utf-8-emacs; -*-

>> --8<---------------cut here---------------end--------------->8---

>> Why the warning is issued is to run (read (current-buffer)) at
>> the beginning of the contents.  This is actually an error but
>> `condition-case' conceals it.

> Hmm, this is all done in a temp buffer,

Yes.  When launching Gnus, the whole contents of the active file
are read into the " *nntpd*" buffer, copied into the temp buffer,
and parsed (see the flow summary attatched in the bottom of this
message for how Gnus behaves when launching).

> with
> `insert-buffer-substring' -- I wonder if the encoding cookie will even
> be honored in this case?

No, it's useless of course.  Moreover, --- I changed my idea
(patching the `gnus-active-to-gnus-format' function so as to
ignore the coding cookie) --- I come to think that the active
file should not contain the ones other than the active infos.
Gnus indeed ignores the coding cookie when parsing active, but
it is due to just a lucky side effect of `read':

(read ";; coding cookie\n\nactive_info\n") => active_info

I.e., `read' ignores comments in the ELisp style and whitespace.
However, in the first place, the active file is neither an ELisp
file nor there is no agreement for a comment style in it.  So, I
think it is better to bind `coding-system-for-(read|write)' while
reading and writing the active file rather than adding a coding
cookie.  Though binding `coding-system-for-(read|write)' would
probably be unnecessary since `gnus-write-active-file' binds
`coding-system-for-write' to `nnmail-active-file-coding-system',
and `gnus-agent-save-active' binds `coding-system-for-read' to
`gnus-agent-file-coding-system' that defaults to `utf-8-emacs'.
Therefore, adding a coding cookie was originally unnecessary,
wasn't it?

Here are how Gnus reads the active file for the nnnil method
observed in my system.  Note that `gnus-agent' is t (the default).

(gnus 1)
  [...]
  (gnus-setup-news nil t nil)
    (gnus-get-unread-articles 1 nil)
      (require 'gnus-agent)
      (with-current-buffer " *nntpd*"
        (gnus-read-active-file-1 '(nnnil) nil)
          (gnus-active-to-gnus-format '(nnnil) hashtb nil t)
            (gnus-agent-save-active '(nnil))
              (gnus-agent-write-active "active-file" hashtb)
                ;; Add a coding cookie.
                (gnus-write-active-file "active-file" hashtb nil)
              (erase-buffer)
              (nnheader-insert-file-contents "active-file")
            (_copy to_ " *nntpd*")
            (_parse it_)

Regards,

P.S.
I'll be not so active in the net for about ten days because of
the national holidays assoc with the era name changing in Japan.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Fri, 26 Apr 2019 06:54:02 GMT) Full text and rfc822 format available.

Message #38 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: eric <at> ericabrahamsen.net, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Fri, 26 Apr 2019 09:53:18 +0300
> Date: Fri, 26 Apr 2019 14:21:11 +0900
> From: Katsumi Yamaoka <yamaoka <at> jpl.org>
> Cc: 35383 <at> debbugs.gnu.org
> 
> > with
> > `insert-buffer-substring' -- I wonder if the encoding cookie will even
> > be honored in this case?
> 
> No, it's useless of course.  Moreover, --- I changed my idea
> (patching the `gnus-active-to-gnus-format' function so as to
> ignore the coding cookie) --- I come to think that the active
> file should not contain the ones other than the active infos.
> Gnus indeed ignores the coding cookie when parsing active, but
> it is due to just a lucky side effect of `read':
> 
> (read ";; coding cookie\n\nactive_info\n") => active_info
> 
> I.e., `read' ignores comments in the ELisp style and whitespace.
> However, in the first place, the active file is neither an ELisp
> file nor there is no agreement for a comment style in it.  So, I
> think it is better to bind `coding-system-for-(read|write)' while
> reading and writing the active file rather than adding a coding
> cookie.  Though binding `coding-system-for-(read|write)' would
> probably be unnecessary since `gnus-write-active-file' binds
> `coding-system-for-write' to `nnmail-active-file-coding-system',
> and `gnus-agent-save-active' binds `coding-system-for-read' to
> `gnus-agent-file-coding-system' that defaults to `utf-8-emacs'.
> Therefore, adding a coding cookie was originally unnecessary,
> wasn't it?

It is not unnecessary, because that file could be visited normally in
Emacs.  So even if you have to bind coding-system-for-read for some
reason (and I admit I don't understand the reasons very well, and in
particular this will not affect 'read' or any other primitive that
is supposed to be reading from an already decoded buffer/string),
there are still valid reasons to have the cookie in the file.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Fri, 26 Apr 2019 08:09:02 GMT) Full text and rfc822 format available.

Message #41 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eric <at> ericabrahamsen.net, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Fri, 26 Apr 2019 17:08:24 +0900
On Fri, 26 Apr 2019 09:53:18 +0300, Eli Zaretskii wrote:
>> Date: Fri, 26 Apr 2019 14:21:11 +0900
>> From: Katsumi Yamaoka <yamaoka <at> jpl.org>
>> Cc: 35383 <at> debbugs.gnu.org
[...]
>> Therefore, adding a coding cookie was originally unnecessary,
>> wasn't it?

> It is not unnecessary, because that file could be visited normally in
> Emacs.  So even if you have to bind coding-system-for-read for some
> reason (and I admit I don't understand the reasons very well, and in
> particular this will not affect 'read' or any other primitive that
> is supposed to be reading from an already decoded buffer/string),
> there are still valid reasons to have the cookie in the file.

Well, I don't know why the active files need a coding cookie, but
what adds it is `gnus-write-active-file' that only gnus-agent and
gnus-cache use; `nnmail-save-active' that many mail back ends use
does not add a coding cookie.  Even so, I have no trouble with my
nnml active file that contains non-ASCII group names.  The file
is utf-8 encoded and it is Emacs' default (IIUC), so I cannot
think any reason why the cookie is required.

(Just in case, we're talking about the gnus-decoded branch.)

Anyway, the error that at least the nnnil active file causes
should be fixed in some way.

Regards,




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Fri, 26 Apr 2019 10:56:02 GMT) Full text and rfc822 format available.

Message #44 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: eric <at> ericabrahamsen.net, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Fri, 26 Apr 2019 13:55:19 +0300
> Date: Fri, 26 Apr 2019 17:08:24 +0900
> From: Katsumi Yamaoka <yamaoka <at> jpl.org>
> Cc: eric <at> ericabrahamsen.net, 35383 <at> debbugs.gnu.org
> 
> > It is not unnecessary, because that file could be visited normally in
> > Emacs.  So even if you have to bind coding-system-for-read for some
> > reason (and I admit I don't understand the reasons very well, and in
> > particular this will not affect 'read' or any other primitive that
> > is supposed to be reading from an already decoded buffer/string),
> > there are still valid reasons to have the cookie in the file.
> 
> Well, I don't know why the active files need a coding cookie, but
> what adds it is `gnus-write-active-file' that only gnus-agent and
> gnus-cache use; `nnmail-save-active' that many mail back ends use
> does not add a coding cookie.  Even so, I have no trouble with my
> nnml active file that contains non-ASCII group names.  The file
> is utf-8 encoded and it is Emacs' default (IIUC), so I cannot
> think any reason why the cookie is required.

You are probably in a UTF-8 locale, so your defaults facilitate
detecting the right encoding.  In non-UTF-8 locales this might not be
that easy, Emacs could guess wrong.

More importantly, utf-8-emacs and utf-8 are not the same, the
differences are subtle, but they do exist.  We use utf-8-emacs for
anything that should support any character representable inside Emacs
buffers and strings, so using it here seems correct.  But if someone
wants to visit this file with "C-x C-f", e.g., to manually fix
something there, they should be able to do that without risking
incorrect decoding.

So I think the cookie should stay, especially as it can never do any
harm in this context, AFAIU.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 29 Apr 2019 04:06:02 GMT) Full text and rfc822 format available.

Message #47 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 28 Apr 2019 21:05:40 -0700
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:

> On Thu, 25 Apr 2019 09:10:13 -0700, Eric Abrahamsen wrote:
>> On 04/25/19 08:48 AM, Katsumi Yamaoka wrote:
>
>>> Warning: Warning - invalid active:
>
>>> for the nnnil method, that is my `gnus-select-method'.  Here are
>>> the contents of ~/News/agent/nnnil/agent.lib/active:
>
>>> --8<---------------cut here---------------start------------->8---
>>> ;; -*- encoding: utf-8-emacs; -*-
>
>>> --8<---------------cut here---------------end--------------->8---
>
>>> Why the warning is issued is to run (read (current-buffer)) at
>>> the beginning of the contents.  This is actually an error but
>>> `condition-case' conceals it.
>
>> Hmm, this is all done in a temp buffer,
>
> Yes.  When launching Gnus, the whole contents of the active file
> are read into the " *nntpd*" buffer, copied into the temp buffer,
> and parsed (see the flow summary attatched in the bottom of this
> message for how Gnus behaves when launching).
>
>> with
>> `insert-buffer-substring' -- I wonder if the encoding cookie will even
>> be honored in this case?
>
> No, it's useless of course.  Moreover, --- I changed my idea
> (patching the `gnus-active-to-gnus-format' function so as to
> ignore the coding cookie) --- I come to think that the active
> file should not contain the ones other than the active infos.
> Gnus indeed ignores the coding cookie when parsing active, but
> it is due to just a lucky side effect of `read':
>
> (read ";; coding cookie\n\nactive_info\n") => active_info
>
> I.e., `read' ignores comments in the ELisp style and whitespace.
> However, in the first place, the active file is neither an ELisp
> file nor there is no agreement for a comment style in it.  So, I
> think it is better to bind `coding-system-for-(read|write)' while
> reading and writing the active file rather than adding a coding
> cookie.  Though binding `coding-system-for-(read|write)' would
> probably be unnecessary since `gnus-write-active-file' binds
> `coding-system-for-write' to `nnmail-active-file-coding-system',
> and `gnus-agent-save-active' binds `coding-system-for-read' to
> `gnus-agent-file-coding-system' that defaults to `utf-8-emacs'.
> Therefore, adding a coding cookie was originally unnecessary,
> wasn't it?
>
> Here are how Gnus reads the active file for the nnnil method
> observed in my system.  Note that `gnus-agent' is t (the default).
>
> (gnus 1)
>   [...]
>   (gnus-setup-news nil t nil)
>     (gnus-get-unread-articles 1 nil)
>       (require 'gnus-agent)
>       (with-current-buffer " *nntpd*"
>         (gnus-read-active-file-1 '(nnnil) nil)
>           (gnus-active-to-gnus-format '(nnnil) hashtb nil t)
>             (gnus-agent-save-active '(nnil))
>               (gnus-agent-write-active "active-file" hashtb)
>                 ;; Add a coding cookie.
>                 (gnus-write-active-file "active-file" hashtb nil)
>               (erase-buffer)
>               (nnheader-insert-file-contents "active-file")
>             (_copy to_ " *nntpd*")
>             (_parse it_)

Hang on, let me slow down here.

The goal is to have Gnus default to writing its active files in
'utf-8-emacs, unless the user has specifically requested otherwise.
`nnmail-active-file-coding-system' governs the "mail" type servers, and
`gnus-agent-file-coding-system' governs the agent. Currently those two
options default to 'raw-text, we'd like them to default to 'utf-8-emacs.

But the active files have to be read correctly, as well, even by a user
who is loading this new code for the first time. That was my original
idea for the coding cookie, though obviously I was confused. Both those
options are used as the value of `coding-system-for-read' when reading
active files. So far I don't have a good solution for "upgrading" old
active files to 'utf-8.

I also realized that there's a `nnmbox-active-file-coding-system' that
defaults to 'binary.

What I find a little perplexing is that, testing with emacs 26, an nnml
active file with multibyte group names gets written to a file that my
terminal recognizes as utf-8, despite the fact that `nnmail-save-active'
sets 'coding-system-for-write' to 'raw-text, and disables multibyte. But
maybe, as Eli mentioned, that has to do with my locale...?

> P.S.
> I'll be not so active in the net for about ten days because of
> the national holidays assoc with the era name changing in Japan.

Hope it's a happy time in Japan! I'm in no rush over here.

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 29 Apr 2019 04:08:01 GMT) Full text and rfc822 format available.

Message #50 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 28 Apr 2019 21:07:23 -0700
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

> Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
>
>> On Thu, 25 Apr 2019 09:10:13 -0700, Eric Abrahamsen wrote:
>>> On 04/25/19 08:48 AM, Katsumi Yamaoka wrote:
>>
>>>> Warning: Warning - invalid active:
>>
>>>> for the nnnil method, that is my `gnus-select-method'.  Here are
>>>> the contents of ~/News/agent/nnnil/agent.lib/active:
>>
>>>> --8<---------------cut here---------------start------------->8---
>>>> ;; -*- encoding: utf-8-emacs; -*-
>>
>>>> --8<---------------cut here---------------end--------------->8---
>>
>>>> Why the warning is issued is to run (read (current-buffer)) at
>>>> the beginning of the contents.  This is actually an error but
>>>> `condition-case' conceals it.
>>
>>> Hmm, this is all done in a temp buffer,
>>
>> Yes.  When launching Gnus, the whole contents of the active file
>> are read into the " *nntpd*" buffer, copied into the temp buffer,
>> and parsed (see the flow summary attatched in the bottom of this
>> message for how Gnus behaves when launching).
>>
>>> with
>>> `insert-buffer-substring' -- I wonder if the encoding cookie will even
>>> be honored in this case?
>>
>> No, it's useless of course.  Moreover, --- I changed my idea
>> (patching the `gnus-active-to-gnus-format' function so as to
>> ignore the coding cookie) --- I come to think that the active
>> file should not contain the ones other than the active infos.
>> Gnus indeed ignores the coding cookie when parsing active, but
>> it is due to just a lucky side effect of `read':
>>
>> (read ";; coding cookie\n\nactive_info\n") => active_info
>>
>> I.e., `read' ignores comments in the ELisp style and whitespace.
>> However, in the first place, the active file is neither an ELisp
>> file nor there is no agreement for a comment style in it.  So, I
>> think it is better to bind `coding-system-for-(read|write)' while
>> reading and writing the active file rather than adding a coding
>> cookie.  Though binding `coding-system-for-(read|write)' would
>> probably be unnecessary since `gnus-write-active-file' binds
>> `coding-system-for-write' to `nnmail-active-file-coding-system',
>> and `gnus-agent-save-active' binds `coding-system-for-read' to
>> `gnus-agent-file-coding-system' that defaults to `utf-8-emacs'.
>> Therefore, adding a coding cookie was originally unnecessary,
>> wasn't it?
>>
>> Here are how Gnus reads the active file for the nnnil method
>> observed in my system.  Note that `gnus-agent' is t (the default).
>>
>> (gnus 1)
>>   [...]
>>   (gnus-setup-news nil t nil)
>>     (gnus-get-unread-articles 1 nil)
>>       (require 'gnus-agent)
>>       (with-current-buffer " *nntpd*"
>>         (gnus-read-active-file-1 '(nnnil) nil)
>>           (gnus-active-to-gnus-format '(nnnil) hashtb nil t)
>>             (gnus-agent-save-active '(nnil))
>>               (gnus-agent-write-active "active-file" hashtb)
>>                 ;; Add a coding cookie.
>>                 (gnus-write-active-file "active-file" hashtb nil)
>>               (erase-buffer)
>>               (nnheader-insert-file-contents "active-file")
>>             (_copy to_ " *nntpd*")
>>             (_parse it_)
>
> Hang on, let me slow down here.
>
> The goal is to have Gnus default to writing its active files in
> 'utf-8-emacs, unless the user has specifically requested otherwise.
> `nnmail-active-file-coding-system' governs the "mail" type servers, and
> `gnus-agent-file-coding-system' governs the agent. Currently those two
> options default to 'raw-text, we'd like them to default to 'utf-8-emacs.

Actually, maybe that's wrong. We don't care how the files are written,
only that, after parsing, the group names are successfully _decoded_ to
'utf-8-emacs. Maybe I'm trying too hard?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 29 Apr 2019 15:39:01 GMT) Full text and rfc822 format available.

Message #53 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 29 Apr 2019 18:38:37 +0300
> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Date: Sun, 28 Apr 2019 21:05:40 -0700
> Cc: 35383 <at> debbugs.gnu.org
> 
> What I find a little perplexing is that, testing with emacs 26, an nnml
> active file with multibyte group names gets written to a file that my
> terminal recognizes as utf-8, despite the fact that `nnmail-save-active'
> sets 'coding-system-for-write' to 'raw-text, and disables multibyte. But
> maybe, as Eli mentioned, that has to do with my locale...?

If the original was a valid UTF-8 sequence, writing it out as raw-text
will leave the byte sequences alone, and then it will look like a
UTF-8 encoded text.  So this is kinda expected in many situations.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 29 Apr 2019 15:42:01 GMT) Full text and rfc822 format available.

Message #56 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 29 Apr 2019 18:41:41 +0300
> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Date: Sun, 28 Apr 2019 21:07:23 -0700
> Cc: 35383 <at> debbugs.gnu.org
> 
> > The goal is to have Gnus default to writing its active files in
> > 'utf-8-emacs, unless the user has specifically requested otherwise.
> > `nnmail-active-file-coding-system' governs the "mail" type servers, and
> > `gnus-agent-file-coding-system' governs the agent. Currently those two
> > options default to 'raw-text, we'd like them to default to 'utf-8-emacs.
> 
> Actually, maybe that's wrong. We don't care how the files are written,
> only that, after parsing, the group names are successfully _decoded_ to
> 'utf-8-emacs. Maybe I'm trying too hard?

When you decode _any_ text by _any_ coding-system, the result is
_always_ utf-8-emacs, because utf-8-emacs is the internal
representation of characters and raw bytes in Emacs buffers and
strings.

We write text out as utf-8-emacs when we don't want to risk the danger
of decoding incorrectly due to local customizations and language
environments.  Text encoded in utf-8-emacs can by definition represent
_any_ character and raw byte that Emacs can read, and the "decoding"
in this case is trivial.

So no, you are not trying too hard, not IMO.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 29 Apr 2019 20:05:01 GMT) Full text and rfc822 format available.

Message #59 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 29 Apr 2019 13:04:31 -0700
On 04/29/19 18:41 PM, Eli Zaretskii wrote:
>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>> Date: Sun, 28 Apr 2019 21:07:23 -0700
>> Cc: 35383 <at> debbugs.gnu.org
>> 
>> > The goal is to have Gnus default to writing its active files in
>> > 'utf-8-emacs, unless the user has specifically requested otherwise.
>> > `nnmail-active-file-coding-system' governs the "mail" type servers, and
>> > `gnus-agent-file-coding-system' governs the agent. Currently those two
>> > options default to 'raw-text, we'd like them to default to 'utf-8-emacs.
>> 
>> Actually, maybe that's wrong. We don't care how the files are written,
>> only that, after parsing, the group names are successfully _decoded_ to
>> 'utf-8-emacs. Maybe I'm trying too hard?
>
> When you decode _any_ text by _any_ coding-system, the result is
> _always_ utf-8-emacs, because utf-8-emacs is the internal
> representation of characters and raw bytes in Emacs buffers and
> strings.

I did know that much! I'm pretty bad at encoding, but not quite that
bad.

> We write text out as utf-8-emacs when we don't want to risk the danger
> of decoding incorrectly due to local customizations and language
> environments.  Text encoded in utf-8-emacs can by definition represent
> _any_ character and raw byte that Emacs can read, and the "decoding"
> in this case is trivial.
>
> So no, you are not trying too hard, not IMO.

So you think Gnus' various *-file-coding-system options should
default to 'utf-8-emacs rather than 'raw-text?

As per your other message, it sounds like active files written as
'raw-text will probably survive being read as 'utf-8-emacs. And if the
user has previously customized those options to something else, the
change in default value won't matter anyway.

What I meant by "trying too hard" is, maybe it's enough to just change
the defaults, and not add any other error checking and guarantees?

Thanks for looking at this,
Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Apr 2019 03:56:02 GMT) Full text and rfc822 format available.

Message #62 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Apr 2019 06:55:00 +0300
> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Cc: yamaoka <at> jpl.org,  35383 <at> debbugs.gnu.org
> Date: Mon, 29 Apr 2019 13:04:31 -0700
> 
> >> Actually, maybe that's wrong. We don't care how the files are written,
> >> only that, after parsing, the group names are successfully _decoded_ to
> >> 'utf-8-emacs. Maybe I'm trying too hard?
> >
> > When you decode _any_ text by _any_ coding-system, the result is
> > _always_ utf-8-emacs, because utf-8-emacs is the internal
> > representation of characters and raw bytes in Emacs buffers and
> > strings.
> 
> I did know that much! I'm pretty bad at encoding, but not quite that
> bad.

Sorry, it was not clear to me, since you talked about "decoding to
utf-8-emacs", which is a kind of tautology.

> So you think Gnus' various *-file-coding-system options should
> default to 'utf-8-emacs rather than 'raw-text?

Not sure about all of them, I don't think I have a clear idea of what
they are used for.  The principle is that we use utf-8-emacs for files
where Emacs records its internal data, and whose primary role is to
allow Emacs to restore its internal data with maximum reliability.
One good example of this is auto-save files, where we use utf-8-emacs
regardless of the actual encoding of the file whose buffer is
auto-saved.

If this principle is not enough to make a decision, please point out
the specific variables you are unsure about, and I will take a look at
their actual usage.

> As per your other message, it sounds like active files written as
> 'raw-text will probably survive being read as 'utf-8-emacs. And if the
> user has previously customized those options to something else, the
> change in default value won't matter anyway.

That is true, but good defaults do matter to new users and new files.

And raw-text is almost never appropriate as the default for
human-readable text.

> What I meant by "trying too hard" is, maybe it's enough to just change
> the defaults, and not add any other error checking and guarantees?

What other checks and guarantees did you consider?  (Sorry, I didn't
read the entire thread, so maybe just point me to the message where
you described this.)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Apr 2019 17:04:01 GMT) Full text and rfc822 format available.

Message #65 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Apr 2019 10:03:46 -0700
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>> Cc: yamaoka <at> jpl.org,  35383 <at> debbugs.gnu.org
>> Date: Mon, 29 Apr 2019 13:04:31 -0700
>>
>> >> Actually, maybe that's wrong. We don't care how the files are written,
>> >> only that, after parsing, the group names are successfully _decoded_ to
>> >> 'utf-8-emacs. Maybe I'm trying too hard?
>> >
>> > When you decode _any_ text by _any_ coding-system, the result is
>> > _always_ utf-8-emacs, because utf-8-emacs is the internal
>> > representation of characters and raw bytes in Emacs buffers and
>> > strings.
>>
>> I did know that much! I'm pretty bad at encoding, but not quite that
>> bad.
>
> Sorry, it was not clear to me, since you talked about "decoding to
> utf-8-emacs", which is a kind of tautology.

True!

>> So you think Gnus' various *-file-coding-system options should
>> default to 'utf-8-emacs rather than 'raw-text?
>
> Not sure about all of them, I don't think I have a clear idea of what
> they are used for.  The principle is that we use utf-8-emacs for files
> where Emacs records its internal data, and whose primary role is to
> allow Emacs to restore its internal data with maximum reliability.
> One good example of this is auto-save files, where we use utf-8-emacs
> regardless of the actual encoding of the file whose buffer is
> auto-saved.
>
> If this principle is not enough to make a decision, please point out
> the specific variables you are unsure about, and I will take a look at
> their actual usage.

I think that's sufficient. I'm really only looking at two variables:
`gnus-agent-file-coding-system', which I'm 100% sure is only used
internally by Gnus/Emacs, and `nnmail-active-file-coding-system', which
I'm 99% sure is only used internally.

>> As per your other message, it sounds like active files written as
>> 'raw-text will probably survive being read as 'utf-8-emacs. And if the
>> user has previously customized those options to something else, the
>> change in default value won't matter anyway.
>
> That is true, but good defaults do matter to new users and new files.
>
> And raw-text is almost never appropriate as the default for
> human-readable text.
>
>> What I meant by "trying too hard" is, maybe it's enough to just change
>> the defaults, and not add any other error checking and guarantees?
>
> What other checks and guarantees did you consider?  (Sorry, I didn't
> read the entire thread, so maybe just point me to the message where
> you described this.)

That's where the encoding cookie idea came from -- if there's no cookie,
read as 'raw-text, otherwise honor the cookie. But as Katsumi Yamaoka
pointed out, the way Gnus reads the file will ignore any encoding
cookie, so that's no good.

To be honest, I didn't have any other ideas.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Apr 2019 17:12:02 GMT) Full text and rfc822 format available.

Message #68 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Apr 2019 20:11:15 +0300
> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Cc: yamaoka <at> jpl.org,  35383 <at> debbugs.gnu.org
> Date: Tue, 30 Apr 2019 10:03:46 -0700
> 
> I think that's sufficient. I'm really only looking at two variables:
> `gnus-agent-file-coding-system', which I'm 100% sure is only used
> internally by Gnus/Emacs, and `nnmail-active-file-coding-system', which
> I'm 99% sure is only used internally.

It sounds like utf-8-emacs is TRT in both cases.

> > What other checks and guarantees did you consider?  (Sorry, I didn't
> > read the entire thread, so maybe just point me to the message where
> > you described this.)
> 
> That's where the encoding cookie idea came from -- if there's no cookie,
> read as 'raw-text, otherwise honor the cookie. But as Katsumi Yamaoka
> pointed out, the way Gnus reads the file will ignore any encoding
> cookie, so that's no good.

You could give up the cookie.  It is only important if the file is
visited by "C-x C-f" etc., which will be rare.  So having a cookie is
a nicety, not a requirement.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Apr 2019 17:20:02 GMT) Full text and rfc822 format available.

Message #71 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Apr 2019 10:19:01 -0700
On 04/30/19 20:11 PM, Eli Zaretskii wrote:
>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>> Cc: yamaoka <at> jpl.org,  35383 <at> debbugs.gnu.org
>> Date: Tue, 30 Apr 2019 10:03:46 -0700
>> 
>> I think that's sufficient. I'm really only looking at two variables:
>> `gnus-agent-file-coding-system', which I'm 100% sure is only used
>> internally by Gnus/Emacs, and `nnmail-active-file-coding-system', which
>> I'm 99% sure is only used internally.
>
> It sounds like utf-8-emacs is TRT in both cases.
>
>> > What other checks and guarantees did you consider?  (Sorry, I didn't
>> > read the entire thread, so maybe just point me to the message where
>> > you described this.)
>> 
>> That's where the encoding cookie idea came from -- if there's no cookie,
>> read as 'raw-text, otherwise honor the cookie. But as Katsumi Yamaoka
>> pointed out, the way Gnus reads the file will ignore any encoding
>> cookie, so that's no good.
>
> You could give up the cookie.  It is only important if the file is
> visited by "C-x C-f" etc., which will be rare.  So having a cookie is
> a nicety, not a requirement.

Right, I think that was the conclusion. But do you have any other advice
about making sure that files previously written as 'raw-text are read
correctly by the updated code? It works on my machine™, but is that just
because my locale is unicode?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Apr 2019 17:45:02 GMT) Full text and rfc822 format available.

Message #74 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Apr 2019 20:44:39 +0300
> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Cc: yamaoka <at> jpl.org,  35383 <at> debbugs.gnu.org
> Date: Tue, 30 Apr 2019 10:19:01 -0700
> 
> > You could give up the cookie.  It is only important if the file is
> > visited by "C-x C-f" etc., which will be rare.  So having a cookie is
> > a nicety, not a requirement.
> 
> Right, I think that was the conclusion. But do you have any other advice
> about making sure that files previously written as 'raw-text are read
> correctly by the updated code? It works on my machine™, but is that just
> because my locale is unicode?

It depends on whether the buffer saved to that file included raw-bytes
(it generally shouldn't).  If not, then there will be no problem.  If
there were raw-bytes, you cannot win, so I wouldn't bother about that
case.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Apr 2019 18:18:02 GMT) Full text and rfc822 format available.

Message #77 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: yamaoka <at> jpl.org, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Apr 2019 11:17:22 -0700
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
>> Cc: yamaoka <at> jpl.org,  35383 <at> debbugs.gnu.org
>> Date: Tue, 30 Apr 2019 10:19:01 -0700
>> 
>> > You could give up the cookie.  It is only important if the file is
>> > visited by "C-x C-f" etc., which will be rare.  So having a cookie is
>> > a nicety, not a requirement.
>> 
>> Right, I think that was the conclusion. But do you have any other advice
>> about making sure that files previously written as 'raw-text are read
>> correctly by the updated code? It works on my machine™, but is that just
>> because my locale is unicode?
>
> It depends on whether the buffer saved to that file included raw-bytes
> (it generally shouldn't).  If not, then there will be no problem.  If
> there were raw-bytes, you cannot win, so I wouldn't bother about that
> case.

Good enough for me! I will swap the defaults to 'utf-8-emacs and push to
the branch -- I haven't gotten to the stage of calling for testers yet,
so with any luck we can still flush out some bugs.

Thanks,
Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 13 May 2019 00:33:01 GMT) Full text and rfc822 format available.

Message #80 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 13 May 2019 09:32:41 +0900
> On 04/23/19 17:12 PM, Katsumi Yamaoka wrote:
>> Though I'm not quite sure
>> yet, the new and the old newsrc.eld files (containing non-ASCII
>> nnml and nnrss groups) seem to be compatible with the new and
>> the old Gnus mutually.

Not mutually but one-way from old Gnus[1] to new Gnus[2] is ok.
To activate the groups of the non-ASCII names for the old Gnus,
I realized I needed to convert those group names to that of

(prin1-to-string (encode-coding-string "Group Name" 'utf-8))

one by one in the .newsrc.eld file before starting the old Gnus.
No other tweak is necessary to switch back to the old Gnus, and
nothing to be done to switch to the new Gnus again.  I verified
it with only nnml and nnrss groups as I tried Gnus of the latest
git master today for the first time in about a month.

[1] currently that of the master and the emacs-26 branches
[2] currentlt that of the scratch/gnus-decoded branch

;; I can switch them by adding site-lisp/gnus (new) to load-path.

Regards,




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 13 May 2019 20:15:02 GMT) Full text and rfc822 format available.

Message #83 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 13 May 2019 13:14:05 -0700
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:

>> On 04/23/19 17:12 PM, Katsumi Yamaoka wrote:
>>> Though I'm not quite sure
>>> yet, the new and the old newsrc.eld files (containing non-ASCII
>>> nnml and nnrss groups) seem to be compatible with the new and
>>> the old Gnus mutually.
>
> Not mutually but one-way from old Gnus[1] to new Gnus[2] is ok.
> To activate the groups of the non-ASCII names for the old Gnus,
> I realized I needed to convert those group names to that of
>
> (prin1-to-string (encode-coding-string "Group Name" 'utf-8))
>
> one by one in the .newsrc.eld file before starting the old Gnus.
> No other tweak is necessary to switch back to the old Gnus, and
> nothing to be done to switch to the new Gnus again.  I verified
> it with only nnml and nnrss groups as I tried Gnus of the latest
> git master today for the first time in about a month.
>
> [1] currently that of the master and the emacs-26 branches
> [2] currentlt that of the scratch/gnus-decoded branch

Thanks for the analysis. I think I only had nnimap groups that were
properly (utf-8) encoded in .newsrc.eld, but still worked with the "old"
Gnus code (in your sense, above). I guess that was different with
nnml/nnrss.

I have some more group sorting issues to fix in the current master code,
then I'll call for testers for the scratch/gnus-decoded branch, then
hopefully this whole thing can go in.

Thanks again,
Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sat, 18 May 2019 20:26:02 GMT) Full text and rfc822 format available.

Message #86 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 18 May 2019 13:25:41 -0700
[Message part 1 (text/plain, inline)]
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

> Katsumi Yamaoka <yamaoka <at> jpl.org> writes:
>
>>> On 04/23/19 17:12 PM, Katsumi Yamaoka wrote:
>>>> Though I'm not quite sure
>>>> yet, the new and the old newsrc.eld files (containing non-ASCII
>>>> nnml and nnrss groups) seem to be compatible with the new and
>>>> the old Gnus mutually.
>>
>> Not mutually but one-way from old Gnus[1] to new Gnus[2] is ok.
>> To activate the groups of the non-ASCII names for the old Gnus,
>> I realized I needed to convert those group names to that of
>>
>> (prin1-to-string (encode-coding-string "Group Name" 'utf-8))
>>
>> one by one in the .newsrc.eld file before starting the old Gnus.
>> No other tweak is necessary to switch back to the old Gnus, and
>> nothing to be done to switch to the new Gnus again.  I verified
>> it with only nnml and nnrss groups as I tried Gnus of the latest
>> git master today for the first time in about a month.
>>
>> [1] currently that of the master and the emacs-26 branches
>> [2] currentlt that of the scratch/gnus-decoded branch
>
> Thanks for the analysis. I think I only had nnimap groups that were
> properly (utf-8) encoded in .newsrc.eld, but still worked with the "old"
> Gnus code (in your sense, above). I guess that was different with
> nnml/nnrss.
>
> I have some more group sorting issues to fix in the current master code,
> then I'll call for testers for the scratch/gnus-decoded branch, then
> hopefully this whole thing can go in.

Before that, I realized that I hadn't updated some of the *Group* buffer
sorting code, for sorting flat groups. The attached patch should go into
master, and then master merged into scratch/gnus-decoded.

Eric

[gnus-group-sorting.diff (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sat, 18 May 2019 22:13:02 GMT) Full text and rfc822 format available.

Message #89 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: Katsumi Yamaoka <yamaoka <at> jpl.org>, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 18 May 2019 23:12:30 +0100
[Message part 1 (text/plain, inline)]
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

> The attached patch should go into master, and then master merged into
> scratch/gnus-decoded.

[...]

> diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el
> index 2f8a260bf1..16d167613e 100644
> --- a/lisp/gnus/gnus-start.el
> +++ b/lisp/gnus/gnus-start.el
> @@ -583,11 +583,11 @@ gnus-subscribe-randomly
>  
>  (defun gnus-subscribe-alphabetically (newgroup)
>    "Subscribe new NEWGROUP and insert it in alphabetical order."
> -  (let ((groups (cdr gnus-newsrc-alist))
> +  (let ((groups (cdr gnus-group-list))
>  	before)
>      (while (and (not before) groups)
> -      (if (string< newgroup (caar groups))
> -	  (setq before (caar groups))
> +      (if (string< newgroup (car groups))
> +	  (setq before (car groups))
>  	(setq groups (cdr groups))))
>      (gnus-subscribe-newsgroup newgroup before)))

I noticed gnus-start.el (already) uses seq.el functions without first
loading the library, so how about the following minor addendum?

[gnus-start.diff (text/x-diff, inline)]
diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el
index 16d167613e..c1e55e4e25 100644
--- a/lisp/gnus/gnus-start.el
+++ b/lisp/gnus/gnus-start.el
@@ -31,6 +31,7 @@
 (require 'gnus-range)
 (require 'gnus-util)
 (require 'gnus-cloud)
+(require 'seq)
 (autoload 'message-make-date "message")
 (autoload 'gnus-agent-read-servers-validate "gnus-agent")
 (autoload 'gnus-agent-save-local "gnus-agent")
@@ -583,12 +584,9 @@ gnus-subscribe-randomly
 
 (defun gnus-subscribe-alphabetically (newgroup)
   "Subscribe new NEWGROUP and insert it in alphabetical order."
-  (let ((groups (cdr gnus-group-list))
-	before)
-    (while (and (not before) groups)
-      (if (string< newgroup (car groups))
-	  (setq before (car groups))
-	(setq groups (cdr groups))))
+  (let ((before (seq-find (lambda (group)
+                            (string< newgroup group))
+                          (cdr gnus-group-list))))
     (gnus-subscribe-newsgroup newgroup before)))
 
 (defun gnus-subscribe-hierarchically (newgroup)
[Message part 3 (text/plain, inline)]
Thanks,

-- 
Basil

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sat, 18 May 2019 23:25:02 GMT) Full text and rfc822 format available.

Message #92 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: Katsumi Yamaoka <yamaoka <at> jpl.org>, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 18 May 2019 16:23:39 -0700
"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>> The attached patch should go into master, and then master merged into
>> scratch/gnus-decoded.
>
> [...]
>
>> diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el
>> index 2f8a260bf1..16d167613e 100644
>> --- a/lisp/gnus/gnus-start.el
>> +++ b/lisp/gnus/gnus-start.el
>> @@ -583,11 +583,11 @@ gnus-subscribe-randomly
>>  
>>  (defun gnus-subscribe-alphabetically (newgroup)
>>    "Subscribe new NEWGROUP and insert it in alphabetical order."
>> -  (let ((groups (cdr gnus-newsrc-alist))
>> +  (let ((groups (cdr gnus-group-list))
>>  	before)
>>      (while (and (not before) groups)
>> -      (if (string< newgroup (caar groups))
>> -	  (setq before (caar groups))
>> +      (if (string< newgroup (car groups))
>> +	  (setq before (car groups))
>>  	(setq groups (cdr groups))))
>>      (gnus-subscribe-newsgroup newgroup before)))
>
> I noticed gnus-start.el (already) uses seq.el functions without first
> loading the library, so how about the following minor addendum?

It does require gnus.el, though, which requires seq, so I figured that
was good enough -- no compiler warnings, anyway.

> (defun gnus-subscribe-alphabetically (newgroup)
> "Subscribe new NEWGROUP and insert it in alphabetical order."
> - (let ((groups (cdr gnus-group-list))
> - before)
> - (while (and (not before) groups)
> - (if (string< newgroup (car groups))
> - (setq before (car groups))
> - (setq groups (cdr groups))))
> + (let ((before (seq-find (lambda (group)
> + (string< newgroup group))
> + (cdr gnus-group-list))))
> (gnus-subscribe-newsgroup newgroup before)))

Looks fine to me! I'll just add this to the patch?

Thanks,
Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sun, 19 May 2019 01:04:01 GMT) Full text and rfc822 format available.

Message #95 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: Katsumi Yamaoka <yamaoka <at> jpl.org>, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 19 May 2019 02:03:28 +0100
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

>> I noticed gnus-start.el (already) uses seq.el functions without first
>> loading the library, so how about the following minor addendum?
>
> It does require gnus.el, though, which requires seq, so I figured that
> was good enough -- no compiler warnings, anyway.

I think it's customary (if not better) for each translation unit to load
its own dependencies, without depending on dependencies to do so, but I
don't know whether this falls in bikeshedding territory.

>> (defun gnus-subscribe-alphabetically (newgroup)
>> "Subscribe new NEWGROUP and insert it in alphabetical order."
>> - (let ((groups (cdr gnus-group-list))
>> - before)
>> - (while (and (not before) groups)
>> - (if (string< newgroup (car groups))
>> - (setq before (car groups))
>> - (setq groups (cdr groups))))
>> + (let ((before (seq-find (lambda (group)
>> + (string< newgroup group))
>> + (cdr gnus-group-list))))
>> (gnus-subscribe-newsgroup newgroup before)))
>
> Looks fine to me! I'll just add this to the patch?

Yes please, thanks,

-- 
Basil




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sun, 19 May 2019 02:58:02 GMT) Full text and rfc822 format available.

Message #98 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: Katsumi Yamaoka <yamaoka <at> jpl.org>, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 18 May 2019 19:56:10 -0700
"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>>> I noticed gnus-start.el (already) uses seq.el functions without first
>>> loading the library, so how about the following minor addendum?
>>
>> It does require gnus.el, though, which requires seq, so I figured that
>> was good enough -- no compiler warnings, anyway.
>
> I think it's customary (if not better) for each translation unit to load
> its own dependencies, without depending on dependencies to do so, but I
> don't know whether this falls in bikeshedding territory.

I don't know if that's policy, TBH, but in the case of Gnus it would
result in a fair amount of churn, I'm not sure it's worth it...




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sun, 19 May 2019 17:03:01 GMT) Full text and rfc822 format available.

Message #101 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: Katsumi Yamaoka <yamaoka <at> jpl.org>, 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 19 May 2019 10:02:10 -0700
On 05/18/19 19:56 PM, Eric Abrahamsen wrote:
> "Basil L. Contovounesios" <contovob <at> tcd.ie> writes:
>
>> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>>
>>>> I noticed gnus-start.el (already) uses seq.el functions without first
>>>> loading the library, so how about the following minor addendum?
>>>
>>> It does require gnus.el, though, which requires seq, so I figured that
>>> was good enough -- no compiler warnings, anyway.
>>
>> I think it's customary (if not better) for each translation unit to load
>> its own dependencies, without depending on dependencies to do so, but I
>> don't know whether this falls in bikeshedding territory.
>
> I don't know if that's policy, TBH, but in the case of Gnus it would
> result in a fair amount of churn, I'm not sure it's worth it...

Okay, I've pushed this to master.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 10 Jun 2019 23:20:02 GMT) Full text and rfc822 format available.

Message #104 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 11 Jun 2019 08:19:04 +0900
Ouch!  `gnus-newsrc-alist' in the .newsrc.eld file turned into a
list of just group names, i.e.:

(setq gnus-newsrc-alist '("nnml:foo" "nnml:bar" "nnml:baz"...))

Ok, I have a backup.  Not investigated it yet sorry, but I guess
this is due to the recent changes in the scratch/gnus-decoded
branch.

On Mon, 10 Jun 2019 00:01:39 -0400, Eric Abrahamsen wrote:
> branch: scratch/gnus-decoded
> commit 2e2ed9f8f2b018e681379d0b4d333038dcc29ebb
> Author: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Commit: Eric Abrahamsen <eric <at> ericabrahamsen.net>

>     Fix/extension to previous commit

>     * lisp/gnus/gnus-start.el (gnus-read-newsrc-el-file): The same
>       decoding needs to be done for group names in gnus-topic-alist.
>       (gnus-gnus-to-quick-newsrc-format): Fix bogus temporary setting of
>       variables; a simply let binding is sufficient.

On Thu, 06 Jun 2019 23:47:59 -0400, Eric Abrahamsen wrote:
> branch: scratch/gnus-decoded
> commit f1c980a979f23676253c71c07a78e4546ee57800
> Author: Eric Abrahamsen <eric <at> ericabrahamsen.net>
> Commit: Eric Abrahamsen <eric <at> ericabrahamsen.net>

>     Preserve group name encoding in newsrc.eld files

>     * lisp/gnus/gnus-start.el (gnus-gnus-to-quick-newsrc-format): Preserve
>       Gnus' earlier odd encoding of group names. Don't change any file
>       formats until it's time to release a new Gnus version.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 11 Jun 2019 00:05:01 GMT) Full text and rfc822 format available.

Message #107 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 10 Jun 2019 17:03:57 -0700
On 06/11/19 08:19 AM, Katsumi Yamaoka wrote:
> Ouch!  `gnus-newsrc-alist' in the .newsrc.eld file turned into a
> list of just group names, i.e.:
>
> (setq gnus-newsrc-alist '("nnml:foo" "nnml:bar" "nnml:baz"...))
>
> Ok, I have a backup.  Not investigated it yet sorry, but I guess
> this is due to the recent changes in the scratch/gnus-decoded
> branch.

Oh yuck, that was a very stupid mistake. I thought I ran this code
before posting... The fix is easy, give me a bit and I'll add another
(tested) commit.

Sorry,
Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 11 Jun 2019 04:35:02 GMT) Full text and rfc822 format available.

Message #110 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 10 Jun 2019 21:33:59 -0700
Katsumi Yamaoka <yamaoka <at> jpl.org> writes:

> Ouch!  `gnus-newsrc-alist' in the .newsrc.eld file turned into a
> list of just group names, i.e.:
>
> (setq gnus-newsrc-alist '("nnml:foo" "nnml:bar" "nnml:baz"...))
>
> Ok, I have a backup.  Not investigated it yet sorry, but I guess
> this is due to the recent changes in the scratch/gnus-decoded
> branch.

Okay, that should be fixed, thanks.

I'm getting more testers!

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 11 Jun 2019 08:10:02 GMT) Full text and rfc822 format available.

Message #113 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 11 Jun 2019 17:09:31 +0900
On Mon, 10 Jun 2019 21:33:59 -0700, Eric Abrahamsen wrote:
> Okay, that should be fixed, thanks.
> I'm getting more testers!

Works fine as before.  Thank you for the fix!




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 17 Jun 2019 06:08:01 GMT) Full text and rfc822 format available.

Message #116 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 16 Jun 2019 23:06:53 -0700
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

> This is part two and the completion of bug#33653, which changed Gnus's
> obarrays into hash tables, and group names from symbols to (encoded)
> strings. The commits in the recently-pushed "scratch/gnus-decoded"
> branch change group names to decoded strings.

I think the code in scratch/gnus-decoded is nearing "ready". I've got a
few testers who have flushed out some good bugs, but it would be nice to
have a couple more.

This change was supposed to involve an update to Gnus' data files
(non-ascii group names would be encoded properly as utf-8). But I ended up
putting in code that maintains the status quo, pending a future bump to
Gnus' version. That will trigger an upgrade that the user has to okay,
and alert them that their data files are no longer backwards-usable.

I do hope to get another tester or two, though!

Eric





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 17 Jun 2019 12:13:01 GMT) Full text and rfc822 format available.

Message #119 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 17 Jun 2019 15:12:20 +0300
Hi Eric,

I would be happy to test for gnus group names.

Sorry, I haven't been following the thread.
So, how can I help testing ?





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 17 Jun 2019 16:24:02 GMT) Full text and rfc822 format available.

Message #122 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 17 Jun 2019 09:22:57 -0700
Deus Max <deusmax <at> gmx.com> writes:

> Hi Eric,
>
> I would be happy to test for gnus group names.
>
> Sorry, I haven't been following the thread.
> So, how can I help testing ?

Cool, thanks! I don't know if you've built Emacs from git before, so
here's complete instructions, assuming something unixy:

git clone https://git.savannah.gnu.org/git/emacs.git
cd emacs
git checkout scratch/gnus-decoded
make (maybe ./autogen.sh is needed?)
[back up your newsrc.eld file, and gnus.registry.eieio if you use it]
src/emacs

Then just start Gnus as normal. Basically I'm just hoping that nothing
"funny" happens. Do what you usually do, maybe kill and yank some
groups, copy messages around, etc. See if anything looks wrong. There's
such an enormous variety of Gnus customization out there, it's important
to get more people with more setups running the code.

Test until you get bored, and let me know if you see anything!

Thanks again,
Eric





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 19 Jun 2019 20:59:02 GMT) Full text and rfc822 format available.

Message #125 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Wed, 19 Jun 2019 23:57:40 +0300
On Mon, Jun 17 2019, Eric Abrahamsen wrote:

> Deus Max <deusmax <at> gmx.com> writes:
>
>> Hi Eric,
>>
>> I would be happy to test for gnus group names.
>>
>> Sorry, I haven't been following the thread.
>> So, how can I help testing ?
>
> Cool, thanks! I don't know if you've built Emacs from git before, so
> here's complete instructions, assuming something unixy:
>
> git clone https://git.savannah.gnu.org/git/emacs.git
> cd emacs
> git checkout scratch/gnus-decoded
> make (maybe ./autogen.sh is needed?)
> [back up your newsrc.eld file, and gnus.registry.eieio if you use it]
> src/emacs
>
> Then just start Gnus as normal. Basically I'm just hoping that nothing
> "funny" happens. Do what you usually do, maybe kill and yank some
> groups, copy messages around, etc. See if anything looks wrong. There's
> such an enormous variety of Gnus customization out there, it's important
> to get more people with more setups running the code.
>
> Test until you get bored, and let me know if you see anything!
>
> Thanks again,
> Eric


Followed your instruction and compiled an emacs binary from the
scratch/gnus-decoded branch.

1. When listing Groups in the Group buffer using (l) or (L), all good.

2. In a group Summary Buffer, when moving (B m) an article to a new
   (non-existent) group in English, Gnus asks, creates group & moves article.

3. Same group Summary Buffer, moving to a Greek named  group, Gnus asks
   with name encoded as numbers and fails to create group. Here is
   '*Message*' buffer:

   : Generating summary...done
   : Loading quail/greek...done
   : No such group: Dias/.  Create it? (y or n) y
   : gnus-read-move-group-name: Couldnt create group Dias/
   : No more articles [2 times]
   : No such group: Dias/.  Create it? (y or n) y
   : gnus-read-move-group-name: Couldnt create group Dias/
   : next-line: End of buffer
   : No more articles [2 times]

Also, trying to save the above text in a separate file, I get (after
setting the coding system to utf-8-emacs-unix):

   : These default coding systems were tried to encode text
   : in the buffer emacs-gnus-group-2019-06-19.txt:
   :   (utf-8-emacs-unix (74 . 4194254) (75 . 4194211) (76 . 4194254) (77 . 4194234) (78
   :   . 4194255) (79 . 4194181) (80 . 4194254) (81 . 4194235) (82 . 4194254) (83 .
   :   4194233) (84 . 4194255))
   : However, each of them encountered characters it couldnt encode:
   :   utf-8-emacs-unix cannot encode these:           ...
   :
   : Click on a character (or switch to this window by M-x other-window
   : and select the characters by RET) to jump to the place it appears,
   : where C-u C-x = will give information about it.
   :
   : Select one of the safe coding systems listed below,
   : or cancel the writing with C-g and edit the buffer
   :    to remove or modify the problematic characters,
   : or specify any other coding system (and risk losing
   :    the problematic characters).
   :
   :   raw-text no-conversion


Hope this helps.
1. comit: 40ad1c0d63804d8d0c30d994907b330ccb952793
2. not using the Gnus registry





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 19 Jun 2019 21:03:01 GMT) Full text and rfc822 format available.

Message #128 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 20 Jun 2019 00:02:26 +0300
On Mon, Jun 17 2019, Eric Abrahamsen wrote:

> Deus Max <deusmax <at> gmx.com> writes:
>
>> Hi Eric,
>>
>> I would be happy to test for gnus group names.
>>
>> Sorry, I haven't been following the thread.
>> So, how can I help testing ?
>
> Cool, thanks! I don't know if you've built Emacs from git before, so
> here's complete instructions, assuming something unixy:
>
> git clone https://git.savannah.gnu.org/git/emacs.git
> cd emacs
> git checkout scratch/gnus-decoded
> make (maybe ./autogen.sh is needed?)
> [back up your newsrc.eld file, and gnus.registry.eieio if you use it]
> src/emacs
>
> Then just start Gnus as normal. Basically I'm just hoping that nothing
> "funny" happens. Do what you usually do, maybe kill and yank some
> groups, copy messages around, etc. See if anything looks wrong. There's
> such an enormous variety of Gnus customization out there, it's important
> to get more people with more setups running the code.
>
> Test until you get bored, and let me know if you see anything!
>
> Thanks again,
> Eric

Hi Eric,

My previous emali converted the messages to ascii.

Here is the Gnus related messages again:

  : Generating summary...done
  : Loading quail/greek...done
  : No such group: Dias/.  Create it? (y or n) y
  : gnus-read-move-group-name: Couldnt create group Dias/
  : No more articles [2 times]
  : No such group: Dias/.  Create it? (y or n) y
  : gnus-read-move-group-name: Couldnt create group Dias/
  : next-line: End of buffer
  : No more articles [2 times]


And the warnings:

  : These default coding systems were tried to encode text
  : in the buffer emacs-gnus-group-2019-06-19.txt:
  :   (utf-8-emacs-unix (74 . 4194254) (75 . 4194211) (76 . 4194254) (77 . 4194234) (78
  :   . 4194255) (79 . 4194181) (80 . 4194254) (81 . 4194235) (82 . 4194254) (83 .
  :   4194233) (84 . 4194255))
  : However, each of them encountered characters it couldnt encode:
  :   utf-8-emacs-unix cannot encode these:           ...
  :
  : Click on a character (or switch to this window by M-x other-window
  : and select the characters by RET) to jump to the place it appears,
  : where C-u C-x = will give information about it.
  :
  : Select one of the safe coding systems listed below,
  : or cancel the writing with C-g and edit the buffer
  :    to remove or modify the problematic characters,
  : or specify any other coding system (and risk losing
  :    the problematic characters).
  :
  :   raw-text no-conversion


Hope they display as I see them.

Dias






Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 19 Jun 2019 21:29:01 GMT) Full text and rfc822 format available.

Message #131 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 20 Jun 2019 00:28:21 +0300
[Message part 1 (text/plain, inline)]
On Mon, Jun 17 2019, Eric Abrahamsen wrote:

> Deus Max <deusmax <at> gmx.com> writes:
>
>> Hi Eric,
>>
>> I would be happy to test for gnus group names.
>>
>> Sorry, I haven't been following the thread.
>> So, how can I help testing ?
>
> Cool, thanks! I don't know if you've built Emacs from git before, so
> here's complete instructions, assuming something unixy:
>
> git clone https://git.savannah.gnu.org/git/emacs.git
> cd emacs
> git checkout scratch/gnus-decoded
> make (maybe ./autogen.sh is needed?)
> [back up your newsrc.eld file, and gnus.registry.eieio if you use it]
> src/emacs
>
> Then just start Gnus as normal. Basically I'm just hoping that nothing
> "funny" happens. Do what you usually do, maybe kill and yank some
> groups, copy messages around, etc. See if anything looks wrong. There's
> such an enormous variety of Gnus customization out there, it's important
> to get more people with more setups running the code.
>
> Test until you get bored, and let me know if you see anything!
>
> Thanks again,
> Eric

Sorry for the multiple posts.
Here's what I was trying to show:

[emacs-GnusGroups-2019-06-20T00-24-16.png (image/png, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Wed, 19 Jun 2019 21:41:01 GMT) Full text and rfc822 format available.

Message #134 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Deus Max <deusmax <at> gmx.com>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Wed, 19 Jun 2019 14:40:36 -0700
On 06/20/19 00:02 AM, Deus Max wrote:
> On Mon, Jun 17 2019, Eric Abrahamsen wrote:
>
>> Deus Max <deusmax <at> gmx.com> writes:
>>
>>> Hi Eric,
>>>
>>> I would be happy to test for gnus group names.
>>>
>>> Sorry, I haven't been following the thread.
>>> So, how can I help testing ?
>>
>> Cool, thanks! I don't know if you've built Emacs from git before, so
>> here's complete instructions, assuming something unixy:
>>
>> git clone https://git.savannah.gnu.org/git/emacs.git
>> cd emacs
>> git checkout scratch/gnus-decoded
>> make (maybe ./autogen.sh is needed?)
>> [back up your newsrc.eld file, and gnus.registry.eieio if you use it]
>> src/emacs
>>
>> Then just start Gnus as normal. Basically I'm just hoping that nothing
>> "funny" happens. Do what you usually do, maybe kill and yank some
>> groups, copy messages around, etc. See if anything looks wrong. There's
>> such an enormous variety of Gnus customization out there, it's important
>> to get more people with more setups running the code.
>>
>> Test until you get bored, and let me know if you see anything!
>>
>> Thanks again,
>> Eric
>
> Hi Eric,
>
> My previous emali converted the messages to ascii.
>
> Here is the Gnus related messages again:
>
>   : Generating summary...done
>   : Loading quail/greek...done
>   : No such group: Dias/」コ�サケ�ケア哩キ.  Create it? (y or n) y
>   : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ

Oh bother. Once again an area I was quite sure I've tested. Can you tell
me which backend you're trying to create the group on?

Thank you very much for testing!

(And don't worry about sending the exact mis-encoded string, it's enough
to know it's mis-encoded.)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 20 Jun 2019 13:01:01 GMT) Full text and rfc822 format available.

Message #137 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: "Deus Max" <deusmax <at> gmx.com>
To: "Eric Abrahamsen" <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 20 Jun 2019 15:00:23 +0200
[Message part 1 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 20 Jun 2019 16:30:02 GMT) Full text and rfc822 format available.

Message #140 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: "Deus Max" <deusmax <at> gmx.com>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 20 Jun 2019 09:29:27 -0700
"Deus Max" <deusmax <at> gmx.com> writes:

>   
>> : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ
>
> Oh bother. Once again an area I was quite sure I've tested. Can you
> tell
> me which backend you're trying to create the group on?
>
> Thank you very much for testing!
>
> (And don't worry about sending the exact mis-encoded string, it's
> enough
> to know it's mis-encoded.)
> 
> 
> 
> Using the nnimap backend

Found the bug, thanks! I have a few other patches I'm working on for
this branch, once those are done I'll push them all together. I
appreciate the help.

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 20 Jun 2019 19:02:02 GMT) Full text and rfc822 format available.

Message #143 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 20 Jun 2019 22:01:11 +0300
On Thu, Jun 20 2019, Eric Abrahamsen wrote:

> "Deus Max" <deusmax <at> gmx.com> writes:
>
>>   
>>> : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ
>>
>> Oh bother. Once again an area I was quite sure I've tested. Can you
>> tell
>> me which backend you're trying to create the group on?
>>
>> Thank you very much for testing!
>>
>> (And don't worry about sending the exact mis-encoded string, it's
>> enough
>> to know it's mis-encoded.)
>> 
>> 
>> 
>> Using the nnimap backend
>
> Found the bug, thanks! I have a few other patches I'm working on for
> this branch, once those are done I'll push them all together. I
> appreciate the help.
>
> Eric


Great !! Glad to help.

Here's another bug:

using (G r) "Rename group" works fine for an English group, but fails
for other lang. characters (Greek as my test case):

Here's a few samples:

  GreeK: (fails)
      Renaming group Σκύλος to ΣκύλοςΓατα...
      Couldn’t rename group Σκύλος to ΣκύλοςΓατα

  English: (OK)
     Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...
     Killed group Dias/MyLifeAsDog
     Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...done


Also multi-language names fail too:

  English+Greek:
     Renaming group Dias/MyLifeAsRat to Dias/MyLifeAsΦως...
     Couldn’t rename group Dias/MyLifeAsRat to Dias/MyLifeAsΦως

Dias




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 20 Jun 2019 19:17:02 GMT) Full text and rfc822 format available.

Message #146 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Deus Max <deusmax <at> gmx.com>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 20 Jun 2019 12:16:44 -0700
On 06/20/19 22:01 PM, Deus Max wrote:
> On Thu, Jun 20 2019, Eric Abrahamsen wrote:
>
>> "Deus Max" <deusmax <at> gmx.com> writes:
>>
>>>   
>>>> : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ
>>>
>>> Oh bother. Once again an area I was quite sure I've tested. Can you
>>> tell
>>> me which backend you're trying to create the group on?
>>>
>>> Thank you very much for testing!
>>>
>>> (And don't worry about sending the exact mis-encoded string, it's
>>> enough
>>> to know it's mis-encoded.)
>>> 
>>> 
>>> 
>>> Using the nnimap backend
>>
>> Found the bug, thanks! I have a few other patches I'm working on for
>> this branch, once those are done I'll push them all together. I
>> appreciate the help.
>>
>> Eric
>
>
> Great !! Glad to help.
>
> Here's another bug:
>
> using (G r) "Rename group" works fine for an English group, but fails
> for other lang. characters (Greek as my test case):
>
> Here's a few samples:
>
>   GreeK: (fails)
>       Renaming group Σκύλος to ΣκύλοςΓατα...
>       Couldn’t rename group Σκύλος to ΣκύλοςΓατα
>
>   English: (OK)
>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...
>      Killed group Dias/MyLifeAsDog
>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...done
>
>
> Also multi-language names fail too:
>
>   English+Greek:
>      Renaming group Dias/MyLifeAsRat to Dias/MyLifeAsΦως...
>      Couldn’t rename group Dias/MyLifeAsRat to Dias/MyLifeAsΦως

Okay, got that one too...




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Fri, 21 Jun 2019 21:00:02 GMT) Full text and rfc822 format available.

Message #149 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Deus Max <deusmax <at> gmx.com>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Fri, 21 Jun 2019 13:59:22 -0700
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

> On 06/20/19 22:01 PM, Deus Max wrote:
>> On Thu, Jun 20 2019, Eric Abrahamsen wrote:
>>
>>> "Deus Max" <deusmax <at> gmx.com> writes:
>>>
>>>>   
>>>>> : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ
>>>>
>>>> Oh bother. Once again an area I was quite sure I've tested. Can you
>>>> tell
>>>> me which backend you're trying to create the group on?
>>>>
>>>> Thank you very much for testing!
>>>>
>>>> (And don't worry about sending the exact mis-encoded string, it's
>>>> enough
>>>> to know it's mis-encoded.)
>>>> 
>>>> 
>>>> 
>>>> Using the nnimap backend
>>>
>>> Found the bug, thanks! I have a few other patches I'm working on for
>>> this branch, once those are done I'll push them all together. I
>>> appreciate the help.
>>>
>>> Eric
>>
>>
>> Great !! Glad to help.
>>
>> Here's another bug:
>>
>> using (G r) "Rename group" works fine for an English group, but fails
>> for other lang. characters (Greek as my test case):
>>
>> Here's a few samples:
>>
>>   GreeK: (fails)
>>       Renaming group Σκύλος to ΣκύλοςΓατα...
>>       Couldn’t rename group Σκύλος to ΣκύλοςΓατα
>>
>>   English: (OK)
>>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...
>>      Killed group Dias/MyLifeAsDog
>>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...done
>>
>>
>> Also multi-language names fail too:
>>
>>   English+Greek:
>>      Renaming group Dias/MyLifeAsRat to Dias/MyLifeAsΦως...
>>      Couldn’t rename group Dias/MyLifeAsRat to Dias/MyLifeAsΦως
>
> Okay, got that one too...

All right, I've pushed these fixes and a bunch of other ones to
scratch/gnus-decoded. If you're still interested in testing, you can
pull in that branch, then either run "make bootstrap" or delete all the
*elc files under lisp/gnus and run a regular "make", then try again.

Unfortunately this led to so many new tweaks and fixes that I'm going to
restart the clock on dogfooding and testing before it's ready to push.

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sat, 22 Jun 2019 14:46:01 GMT) Full text and rfc822 format available.

Message #152 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 22 Jun 2019 17:44:39 +0300
On Fri, Jun 21 2019, Eric Abrahamsen wrote:

> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>> On 06/20/19 22:01 PM, Deus Max wrote:
>>> On Thu, Jun 20 2019, Eric Abrahamsen wrote:
>>>
>>>> "Deus Max" <deusmax <at> gmx.com> writes:
>>>>
>>>>>   
>>>>>> : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ
>>>>>
>>>>> Oh bother. Once again an area I was quite sure I've tested. Can you
>>>>> tell
>>>>> me which backend you're trying to create the group on?
>>>>>
>>>>> Thank you very much for testing!
>>>>>
>>>>> (And don't worry about sending the exact mis-encoded string, it's
>>>>> enough
>>>>> to know it's mis-encoded.)
>>>>> 
>>>>> 
>>>>> 
>>>>> Using the nnimap backend
>>>>
>>>> Found the bug, thanks! I have a few other patches I'm working on for
>>>> this branch, once those are done I'll push them all together. I
>>>> appreciate the help.
>>>>
>>>> Eric
>>>
>>>
>>> Great !! Glad to help.
>>>
>>> Here's another bug:
>>>
>>> using (G r) "Rename group" works fine for an English group, but fails
>>> for other lang. characters (Greek as my test case):
>>>
>>> Here's a few samples:
>>>
>>>   GreeK: (fails)
>>>       Renaming group Σκύλος to ΣκύλοςΓατα...
>>>       Couldn’t rename group Σκύλος to ΣκύλοςΓατα
>>>
>>>   English: (OK)
>>>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...
>>>      Killed group Dias/MyLifeAsDog
>>>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...done
>>>
>>>
>>> Also multi-language names fail too:
>>>
>>>   English+Greek:
>>>      Renaming group Dias/MyLifeAsRat to Dias/MyLifeAsΦως...
>>>      Couldn’t rename group Dias/MyLifeAsRat to Dias/MyLifeAsΦως
>>
>> Okay, got that one too...
>
> All right, I've pushed these fixes and a bunch of other ones to
> scratch/gnus-decoded. If you're still interested in testing, you can
> pull in that branch, then either run "make bootstrap" or delete all the
> *elc files under lisp/gnus and run a regular "make", then try again.
>
> Unfortunately this led to so many new tweaks and fixes that I'm going to
> restart the clock on dogfooding and testing before it's ready to push.
>
> Eric

dogfooding ??




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sat, 22 Jun 2019 16:10:02 GMT) Full text and rfc822 format available.

Message #155 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Deus Max <deusmax <at> gmx.com>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 22 Jun 2019 09:09:05 -0700
Deus Max <deusmax <at> gmx.com> writes:

> On Fri, Jun 21 2019, Eric Abrahamsen wrote:
>
>> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>>
>>> On 06/20/19 22:01 PM, Deus Max wrote:
>>>> On Thu, Jun 20 2019, Eric Abrahamsen wrote:
>>>>
>>>>> "Deus Max" <deusmax <at> gmx.com> writes:
>>>>>
>>>>>>   
>>>>>>> : gnus-read-move-group-name: Couldnt create group Dias/」コ�サケ�ケア哩キ
>>>>>>
>>>>>> Oh bother. Once again an area I was quite sure I've tested. Can you
>>>>>> tell
>>>>>> me which backend you're trying to create the group on?
>>>>>>
>>>>>> Thank you very much for testing!
>>>>>>
>>>>>> (And don't worry about sending the exact mis-encoded string, it's
>>>>>> enough
>>>>>> to know it's mis-encoded.)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Using the nnimap backend
>>>>>
>>>>> Found the bug, thanks! I have a few other patches I'm working on for
>>>>> this branch, once those are done I'll push them all together. I
>>>>> appreciate the help.
>>>>>
>>>>> Eric
>>>>
>>>>
>>>> Great !! Glad to help.
>>>>
>>>> Here's another bug:
>>>>
>>>> using (G r) "Rename group" works fine for an English group, but fails
>>>> for other lang. characters (Greek as my test case):
>>>>
>>>> Here's a few samples:
>>>>
>>>>   GreeK: (fails)
>>>>       Renaming group Σκύλος to ΣκύλοςΓατα...
>>>>       Couldn’t rename group Σκύλος to ΣκύλοςΓατα
>>>>
>>>>   English: (OK)
>>>>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...
>>>>      Killed group Dias/MyLifeAsDog
>>>>      Renaming group Dias/MyLifeAsDog to Dias/MyLifeAsRat...done
>>>>
>>>>
>>>> Also multi-language names fail too:
>>>>
>>>>   English+Greek:
>>>>      Renaming group Dias/MyLifeAsRat to Dias/MyLifeAsΦως...
>>>>      Couldn’t rename group Dias/MyLifeAsRat to Dias/MyLifeAsΦως
>>>
>>> Okay, got that one too...
>>
>> All right, I've pushed these fixes and a bunch of other ones to
>> scratch/gnus-decoded. If you're still interested in testing, you can
>> pull in that branch, then either run "make bootstrap" or delete all the
>> *elc files under lisp/gnus and run a regular "make", then try again.
>>
>> Unfortunately this led to so many new tweaks and fixes that I'm going to
>> restart the clock on dogfooding and testing before it's ready to push.
>>
>> Eric
>
> dogfooding ??

Ha -- "eating your own dogfood", ie I'll spend a week or two using this
branch as my daily driver, and see if anything explodes. I'm not sure
who started using that term, and why it has to be dog food, in particular.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sun, 23 Jun 2019 10:41:01 GMT) Full text and rfc822 format available.

Message #158 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 23 Jun 2019 13:27:24 +0300
On Sat, Jun 22 2019, Eric Abrahamsen wrote:

>>> All right, I've pushed these fixes and a bunch of other ones to
>>> scratch/gnus-decoded. If you're still interested in testing, you can
>>> pull in that branch, then either run "make bootstrap" or delete all the
>>> *elc files under lisp/gnus and run a regular "make", then try again.
>>>
>>> Unfortunately this led to so many new tweaks and fixes that I'm going to
>>> restart the clock on dogfooding and testing before it's ready to push.
>>>
>>> Eric
>>
>> dogfooding ??
>
> Ha -- "eating your own dogfood", ie I'll spend a week or two using this
> branch as my daily driver, and see if anything explodes. I'm not sure
> who started using that term, and why it has to be dog food, in particular.

:-)  I like that. Funny !

Pulled in and bootstrapped. Here's what I found:

1. From a Summary Buffer used "B m" to move articles to a new Greek
   group. 
   - The confirmation of this action disappears to the message
     "nnimap read...". 
     : No such group: nnimap+AIA:ΖωηΕνΤάφο.  Create it? (y or n) y
     : Moving to nnimap+AIA:ΖωηΕνΤάφο: (27666)...
     : nnimap read 0k from imap-mail.outlook.com (initial sync of 1
     :   group; please wait)
   
2. The article count for the encoded group is +1 the actual number.

3. In the Group buffer typing "j" (gnus-group-jump-to-group), creates
   the buffer with a "K" and is not usable.  ex:
   :  K     *: Dias/CrappyΚώλος

   From the *Messages* buffer (No unknown folder ??):
   : Retrieving newsgroup: Dias/CrappyΚώλος...
   : gnus-select-newsgroup: Couldn’t activate group Dias/CrappyΚώλος: 
   :   NO unknown folder

4. Further, when trying to move articles to above group:
   : No such group: Dias/CrappyΚώλος.  Create it? (y or n) n
   : gnus-read-move-group-name: No such group: Dias/CrappyΚώλος

Dias






Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 08 Jul 2019 03:10:01 GMT) Full text and rfc822 format available.

Message #161 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Deus Max <deusmax <at> gmx.com>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sun, 07 Jul 2019 20:09:13 -0700
On 06/23/19 13:27 PM, Deus Max wrote:
> On Sat, Jun 22 2019, Eric Abrahamsen wrote:
>
>>>> All right, I've pushed these fixes and a bunch of other ones to
>>>> scratch/gnus-decoded. If you're still interested in testing, you can
>>>> pull in that branch, then either run "make bootstrap" or delete all the
>>>> *elc files under lisp/gnus and run a regular "make", then try again.
>>>>
>>>> Unfortunately this led to so many new tweaks and fixes that I'm going to
>>>> restart the clock on dogfooding and testing before it's ready to push.
>>>>
>>>> Eric
>>>
>>> dogfooding ??
>>
>> Ha -- "eating your own dogfood", ie I'll spend a week or two using this
>> branch as my daily driver, and see if anything explodes. I'm not sure
>> who started using that term, and why it has to be dog food, in particular.
>
> :-)  I like that. Funny !
>
> Pulled in and bootstrapped. Here's what I found:
>
> 1. From a Summary Buffer used "B m" to move articles to a new Greek
>    group. 
>    - The confirmation of this action disappears to the message
>      "nnimap read...". 
>      : No such group: nnimap+AIA:ΖωηΕνΤάφο.  Create it? (y or n) y
>      : Moving to nnimap+AIA:ΖωηΕνΤάφο: (27666)...
>      : nnimap read 0k from imap-mail.outlook.com (initial sync of 1
>      :   group; please wait)
>    
> 2. The article count for the encoded group is +1 the actual number.
>
> 3. In the Group buffer typing "j" (gnus-group-jump-to-group), creates
>    the buffer with a "K" and is not usable.  ex:
>    :  K     *: Dias/CrappyΚώλος
>
>    From the *Messages* buffer (No unknown folder ??):
>    : Retrieving newsgroup: Dias/CrappyΚώλος...
>    : gnus-select-newsgroup: Couldn’t activate group Dias/CrappyΚώλος: 
>    :   NO unknown folder
>
> 4. Further, when trying to move articles to above group:
>    : No such group: Dias/CrappyΚώλος.  Create it? (y or n) n
>    : gnus-read-move-group-name: No such group: Dias/CrappyΚώλος

Sorry for my slow response, and thank you for the testing! This was very
thorough, and I will definitely be coming back to you for testing in the
future :)

So far as I know, all the behaviors you've noted here are bugs, but I've
also been able to reproduce the bugs in the Emacs 26 branch. Which
doesn't mean I won't fix them -- it would be a pleasure to fix bugs that
I didn't cause myself, for once -- but I might hold off on them until
this branch has landed. One thing at a time...

Thanks again!

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Mon, 08 Jul 2019 19:48:01 GMT) Full text and rfc822 format available.

Message #164 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Deus Max <deusmax <at> gmx.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Mon, 08 Jul 2019 22:46:37 +0300
On Sun, Jul 07 2019, Eric Abrahamsen wrote:

>
> Sorry for my slow response, and thank you for the testing! This was very
> thorough, and I will definitely be coming back to you for testing in the
> future :)
>
> So far as I know, all the behaviors you've noted here are bugs, but I've
> also been able to reproduce the bugs in the Emacs 26 branch. Which
> doesn't mean I won't fix them -- it would be a pleasure to fix bugs that
> I didn't cause myself, for once -- but I might hold off on them until
> this branch has landed. One thing at a time...
>
> Thanks again!
>
> Eric

Sure, happy to help.
For me Gnus is one of Emacs' underappreciated gems.

Let me know when its done.











Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 23 Jul 2019 23:53:02 GMT) Full text and rfc822 format available.

Message #167 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 23 Jul 2019 16:52:37 -0700
By god, I think this branch (scratch/gnus-decoded) is done. I've been
running it for several weeks, have added some additional fixes, and have
merged a few times from master to keep up with development.

I just rebased onto master (it went better than I expected, the
merge-from-master commits were ignored), so now I have a local branch
with 19 commits in it, some of which are just WIP commits, others small
fixes and tweaks to previous commits in the branch.

If anyone wants to see this again I can push a new branch (say
feature/gnus-decoded) with either squashed or unsquashed commits, or I
can post patches here.

I'm assuming the final branch should be squashed. One thing about the
code here is that it preserves backwards compatibility with Gnus' files
(newsrc.eld, the registry persistence file if that's used, and the agent
category files), keeping non-ascii group names written in an encoded
format. My idea was that, as Emacs 27 is released, we could bump the
Gnus version a point, which would trigger `gnus-convert-old-newsrc' (or
maybe this would be a clean?), and the permanent decoding could be done
there -- the user would at least be warned that their files might not be
backwards compatible.

Anyway, if anyone has any opinions on any of this, let me know!

Eric




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Tue, 30 Jul 2019 23:01:05 GMT) Full text and rfc822 format available.

Message #170 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: 35383 <at> debbugs.gnu.org 
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Tue, 30 Jul 2019 16:00:30 -0700
[Message part 1 (text/plain, inline)]
Okay, I decided to squash into two commits: the first holding the vast
majority of the changes (but still fairly atomic), and the second
holding some temporary backward compatibility tomfoolery, which should
be replaced by a proper upgrade (including an uptick to Gnus' version)
before 27 is released.

Speak now! Or... file bug reports later.

[0002-Temporarily-preserve-encoded-Gnus-group-names-in-Gnu.patch (text/x-patch, attachment)]
[0001-Remove-Gnus-group-name-encoding-decoding.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 01 Aug 2019 12:08:01 GMT) Full text and rfc822 format available.

Message #173 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 01 Aug 2019 14:07:32 +0200
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

> Okay, I decided to squash into two commits: the first holding the vast
> majority of the changes (but still fairly atomic), and the second
> holding some temporary backward compatibility tomfoolery, which should
> be replaced by a proper upgrade (including an uptick to Gnus' version)
> before 27 is released.
>
> Speak now! Or... file bug reports later.

Wow.  That's a huge patch.  :-)

I've only...  skimmed it (if you can call it that), but the main idea is
sound.  As there isn't any test harness for this, really, I guess the
only way to find out whether there's any fallout from this change is to
apply it and see what breaks, and then fix that.  So when it's applied
you should be on hand to quickly fix breakages over the next week or
so.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Thu, 01 Aug 2019 16:23:01 GMT) Full text and rfc822 format available.

Message #176 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Thu, 01 Aug 2019 09:22:03 -0700
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>> Okay, I decided to squash into two commits: the first holding the vast
>> majority of the changes (but still fairly atomic), and the second
>> holding some temporary backward compatibility tomfoolery, which should
>> be replaced by a proper upgrade (including an uptick to Gnus' version)
>> before 27 is released.
>>
>> Speak now! Or... file bug reports later.
>
> Wow.  That's a huge patch.  :-)

Yes it is! Though most of it is just removing calls to
`gnus-group-decoded-name' and associated let bindings. The part I'm
least confident about is the removal of
`nnmail-group-names-not-encoded-p', since there's more filesystem
interaction.

> I've only... skimmed it (if you can call it that), but the main idea is
> sound.  As there isn't any test harness for this, really, I guess the
> only way to find out whether there's any fallout from this change is to
> apply it and see what breaks, and then fix that.  So when it's applied
> you should be on hand to quickly fix breakages over the next week or
> so.  :-)

I haven't come up with any good method of automated testing. I've
started putting "interactive" tests in the gnus-mock package, where you
start up a Gnus instance and then run the tests and watch them go. I
can't think of any better approach right now.

I will definitely be on call for the next few weeks!

Eric





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Sat, 03 Aug 2019 21:55:01 GMT) Full text and rfc822 format available.

Message #179 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Sat, 03 Aug 2019 14:53:58 -0700
On 08/01/19 14:07 PM, Lars Ingebrigtsen wrote:
> Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:
>
>> Okay, I decided to squash into two commits: the first holding the vast
>> majority of the changes (but still fairly atomic), and the second
>> holding some temporary backward compatibility tomfoolery, which should
>> be replaced by a proper upgrade (including an uptick to Gnus' version)
>> before 27 is released.
>>
>> Speak now! Or... file bug reports later.
>
> Wow.  That's a huge patch.  :-)
>
> I've only...  skimmed it (if you can call it that), but the main idea is
> sound.  As there isn't any test harness for this, really, I guess the
> only way to find out whether there's any fallout from this change is to
> apply it and see what breaks, and then fix that.  So when it's applied
> you should be on hand to quickly fix breakages over the next week or
> so.  :-)

Bombs away...




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#35383; Package emacs. (Fri, 27 Sep 2019 14:59:01 GMT) Full text and rfc822 format available.

Message #182 received at 35383 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 35383 <at> debbugs.gnu.org
Subject: Re: bug#35383: 27.0.50; Complete process of decoding Gnus group names
Date: Fri, 27 Sep 2019 16:58:20 +0200
Eric Abrahamsen <eric <at> ericabrahamsen.net> writes:

>> I've only...  skimmed it (if you can call it that), but the main idea is
>> sound.  As there isn't any test harness for this, really, I guess the
>> only way to find out whether there's any fallout from this change is to
>> apply it and see what breaks, and then fix that.  So when it's applied
>> you should be on hand to quickly fix breakages over the next week or
>> so.  :-)
>
> Bombs away...

It seemed like everything worked fine.  :-) So I'm closing this bug
report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Added tag(s) fixed. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Fri, 27 Sep 2019 14:59:02 GMT) Full text and rfc822 format available.

bug marked as fixed in version 27.1, send any further explanations to 35383 <at> debbugs.gnu.org and Eric Abrahamsen <eric <at> ericabrahamsen.net> Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Fri, 27 Sep 2019 14:59:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 26 Oct 2019 11:24:08 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 176 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.