GNU bug report logs - #40676
28.0.50; gnus locks when reading email

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40676 in the body.
You can then email your comments to 40676 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alex Branham <alex.branham <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; gnus locks when reading email
Date: Thu, 16 Apr 2020 19:18:18 -0400

Hello -

gnus locks when reading email (when trying to display the message
contents) and asks me:

Buffer " *temp*" has a running process; kill it? (y or n) y

This seems to happen independently of backend, as I use nnimap and nntp.
It also happens when reading through debbugs-gnu.

Specifically, to reproduce, I do:

M-x gnus
RET (this opens an inbox)
RET (this tries to open an email)

I can then see the new window & buffer pop up with initial content like:

From: <foo>
Subject: <bar>
To: <baz>
Date: <fooier>
Reply-To: <barier>
X-Boundary: <bazier>

but it's not font locked and the rest of the email doesn't show until I
hit y or n (the email shows after hitting y or n, doesn't seem to make a
difference).

Thanks,
Alex

Message #8 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Fri, 17 Apr 2020 09:51:18 +0300

> From: Alex Branham <alex.branham <at> gmail.com>
> Date: Thu, 16 Apr 2020 19:18:18 -0400
> 
> gnus locks when reading email (when trying to display the message
> contents) and asks me:
> 
> Buffer " *temp*" has a running process; kill it? (y or n) y

What does list-processes display in that situation?

Message #11 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Fri, 17 Apr 2020 12:51:26 +0100

reassign 40676 emacs,gnus
quit

Alex Branham <alex.branham <at> gmail.com> writes:

> gnus locks when reading email (when trying to display the message
> contents) and asks me:
>
> Buffer " *temp*" has a running process; kill it? (y or n) y
>
> This seems to happen independently of backend, as I use nnimap and nntp.
> It also happens when reading through debbugs-gnu.
>
> Specifically, to reproduce, I do:
>
> M-x gnus
> RET (this opens an inbox)
> RET (this tries to open an email)

I can't reproduce this from 'emacs -Q'; could some element of your
configuration be giving rise to this?  Can you try reproducing the
problem from 'emacs -Q'?  Here's what I tried:

0. HOME=$(mktemp -d) emacs -Q
1. (setq gnus-select-method '(nntp "news.gwene.org")) C-x C-e
2. M-x gnus RET
3. ^ C-s gwene RET RET (browse gwene groups)
4. C-s gmane.emacs.devel RET (search for emacs-devel group)
5. u q q (subscribe and return to Group buffer)
6. RET 10 RET RET (select first out of 10 latest articles)

Thanks,

-- 
Basil

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw3d scroll bars) of 2020-04-16 built on thunk

Message #18 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Fri, 17 Apr 2020 08:09:28 -0700

Alex Branham <alex.branham <at> gmail.com> writes:

> Hello -
>
> gnus locks when reading email (when trying to display the message
> contents) and asks me:
>
> Buffer " *temp*" has a running process; kill it? (y or n) y
>
> This seems to happen independently of backend, as I use nnimap and nntp.
> It also happens when reading through debbugs-gnu.
>
> Specifically, to reproduce, I do:
>
> M-x gnus
> RET (this opens an inbox)
> RET (this tries to open an email)
>
> I can then see the new window & buffer pop up with initial content like:
>
> From: <foo>
> Subject: <bar>
> To: <baz>
> Date: <fooier>
> Reply-To: <barier>
> X-Boundary: <bazier>
>
> but it's not font locked and the rest of the email doesn't show until I
> hit y or n (the email shows after hitting y or n, doesn't seem to make a
> difference).

FWIW, a " *temp*" buffer is only created in mm-util.el, in
`mm-find-buffer-file-coding-system'. My guess is that some mime handler
is calling an external process to do some sort of decoding of article
content or attachment, and that's somehow getting attached to the temp
buffer incorrectly.

Total guess!

Message #21 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Fri, 17 Apr 2020 17:14:54 +0200

>>>>> On Fri, 17 Apr 2020 08:09:28 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:

    Eric> FWIW, a " *temp*" buffer is only created in mm-util.el, in
    Eric> `mm-find-buffer-file-coding-system'. My guess is that some mime handler
    Eric> is calling an external process to do some sort of decoding of article
    Eric> content or attachment, and that's somehow getting attached to the temp
    Eric> buffer incorrectly.

'with-temp-buffer' also creates such buffers, so it could be any
number of things. The output of 'list-processes' will tell us.

Robert

Message #24 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Fri, 17 Apr 2020 08:22:33 -0700

Robert Pluim <rpluim <at> gmail.com> writes:

>>>>>> On Fri, 17 Apr 2020 08:09:28 -0700, Eric Abrahamsen <eric <at> ericabrahamsen.net> said:
>
>     Eric> FWIW, a " *temp*" buffer is only created in mm-util.el, in
>     Eric> `mm-find-buffer-file-coding-system'. My guess is that some mime handler
>     Eric> is calling an external process to do some sort of decoding of article
>     Eric> content or attachment, and that's somehow getting attached to the temp
>     Eric> buffer incorrectly.
>
> 'with-temp-buffer' also creates such buffers, so it could be any
> number of things. The output of 'list-processes' will tell us.

Oh, I didn't know that, thanks! Looks like those buffer names are done
with `generate-new-buffer-name', though (I just tested and they all had
numbers appended), so I mm-util theory might not be 100% a red herring.
Pink herring? Anyway...

Message #27 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Alex Branham <alex.branham <at> gmail.com>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Fri, 17 Apr 2020 13:26:30 -0400

> I can't reproduce this from 'emacs -Q'; could some element of your
> configuration be giving rise to this?  Can you try reproducing the
> problem from 'emacs -Q'?  Here's what I tried:
>
> 0. HOME=$(mktemp -d) emacs -Q
> 1. (setq gnus-select-method '(nntp "news.gwene.org")) C-x C-e
> 2. M-x gnus RET
> 3. ^ C-s gwene RET RET (browse gwene groups)
> 4. C-s gmane.emacs.devel RET (search for emacs-devel group)
> 5. u q q (subscribe and return to Group buffer)
> 6. RET 10 RET RET (select first out of 10 latest articles)

Thanks. I can't reproduce it that way. I don't think I have anything too
terribly crazy in my gnus config but I'll try to make time to go through
it.

> What does list-processes display in that situation?

Eli, the output of list-processes (removing some non-gnus related stuff) is:

*nnimap*        --      open     *nnimap localhost nil  *nntpd** --           Main         (network connection to localhost:imap)
dns             --      open     *temp*                   --           Main         (datagram connection to 192.168.1.1:domain)
dns<1>          --      open     *temp*-198741            --           Main         (datagram connection to 192.168.1.1:domain)
dns<2>          --      open     *temp*-84789             --           Main         (datagram connection to 192.168.1.1:domain)
dns<3>          --      open     *temp*-23796             --           Main         (datagram connection to 192.168.1.1:domain)
nntpd           --      open     *server news.gmane.io nntp  *nntpd** --           Main         (network connection to news.gmane.io:nntp)
server          --      listen  --                        --           Main         (network server on /run/user/1000/emacs/server)

At this point, gnus was complaining about the *temp*-23796 buffer
process having a running process.

I'm not sure if it's related, but recently-ish (after Emacs 26 but
before Emacs 27) gnus asks me on every open a question concerning
localhost certificates. I select "session".  Here's the output from the
message buffer:

Opening connection to localhost...
Continue connecting? (always, session only, no, details, ?): 
Accepting certificate for localhost:imap this session only.

Thanks, and let me know if I can provide any other info,
Alex

Message #30 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Alex Branham <alex.branham <at> gmail.com>
Cc: "Basil L. Contovounesios" <contovob <at> tcd.ie>, Eli Zaretskii <eliz <at> gnu.org>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sat, 18 Apr 2020 12:26:07 +0200

>>>>> On Fri, 17 Apr 2020 13:26:30 -0400, Alex Branham <alex.branham <at> gmail.com> said:

    Alex> Thanks. I can't reproduce it that way. I don't think I have anything too
    Alex> terribly crazy in my gnus config but I'll try to make time to go through
    Alex> it.

My crystal ball says you have one or more of the gnus-gravatar
variables enabled, and you have a slow DNS, in which case customizing
'gravatar-service' to something other than 'libravatar will help.

(given the troubles the gravatar changes are causing, maybe we should
flip the default away from libravatar for the moment)

    >> What does list-processes display in that situation?

    Alex> Eli, the output of list-processes (removing some non-gnus related stuff) is:

    Alex> *nnimap*        --      open     *nnimap localhost nil  *nntpd** --           Main         (network connection to localhost:imap)
    Alex> dns             --      open     *temp*                   --           Main         (datagram connection to 192.168.1.1:domain)
    Alex> dns<1>          --      open     *temp*-198741            --           Main         (datagram connection to 192.168.1.1:domain)
    Alex> dns<2>          --      open     *temp*-84789             --           Main         (datagram connection to 192.168.1.1:domain)
    Alex> dns<3>          --      open     *temp*-23796             --           Main         (datagram connection to 192.168.1.1:domain)
    Alex> nntpd           --      open     *server news.gmane.io nntp  *nntpd** --           Main         (network connection to news.gmane.io:nntp)
    Alex> server          --      listen  --                        --           Main         (network server on /run/user/1000/emacs/server)

    Alex> At this point, gnus was complaining about the *temp*-23796 buffer
    Alex> process having a running process.

Although looking at the relevant code in dns.el, that should kill the
process unconditionally before killing the buffer, so I donʼt
understand why the process is still hanging around. It does:

          (condition-case nil
              (delete-process process)
            (error nil))

Sledgehammer patch to stop the querying below.

    Alex> I'm not sure if it's related, but recently-ish (after Emacs 26 but
    Alex> before Emacs 27) gnus asks me on every open a question concerning
    Alex> localhost certificates. I select "session".  Here's the output from the
    Alex> message buffer:

    Alex> Opening connection to localhost...
    Alex> Continue connecting? (always, session only, no, details, ?): 
    Alex> Accepting certificate for localhost:imap this session only.

Given that this is your local imap server, I think you can get away
with saying 'always', and the network security manager will stop
prompting.

Robert

diff --git a/lisp/net/dns.el b/lisp/net/dns.el
index 53ea0b19b5..4e0e1d1b0b 100644
--- a/lisp/net/dns.el
+++ b/lisp/net/dns.el
@@ -367,12 +365,13 @@ dns-make-network-process
 	  :coding 'binary
 	  :buffer (current-buffer)
 	  :host server
+          :noquery t
 	  :service "domain"
 	  :type 'datagram)

Message #33 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Alex Branham <alex.branham <at> gmail.com>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: "Basil L. Contovounesios" <contovob <at> tcd.ie>, Eli Zaretskii <eliz <at> gnu.org>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sat, 18 Apr 2020 07:50:48 -0400

On Sat 18 Apr 2020 at 12:26, Robert Pluim <rpluim <at> gmail.com> wrote:

>>>>>> On Fri, 17 Apr 2020 13:26:30 -0400, Alex Branham <alex.branham <at> gmail.com> said:
>
>     Alex> Thanks. I can't reproduce it that way. I don't think I have anything too
>     Alex> terribly crazy in my gnus config but I'll try to make time to go through
>     Alex> it.
>
> My crystal ball says you have one or more of the gnus-gravatar
> variables enabled, and you have a slow DNS, in which case customizing
> 'gravatar-service' to something other than 'libravatar will help.

Thanks! Changing gnus-treat-from-gravatar to nil (from 'head) solves the
issue so it seems your crystal ball is correct.

Alex

Message #36 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: "Basil L. Contovounesios" <contovob <at> tcd.ie>,
 Alex Branham <alex.branham <at> gmail.com>, Philip K <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sun, 19 Jul 2020 04:20:39 +0200

Robert Pluim <rpluim <at> gmail.com> writes:

> My crystal ball says you have one or more of the gnus-gravatar
> variables enabled, and you have a slow DNS, in which case customizing
> 'gravatar-service' to something other than 'libravatar will help.

Oh, deer.  I just tried

  (setq gnus-treat-from-gravatar 'head)

and the behaviour in Gnus is totally unacceptable -- it stops and thinks
for a second after rendering the header, and then renders the body of
the message.

This is a regression -- the new libavatar code obviously can't work this
way.  If it's taking a lot of time, it has to run asynchronously after
the rest of the article has finished rendering.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #39 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: "Philip K." <philip <at> warpmail.net>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, eliz <at> gnu.org, alex.branham <at> gmail.com,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sun, 19 Jul 2020 10:15:49 +0200

That was part of the rationale behind #40355, but the best way I see to
fix this would be to implement asynchronous DNS, since a libravatar
lookup has two phases (DNS lookup + image retrieval), compared to
Gravatar's single request.

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Robert Pluim <rpluim <at> gmail.com> writes:
>
>> My crystal ball says you have one or more of the gnus-gravatar
>> variables enabled, and you have a slow DNS, in which case customizing
>> 'gravatar-service' to something other than 'libravatar will help.
>
> Oh, deer.  I just tried
>
>   (setq gnus-treat-from-gravatar 'head)
>
> and the behaviour in Gnus is totally unacceptable -- it stops and thinks
> for a second after rendering the header, and then renders the body of
> the message.
>
> This is a regression -- the new libavatar code obviously can't work this
> way.  If it's taking a lot of time, it has to run asynchronously after
> the rest of the article has finished rendering.


-- 
	Philip K.

Message #42 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Philip K." <philip <at> warpmail.net>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, eliz <at> gnu.org, alex.branham <at> gmail.com,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sun, 19 Jul 2020 15:02:35 +0200

"Philip K." <philip <at> warpmail.net> writes:

> That was part of the rationale behind #40355, but the best way I see to
> fix this would be to implement asynchronous DNS, since a libravatar
> lookup has two phases (DNS lookup + image retrieval), compared to
> Gravatar's single request.

A cache would be a band-aid -- it still will have to do these lookups
occasionally, and the user experience in Gnus suffers.

As it stands, librgravatar shouldn't be the default gravatar provider.

But, yes, Emacs should have asynchronous DNS support, and adding that
probably isn't too difficult, I'd have thought?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #45 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: "Philip K." <philip <at> warpmail.net>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, eliz <at> gnu.org, alex.branham <at> gmail.com,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sun, 19 Jul 2020 15:37:41 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> "Philip K." <philip <at> warpmail.net> writes:
>
>> That was part of the rationale behind #40355, but the best way I see to
>> fix this would be to implement asynchronous DNS, since a libravatar
>> lookup has two phases (DNS lookup + image retrieval), compared to
>> Gravatar's single request.
>
> A cache would be a band-aid -- it still will have to do these lookups
> occasionally, and the user experience in Gnus suffers.

Another idea would be to only wait for a few milliseconds (or whatever
is reasonable) and only return an image if the backend manages to find
something in that time. But the request is still finished and cached for
the next query.

> As it stands, librgravatar shouldn't be the default gravatar provider.

I'm somewhat split on this. On the one hand, the reason I implemented
libravatar support is to increase it's audience and make more people
aware if it's existence, as a free and federated alternative to
Gravatar.

On the other hand, there is a privacy issue in the system, as explained
by [0]. The problem basically is that if I send you an email and set up
a libravatar server, by querying my server, I can know if you opened my
message or not. I can imagine spammers being very interested in
something like this.

> But, yes, Emacs should have asynchronous DNS support, and adding that
> probably isn't too difficult, I'd have thought?

I started writing something a few months ago, but didn't have the time
to finish it. But you're right, it shouldn't be too much work to come up
with a rough draft.

[0] https://lobste.rs/s/nwgljm/libravatar_federated_avatar_hosting#c_00fsba

-- 
	Philip K.

Message #48 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Philip K." <philip <at> warpmail.net>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, eliz <at> gnu.org, alex.branham <at> gmail.com,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Sun, 19 Jul 2020 15:52:34 +0200

"Philip K." <philip <at> warpmail.net> writes:

>> As it stands, librgravatar shouldn't be the default gravatar provider.
>
> I'm somewhat split on this. On the one hand, the reason I implemented
> libravatar support is to increase it's audience and make more people
> aware if it's existence, as a free and federated alternative to
> Gravatar.
>
> On the other hand, there is a privacy issue in the system, as explained
> by [0]. The problem basically is that if I send you an email and set up
> a libravatar server, by querying my server, I can know if you opened my
> message or not. I can imagine spammers being very interested in
> something like this.

Indeed -- it makes libravatar a much worse interface than the central
Gravatar service (where you just have to trust the Gravatar people
somewhat, instead of the entire internet).

So that's another reason not to make libravatar the default provider.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #51 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org, eliz <at> gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 10:49:24 +0200

>>>>> On Sun, 19 Jul 2020 15:02:35 +0200, Lars Ingebrigtsen <larsi <at> gnus.org> said:

    Lars> "Philip K." <philip <at> warpmail.net> writes:
    >> That was part of the rationale behind #40355, but the best way I see to
    >> fix this would be to implement asynchronous DNS, since a libravatar
    >> lookup has two phases (DNS lookup + image retrieval), compared to
    >> Gravatar's single request.

    Lars> A cache would be a band-aid -- it still will have to do these lookups
    Lars> occasionally, and the user experience in Gnus suffers.

    Lars> As it stands, librgravatar shouldn't be the default gravatar provider.

    Lars> But, yes, Emacs should have asynchronous DNS support, and adding that
    Lars> probably isn't too difficult, I'd have thought?

emacs calls getaddrinfo_a already if itʼs available, when creating
connections, but I donʼt think thereʼs a lisp-level interface to it
(and itʼs not available on macOS, weʼd have to implement it using
Apple's platform-specific API).

If you were thinking of using the emacs threading API, I tried hooking
that up to dns.el, but the results were not great. [1]

Robert

Footnotes:
[1]  Iʼm saying this so that Lars will now go away and implement it :-)

Message #54 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org, eliz <at> gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 11:05:58 +0200

Robert Pluim <rpluim <at> gmail.com> writes:

> emacs calls getaddrinfo_a already if itʼs available, when creating
> connections, but I donʼt think thereʼs a lisp-level interface to it
> (and itʼs not available on macOS, weʼd have to implement it using
> Apple's platform-specific API).

Yes, I was thinking that we could use Emacs' normal process support for
it, but just call getaddrinf_a and then return the results.  I haven't
actually looked at how to do that, but it seems like it should
conceptually makes sense.

For instance, to make it very much like a normal network process, Emacs
could even just output the DNS data as bytes in the process buffer.  (We
have code to parse DNS protocol data in dns.el already.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #57 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 11:23:02 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> For instance, to make it very much like a normal network process, Emacs
> could even just output the DNS data as bytes in the process buffer.  (We
> have code to parse DNS protocol data in dns.el already.)

Actually, that'd make no sense, because we don't have access to the raw
DNS packages on the C side.

So the process could just output the result as ASCII, perhaps.  That is,
a call to

(make-network-process :lookup-address "gnu.org")

would result in

209.51.188.148^@2001:470:142:3::a^@

or something in the buffer.  Or perhaps

ipv4-address^@209.51.188.148^@ipv6-address^@2001:470:142:3::a^@

or however we want to do this.

This all reminds me of a not-very-related issue: Emacs needs support for
DNS-over-HTTPS/TLS, I guess.  Emacs shouldn't have weaker network
security than web browsers.

DoH is trivial to implement, but we do want to be efficient, which may
make it somewhat of a bigger project.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #60 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 11:33:31 +0200

>>>>> On Mon, 20 Jul 2020 11:23:02 +0200, Lars Ingebrigtsen <larsi <at> gnus.org> said:

    Lars> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
    >> For instance, to make it very much like a normal network process, Emacs
    >> could even just output the DNS data as bytes in the process buffer.  (We
    >> have code to parse DNS protocol data in dns.el already.)

    Lars> Actually, that'd make no sense, because we don't have access to the raw
    Lars> DNS packages on the C side.

    Lars> So the process could just output the result as ASCII, perhaps.  That is,
    Lars> a call to

    Lars> (make-network-process :lookup-address "gnu.org")

    Lars> would result in

    Lars> 209.51.188.148^@2001:470:142:3::a^@

    Lars> or something in the buffer.  Or perhaps

    Lars> ipv4-address^@209.51.188.148^@ipv6-address^@2001:470:142:3::a^@

    Lars> or however we want to do this.

You donʼt like `network-lookup-address-info'? Just asynchifying that
should be enough.

Robert

Message #63 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 11:36:27 +0200

>>>>> On Mon, 20 Jul 2020 11:33:31 +0200, Robert Pluim <rpluim <at> gmail.com> said:

>>>>> On Mon, 20 Jul 2020 11:23:02 +0200, Lars Ingebrigtsen <larsi <at> gnus.org> said:
    Lars> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
    >>> For instance, to make it very much like a normal network process, Emacs
    >>> could even just output the DNS data as bytes in the process buffer.  (We
    >>> have code to parse DNS protocol data in dns.el already.)

    Lars> Actually, that'd make no sense, because we don't have access to the raw
    Lars> DNS packages on the C side.

    Lars> So the process could just output the result as ASCII, perhaps.  That is,
    Lars> a call to

    Lars> (make-network-process :lookup-address "gnu.org")

    Lars> would result in

    Lars> 209.51.188.148^@2001:470:142:3::a^@

    Lars> or something in the buffer.  Or perhaps

    Lars> ipv4-address^@209.51.188.148^@ipv6-address^@2001:470:142:3::a^@

    Lars> or however we want to do this.

    Robert> You donʼt like `network-lookup-address-info'? Just asynchifying that
    Robert> should be enough.

Actually, no, that wouldnʼt work since network-lookup-address-info can
only lookup 'A' and 'AAAA' records, and gravatar needs 'SRV'.

Robert

Message #66 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 11:41:47 +0200

Robert Pluim <rpluim <at> gmail.com> writes:

> You donʼt like `network-lookup-address-info'? Just asynchifying that
> should be enough.

I like that function.  :-)  I just see how to fit that into a
process-like async context.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #69 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 11:43:35 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Robert Pluim <rpluim <at> gmail.com> writes:
>
>> You donʼt like `network-lookup-address-info'? Just asynchifying that
>> should be enough.
>
> I like that function.  :-)  I just see how to fit that into a
> process-like async context.

There should be a "don't" after the "just" there, I guess.  :-/

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #72 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: contovob <at> tcd.ie, alex.branham <at> gmail.com, "Philip K." <philip <at> warpmail.net>,
 40676 <at> debbugs.gnu.org
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Mon, 20 Jul 2020 13:02:44 +0200

Robert Pluim <rpluim <at> gmail.com> writes:

> Actually, no, that wouldnʼt work since network-lookup-address-info can
> only lookup 'A' and 'AAAA' records, and gravatar needs 'SRV'.

I should have actually looked at the libravatar code -- it uses dns.el.
Which can be rewritten to be asynchronous, since it just does network
chatter via processes...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #75 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Philip K." <philip <at> warpmail.net>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, 40676 <at> debbugs.gnu.org, eliz <at> gnu.org,
 alex.branham <at> gmail.com
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Wed, 29 Jul 2020 08:48:58 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Indeed -- it makes libravatar a much worse interface than the central
> Gravatar service (where you just have to trust the Gravatar people
> somewhat, instead of the entire internet).
>
> So that's another reason not to make libravatar the default provider.

I've now changed the default to use gravatar, since that doesn't have
the same security implications.

Emacs should still grow an async DNS implementation (reachable from Lisp
land), though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #78 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Philip K." <philip <at> warpmail.net>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, eliz <at> gnu.org, 40676 <at> debbugs.gnu.org,
 alex.branham <at> gmail.com
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Thu, 30 Jul 2020 03:43:13 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Emacs should still grow an async DNS implementation (reachable from Lisp
> land), though.

I've now added a function dns-query-asynchronous to dns.el to have an
asynchronous implementation that works across all Emacs builds.

However, I'm not pushing it yet until somebody makes a decision as to
whether the Emacs git repo is to be recovered from backup, or we just
carry on as before after the mis-merge into emacs-27 a few hours ago...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #81 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, philip <at> warpmail.net,
 40676 <at> debbugs.gnu.org, alex.branham <at> gmail.com
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Wed, 29 Jul 2020 23:11:20 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Indeed -- it makes libravatar a much worse interface than the central
  > > Gravatar service (where you just have to trust the Gravatar people
  > > somewhat, instead of the entire internet).
  > >
  > > So that's another reason not to make libravatar the default provider.

I don't know about libravatar, but ISTR that gravatar was found
to do some sort of surveillance of users.  Is that correct?
Was libravatar intended as a replacement to respect users' privacy
more than gravatar?

But it seems there is a problem with using libravatar, right?

If that is the case, we should not think of "use gravatar"
as a _solution_.  It might fix an urgent problem, but we
would not want it as a permanent change.  A real solution 
is to make libravatar work well.

If what is needed is async DNS lookup, how about running
a shell command asynchronously to do it?


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Message #84 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Philip K." <philip <at> warpmail.net>
Cc: contovob <at> tcd.ie, rpluim <at> gmail.com, 40676 <at> debbugs.gnu.org, eliz <at> gnu.org,
 alex.branham <at> gmail.com
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Thu, 30 Jul 2020 05:30:18 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> I've now added a function dns-query-asynchronous to dns.el to have an
> asynchronous implementation that works across all Emacs builds.

And now I've rewritten gravatar.el to be asynchronous, too, so I'm
closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #91 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: philipk <at> posteo.net (Philip K.)
To: rms <at> gnu.org
Cc: contovob <at> tcd.ie, larsi <at> gnus.org, 40676 <at> debbugs.gnu.org, rpluim <at> gmail.com,
 alex.branham <at> gmail.com
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Thu, 30 Jul 2020 10:32:51 +0200

Richard Stallman <rms <at> gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Indeed -- it makes libravatar a much worse interface than the central
>   > > Gravatar service (where you just have to trust the Gravatar people
>   > > somewhat, instead of the entire internet).
>   > >
>   > > So that's another reason not to make libravatar the default provider.
>
> I don't know about libravatar, but ISTR that gravatar was found
> to do some sort of surveillance of users.

I'm not sure if it was proven, but just like Google Analytics or the
Facebook Like-button, they offer a service with a hidden fee -- that
they can or cannot exploit. 

In the case of Emacs' Gravatar Integration, they get "notified" whenever
someone opens a message from person X (at least once), meaning they
could more easily reconstruct a database of who communicates with whom.

> Is that correct?  Was libravatar intended as a replacement to respect
> users' privacy more than gravatar?

From libravatar's web page:

        FREEDOM AND FEDERATION

        How is Libravatar different from Gravatar though? The main
        difference is that while Libravatar.org is an online avatar
        hosting service just like Gravatar, the software that powers the
        former is also available for download under a free software
        license.

        Why would you want to run your own instance? Our belief is that
        centralised approaches don't put users in control. If you own
        your own domain name, you control how mail is delivered to your
        domain. You should also be able to control how avatars are
        served to other websites.

So kind of, thought it seems to be more about controlling the software
you use than privacy per se.

> But it seems there is a problem with using libravatar, right?

Yes, the same attack as above with Gravatar (getting notified when you
open a message), just that a third party can set up their own libravatar
server and get notified when I open their email, instead of one
centralised service. So kind of like tracking bits in HTML messages.

> If that is the case, we should not think of "use gravatar"
> as a _solution_.  It might fix an urgent problem, but we
> would not want it as a permanent change.  A real solution 
> is to make libravatar work well.

That would require an architectural change to libravatar itself. It's
current mechanism to retrieve an image for user <at> domain.tld is:

1. Check if domain.tld has a libravatar service (via DNS)
2. If yes, use that server, otherwise default to libravatar's server
3. Send a request to the server using the MD5/SHA256 hash of
   user <at> domain.tld

Steps 1./2. are the trick to federation, but the crux of the issue as
well. 

> If what is needed is async DNS lookup, how about running
> a shell command asynchronously to do it?

I guess that would also be possible, but wouldn't that require a DNS
tool to be installed on the system?

-- 
	Philip K.

Message #94 received at 40676 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: philipk <at> posteo.net (Philip K.)
Cc: contovob <at> tcd.ie, larsi <at> gnus.org, 40676 <at> debbugs.gnu.org,
 alex.branham <at> gmail.com, rpluim <at> gmail.com
Subject: Re: bug#40676: 28.0.50; gnus locks when reading email
Date: Thu, 30 Jul 2020 23:24:17 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > But it seems there is a problem with using libravatar, right?

  > Yes, the same attack as above with Gravatar (getting notified when you
  > open a message), just that a third party can set up their own libravatar
  > server and get notified when I open their email, instead of one
  > centralised service. So kind of like tracking bits in HTML messages.

I guess this is not a big practical difference.
Simply using libravatar instead of gravatar doesn't seem to protect privacy.
You've explained that perhaps some redesign could make it protect privacy,
but it seems to me that the difference _at present_ is not crucial.

Thanks.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.