GNU bug report logs - #78366
30.1; auth-source-xoauth2-plugin conflicts with multiple Google accounts

Previous Next

To reply to this bug, email your comments to 78366 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Anush V <j <at> gnu.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.1; auth-source-xoauth2-plugin conflicts with multiple Google
 accounts
Date: Sat, 10 May 2025 22:45:43 -0400

Hello Emacs maintainers,

I have two google mail accounts, the first one uses app passwords to
authenticate, and the other uses oauth (because it doesn't support app
passwords)

When I enable auth-source-xoauth2-plugin-mode, I’m able to send email
from second account without any issues.  But i’m unable to send emails
from the first account.  When I disable auth-source-xoauth2-plugin i’m
able to send email from the first account.

I think that enabling xoauth2 shouldn't interfere with other
authentication methods, so this could be a bug.

Thank you for your time

* * *

In GNU Emacs 30.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43,
cairo version 1.18.2)
System Description: Guix System

Configured using:
 'configure
 CONFIG_SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash
 SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash --prefix=/gnu/store/lq0nwm8qkj9cmyjm85z3dcqrjnglhcym-emacs-next-pgtk-30.1-rc1-2.7144e84 --enable-fast-install --with-pgtk --with-cairo --with-modules --with-native-compilation=aot --disable-build-details'

--
Regards,
Anush

Message #8 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Anush V <j <at> gnu.org>, 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Sat, 10 May 2025 20:36:25 -0700

[Message part 1 (text/plain, inline)]

Hi Anush,

Anush V <j <at> gnu.org> writes:

> Hello Emacs maintainers,
>
> I have two google mail accounts, the first one uses app passwords to
> authenticate, and the other uses oauth (because it doesn't support app
> passwords)
>
> When I enable auth-source-xoauth2-plugin-mode, I’m able to send email
> from second account without any issues.  But i’m unable to send emails
> from the first account.  When I disable auth-source-xoauth2-plugin i’m
> able to send email from the first account.
>
> I think that enabling xoauth2 shouldn't interfere with other
> authentication methods, so this could be a bug.
>
> Thank you for your time
>
> * * *
>
> In GNU Emacs 30.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43,
> cairo version 1.18.2)
> System Description: Guix System
>
> Configured using:
>  'configure
>  CONFIG_SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash
>  SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash --prefix=/gnu/store/lq0nwm8qkj9cmyjm85z3dcqrjnglhcym-emacs-next-pgtk-30.1-rc1-2.7144e84 --enable-fast-install --with-pgtk --with-cairo --with-modules --with-native-compilation=aot --disable-build-details'
>
> --
> Regards,
> Anush
>
>

Thanks for your report!  Ideally auth-source-xoauth2-plugin should only
be in effect when your auth-source entry has "auth" set to "xoauth2".
Can you check whether you happen to set that for your auth-source entry
for the account using app password?  Sharing your auth-source entries
with your personal information removed would be helpful.

It would also help to share some of the error logs following instruction
in the "Debugging" section in README.org[1].  Please be careful not to
share any personal information from the logs.

[1] https://gitlab.com/manphiz/auth-source-xoauth2-plugin/-/blob/main/README.org?ref_type=heads

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Anush V <j <at> gnu.org>
To: Xiyue Deng <manphiz <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Mon, 12 May 2025 19:49:14 -0400

> From: Xiyue Deng <manphiz <at> gmail.com>
> Date: Sat, 10 May 2025 20:36:25 -0700
>
> Hi Anush,
>
> Anush V <j <at> gnu.org> writes:
>
>> Hello Emacs maintainers,
>>
>> I have two google mail accounts, the first one uses app passwords to
>> authenticate, and the other uses oauth (because it doesn't support app
>> passwords)
>>
>> When I enable auth-source-xoauth2-plugin-mode, I’m able to send email
>> from second account without any issues.  But i’m unable to send emails
>> from the first account.  When I disable auth-source-xoauth2-plugin i’m
>> able to send email from the first account.
>>
>> I think that enabling xoauth2 shouldn't interfere with other
>> authentication methods, so this could be a bug.
>>
>> Thank you for your time
>>
>> * * *
>>
>> In GNU Emacs 30.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43,
>> cairo version 1.18.2)
>> System Description: Guix System
>>
>> Configured using:
>>  'configure
>>  CONFIG_SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash
>>  SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash --prefix=/gnu/store/lq0nwm8qkj9cmyjm85z3dcqrjnglhcym-emacs-next-pgtk-30.1-rc1-2.7144e84 --enable-fast-install --with-pgtk --with-cairo --with-modules --with-native-compilation=aot --disable-build-details'
>>
>> --
>> Regards,
>> Anush
>>
>>
>
> Thanks for your report!  Ideally auth-source-xoauth2-plugin should only
> be in effect when your auth-source entry has "auth" set to "xoauth2".
> Can you check whether you happen to set that for your auth-source entry
> for the account using app password?  Sharing your auth-source entries
> with your personal information removed would be helpful.
>
> It would also help to share some of the error logs following instruction
> in the "Debugging" section in README.org[1].  Please be careful not to
> share any personal information from the logs.
>
> [1]
> https://gitlab.com/manphiz/auth-source-xoauth2-plugin/-/blob/main/README.org?ref_type=heads


I did some debugging.  According to comment “;; A string result is an
error.” in the function smtpmail-try-auth-methods, the function call
(smtpmail-try-auth-method process 'xoauth2 "user" "password") should
return a string when authentication fails.  However, it currently
returns a list instead, which prevents other authentication methods
from being tried.

As requested,

auth source entries.

     # for first account (app password authentication)
     machine imap.gmail.com login <first_email> port 993 password <app password>
     machine smtp.gmail.com login <first_email> port 587 password <app password>

     # second account (xoauth2 authentication)
     machine imap.gmail.com user <second_email> port 993 auth xoauth2 auth-url https://accounts.google.com/o/oauth2/auth token-url https://accounts.google.com/o/oauth2/token client-id <client_id> client-secret <client_secret> redirect-uri http://localhost scope https://mail.google.com
     machine smtp.gmail.com user <second_email> port 587 auth xoauth2 auth-url https://accounts.google.com/o/oauth2/auth token-url https://accounts.google.com/o/oauth2/token client-id <client_id> client-secret <client_secret> redirect-uri http://localhost scope https://mail.google.com


Logs when I send email from account with app password
(auth-source-xoauth2-plugin-mode enabled):


     Gnus is unplugged; really send queue? (y or n) y
     Sending message 1 of 1...
     Sending via mail...
     auth-source-search: found 4 backends matching (:max 1 :host "smtp.gmail.com" :port "587")
     Advising auth-source-search
     auth-source-netrc-parse: using CACHED file data for <path_to_authinfo.gpg>
     auth-source-search-backend: got 1 (max 1) in netrc:<path_to_authinfo.gpg> matching (:max 1 :host "smtp.gmail.com" :port "587")
     Matched auth data: (:host "smtp.gmail.com" :user "<second_email>" :port "587" :auth "xoauth2"
            :auth-url "https://accounts.google.com/o/oauth2/auth"
            :token-url "https://accounts.google.com/o/oauth2/token"
            :client-id
            "<client_id>"
            :client-secret "<client_secret>"
            :redirect-uri "http://localhost" :scope
            "https://mail.google.com")

     :auth set to ‘xoauth2’.  Will get access token.
     Using oauth2 to auth and store token...
     Decrypting <oauth2.plstore>...done
     oauth2 token: #s(oauth2-token
        [#<buffer  plstore
                   <oauth2.plstore>>
                   (("<string1>"
                     :secret-access-token t :secret-refresh-token t
                     :secret-access-response t))
                   nil
                   (("<string1>"
                     :access-token
                     "<string4>"
                     :refresh-token
                     "<string3>"
                     :access-response
                     ((access_token
                       . "<string5>")
                      (expires_in . 3599)
                      (refresh_token
                       . "<string3>")
                      (scope . "https://mail.google.com/")
                      (token_type . "Bearer"))))
                   (("<string1>"
                     :access-token
                     "<string4>"
                     :refresh-token
                     "<string3>"
                     :access-response
                     ((access_token
                       . "<string5>")
                      (expires_in . 3599)
                      (refresh_token
                       . "<string3>")
                      (scope . "https://mail.google.com/")
                      (token_type . "Bearer"))))]
        "<string1>"
        "<client_id>"
        "<client_secret>"
        "<string4>"
        "<string3>"
        "https://accounts.google.com/o/oauth2/token"
        ((access_token
          . "<string5>")
         (expires_in . 3599)
         (refresh_token
          . "<string3>")
         (scope . "https://mail.google.com/") (token_type . "Bearer")))

     Refreshing token...
     Contacting host: accounts.google.com:443
     Saving file <oauth2.plstore>...
     Wrote <oauth2.plstore>
     Refresh successful.
     oauth2 token after refresh: #s(oauth2-token
        [#<buffer  plstore
                   <oauth2.plstore>>
                   (("<string1>"
                     :secret-access-token t :secret-refresh-token t
                     :secret-access-response t))
                   nil
                   (("<string1>"
                     :access-token
                     "<string2>"
                     :refresh-token
                     "<string3>"
                     :access-response
                     ((access_token
                       . "<string5>")
                      (expires_in . 3599)
                      (refresh_token
                       . "<string3>")
                      (scope . "https://mail.google.com/")
                      (token_type . "Bearer"))))
                   (("<string1>"
                     :access-token
                     "<string2>"
                     :refresh-token
                     "<string3>"
                     :access-response
                     ((access_token
                       . "<string5>")
                      (expires_in . 3599)
                      (refresh_token
                       . "<string3>")
                      (scope . "https://mail.google.com/")
                      (token_type . "Bearer"))))]
        "<string1>"
        "<client_id>"
        "<client_secret>"
        "<string2>"
        "<string3>"
        "https://accounts.google.com/o/oauth2/token"
        ((access_token
          . "<string5>")
         (expires_in . 3599)
         (refresh_token
          . "<string3>")
         (scope . "https://mail.google.com/") (token_type . "Bearer")))

     Updating :secret with access-token: <string2>
     Updating auth-source-search results.
     auth-source-search: found 1 results (max 1) matching (:max 1 :host "smtp.gmail.com" :port "587")
     auth-source-search: found 4 backends matching (:host "smtp.gmail.com" :port "587" :user "<first_email>" :max 1 :require nil :create nil)
     Advising auth-source-search
     auth-source-netrc-parse: using CACHED file data for <path_to_authinfo.gpg>
     auth-source-search-backend: got 1 (max 1) in netrc:<path_to_authinfo.gpg> matching (:host "smtp.gmail.com" :port "587" :user "<first_email>" :max 1 :require nil :create nil)
     Matched auth data: (:host "smtp.gmail.com" :user "<first_email>" :port "587"
            :secret
            #[0 "<secret1>"
                [(nil)
                 "<string6>"
                 auth-source--deobfuscate]
                3])

     Updating auth-source-search results.
     auth-source-search: found 1 results (max 1) matching (:host "smtp.gmail.com" :port "587" :user "<first_email>" :max 1 :require nil :create nil)
     smtpmail-send-it: Sending failed: 535-5.7.8 Username and Password not accepted. For more information, go to
     535 5.7.8  https://support.google.com/mail/?p=BadCredentials <string6> - gsmtp


Logs when I send email from account with app password
(auth-source-xoauth2-plugin-mode disabled):


     Auth-Source-Xoauth2-Plugin mode disabled
     Gnus is unplugged; really send queue? (y or n) y
     Sending message 1 of 1...
     Sending via mail...
     auth-source-search: found 1 CACHED results matching (:max 1 :host "smtp.gmail.com" :port "587")
     auth-source-search: found 1 CACHED results matching (:host "smtp.gmail.com" :port "587" :user "<first_email>" :max 1 :require nil :create nil)
     Sending email
     Sending email done
     Sending...done
     No more newsgroups


--
Regards,
Anush

Message #14 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Anush V <j <at> gnu.org>
Cc: 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Tue, 13 May 2025 09:55:24 -0700

[Message part 1 (text/plain, inline)]

Hi Anush,

Anush V <j <at> gnu.org> writes:

>> From: Xiyue Deng <manphiz <at> gmail.com>
>> Date: Sat, 10 May 2025 20:36:25 -0700
>>
>> Hi Anush,
>>
>> Anush V <j <at> gnu.org> writes:
>>
>>> Hello Emacs maintainers,
>>>
>>> I have two google mail accounts, the first one uses app passwords to
>>> authenticate, and the other uses oauth (because it doesn't support app
>>> passwords)
>>>
>>> When I enable auth-source-xoauth2-plugin-mode, I’m able to send email
>>> from second account without any issues.  But i’m unable to send emails
>>> from the first account.  When I disable auth-source-xoauth2-plugin i’m
>>> able to send email from the first account.
>>>
>>> I think that enabling xoauth2 shouldn't interfere with other
>>> authentication methods, so this could be a bug.
>>>
>>> Thank you for your time
>>>
>>> * * *
>>>
>>> In GNU Emacs 30.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43,
>>> cairo version 1.18.2)
>>> System Description: Guix System
>>>
>>> Configured using:
>>>  'configure
>>>  CONFIG_SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash
>>>  SHELL=/gnu/store/m0xdsa8cfq6mq1kxgxmpmpg71la4f0b9-bash-minimal-5.1.16/bin/bash --prefix=/gnu/store/lq0nwm8qkj9cmyjm85z3dcqrjnglhcym-emacs-next-pgtk-30.1-rc1-2.7144e84 --enable-fast-install --with-pgtk --with-cairo --with-modules --with-native-compilation=aot --disable-build-details'
>>>
>>> --
>>> Regards,
>>> Anush
>>>
>>>
>>
>> Thanks for your report!  Ideally auth-source-xoauth2-plugin should only
>> be in effect when your auth-source entry has "auth" set to "xoauth2".
>> Can you check whether you happen to set that for your auth-source entry
>> for the account using app password?  Sharing your auth-source entries
>> with your personal information removed would be helpful.
>>
>> It would also help to share some of the error logs following instruction
>> in the "Debugging" section in README.org[1].  Please be careful not to
>> share any personal information from the logs.
>>
>> [1]
>> https://gitlab.com/manphiz/auth-source-xoauth2-plugin/-/blob/main/README.org?ref_type=heads
>
>
> I did some debugging.  According to comment “;; A string result is an
> error.” in the function smtpmail-try-auth-methods, the function call
> (smtpmail-try-auth-method process 'xoauth2 "user" "password") should
> return a string when authentication fails.  However, it currently
> returns a list instead, which prevents other authentication methods
> from being tried.
>
> [...]

Thanks for the investigation! If the try-method don't fallback to the
other supported auth methods then it will break.  I tried to read the
code of `smtpmail-try-auth-method' for xoauth2[1], and besides lacking
encoding user and password into utf-8, I don't see anything obviously
different from plain[2] and not sure why it would return a list.

Would be great if any Emacs developer can shed some light here.

[1] https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/mail/smtpmail.el#n643
[2] https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/mail/smtpmail.el#n628

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Xiyue Deng <manphiz <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Wed, 14 May 2025 14:50:21 +0200

>>>>> On Tue, 13 May 2025 09:55:24 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
    >> 
    >> 
    >> I did some debugging.  According to comment “;; A string result is an
    >> error.” in the function smtpmail-try-auth-methods, the function call
    >> (smtpmail-try-auth-method process 'xoauth2 "user" "password") should
    >> return a string when authentication fails.  However, it currently
    >> returns a list instead, which prevents other authentication methods
    >> from being tried.
    >> 
    >> [...]

    Xiyue> Thanks for the investigation! If the try-method don't fallback to the
    Xiyue> other supported auth methods then it will break.  I tried to read the
    Xiyue> code of `smtpmail-try-auth-method' for xoauth2[1], and besides lacking
    Xiyue> encoding user and password into utf-8, I don't see anything obviously
    Xiyue> different from plain[2] and not sure why it would return a list.

utf-8 encoding is not necessary for xoauth2, see commit
745847ba8eca27e981a50ad91b628bbce35bb0f3

    Xiyue> Would be great if any Emacs developer can shed some light here.

`smtpmail-try-auth-method' will call `smtpmail-command-or-throw
process', which will return a list upon success, and string upon
failure. That decision is made by `smtpmail-ok-p', so thatʼs the
function you should check. Setting `smtpmail-debug-info' to t will get
you a trace buffer of the SMTP commands, that should help.

Robert
-- 

Message #20 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Thu, 15 May 2025 00:17:02 -0700

[Message part 1 (text/plain, inline)]

Hi Robert,

Robert Pluim <rpluim <at> gmail.com> writes:

>>>>>> On Tue, 13 May 2025 09:55:24 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
>     >> 
>     >> 
>     >> I did some debugging.  According to comment “;; A string result is an
>     >> error.” in the function smtpmail-try-auth-methods, the function call
>     >> (smtpmail-try-auth-method process 'xoauth2 "user" "password") should
>     >> return a string when authentication fails.  However, it currently
>     >> returns a list instead, which prevents other authentication methods
>     >> from being tried.
>     >> 
>     >> [...]
>
>     Xiyue> Thanks for the investigation! If the try-method don't fallback to the
>     Xiyue> other supported auth methods then it will break.  I tried to read the
>     Xiyue> code of `smtpmail-try-auth-method' for xoauth2[1], and besides lacking
>     Xiyue> encoding user and password into utf-8, I don't see anything obviously
>     Xiyue> different from plain[2] and not sure why it would return a list.
>
> utf-8 encoding is not necessary for xoauth2, see commit
> 745847ba8eca27e981a50ad91b628bbce35bb0f3
>
>     Xiyue> Would be great if any Emacs developer can shed some light here.
>
> `smtpmail-try-auth-method' will call `smtpmail-command-or-throw
> process', which will return a list upon success, and string upon
> failure. That decision is made by `smtpmail-ok-p', so thatʼs the
> function you should check. Setting `smtpmail-debug-info' to t will get
> you a trace buffer of the SMTP commands, that should help.
>
> Robert
> -- 

Thanks for the insights! I managed to reproduce the issue, and during
debugging I got the list from the reply as Anush mentioned.  Turned out
that the return code was 334 server challenge[1], so it was waiting for
the correct user and password.  Sometimes this was directly considered
authentication unsuccessful for Gmail[2].  `smtpmail-ok-p' considers a
return code less than 400 as successful, and only has challenge handling
implemented in cram-md5.  As we should be providing the correct
credentials directly in xoauth2, 334 is effectively a failure.

Maybe in `smtpmail-try-auth-method' for xoauth2, if we see return code
334, we should change the return value to "535 5.7.8 Authentication
credentials invalid".  Would like to see whether the Emacs maintainers
this is a good idea.

Meanwhile, a workaround for auth-source-xoauth2-plugin is to put xoauth2
as the last entry in `smtpmail-auth-supported'.  Anush, can you try
enable auth-source-xoauth2-plugin and eval `(setq smtp-auth-supported
'(cram-md5 plain login xoauth2))' and see if this helps?

[1] https://en.wikipedia.org/wiki/List_of_SMTP_server_return_codes#%E2%80%94_3yz_Positive_intermediate
[2] https://support.google.com/mail/thread/204884242/334-authentication-unsuccessful?hl=en 
[3] https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/mail/smtpmail.el#n662

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Xiyue Deng <manphiz <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Thu, 15 May 2025 11:42:42 +0200

>>>>> On Thu, 15 May 2025 00:17:02 -0700, Xiyue Deng <manphiz <at> gmail.com> said:

    Xiyue> Thanks for the insights! I managed to reproduce the issue, and during
    Xiyue> debugging I got the list from the reply as Anush mentioned.  Turned out
    Xiyue> that the return code was 334 server challenge[1], so it was waiting for
    Xiyue> the correct user and password.  Sometimes this was directly considered
    Xiyue> authentication unsuccessful for Gmail[2].  `smtpmail-ok-p' considers a
    Xiyue> return code less than 400 as successful, and only has challenge handling
    Xiyue> implemented in cram-md5.  As we should be providing the correct
    Xiyue> credentials directly in xoauth2, 334 is effectively a failure.

    Xiyue> Maybe in `smtpmail-try-auth-method' for xoauth2, if we see return code
    Xiyue> 334, we should change the return value to "535 5.7.8 Authentication
    Xiyue> credentials invalid".  Would like to see whether the Emacs maintainers
    Xiyue> this is a good idea.

Itʼs either that, or change `smtpmail-ok-p' to accept a second
optional parameter for which codes to accept for success, which seems
like overkill here.

Iʼm confused as to why Gmail is returning 334 here, if as you say the
credentials should be correct. Would answering the challenge help?

    Xiyue> Meanwhile, a workaround for auth-source-xoauth2-plugin is to put xoauth2
    Xiyue> as the last entry in `smtpmail-auth-supported'.  Anush, can you try
    Xiyue> enable auth-source-xoauth2-plugin and eval `(setq smtp-auth-supported
    Xiyue> '(cram-md5 plain login xoauth2))' and see if this helps?

If this fixes the issue, you could put this workaround in your xoauth2
package.

Robert
-- 

Message #26 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Anush V <j <at> gnu.org>
To: Xiyue Deng <manphiz <at> gmail.com>
Cc: Robert Pluim <rpluim <at> gmail.com>, 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Thu, 15 May 2025 19:01:23 -0400

> From: Xiyue Deng <manphiz <at> gmail.com>
> Date: Thu, 15 May 2025 00:17:02 -0700
>
> Anush, can you try enable auth-source-xoauth2-plugin and eval `(setq
> smtp-auth-supported '(cram-md5 plain login xoauth2))' and see if
> this helps?


`(setq smtp-auth-supported '(cram-md5 plain login xoauth2))' didn’t
work.  You probably meant `(setq smtpmail-auth-supported '(cram-md5
plain login xoauth2))' and it worked.  Able to send email from both
the accounts.

--
Regards,
Anush

Message #29 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Fri, 16 May 2025 01:11:44 -0700

[Message part 1 (text/plain, inline)]

Hi Robert,

Robert Pluim <rpluim <at> gmail.com> writes:

>>>>>> On Thu, 15 May 2025 00:17:02 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
>
>     Xiyue> Thanks for the insights! I managed to reproduce the issue, and during
>     Xiyue> debugging I got the list from the reply as Anush mentioned.  Turned out
>     Xiyue> that the return code was 334 server challenge[1], so it was waiting for
>     Xiyue> the correct user and password.  Sometimes this was directly considered
>     Xiyue> authentication unsuccessful for Gmail[2].  `smtpmail-ok-p' considers a
>     Xiyue> return code less than 400 as successful, and only has challenge handling
>     Xiyue> implemented in cram-md5.  As we should be providing the correct
>     Xiyue> credentials directly in xoauth2, 334 is effectively a failure.
>
>     Xiyue> Maybe in `smtpmail-try-auth-method' for xoauth2, if we see return code
>     Xiyue> 334, we should change the return value to "535 5.7.8 Authentication
>     Xiyue> credentials invalid".  Would like to see whether the Emacs maintainers
>     Xiyue> this is a good idea.
>
> Itʼs either that, or change `smtpmail-ok-p' to accept a second
> optional parameter for which codes to accept for success, which seems
> like overkill here.
>

In this case it's more like which codes not to accept (334), but I agree
`smtpmail-ok-p' is probably the wrong place to handle that.

I'll work on a patch for `smtpmail-try-auth-method' later.

> Iʼm confused as to why Gmail is returning 334 here, if as you say the
> credentials should be correct. Would answering the challenge help?
>

Oh I wasn't clear.  Actually in this case the credentials provided
through xoauth2 is incorrect - the password set should be the access
token but instead it was the app password.  Gmail handles this by
returning 334 and waiting for the correct access token instead of
directly failing with 535.  I'm not sure why Gmail does this.  Probably
it's more tolerant if a user is using xoauth2.

>     Xiyue> Meanwhile, a workaround for auth-source-xoauth2-plugin is to put xoauth2
>     Xiyue> as the last entry in `smtpmail-auth-supported'.  Anush, can you try
>     Xiyue> enable auth-source-xoauth2-plugin and eval `(setq smtp-auth-supported
>     Xiyue> '(cram-md5 plain login xoauth2))' and see if this helps?
>
> If this fixes the issue, you could put this workaround in your xoauth2
> package.
>

This works in my local test, and I think Anush also confirmed it worked.
I'll update my plugin soon.

> Robert
> -- 

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Anush V <j <at> gnu.org>
Cc: Robert Pluim <rpluim <at> gmail.com>, 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Fri, 16 May 2025 01:11:59 -0700

[Message part 1 (text/plain, inline)]

Hi Anush,

Anush V <j <at> gnu.org> writes:

>> From: Xiyue Deng <manphiz <at> gmail.com>
>> Date: Thu, 15 May 2025 00:17:02 -0700
>>
>> Anush, can you try enable auth-source-xoauth2-plugin and eval `(setq
>> smtp-auth-supported '(cram-md5 plain login xoauth2))' and see if
>> this helps?
>
>
> `(setq smtp-auth-supported '(cram-md5 plain login xoauth2))' didn’t
> work.

It was a typo, sorry.

> You probably meant `(setq smtpmail-auth-supported '(cram-md5
> plain login xoauth2))' and it worked.  Able to send email from both
> the accounts.
>

And thanks for confirming.  I'll update my plugin soon.

> --
> Regards,
> Anush

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #35 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Fri, 16 May 2025 02:58:39 -0700

[Message part 1 (text/plain, inline)]

Xiyue Deng <manphiz <at> gmail.com> writes:

> Hi Robert,
>
> Robert Pluim <rpluim <at> gmail.com> writes:
>
>>>>>>> On Thu, 15 May 2025 00:17:02 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
>>
>>     Xiyue> Thanks for the insights! I managed to reproduce the issue, and during
>>     Xiyue> debugging I got the list from the reply as Anush mentioned.  Turned out
>>     Xiyue> that the return code was 334 server challenge[1], so it was waiting for
>>     Xiyue> the correct user and password.  Sometimes this was directly considered
>>     Xiyue> authentication unsuccessful for Gmail[2].  `smtpmail-ok-p' considers a
>>     Xiyue> return code less than 400 as successful, and only has challenge handling
>>     Xiyue> implemented in cram-md5.  As we should be providing the correct
>>     Xiyue> credentials directly in xoauth2, 334 is effectively a failure.
>>
>>     Xiyue> Maybe in `smtpmail-try-auth-method' for xoauth2, if we see return code
>>     Xiyue> 334, we should change the return value to "535 5.7.8 Authentication
>>     Xiyue> credentials invalid".  Would like to see whether the Emacs maintainers
>>     Xiyue> this is a good idea.
>>
>> Itʼs either that, or change `smtpmail-ok-p' to accept a second
>> optional parameter for which codes to accept for success, which seems
>> like overkill here.
>>
>
> In this case it's more like which codes not to accept (334), but I agree
> `smtpmail-ok-p' is probably the wrong place to handle that.
>
> I'll work on a patch for `smtpmail-try-auth-method' later.
>

A draft patch is attached, please take a look.

-- 
Regards,
Xiyue Deng

[0001-Make-xoauth2-auth-fail-when-a-smtp-server-replies-33.patch (text/x-diff, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Mon, 26 May 2025 01:01:50 -0700

[Message part 1 (text/plain, inline)]

Xiyue Deng <manphiz <at> gmail.com> writes:

> Xiyue Deng <manphiz <at> gmail.com> writes:
>
>> Hi Robert,
>>
>> Robert Pluim <rpluim <at> gmail.com> writes:
>>
>>>>>>>> On Thu, 15 May 2025 00:17:02 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
>>>
>>>     Xiyue> Thanks for the insights! I managed to reproduce the issue, and during
>>>     Xiyue> debugging I got the list from the reply as Anush mentioned.  Turned out
>>>     Xiyue> that the return code was 334 server challenge[1], so it was waiting for
>>>     Xiyue> the correct user and password.  Sometimes this was directly considered
>>>     Xiyue> authentication unsuccessful for Gmail[2].  `smtpmail-ok-p' considers a
>>>     Xiyue> return code less than 400 as successful, and only has challenge handling
>>>     Xiyue> implemented in cram-md5.  As we should be providing the correct
>>>     Xiyue> credentials directly in xoauth2, 334 is effectively a failure.
>>>
>>>     Xiyue> Maybe in `smtpmail-try-auth-method' for xoauth2, if we see return code
>>>     Xiyue> 334, we should change the return value to "535 5.7.8 Authentication
>>>     Xiyue> credentials invalid".  Would like to see whether the Emacs maintainers
>>>     Xiyue> this is a good idea.
>>>
>>> Itʼs either that, or change `smtpmail-ok-p' to accept a second
>>> optional parameter for which codes to accept for success, which seems
>>> like overkill here.
>>>
>>
>> In this case it's more like which codes not to accept (334), but I agree
>> `smtpmail-ok-p' is probably the wrong place to handle that.
>>
>> I'll work on a patch for `smtpmail-try-auth-method' later.
>>
>
> A draft patch is attached, please take a look.
>

Friendly ping.  Does the patch look acceptable for smtpmail.el?

P.S. I have auth-source-xoauth2-plugin 0.2.1 released with the
workaround.  Please check it out.

> -- 
> Regards,
> Xiyue Deng
> From 8de2535105c1fac14ab6c5fef792435b21a0861f Mon Sep 17 00:00:00 2001
> From: Xiyue Deng <manphiz <at> gmail.com>
> Date: Fri, 16 May 2025 02:48:52 -0700
> Subject: [PATCH] Make xoauth2 auth fail when a smtp server replies 334
>  (bug#78366)
>
> * lisp/mail/smtpmail.el (smtpmail-try-auth-method): Throws error 535
> when receiving a "334 server challenge" reply.
> ---
>  lisp/mail/smtpmail.el | 20 +++++++++++++++-----
>  1 file changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/lisp/mail/smtpmail.el b/lisp/mail/smtpmail.el
> index 9337ee9401a..eda91793d17 100644
> --- a/lisp/mail/smtpmail.el
> +++ b/lisp/mail/smtpmail.el
> @@ -642,11 +642,21 @@ smtpmail-try-auth-method
>  
>  (cl-defmethod smtpmail-try-auth-method
>    (process (_mech (eql 'xoauth2)) user password)
> -  (smtpmail-command-or-throw
> -   process
> -   (concat "AUTH XOAUTH2 "
> -           (base64-encode-string
> -            (concat "user=" user "\1auth=Bearer " password "\1\1") t))))
> +  (let ((ret (smtpmail-command-or-throw
> +              process
> +              (concat "AUTH XOAUTH2 "
> +                      (base64-encode-string
> +                       (concat "user=" user "\1auth=Bearer " password "\1\1")
> +                       t)))))
> +    (if (eq (car ret) 334)
> +        ;; When a server returns 334 server challenge, it usually means
> +        ;; the credentials it received was wrong (e.g. was an actual
> +        ;; password instead of an access token).  In such case, we
> +        ;; should return a string with 535 to indicate a failure so that
> +        ;; smtpmail will try other authentication mechanisms.  See also
> +        ;; https://debbugs.gnu.org/78366.
> +        (throw 'done "535 5.7.8 Authentication credentials invalid")
> +      ret)))
>  
>  (defun smtpmail-response-code (string)
>    (when string
> -- 
> 2.47.2
>

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Xiyue Deng <manphiz <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Mon, 26 May 2025 12:37:52 +0200

>>>>> On Mon, 26 May 2025 01:01:50 -0700, Xiyue Deng <manphiz <at> gmail.com> said:

    Xiyue> Friendly ping.  Does the patch look acceptable for smtpmail.el?

    Xiyue> P.S. I have auth-source-xoauth2-plugin 0.2.1 released with the
    Xiyue> workaround.  Please check it out.

Looks good to me.

Robert
-- 

Message #44 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, j <at> gnu.org, manphiz <at> gmail.com
Subject: Re: bug#78366: 30.1;
 auth-source-xoauth2-plugin conflicts with multiple Google accounts
Date: Mon, 26 May 2025 15:14:53 +0300

> Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
> From: Robert Pluim <rpluim <at> gmail.com>
> Date: Mon, 26 May 2025 12:37:52 +0200
> 
> >>>>> On Mon, 26 May 2025 01:01:50 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
> 
>     Xiyue> Friendly ping.  Does the patch look acceptable for smtpmail.el?
> 
>     Xiyue> P.S. I have auth-source-xoauth2-plugin 0.2.1 released with the
>     Xiyue> workaround.  Please check it out.
> 
> Looks good to me.

Thanks, feel free to install on the master branch.

Message #51 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 78366 <at> debbugs.gnu.org, j <at> gnu.org, manphiz <at> gmail.com
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Mon, 26 May 2025 15:16:46 +0200

tags 78366 fixed
close 78366 31.1
quit

>>>>> On Mon, 26 May 2025 15:14:53 +0300, Eli Zaretskii <eliz <at> gnu.org> said:

    >> Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
    >> From: Robert Pluim <rpluim <at> gmail.com>
    >> Date: Mon, 26 May 2025 12:37:52 +0200
    >> 
    >> >>>>> On Mon, 26 May 2025 01:01:50 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
    >> 
    Xiyue> Friendly ping.  Does the patch look acceptable for smtpmail.el?
    >> 
    Xiyue> P.S. I have auth-source-xoauth2-plugin 0.2.1 released with the
    Xiyue> workaround.  Please check it out.
    >> 
    >> Looks good to me.

    Eli> Thanks, feel free to install on the master branch.

Done.

Robert
-- 

Pushed to master.

53371c95946 2025-05-26T15:07:24+02:00 "Make xoauth2 auth fail when a smtp server replies 334 (Bug#78366)"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=53371c959462a677a29ee869b3b6627facf3ed79

Message #54 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Robert Pluim <rpluim <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>
Cc: j <at> gnu.org, 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Mon, 26 May 2025 21:11:00 -0700

[Message part 1 (text/plain, inline)]

Hi Robert, Eli,

Robert Pluim <rpluim <at> gmail.com> writes:

> tags 78366 fixed
> close 78366 31.1
> quit
>
>>>>>> On Mon, 26 May 2025 15:14:53 +0300, Eli Zaretskii <eliz <at> gnu.org> said:
>
>     >> Cc: 78366 <at> debbugs.gnu.org, Anush V <j <at> gnu.org>
>     >> From: Robert Pluim <rpluim <at> gmail.com>
>     >> Date: Mon, 26 May 2025 12:37:52 +0200
>     >> 
>     >> >>>>> On Mon, 26 May 2025 01:01:50 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
>     >> 
>     Xiyue> Friendly ping.  Does the patch look acceptable for smtpmail.el?
>     >> 
>     Xiyue> P.S. I have auth-source-xoauth2-plugin 0.2.1 released with the
>     Xiyue> workaround.  Please check it out.
>     >> 
>     >> Looks good to me.
>
>     Eli> Thanks, feel free to install on the master branch.
>
> Done.
>
> Robert
> -- 
>
> Pushed to master.
>
> 53371c95946 2025-05-26T15:07:24+02:00 "Make xoauth2 auth fail when a smtp server replies 334 (Bug#78366)"
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=53371c959462a677a29ee869b3b6627facf3ed79
>

Thanks for applying!  As this can also be considered a regression in
smtpmail.el, do you think it's OK to install to 30-branch as well?

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Message #57 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Xiyue Deng <manphiz <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, j <at> gnu.org, 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Tue, 27 May 2025 12:32:22 +0200

>>>>> On Mon, 26 May 2025 21:11:00 -0700, Xiyue Deng <manphiz <at> gmail.com> said:

    Xiyue> Thanks for applying!  As this can also be considered a regression in
    Xiyue> smtpmail.el, do you think it's OK to install to 30-branch as well?

For it to be a regression there has to be a version in which it worked
previously, which I donʼt think is the case. But as a bugfix I can see
that applying it to emacs-30 makes sense, and I think itʼs
low-risk. Eli?

Robert
-- 

Message #60 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: 78366 <at> debbugs.gnu.org, j <at> gnu.org, manphiz <at> gmail.com
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Tue, 27 May 2025 14:53:28 +0300

> From: Robert Pluim <rpluim <at> gmail.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  78366 <at> debbugs.gnu.org,  j <at> gnu.org
> Date: Tue, 27 May 2025 12:32:22 +0200
> 
> >>>>> On Mon, 26 May 2025 21:11:00 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
> 
>     Xiyue> Thanks for applying!  As this can also be considered a regression in
>     Xiyue> smtpmail.el, do you think it's OK to install to 30-branch as well?
> 
> For it to be a regression there has to be a version in which it worked
> previously, which I donʼt think is the case.

Right.

> But as a bugfix I can see that applying it to emacs-30 makes sense,
> and I think itʼs low-risk. Eli?

Are we absolutely sure the status we correct should never-ever be
reported as itself?  If we are sure, feel free to backport the fix to
the emacs-30 branch.

Thanks.

Message #63 received at 78366 <at> debbugs.gnu.org (full text, mbox):

From: Xiyue Deng <manphiz <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Robert Pluim <rpluim <at> gmail.com>
Cc: j <at> gnu.org, 78366 <at> debbugs.gnu.org
Subject: Re: bug#78366: 30.1; auth-source-xoauth2-plugin conflicts with
 multiple Google accounts
Date: Tue, 27 May 2025 10:08:54 -0700

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Robert Pluim <rpluim <at> gmail.com>
>> Cc: Eli Zaretskii <eliz <at> gnu.org>,  78366 <at> debbugs.gnu.org,  j <at> gnu.org
>> Date: Tue, 27 May 2025 12:32:22 +0200
>> 
>> >>>>> On Mon, 26 May 2025 21:11:00 -0700, Xiyue Deng <manphiz <at> gmail.com> said:
>> 
>>     Xiyue> Thanks for applying!  As this can also be considered a regression in
>>     Xiyue> smtpmail.el, do you think it's OK to install to 30-branch as well?
>> 
>> For it to be a regression there has to be a version in which it worked
>> previously, which I donʼt think is the case.
>
> Right.
>

Ack.  Should have called it a bug-fix.

>> But as a bugfix I can see that applying it to emacs-30 makes sense,
>> and I think itʼs low-risk. Eli?
>
> Are we absolutely sure the status we correct should never-ever be
> reported as itself?  If we are sure, feel free to backport the fix to
> the emacs-30 branch.
>

I think in the case of smtpmail-try-auth-method, it never handles a
status that requires further input, so status like 334 (where < 400 is
considered a success) is essentially a failure; plus cram-md5 is already
doing something similar.  Hope this helps to confirm that this bug-fix
is low-risk.

> Thanks.

-- 
Regards,
Xiyue Deng

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.