Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 11:30:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 07:30:01 2026 Received: from localhost ([127.0.0.1]:37019 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4zSD-00063H-64 for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 07:30:01 -0400 Received: from mail-wm2-x10.google.com ([2a00:1450:4864:31::10]:36126) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x4zSA-00062r-Hr for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 07:29:59 -0400 Received: by mail-wm2-x10.google.com with SMTP id 5b1f17b1804b1-49d097b4939so3628645e9.0 for <81684 <at> debbugs.gnu.org>; Fri, 11 Sep 2026 04:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789126192; x=1789730992; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ennCycge8iM0p6GL0/8XxurnwtXFuYsJJx1YtvkOx1s=; b=cllqAzx/x7hvWYLSDUS6g3G7MUnCW2RnlIaYfLILljIvbtH4U1CHX2poZvHCmJZQjB xP+J905d9JhIuJQjqDYCVrYAago4/sHkJF+DgHoDohd4oqmc1EOC9gt6e4wS+CIIimJ1 g7H+iW8pnR7W4i7wPV/4AAKOw5FmSaJ7CxlOpt3PVsDsV7+hvRGY2SIjPxzeP33MbuV5 /Hx5zBkDB4QP3ysuD5C1Etkd7DMxCOKJoAGKJElVpkLeuL1JbPtpl58yw2dbOLdqKrQw g4IQck3Kj4t045a2+tmHc4fjJSpUiwL03oiWKdDXHA0+ppn6Ot2Xkpil2Sm4ZWOEPHph xbvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789126192; x=1789730992; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ennCycge8iM0p6GL0/8XxurnwtXFuYsJJx1YtvkOx1s=; b=S62sTlZzCZmAi2kXBDK60Kcr99RCIBZaxTe3MsWBlWtZkSqQdWZN5OHhKxMuXvIOSh yTU/MaCC6xqQUHqf4mtmhDIBjuJ+3biKPLB2r86WHiWj0PFy/t8pMZuM9IWWFOmLU/i9 zvFhQHESAXs/WEg2xJbss6NW8pAIvWLMNbUTE5IkMG+1MYWfsUYAeLaZCEdsxFuLEeAN 1VWI/bqcfNC0u6oxlcR/cKYyP4ZHHlpgJn+Ea2F0jt5UWY8QYeX/NQprQbjKf7257OrW xQ+0x3ImliCrT6It3njxE0/JrY8jAqZvMsqDrqmQbrqyIp+2W+Q7Jd7BRSS/gnl4yMm3 IeVg== X-Gm-Message-State: AFuF++kWLklIKhVboQa4txQ3N0tuucmLU7iKnxYa7MjUvmcZAVXfUGoG nMa6kJdDuiPzMc4WrzCQGESTen1py0t7Y0xQEWTCal2WC56KfuXDRHTaIdJ1+g== X-Gm-Gg: AYBFou12CM66dtZNyM1l1H/miEvic68D/ztjiI8zW3kWMI7sTtJJoRn08YAGZHtky/r 4IRwXbRfMZk9GT5jzuAC7BZtTcyEAIdQVV/oUaw5tkWrBfHGHQ1phZLy32x5dVlvbnf64w2gwf/ BE/ej7Juggbj8fk6vjIQftU7HyjKjLEeN1b7JQSba1vrMrqueEnrQMmzqkjLgIQZTF91oFb4rNR 8pCpZxxnN/LRSV6+yM5+jIN68OJ90uDnEVnJptaB3J5bnQ7q/LztCgmcPW3mhfASJOsRSm+X9/Y doxBLPqEZTJ7EJB6V+KIES4CvzOGkeZqFz+AGwFNqZsX/IGXFDSEiUKtdXqLc7jDJcHXPQ16tD+ WJAFYjwhYgij05ZtGgG3TDk52BRa2n4g1B8U0QZiCLQxD/Fi4SJg5U2v7Cig/YdzOYSqIdLTomt nYL38RtKFysBI6XbiOtdGYYfhzS/SZ6u4FcOk5zdcjTuyg5BVbNMsVxNIUTPIl+nTWwhRBvRxNb xpAw9YgtXQcg+aLNc62XqSj6tZKEU+2HFVOhYlukW3upODSMPVv6WbP73Z++3HOURYGRDGdxnxa Dw== X-Received: by 2002:a05:600d:8654:20b0:495:6e68:5df2 with SMTP id 5b1f17b1804b1-49e619995e3mr32469475e9.12.1789126191957; Fri, 11 Sep 2026 04:29:51 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef2b15b665c88f1743cecc31.dip0.t-ipconnect.de. [2003:ea:ef2b:15b6:65c8:8f17:43ce:cc31]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d26b7332asm147055245e9.0.2026.09.11.04.29.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2026 04:29:51 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: Encoding In-Reply-To: <868q585a41.fsf@HIDDEN> References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN> <87ecf0p7rp.fsf@HIDDEN> <86bja45j17.fsf@HIDDEN> <874ifwp5n6.fsf@HIDDEN> <868q585a41.fsf@HIDDEN> Content-Language: en-US Date: Fri, 11 Sep 2026 13:29:50 +0200 Message-ID: <87tsnwnhzl.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Nickolai Dobrynin <ndobrynin@HIDDEN> >> Cc: 81684 <at> debbugs.gnu.org >> Date: Fri, 11 Sep 2026 10:13:33 +0200 >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> > Let's talk about encoding while sending first, okay? That's what the >> > original exchange was about, AFAIR. When we send something to IMAP >> > with "UID SEARCH CHARSET UTF-8", we should encode the payload using >> > UTF-8, don't you agree? >> >> Yes. >> >> In general, >> >> 1. All quoted strings should be 'utf-8 encoded, and >> 2. all literals should be encoded using 'no-conversion. > > What do you mean by "literals" in this context? At any given time, the IMAP command parser on the server side operates in one of two modes. The line-based mode is one where the parser is processing a single command per line, and the lines are CRLF-separated. In the other mode, the parser counts *arbitrary* incoming bytes until a prespecified number of them is reached. Such sequence of bytes is called "literal". Now, I'd like to make a correction. All literals should be encoded using 'no-conversion UNLESS we know in advance that the literal contains text. This text-holding literal should be encoded via 'utf-8 instead.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 10:57:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 06:57:28 2026 Received: from localhost ([127.0.0.1]:60600 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4ywi-0003zv-Bw for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 06:57:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53442) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1x4ywg-0003zN-DS for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 06:57:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1x4ywb-0003Ml-3j; Fri, 11 Sep 2026 06:57:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=qZ+Cex7i3sN3fe733hYwtIKwxbwIFtKUpkmklSaiE/s=; b=Xkp+N9uKeFms guJ8qiUbvgFV4DsyUVMc7a1WGEWOMSRqsW9JeLqjHES2t+0hX4JPgr4vSYzbToOGhPJiXVbM/Rd7t MUCdBMKmwBLivLhDmtjUKvl2c/sOaXLmPMAeEfAepM1CpbfQHrg4bfnl9UG23zHXmSO6j2xAgZ1hH y8BWZbdn6F+MjDr5AkVn8OCw1XSd/5RdI0Vy+R8dlsUY2bpFSU3Sbf9PHgiV5nKG9BWuyDVbyD4q4 HCIuENGY1n6uzRLM2L/qgtL7bbOurieRg5P/rXkSuRQy2ZOMNj/VF46TL/qTtP1adcvT0GsD7tD7W 4BYM39tauPdLiw5dK55QYA==; Date: Fri, 11 Sep 2026 13:57:18 +0300 Message-Id: <868q585a41.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <874ifwp5n6.fsf@HIDDEN> (message from Nickolai Dobrynin on Fri, 11 Sep 2026 10:13:33 +0200) Subject: Re: Encoding References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN> <87ecf0p7rp.fsf@HIDDEN> <86bja45j17.fsf@HIDDEN> <874ifwp5n6.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Cc: 81684 <at> debbugs.gnu.org > Date: Fri, 11 Sep 2026 10:13:33 +0200 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > Let's talk about encoding while sending first, okay? That's what the > > original exchange was about, AFAIR. When we send something to IMAP > > with "UID SEARCH CHARSET UTF-8", we should encode the payload using > > UTF-8, don't you agree? > > Yes. > > In general, > > 1. All quoted strings should be 'utf-8 encoded, and > 2. all literals should be encoded using 'no-conversion. What do you mean by "literals" in this context?
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.
Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 10:56:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 06:56:00 2026
Received: from localhost ([127.0.0.1]:60399 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1x4yvH-0003uR-CR
for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 06:56:00 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:58800)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1x4yvF-0003u8-0u
for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 06:55:57 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1x4yv9-00030U-Hd; Fri, 11 Sep 2026 06:55:51 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=UTDKAKwoL0kU60zWzujr7geAs4MgrsDdjbnng1amukE=; b=VSkO+mut9xX2
fzRwecm6eBqYMwT64n08scyQjQ1r9QbM/1OY369AzHm+n40L24f/TUP1OVDrhLy0eiSe1jEKLP14T
Nu8WQKxMeYsNCXv52wRX12k346sIBUNGGdrkxJ8NVwvqD+ikuW9aLcwjDScfL5hmTJEqIHOPyIM7h
UQRad80M8DfXdPf8K+dbxJ7OYSj/u+OkcnnM04tr65MXgEIsCybzneM8HWmOVoYApMD2soeCHh/O3
dUes56lBn+hBPjbFaVUXJ6tG0FthmDR6GFc4264LwbEy5F+750tpwZ1fvH4HzpRyxryXfsPYz8msC
xAXfaQq4q+dpJMQc75Y4XA==;
Date: Fri, 11 Sep 2026 13:55:47 +0300
Message-Id: <86a4po5a6k.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Nickolai Dobrynin <ndobrynin@HIDDEN>
In-Reply-To: <878q58p68k.fsf@HIDDEN> (message from Nickolai Dobrynin
on Fri, 11 Sep 2026 10:00:43 +0200)
Subject: Re: Encoding
References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN>
<87ecf0p7rp.fsf@HIDDEN> <86bja45j17.fsf@HIDDEN>
<878q58p68k.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 81684
Cc: 81684 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Nickolai Dobrynin <ndobrynin@HIDDEN>
> Cc: 81684 <at> debbugs.gnu.org
> Date: Fri, 11 Sep 2026 10:00:43 +0200
>
> Eli Zaretskii <eliz@HIDDEN> writes:
>
> >> From: Nickolai Dobrynin <ndobrynin@HIDDEN>
> >> Cc: 81684 <at> debbugs.gnu.org
> >> Date: Fri, 11 Sep 2026 09:27:38 +0200
> >>
> >> Eli Zaretskii <eliz@HIDDEN> writes:
> >>
> >> > So I think either nnimap-open-connection-1 should set the
> >> > encoding/decoding to use UTF-8 instead, or when we actually send
> >> > strings we need to encode them by UTF-8, and where we receive strings
> >> > we should decode.
> >>
> >> Treating an entire response as UTF-8 (if I understood you right) puts
> >> us on a collision course not only with other IMAP4rev1 extensions, like
> >> BINARY, but also with IMAP4rev2 and, theoretically, even with the
> >> existing IMAP4rev1 protocol. If someone, for example, sends you a
> >> message of type
> >>
> >> Content-Type: application/octet-stream
> >> Content-Transfer-Encoding: binary
> >>
> >> and the client requests it, the server may respond by sending the raw
> >> binary data back. Nothing in rev1, as far as I can tell, (and in rev2,
> >> of course) prohibits this. That said, the likelihood of you getting this
> >> message now, in 2026, is low, but it will increase.
> >
> > Sorry, I don't follow. If we get a message with
> > Content-Transfer-Encoding, we should decode as that header says. But
> > we _should_ decode ("binary" means we should use 'no-conversion').
>
> Ah! Sorry, I misspoke. I meant "we should not decode using
> 'utf-8". Please replace "we should not decode" with "we should decode
> with 'no-conversion RATHER THAN 'utf-8".
Yes, of course. That goes without saying.
But the current code neither encodes nor decodes, regardless of
Content-Transfer-Encoding, AFAICT.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 08:13:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 04:13:47 2026 Received: from localhost ([127.0.0.1]:50772 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4wOI-0002AW-U2 for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 04:13:47 -0400 Received: from mail-wm2-x10.google.com ([2a00:1450:4864:31::10]:36418) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x4wOE-0002AG-6q for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 04:13:45 -0400 Received: by mail-wm2-x10.google.com with SMTP id 5b1f17b1804b1-49ce364488dso1824125e9.0 for <81684 <at> debbugs.gnu.org>; Fri, 11 Sep 2026 01:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789114416; x=1789719216; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=s3Bc+pOQ3ksZHTpXuaUucVEfCayrzT4MnArRRU5xkkU=; b=rb6+YlZz2XZDS1MHu+OtzVrfjzorOcS/BMSJMCvSOyLO7oao7X0fBFl84pXncLimtC 34zM8Sk7pelZ9tqG6q5ZKzYWO1gFjAa7C8J1G/XyG9k8wd/h4y5elOLU/q2XWeYAQ85z 3YiEeEQ4Ilw9IlDA1oXLRK6tv7FEXaIGUWjx+kEFjXRTS2G9GOt+uni3pg1rf+GqglGH ztqRaOgzH7RC8hCz5RRyle62RXnmZczS9TfgFDLtpH2OCFsY9aqN9srkud/565MveivJ nzKpCqqFsHTVk8jpvDRKheaqebTNHYYxT8VJI0LhFOpk4iYqKL43bRcEEy2r6W8RBINk Hmmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789114416; x=1789719216; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=s3Bc+pOQ3ksZHTpXuaUucVEfCayrzT4MnArRRU5xkkU=; b=PaIlMUj1D0cKCRxlsfAJF0/XjSDH1l2grFlGRMTp1YySMRXQRvRqsSg1YURjLtkM8u DzBtH+wDzB6Of51d/f40wA7FTcZUAKBSdRzKTmCQcCtMwTmP2tvhW+fhvWTj2z5xguqD cvxE5X/VRSzWXH8N3zB/UZ6laYasnKjIfh8jOwB9OQ1tJya5blNjTqgc15Kq4ZcSPp+H KL0dooNv3vhSB5ehrjTdh27dVngrBgtrVIlqfoGVBjgr6GpOVrNUIcDjGfu4MXkofPa/ yUbHMxayJvLSsmSZnT7Vn1Bb33FqnPmmSefuRofi3XUuNcXK2iY4A+HzKnCgpcbE3MKl cuiw== X-Gm-Message-State: AFuF++lHWQwHwQucqvgHfraaagmrRTVfrOSaI1KqeJI75E0hS5P1TFCa 6uE6X+AGWEP9I3Zk5sthiKf37WXeQTcr+npy6JBwRZL5VjstwtR6YFR096LxuQ== X-Gm-Gg: AYBFou3b0OL1F6K03dXGo2RdHXQm3wnFTWPdlAKvyj66/u5obZwRb1nQH6i/rMlfxBL fNKIBpFoH6nkSSNYHB44IDvlg2WiEWkHpqrbV2X3gSxZgeIcsbBK4lTGMGUaF85HLbXUN244bBW Fo5tKGPTMVPVyUTnGhpl0ieQoTNuxBdOP+45fwl9VubMZdhIyR07WBamar49SqpeP+1w/paxTn0 0t2HcyxhaMq20ztKCfSxbuxekNw10S2zE8D7R0enX2QAY1dL2W1Y1qsfRZhNfosxYefVud00TGF yNR6wuOcGbcw95FlLQM27gl/XNQI2uwuTBvaZgLIXQbtKUeRTuSZiDkcT0VpPPz94O59+byKlmt HEGKpQpcfc+r8mHqIVqRDYRk+gIldxlvs8lOH7etKGmPht3NpsR5+WOgZJ2atDcpb9VjfRPblYn ornx9OWIfeTRWEwwU/1th5irq42IjtW0vyud9x4uknTpzF2JMJmxZtEAnJEQWP3o2KxJ5acZRbD 7MmPVZ9N/AvBaAWLKbdZvgnyam0cVPH+QdChrMkiuMYZs8YkSjG61SFidpPzzkbKP0evT/+Mj/1 lM4= X-Received: by 2002:a05:600c:3acb:b0:49e:6072:fce6 with SMTP id 5b1f17b1804b1-49e60d22d9dmr33970585e9.0.1789114415698; Fri, 11 Sep 2026 01:13:35 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef2b15b665c88f1743cecc31.dip0.t-ipconnect.de. [2003:ea:ef2b:15b6:65c8:8f17:43ce:cc31]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d26be3e0asm144134135e9.1.2026.09.11.01.13.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2026 01:13:35 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: Encoding In-Reply-To: <86bja45j17.fsf@HIDDEN> References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN> <87ecf0p7rp.fsf@HIDDEN> <86bja45j17.fsf@HIDDEN> Content-Language: en-US Date: Fri, 11 Sep 2026 10:13:33 +0200 Message-ID: <874ifwp5n6.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Eli Zaretskii <eliz@HIDDEN> writes: > Let's talk about encoding while sending first, okay? That's what the > original exchange was about, AFAIR. When we send something to IMAP > with "UID SEARCH CHARSET UTF-8", we should encode the payload using > UTF-8, don't you agree? Yes. In general, 1. All quoted strings should be 'utf-8 encoded, and 2. all literals should be encoded using 'no-conversion. PS With UTF8=ACCEPT enabled, we can't include the CHARSET directive anymore, so the command becomes UID SEARCH "<payload>"
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.
Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 08:00:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 04:00:57 2026
Received: from localhost ([127.0.0.1]:50294 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1x4wBt-0001Wk-GR
for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 04:00:57 -0400
Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]:60615)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>)
id 1x4wBr-0001WS-4E
for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 04:00:56 -0400
Received: by mail-wm1-x32c.google.com with SMTP id
5b1f17b1804b1-49b0d8bc2aaso7365505e9.0
for <81684 <at> debbugs.gnu.org>; Fri, 11 Sep 2026 01:00:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20251104; t=1789113649; x=1789718449; darn=debbugs.gnu.org;
h=content-type:mime-version:message-id:date:content-language
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to:content-type;
bh=icD/JYrObP/QBd5q57qeFbKLIGdjD/rHYME6o0qAYqA=;
b=GiI/yfKlH7MZg4ovTPYA11Dqutk5vWUb5hjagGKbsT6cAq17/d3FK5M/HvnusZqNZI
AMde1sz21nJJ+/sMg+pzDpt9drEFBPMPwySq+9+113YL1MZsZecJ60g2tgPmplvPY7iE
RbD5k9RtpWU3eNB/aEcNc9obKjmsjH8v8Q0YKStDTXEkO5cMl13CEB4072n7hlaDNIcG
aYn1O4bmCh0cyEaOEq1Rz1dqQL5cxNynpUr0GV5e6IN3ddorLfqn9kGipzS7BAGvxgD3
Inl30hCFZ/3aMA3Gy/qS2ZNiH0ddMpYuONJ38OKqS1LjL8m4FJfOAkZp0htEf/V3KqbU
Dk+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20251104; t=1789113649; x=1789718449;
h=content-type:mime-version:message-id:date:content-language
:references:in-reply-to:subject:cc:to:from:x-gm-gg
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to
:content-type;
bh=icD/JYrObP/QBd5q57qeFbKLIGdjD/rHYME6o0qAYqA=;
b=H2WTtIFeEjqDo40NK47TPTWuAYRr9KpQKbNG3K2CKZoIdSoRnA5TOr3uEcj0UWgavp
GxQzLEr6YBoDd2t2cYMLjBhX3spHxaiZo6osbJWltiLCJp/OFOuzqOk1EpJj4NcoT7xl
7Sh8lrQHv91UT9rcmq31Bz8orv6N64yzKcIfaKAxiuBd++/FFPOGFNZ3q9UHCtIuHrYM
VYVGr8Qv1nNIDIWEAT0CDylsIvdJ5EWgGz9vyzKwTAqiarRCPAF16tPjTIFQITf/Pcnb
y+4s1/Pf0qgdFyeB0CGRd0KpTtLjXVH/ihkjLSf50kgB2OXIp2ypyEvFqW9G+GbeIP2d
qTNw==
X-Gm-Message-State: AFuF++n8UvIY8J7o5cCoThE4F/b+zfniSl7Y0ikhlB6M6jVeDY+31t9x
hNgl/UnvXhKQMEasX+mwiXpwN3Kuueav37iqKhdiTEme8ddWxEQB0RFZoYCWsw==
X-Gm-Gg: AYBFou2590ISY8xsYIYbncDDcbh4KJ/BkpF0znXEne3ac0r5SIejBiE66R+9hda1wZV
BgVmcSjK2xmBzKBeu1Rnl0Eu4nT4u6AY+Te87IBzEwguXUjNsUvXmea1yILIiWY2lOtesj3Lq+9
gcjxSnkQ5wjlF1GHeX8HGM+v+tWZTlsiCjXCMKZHMP+xUAdl8JR8E4WGjdHel64b8K4zBx4D2mM
J8AZoZQxIdJl8jW6fGjoGem/4LQzONJWq5P7a387Ou+q+GWfGR+MgphaUTYyPq5WjdP8uiBCtLv
gqYgEW6wX4KZH3Qzr9yXj4X0HZzQFLn9uX/FQDN7OVb3aesJLUBN7g7zgGdwrpA2bwqVsczxCTl
upc9bmmBrWrdTY4HKokQKEQ6enRryKmezYYFJoCsNs51v1h01hj1FeVotBHbKfPiJxFXnFKJnsM
rmd6iCUQ9eq63bBbQZWtKoulfmG7sMTLh20WEM9fkCGzN8MebzpVUq1sgQ9gaJruFL0S1j9BV82
jhARVZEAxiApAAIT5bu8Yg56hYXiUHxlvSnNOZePKqLtx90mMnzu8V1DrnC5btT7pilTiR54VES
dQ==
X-Received: by 2002:a05:600c:3b27:b0:499:8b13:3a98 with SMTP id
5b1f17b1804b1-49e6197f60cmr34717105e9.4.1789113644879;
Fri, 11 Sep 2026 01:00:44 -0700 (PDT)
Received: from nova-laptop.dobrynin.net
(p200300eaef2b15b665c88f1743cecc31.dip0.t-ipconnect.de.
[2003:ea:ef2b:15b6:65c8:8f17:43ce:cc31])
by smtp.gmail.com with ESMTPSA id
5b1f17b1804b1-49d26c2a419sm148379035e9.7.2026.09.11.01.00.44
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 11 Sep 2026 01:00:44 -0700 (PDT)
From: Nickolai Dobrynin <ndobrynin@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: Encoding
In-Reply-To: <86bja45j17.fsf@HIDDEN>
References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN>
<87ecf0p7rp.fsf@HIDDEN> <86bja45j17.fsf@HIDDEN>
Content-Language: en-US
Date: Fri, 11 Sep 2026 10:00:43 +0200
Message-ID: <878q58p68k.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 81684
Cc: 81684 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: Nickolai Dobrynin <ndobrynin@HIDDEN>
>> Cc: 81684 <at> debbugs.gnu.org
>> Date: Fri, 11 Sep 2026 09:27:38 +0200
>>
>> Eli Zaretskii <eliz@HIDDEN> writes:
>>
>> > So I think either nnimap-open-connection-1 should set the
>> > encoding/decoding to use UTF-8 instead, or when we actually send
>> > strings we need to encode them by UTF-8, and where we receive strings
>> > we should decode.
>>
>> Treating an entire response as UTF-8 (if I understood you right) puts
>> us on a collision course not only with other IMAP4rev1 extensions, like
>> BINARY, but also with IMAP4rev2 and, theoretically, even with the
>> existing IMAP4rev1 protocol. If someone, for example, sends you a
>> message of type
>>
>> Content-Type: application/octet-stream
>> Content-Transfer-Encoding: binary
>>
>> and the client requests it, the server may respond by sending the raw
>> binary data back. Nothing in rev1, as far as I can tell, (and in rev2,
>> of course) prohibits this. That said, the likelihood of you getting this
>> message now, in 2026, is low, but it will increase.
>
> Sorry, I don't follow. If we get a message with
> Content-Transfer-Encoding, we should decode as that header says. But
> we _should_ decode ("binary" means we should use 'no-conversion').
Ah! Sorry, I misspoke. I meant "we should not decode using
'utf-8". Please replace "we should not decode" with "we should decode
with 'no-conversion RATHER THAN 'utf-8".
>> The only future-proof way that I see is to follow the ABNF grammar for
>> rev1 and rev2, side by side, and decode only the parts that don't
>> include the "literal8" non-terminal in the hierarchy.
>>
>> The parts that are now text-only but are not required to stay that way
>> based on the grammars should not be decoded.
Likewise (see above).
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.
Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 07:44:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 03:44:47 2026
Received: from localhost ([127.0.0.1]:49362 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1x4vwE-0000R7-WF
for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 03:44:47 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:58482)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1x4vwB-0000Qt-S2
for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 03:44:45 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1x4vw6-0001o2-Ei; Fri, 11 Sep 2026 03:44:38 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=sPE81g+t1pcHVPM0cEyJ/3ky5Hj5wi1Z2tYA3io3pg0=; b=kepyOlZ7C6/S
ohdBxbiUgxgGdRCu2xPhJ4iqRfh3iHZmCLxB6wukrhrbt7Z9pI3QUCoOO0pAllaCEGYZeSPUXec9Y
tvBw7tiXyGC/ky96fx3QwyfQXSb77fPcO2QD6JWqbK9bNPAO9fb8R/yvOT7YrhtYRuHgN3NiKz+ml
TDFSImn4DPJ0gDOaVKynoiF9FN1bqVHsXGVN2qQQRQR/3SXdQravMMRqAUBcOLlwKrKEhTq1E6Bml
TJPma1QBudExR4GlmTAS6yWih4tdZNd5RBZRupykEQh0En1zKnR49Q1Er4mUaHbdisGPubrgfsiWl
mJsnOWlEyT80qY5ebEPvYg==;
Date: Fri, 11 Sep 2026 10:44:36 +0300
Message-Id: <86bja45j17.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Nickolai Dobrynin <ndobrynin@HIDDEN>
In-Reply-To: <87ecf0p7rp.fsf@HIDDEN> (message from Nickolai Dobrynin
on Fri, 11 Sep 2026 09:27:38 +0200)
Subject: Re: Encoding
References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN>
<87ecf0p7rp.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 81684
Cc: 81684 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Nickolai Dobrynin <ndobrynin@HIDDEN>
> Cc: 81684 <at> debbugs.gnu.org
> Date: Fri, 11 Sep 2026 09:27:38 +0200
>
> Eli Zaretskii <eliz@HIDDEN> writes:
>
> > So I think either nnimap-open-connection-1 should set the
> > encoding/decoding to use UTF-8 instead, or when we actually send
> > strings we need to encode them by UTF-8, and where we receive strings
> > we should decode.
>
> Treating an entire response as UTF-8 (if I understood you right) puts
> us on a collision course not only with other IMAP4rev1 extensions, like
> BINARY, but also with IMAP4rev2 and, theoretically, even with the
> existing IMAP4rev1 protocol. If someone, for example, sends you a
> message of type
>
> Content-Type: application/octet-stream
> Content-Transfer-Encoding: binary
>
> and the client requests it, the server may respond by sending the raw
> binary data back. Nothing in rev1, as far as I can tell, (and in rev2,
> of course) prohibits this. That said, the likelihood of you getting this
> message now, in 2026, is low, but it will increase.
Sorry, I don't follow. If we get a message with
Content-Transfer-Encoding, we should decode as that header says. But
we _should_ decode ("binary" means we should use 'no-conversion').
> The only future-proof way that I see is to follow the ABNF grammar for
> rev1 and rev2, side by side, and decode only the parts that don't
> include the "literal8" non-terminal in the hierarchy.
>
> The parts that are now text-only but are not required to stay that way
> based on the grammars should not be decoded.
Sorry, you lost me here.
Let's talk about encoding while sending first, okay? That's what the
original exchange was about, AFAIR. When we send something to IMAP
with "UID SEARCH CHARSET UTF-8", we should encode the payload using
UTF-8, don't you agree?
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 11 Sep 2026 07:27:51 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 11 03:27:51 2026 Received: from localhost ([127.0.0.1]:40182 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4vfq-0006ql-PX for submit <at> debbugs.gnu.org; Fri, 11 Sep 2026 03:27:51 -0400 Received: from mail-wm2-x10.google.com ([2a00:1450:4864:31::10]:43540) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x4vfn-0006qR-6D for 81684 <at> debbugs.gnu.org; Fri, 11 Sep 2026 03:27:48 -0400 Received: by mail-wm2-x10.google.com with SMTP id 5b1f17b1804b1-49cd5462b69so3548455e9.1 for <81684 <at> debbugs.gnu.org>; Fri, 11 Sep 2026 00:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789111661; x=1789716461; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=dQFuWx9UD1CpLjd6x+RNm5J9ZRuCPKUdAPu8UEh6wkU=; b=nfoKao1U7jkIvz8vU8HpZeTrxxgi40hY3F5lO4x15i9+HXGxqeu1SUt9eZOUHENrg/ CS9D8dM70n6sYaYwVQLMJIXZJNeegvAr+BaVh/aR5/rojK0NjYtmx5wct/mL7oP5lYS3 Ok+Mec2GHtyGDOqBtqXhKG6L30/9QiBYUpUmLgAqDulerWf7tjqT4aXXcdlhallPqw2H ujfA6X4M7K/EFEhHcSbc9AxSFSIX/TJ529Il3+72otglCivYW2bmK7PUus0txWa8ut7b Veq+VRDFm9r9Seij40wa4sT0GZ5VqILxVsLqBQ4SbSqQQq7mgxJ9yLBbT5Ho+Fnt+4vb GFKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789111661; x=1789716461; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=dQFuWx9UD1CpLjd6x+RNm5J9ZRuCPKUdAPu8UEh6wkU=; b=FB7cSgk0FAPbp8+7jKkx0itN5OR68kdQQc01nwIzm5F7JsmJKdZoQSCcJ8dOQfmDD5 KLlxAvWV0JHpH4L9+g/sTLQ+vA3CddWPTGk4JrR6/CraVDkIXyuV6CkT1k+myVbxXlUX 9PA13Osg1JWR92k3LG9v1sVapF1Jd0FEYIhzSwOf1fyCKHlPr18aao0yMiQ+/iPjfF3/ 5dtx1+9s/w4+3Neoh6iVs+UR+LfmkQw/gQHUkHPiI16rESHsha+mYJsTmD7RQwUmaca+ prUwawJqBZjU5ZVW2YHxSBq9lXm9ub2E0x/ErDjqOwqFOdjZgmHI8o1ihf7F+ndFieio 8fUA== X-Gm-Message-State: AFuF++mzDXaMAu5v0CRt8eSANyzHnaS/spQ0jy6f6LdVAYdyqGyJqDG+ xNwsInReMlmSlmNq8LKN1nlkHBqF5v35skrYBftYc2W3oM7WFm8fiI59XN3WOw== X-Gm-Gg: AYBFou1nslLa99GiLTNgBFyjviKxLGvAPhAHhc+E0iveFEBOpWtzVEZYQwVRkinScCH Y8vgeu3Pu1S9uoPVG414McQ7Epgjeczy5o/2ZlZrhrNwQRdI/IjxiyYA09VHIKeBdxM5fLmRsCA ll+l0F+rwt2K6AOCiJp2zvUfm4P7d3yQpxJ8f92u1eLCVhgR/k1uUpdjR23+y8sneuBba0hUKvH TeLIcPnrfchM+8AuiWk1avTCund8i6W6jo4SXbmU/USXzkBCiWnLkq5o0Ub9ifQ39g5g1WaqQTf H0JAKi8Ihbp7R/Ew/cAFtV3oMYysYFU3ZioyVAM5BAfAF0Tg7eSiULN0DQUKXxxU/CwZFKT1Dv+ ZApH6CdwwIpdaeShUDfgpMqKr30FhGzKdVWLamBJr9rkr2xkizKI9wCLPGMz31yEtL69WCUQCXf ECYexTVeoKQS6UZCHImjOw2+QIVWuwOVUlEF9OcwU2U+WEA2N/Z+9RM8kp8gKe0SHPqGr+n/JlE lkmovEHNmD+2kpjyWzehGJeIOz8PBX+004M50LK07dPwDqeqOIg38o3Gt/ifaheFu68UN3RgslY EW0= X-Received: by 2002:a05:600c:3f0a:b0:49d:1012:893a with SMTP id 5b1f17b1804b1-49e619941b3mr66742585e9.7.1789111660449; Fri, 11 Sep 2026 00:27:40 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef2b15b665c88f1743cecc31.dip0.t-ipconnect.de. [2003:ea:ef2b:15b6:65c8:8f17:43ce:cc31]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d26c39086sm120006245e9.10.2026.09.11.00.27.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2026 00:27:39 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: Encoding In-Reply-To: <86se3h5vhv.fsf@HIDDEN> References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN> Content-Language: en-US Date: Fri, 11 Sep 2026 09:27:38 +0200 Message-ID: <87ecf0p7rp.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Eli Zaretskii <eliz@HIDDEN> writes: > So I think either nnimap-open-connection-1 should set the > encoding/decoding to use UTF-8 instead, or when we actually send > strings we need to encode them by UTF-8, and where we receive strings > we should decode. Treating an entire response as UTF-8 (if I understood you right) puts us on a collision course not only with other IMAP4rev1 extensions, like BINARY, but also with IMAP4rev2 and, theoretically, even with the existing IMAP4rev1 protocol. If someone, for example, sends you a message of type Content-Type: application/octet-stream Content-Transfer-Encoding: binary and the client requests it, the server may respond by sending the raw binary data back. Nothing in rev1, as far as I can tell, (and in rev2, of course) prohibits this. That said, the likelihood of you getting this message now, in 2026, is low, but it will increase. The only future-proof way that I see is to follow the ABNF grammar for rev1 and rev2, side by side, and decode only the parts that don't include the "literal8" non-terminal in the hierarchy. The parts that are now text-only but are not required to stay that way based on the grammars should not be decoded. What do you think?
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 10 Sep 2026 10:42:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 10 06:42:26 2026 Received: from localhost ([127.0.0.1]:38440 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4cEb-0002Pf-Ok for submit <at> debbugs.gnu.org; Thu, 10 Sep 2026 06:42:26 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:56395) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x4cEW-0002PL-RW for 81684 <at> debbugs.gnu.org; Thu, 10 Sep 2026 06:42:23 -0400 Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-4843e9c5960so6520627f8f.0 for <81684 <at> debbugs.gnu.org>; Thu, 10 Sep 2026 03:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789036934; x=1789641734; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=nXQhdNyYt12nQXe+61GH20FGXrs6ziQIvIrtiHDOr5c=; b=ZK1xxl667N0S7/hiXiUGrjnnVEAAXTft0lmLp6ad3Ljwm2gkYgXi7CruSpEU4h9fkL 4ptYT6G+AwTUeD5waQG2FOayfCaTD1U8V/ByAiK8nRaX5BCpUteLP0IDFqV2+vw1esUA Jh18RjbJIdn48zkZrzbLO2NZAioke/da5q4IZPY5zqc5Wx0SxQ+F3ossTz8MQTpKllA3 5KDb2YQC3TaceZ1VmbNqcOngRt9toxGg/VcOlJcLsemIuN2lUTBxaSDCtd973iDHXOnO rE5vCVCaUX40LbQ3ni77LmMz8oA1AKvOmBlGk8BWjMlbG4uNIKCyv6H05ZN/6d11pz1Y ccmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789036934; x=1789641734; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=nXQhdNyYt12nQXe+61GH20FGXrs6ziQIvIrtiHDOr5c=; b=Rb4cGpv0b/eumstOIaM5t9+/xWWEowT0UyVatsWVHrx02SaeOupRGEYVxP8/ybzANa QEKAOaxQboLcHtrmfeXXD5FgSWPPWPlXrq2S2DVWP+DHsVoe8d8+CgZx1qnS6kHzHQ8H aspw3I92wRq3u8fLeXPpGSGh7SpsOBPuv0xlKKJrIXH2K2dTNwp6f2tT33KHZS7jEuMZ 6AkkcEuXaIP1UUjyl40xIztPmUIiXROEivmalz+98KpVl8AeVRlJh465OfoPc5zlLMY7 G/2x8/3pyIV8CiFnfzgfdbIcf0hHhfowIzT3ID/oEHytiPdM8SrnTCRKX6O7I89+coy7 KXsg== X-Gm-Message-State: AFuF++msQDWoa1B3NRLxWc4+RcWgdGVME9zjskSFoaJ90tZjR27DB8/P qILljq6zoJz/14SngZDmvnC4SjthquVobphabNJ38rF0RWY+cyiQLT7w8iia3g== X-Gm-Gg: AYBFou1uzR/SC9nxxazpeiRyg84b8GZDXJQEr9KSbVBPNPW6hWQMjj+hlAWDmxf7IuN Zdrxsih5qgyC8tLE4F//fOdygDxl9c44Fg6I6tpv167Q3LNncJkiKQJqVndmQJYH0rI7f7ftIYU i+egjMRZ+xvsmPImMxJZxne7zF8Mx4JSiq51Ho96/USW1rZc+oLIA2rXFPRaGtwyxRo/tRwqVS4 1l9U//jjpoKz316FBsJjYCYbgwQszcsl5Y0f/z7+qLiz0PkORC0M6NYcPUWva+9avpAoETya0zI DvML/tglUZhJ51wuZBUaWS51pENt7fDo3GUxJnaPNW/O8rZdYXR8hltzcbCIsvlEATZ+Hu//Z9P l+xG9OQtbsK3qt3o9XRu9d4x70amN98oT6+UU03am0hzYgJMoDFQu7XSmtrbihonPRBbwvL/mab p7SEYske6Vu1EILbXRP+ljizhOtIYWzhPrHTH8TYVik1EuWCYoRL8X7oAIa1aIDEHrXCOxxY5Fl RnYSlZzxWk115WY+IvhcYNV9gAcTrjbiqFzk7ytuJQaXtAedZXzg/g3kDUVu9rxdPAoUVKP0cuX IA== X-Received: by 2002:a05:6000:2f88:b0:485:8c16:5eee with SMTP id ffacd0b85a97d-4858c1660b1mr40146551f8f.40.1789036934277; Thu, 10 Sep 2026 03:42:14 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef2b15ed9480136a80189352.dip0.t-ipconnect.de. [2003:ea:ef2b:15ed:9480:136a:8018:9352]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4858d2693e8sm49268716f8f.3.2026.09.10.03.42.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 03:42:13 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: Encoding In-Reply-To: <86se3h5vhv.fsf@HIDDEN> References: <87wlsty26d.fsf@HIDDEN> <86se3h5vhv.fsf@HIDDEN> Content-Language: en-US Date: Thu, 10 Sep 2026 12:42:11 +0200 Message-ID: <87pkylxu9o.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Eli Zaretskii <eliz@HIDDEN> writes: > Aren't other clients of nnimap.el call > encode-coding-string and decode-coding-string on the strings they send > or receive from IMAP servers? Or maybe gnus-search is its only > client? I don't know the answer to this off the top of my head. I would have to look. > I hope the above makes sense, Yes, it very much does. All the more reason to go back to the drawing board.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 10 Sep 2026 09:03:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 10 05:03:28 2026 Received: from localhost ([127.0.0.1]:58761 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4agl-0004CE-1O for submit <at> debbugs.gnu.org; Thu, 10 Sep 2026 05:03:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54716) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1x4agf-0004AK-3z for 81684 <at> debbugs.gnu.org; Thu, 10 Sep 2026 05:03:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1x4agZ-0001Mg-Oc; Thu, 10 Sep 2026 05:03:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=HmOklejWQEz1jEfSPv8AFCLHbhca1p5bL+v97kY2P0A=; b=aNT99DH8ZFN/ 2OcC9RD+mT23fVg9fF8FS1/6H14BfZbUmVRzmxp6hnQg8xT4aasHVvjKZi4jHLlAgcTfotk6MZgtd KAExv2Pfqbxb8gFTOLYhsE0qrCEgsxJe0Fb97zigCyfFrU6IjTteH2gFPu8EuiWB6P46xEO+cUupL 2N7TUKENY+BMdOsqbVZE1VBHNXUXhXuSEK12Zxp8Hxl+quYI/K6+FjeboLN0KH+JCOD8rNMCZ1DC7 jPFBY4Z2RZJRXozqLJmdOmBUQce/0hqydeSC+NyDI1Qxp0BCsKWf8pNyypTaEYrKjytd28JQgVm+q 6lpkXrB9pXMkgND1W5QbWQ==; Date: Thu, 10 Sep 2026 12:03:08 +0300 Message-Id: <86se3h5vhv.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <87wlsty26d.fsf@HIDDEN> (message from Nickolai Dobrynin on Thu, 10 Sep 2026 09:51:22 +0200) Subject: Re: Encoding References: <87wlsty26d.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Cc: 81684 <at> debbugs.gnu.org > Date: Thu, 10 Sep 2026 09:51:22 +0200 > > I am still testing, and before I post the new patches, I'd like to > address the query string encoding issue you brought up and see if you > agree. > > Please refer to `gnus-search-imap-search-command' in gnus-search.el and > `nnimap-send-command' in nnimap.el. > > Based on what I've found, I believe that _no_ explicit encoding is > necessary for both the query string and a text command to be passed to > the IMAP process because > > 1. the IMAP process has its encode/decode coding systems set to 'binary, > a.k.a. 'no-conversion, a.k.a. 'raw-text-unix. Any such process accepts > verbatim whatever they are given and does not care if the input is ASCII > or not. See > > g(elisp)Text Representations > > In particular, the process does not care what coding system the > underlying OS uses; > > 2. the internal IMAP process logic has already been tested extensively > against non-ASCII input. This kind of input was being passed to that > process all along inside > > UID SEARCH CHARSET UTF-8 "<string>" > > command. Sorry, I don't see how this could be TRT. If the process's coding-systems are 'binary, it means that the calling application needs to encode and decode the text it sends/receives from the process. Otherwise, we risk problems, at least with some locales. The fact that we use "UID SEARCH CHARSET UTF-8" probably means that the string should be encoded in UTF-8. Using 'binary without explicit encoding/decoding will only work as long as all the characters in the string are within the Unicode 0..#x10FFFF range, which is in most cases true, but not always. So I think either nnimap-open-connection-1 should set the encoding/decoding to use UTF-8 instead, or when we actually send strings we need to encode them by UTF-8, and where we receive strings we should decode. Aren't other clients of nnimap.el call encode-coding-string and decode-coding-string on the strings they send or receive from IMAP servers? Or maybe gnus-search is its only client? I hope the above makes sense, because I know nothing about nnimap and gnus-search-imap-search-command, and this code survived unaltered since 2010.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 10 Sep 2026 07:51:34 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 10 03:51:34 2026 Received: from localhost ([127.0.0.1]:51359 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x4ZZG-0007PM-4W for submit <at> debbugs.gnu.org; Thu, 10 Sep 2026 03:51:34 -0400 Received: from mail-wr2-x10.google.com ([2a00:1450:4864:30::10]:45025) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x4ZZD-0007P8-NV for 81684 <at> debbugs.gnu.org; Thu, 10 Sep 2026 03:51:32 -0400 Received: by mail-wr2-x10.google.com with SMTP id ffacd0b85a97d-482f6356256so1253479f8f.1 for <81684 <at> debbugs.gnu.org>; Thu, 10 Sep 2026 00:51:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789026685; x=1789631485; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language:subject :cc:to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=5L42QrW9Xn8IpM8cu9c4KV5RTnF84Au5gMA++iUbMUs=; b=FrD9q6VGhLdurb9ExYPTXwqZjW9qu2qAz4aU30xeM2DlxJe+s01Pj+/pF9Na8GZDWz 3MXX6hyrShvJx88OGNrpGJLw22+iweoYYjDfKEVRCQ90fj/4thfg1sG+bRyabi8iDgfI mMSNCZeQSayAk8iECCt9LkPZaWC/h1/ZT1UrrKztTJxlubrg2ap/0StPY7Pq1qllG+k2 VBj/N0poS44xSpcxVmbc8sIGI+EKEP3LqPmHczK6KxUkUfd5kIxIzo9OaR2x5ZWI+aLY 87WmuaQlXBaNRRDgCv8r+1ilNoTICLIMhweY5O8EVaKlCxZ3MYhC3GZjyqnhla6PGlw9 xDrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789026685; x=1789631485; h=content-type:mime-version:message-id:date:content-language:subject :cc:to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=5L42QrW9Xn8IpM8cu9c4KV5RTnF84Au5gMA++iUbMUs=; b=gU4h1+An1RQEm4nmnwRcOtZcqudggBQeDQ11ocxlFIEwCCH1RdCqhKvlQRQVyaoCNl Npp036EBgMuTLZOLhgx9uMcrmz+TdphZ1pYTGmyaSpbrNzarY2gtY5N7+nR1EMYoOtn7 0RezDShzmlTe0wFhKxclwVL+GsdS5pfxZ0/fH4szoaty6TOje3cuK5ptnh7MJxf+ozza ABKtLfsi+A4Xkx+sCs0HDExEgoVg/DVOA+Vf91L+Yp8Cv8Ym5hj6n6kNZvNhfaIoFurL BnmpbQ3syQLxsRFvWXD5O9KvbAdmT0OPKDF3isscAqhMGFD9ZURrqxctYb5eyyakBGoM YpUQ== X-Gm-Message-State: AFuF++l8E86B9hR+HuvbdEm/SGBHwsxxU55LsL8wuZ3/Z1CxYZ7PE/Cz 8hUAAIf8omdImUPHys+j9U3S52MY39guI4BUtq5gs46H9c3e/MR1AXXUna0vQg== X-Gm-Gg: AYBFou3yUDaHvW7XMs5pyItSNAWPcNDAfJkTPdQvqdaRuN29mCJFwt1hvrAMfnqM0LP sq2Ai3LmPtZKcY3rFkpR0r+xWWyEJqHvuBoPthhkKGPIVXpzwM/BsBQqsaF7UYH6EpwCnm0gNJB 2EiIanq/ruRjF6unUmQVmUEd/vJQoJt+sw4J++seCZgWmFu4kln+TEA+Kqo4SmHQpxmRE2yGfja hgHf/cDqpBa59FkGDlYhay9lh/0qpaTPQ2TUubDUwho7qVYh540ybb7AzSZPTeXgU4KElM4nXpH 7Rf1qYjjXVclJFcFSaIEMh1WoRBAMA3VZyDms7uWu2Dh0ummNzynvZfqnu4+ppRNKAR3hV5wIZ0 9ws75kpX0ClmKMIlqxcCSdEoK8p9eU74L/t7hv7npiigJu6keQ/MQsHErHyRoXHq/Ly7h7CeuF0 ShJipDhvXIPBf0dsa5KvCCm6WVLo5cE/8o5lelt/zwEN0mXgpByeIiiEPhMEq+WRMi8UrEZiGTG SLAMXmifT4jDpEpPTZk4LhluwhJjxtG5RS++x0j+7TheAz3tByumGbtfe1Z0W6sXnVUatHoU2tt MQ== X-Received: by 2002:a05:600c:164a:b0:49b:8f5e:51fb with SMTP id 5b1f17b1804b1-49d26d863e3mr21463125e9.3.1789026684981; Thu, 10 Sep 2026 00:51:24 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef2b15ed9480136a80189352.dip0.t-ipconnect.de. [2003:ea:ef2b:15ed:9480:136a:8018:9352]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20db6e76sm126909325e9.3.2026.09.10.00.51.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 00:51:24 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Encoding Content-Language: en-US Date: Thu, 10 Sep 2026 09:51:22 +0200 Message-ID: <87wlsty26d.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Hi Eli, I am still testing, and before I post the new patches, I'd like to address the query string encoding issue you brought up and see if you agree. Please refer to `gnus-search-imap-search-command' in gnus-search.el and `nnimap-send-command' in nnimap.el. Based on what I've found, I believe that _no_ explicit encoding is necessary for both the query string and a text command to be passed to the IMAP process because 1. the IMAP process has its encode/decode coding systems set to 'binary, a.k.a. 'no-conversion, a.k.a. 'raw-text-unix. Any such process accepts verbatim whatever they are given and does not care if the input is ASCII or not. See g(elisp)Text Representations In particular, the process does not care what coding system the underlying OS uses; 2. the internal IMAP process logic has already been tested extensively against non-ASCII input. This kind of input was being passed to that process all along inside UID SEARCH CHARSET UTF-8 "<string>" command. Many thanks. PS Two corrections: in my previous posts, I wrote that the two SELECT statements are equivalent and that Gnus does not send binary literals out. This is what I believed initially, but now I take these assertions back.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 3 Sep 2026 03:51:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 02 23:51:51 2026 Received: from localhost ([127.0.0.1]:48883 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x1yUQ-00086V-WE for submit <at> debbugs.gnu.org; Wed, 02 Sep 2026 23:51:51 -0400 Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]:54626) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x1yUP-000857-Da for 81684 <at> debbugs.gnu.org; Wed, 02 Sep 2026 23:51:49 -0400 Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-490cf322ed0so19666145e9.1 for <81684 <at> debbugs.gnu.org>; Wed, 02 Sep 2026 20:51:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788407503; x=1789012303; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=+BrX8pIrGzpw2XBB2e2cCNnVzkiLultJqLGv+S48nSg=; b=j9+3OjAyEjlhDBfa7IZjWzToSs9H/O2nVcZoVuzoJOq/TsAdIYafZn7JrnJpQroAdq bJtcaREOIZTFYYROdKGn2PMfP6tLfsg4PqCuDwC0ioeAHuiG64aj0eSnZ3PglpLxcfqe 4otNnkntJoMCwioS69HuqxP1wFn9Odpdom2/ykjldkjkjkULFrc6UgtQOX1av+EN1fMT VfMdogvxrxigPTKvlmk6dtoyeU+bUfWjuG+RKdZclIWgXCorSXXig38aubWJTj8IhTSq LE/BGshk0KBkvScFeuZPsc/T3XXaJiFwEj6b0htAw6mJniFLTT5FKy8x4h1s6MX6o1dA krNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788407503; x=1789012303; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=+BrX8pIrGzpw2XBB2e2cCNnVzkiLultJqLGv+S48nSg=; b=bjKzt0m1dlT9qcQ8qxqiBmYZb4TGtgGBWzrMbDLy5SLr9kuu5IN+6PYuoe6bRrOD6q KTRI4bKfQEXn+ugiunzM7IVvkmSK7HLmQXdNX9kmdKI2kaYKruRHrqhpzjgK0rL7t/rd X1wxP3de08bHTDRfaKOBhZdRiY9y/3iyP0Jwrzt/4TYH2cJMdUnuqgTvoK4Ox5UfBT5g BY5ZugPsttp7WJRUMkrFUtk/DUwbDFFOBhe07hsyMXFmaLdPF4PoZuVAm9j9pXuJy6mP ENpVhPgT2EsVmkFE8IEz5qlJNRMXXG21i9ZHRyLOM1ppufMiORt2ELpZYzEmeX1I6h4I Uajw== X-Gm-Message-State: AFuF++nKtA0u0uxwi4q9MxaVXhCuFoaCDHTQSjpylWrDGCxEIGtV9954 cYmn5Y4WEU1slnkVnGCqEQ/9Pz0NVjVa7oB+adD7SxgcNVT/jqk7yv9FXfI16Q== X-Gm-Gg: AYBFou26J5XthacPDPa4ngHU6NkmkiDPXfMtedr80ki70FNNPgJNslw8/LpSjp02ffJ c4gCWE5dZj7aR7kcVxXjewZ6wIu4lKMBAwZwwtKv6+GDIqG/Cnzz+vi8v5DGHawPnmE9GU/oqkq m24KA7cG0oD2oNV3PMlxI3H0baJgt5RbqgPxniMa1f31QBlsxT+z0zFQ1DFXJOOlG66NWS+ZGQB Cuh+omeoY2yvRJpw4LdPztL+JX/DUOvYQQWnyvrvYMTNj2a7EPmsKGle0QrhnzLUsmDHFf3KLeg lHYAdKy1lH3DSjGG9qJuoNI9UdmESdnZtYOsdyRKV0dM1I2wREmCvRVU+E1vJdSH+7P9YH0kYzj 5NYGTJxVeLpMTTJhcIMJ1LMIe04tggLTCR2Cma63zwwXdWEshIPce0LBmjnrgf8aojWGWgfM3M9 CnA6AWW8LaXVqPaGrhsbHu68qHVjq+WggNkb2eXpKWVTgNzvs/VX1P6WL/8oW5mYXc64CWtngSZ wQqv1jLY/OMndQV+Y17rYHEFi94S0+Fh3S+WrKeP3qOaynwH6nZv2iLWPryS1btv+Zd X-Received: by 2002:a05:600c:154c:b0:49b:5521:785d with SMTP id 5b1f17b1804b1-49ce57f4d96mr159163415e9.4.1788407503107; Wed, 02 Sep 2026 20:51:43 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef24300f199558991b3267eb.dip0.t-ipconnect.de. [2003:ea:ef24:300f:1995:5899:1b32:67eb]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cee5d4938sm37845815e9.2.2026.09.02.20.51.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 20:51:42 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail In-Reply-To: <86mru4ain9.fsf@HIDDEN> (Eli Zaretskii's message of "Sat, 29 Aug 2026 21:25:30 +0300") References: <878q5xiz39.fsf@HIDDEN> <86h5kdb4w6.fsf@HIDDEN> <87bjakesm0.fsf@HIDDEN> <86mru4ain9.fsf@HIDDEN> Content-Language: en-US Date: Thu, 03 Sep 2026 05:51:41 +0200 Message-ID: <877bl3uh4i.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Sorry, I found some issues with the patch. I will fix them and send a new one for your review.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.
Received: (at 81684) by debbugs.gnu.org; 1 Sep 2026 04:10:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 01 00:10:17 2026
Received: from localhost ([127.0.0.1]:51077 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1x1FpA-00062w-AM
for submit <at> debbugs.gnu.org; Tue, 01 Sep 2026 00:10:17 -0400
Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]:46536)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>)
id 1x1Fp4-0005wi-7w
for 81684 <at> debbugs.gnu.org; Tue, 01 Sep 2026 00:10:14 -0400
Received: by mail-wm1-x32c.google.com with SMTP id
5b1f17b1804b1-49557167508so45469455e9.1
for <81684 <at> debbugs.gnu.org>; Mon, 31 Aug 2026 21:10:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20251104; t=1788235804; x=1788840604; darn=debbugs.gnu.org;
h=content-type:mime-version:message-id:date:content-language
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to:content-type;
bh=Hr8KkYzmBewuFJ3lt9+eTrM71eyUH/jdUEnALcQzprA=;
b=geEQsdirCeHuhtlHPGrmj29n8J9YsewxHnGOeaj1LpbXbNr30/fOKZnsBcUH/Zn8sB
GdPGVZt/3/Dc3Vs5RoXuajl7jh0q9RxdUO1Bxye/73aKo2HayyVrknbIXhSjRV6XCqXd
lgu1sDIKSV8BCFshUJE3iqPdLpX0IPXErcY7Nzw8sbTiazQck3QT4IxslkkQXmJuwT1c
0Tgd2VVscVMy8oYw9Tsmlan/7nOXNZKqBrvnByHmFIdsIFMmjKeZal1EF5osM1Q/BOHj
30L8nt2yGnAMNeR8wRHyeNOGu4te2o3FggP1fcejVWaguCFjCprG4+N0NWrt1U/87Vyc
Ovog==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20251104; t=1788235804; x=1788840604;
h=content-type:mime-version:message-id:date:content-language
:references:in-reply-to:subject:cc:to:from:x-gm-gg
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to
:content-type;
bh=Hr8KkYzmBewuFJ3lt9+eTrM71eyUH/jdUEnALcQzprA=;
b=VM/F1xmtHaiyC4rjRN6RVdG6nzX01Jcn/JyZ7cDnd8Xy0npZIuSxNiiCki8glqtBWD
kWYcwDjZ8XCLnPPfFI/mwVu9G2nULGArDhJEhqhXYWxik9eRC0+oIiQ5TAj8lhYEe4nA
7d5erK+OuyMa/499y+nzbjY2fUxrsKshGGEf/aD6gL2U5E1HuReXCH+rxGSd2MDVlTE4
B1Gf2eqyKvuhki31ukihkPnM3rwhyD8XAZqSovsebUKG+P9PayV8LVqRclcVLtSXovmj
/3PcO058a1hfs1jFX0BR0Z1NYA/yS/b/hXau0yJfDHkMrjyhkMwbLxKISt0z9plqrYp+
Pzkw==
X-Gm-Message-State: AFuF++mB8F8pmKg0qsMUhssHeVr+DaYXBSlvFqHBm/qxbOg4EJ8+biTW
tCiJ6Du/kaUE4FYgDs0y5OXMeraWxpttVaLh0lx4eFxNX7cJOmHZnFVgMca7RQ==
X-Gm-Gg: AR+sD10fsMaCeV5i4uQU0Kl5C3DvXomz+UgNQsbgO1nXktVBa/M5wp0TwxFPnOvpvGX
qIeboRI0ZtJrDSglFeIbWPrDc8d0tv68GR80z+Hy4TRtHi3cgWvlsxzABMV6KudqpD0FvzOZloc
p8Rr/7ex515Q4iCnewTKy9suVbuY7oE3Nv1aG8bPwGk8j7Cmr7WNiRBu0BJe/BmLIYjjWtL9WAP
gq6pbtZcAAaXOtmTw881KQ7O+6ud0V5PkHRUEQ6Yw0SDCFRKOiaE0HyIqQBvW9pgcNOoIRushE5
6Ko7VP+qVggG9bERRNrJj+/1VObAtp5q00R/DN+3oqSusQwuRECuQHvBkP/E4t1H6D0ASnBmNKi
rF9oUh0uymId5O5If3Td3/Qpg3xiMRBvJrwaQSpG2BuS5FYarLxHkAdp+S3A17TTDJqRhUs21Q3
lqsXuB3h0qdPcXCp2992MLlpgp8uE2T99425f0ojB1nqKkwTBikUjWTvW934gADwcUA2REFrCo9
iunNUQ0iRHLuoZ9Pnd51WyAHSRL/W/P8vNhWm4scrUgOaY9KIdDVLm2GbYzqF8mVAQ=
X-Received: by 2002:a05:600d:6402:20b0:499:a277:e8b5 with SMTP id
5b1f17b1804b1-49cdc42f448mr76319245e9.3.1788235803345;
Mon, 31 Aug 2026 21:10:03 -0700 (PDT)
Received: from nova-laptop.dobrynin.net
(p200300eaef2430693b89d385f5cba8b7.dip0.t-ipconnect.de.
[2003:ea:ef24:3069:3b89:d385:f5cb:a8b7])
by smtp.gmail.com with ESMTPSA id
5b1f17b1804b1-49b91c510c9sm212107285e9.0.2026.08.31.21.10.01
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 31 Aug 2026 21:10:02 -0700 (PDT)
From: Nickolai Dobrynin <ndobrynin@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8
support causes non-ASCII IMAP queries made against UTF8-capable servers to
fail
In-Reply-To: <86mru4ain9.fsf@HIDDEN>
References: <878q5xiz39.fsf@HIDDEN> <86h5kdb4w6.fsf@HIDDEN>
<87bjakesm0.fsf@HIDDEN> <86mru4ain9.fsf@HIDDEN>
Content-Language: en-US
Date: Tue, 01 Sep 2026 06:10:00 +0200
Message-ID: <8733vtipd3.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 81684
Cc: 81684 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)
--=-=-=
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Hi Eli,
Please see attached. I hope this patch addresses the discussions we
had. And my explanation follows.
USER-VISIBLE CHANGE.
This patch adds experimental (see below) support for non-ASCII IMAP
queries. See RFC 9755. `nnimap-utf8-native' must be non-`nil'. Setting
`nnimap-utf8-native' to `nil' (which is the default) skips the new logic
completely. The server must offer IMAP4rev1 and UTF8=3DACCEPT. IMAP4rev2
and IMAP4rev1 + UTF8=3DONLY are unsupported. The servers that operate as
either IMAP4rev2 or IMAP4rev1 + UTF8=3DONLY are treated like
`nnimap-utf8-native' is `nil'.
INTERNAL NOTES.
The patch delivers the bare minimum, just enough to get the new thing
going. It implements a _compatibility mode_ known as UTF8=3DACCEPT. In
this mode, for example, the commands "SELECT Entw=FCrfe" and "SELECT
Entw+APw-rfe" are both permitted and treated equally.
While, likely, still a few years out, the patch will stop working when
IMAP4rev1 either switches to UTF8=3DONLY or gets removed in favor of
IMAP4rev2. More is needed to make the current setup stick. At a minimum,
the UTF8=3DONLY mode must be added.
Since the importance of RFC 9755 is discussed extensively online, I'll
just bring up a few quick points here. This standard is important for at
least two reasons. First: ease of use. The mail clients no longer have
to go out of their way and rely on the so-called "binary literals" for
the transmission of non-ASCII content and non-printable data. To my
surprise, Gnus fully lacks this capability, though it's _partially_
compensated for through the use of charset directive. By contrast, under
RFC 9755, the client just packs everything into a UTF-8-encoded string
and sends it out as a single command. Second: efficiency. Transmitting
binary data no longer takes an extra roundtrip or additional handling
involving LITERAL+ to avoid that roundtrip. Instead, as before, the
client packs everything into a UTF-8-encoded string and sends it as a
single command. Very simple.
To visualize the relationship between UTF8=3DACCEPT and UTF8=3DONLY, think
of a bridge connecting IMAP4rev1 on the left to IMAP4rev2 on the
right. UTF8=3DACCEPT is the midpoint of the bridge. UTF8=3DONLY is the
right-side landing. While UTF8=3DACCEPT works "both ways", UTF8=3DONLY was
designed with the express purpose to mimic IMAP4rev2.
Unfortunately, I have no immediate access to either an IMAP4rev2-capable
server or one with UTF8=3DONLY. I will do what I can to solve that. As the
saying goes, "We will cross that bridge ...".
Even more is necessary for other RFC9755 features to start working,
including non-ASCII credentials and addresses.
Please let me know if the patch is ok, and I will send the copyright
form back to you.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=gnus-imap-rfc9755.patch
Content-Description: New patch for RFC 9755
diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el
index 2e9adeec23d..1f5312f7e36 100644
--- a/lisp/gnus/gnus-search.el
+++ b/lisp/gnus/gnus-search.el
@@ -796,7 +796,16 @@ gnus-search-nnselect
nil)
(defclass gnus-search-imap (gnus-search-engine)
- ((literal-plus
+ ((utf8-native
+ :initarg :utf8-native
+ :initform nil
+ :type boolean
+ :documentation
+ "Can this search engine handle international characters in
+ queries and message headers (RFC 9755)? This slot is set
+ automatically by the imap server, and can be disabled by
+ setting `nnimap-utf8-native' to `nil'.")
+ (literal-plus
:initarg :literal-plus
:initform nil
:type boolean
@@ -1103,6 +1112,8 @@ gnus-search-run-search
;; `nnimap-open-connection', but it's too frustrating to try to
;; get to the server from the process buffer.
(with-current-buffer (nnimap-buffer)
+ (setf (slot-value engine 'utf8-native)
+ (when (nnimap-utf8-native-mode-p) t))
(setf (slot-value engine 'literal-plus)
(when (nnimap-capability "LITERAL+") t))
;; MULTISEARCH not yet implemented.
@@ -1160,13 +1171,17 @@ gnus-search-run-search
(cl-defmethod gnus-search-imap-search-command ((engine gnus-search-imap)
(query string))
"Create the IMAP search command for QUERY.
-Currently takes into account support for the LITERAL+ capability.
-Other capabilities could be tested here."
- (with-slots (literal-plus) engine
+Currently takes into account support for the LITERAL+ and UTF8-NATIVE
+capabilities. Other capabilities could be tested here."
+ (with-slots (literal-plus utf8-native) engine
(when (and literal-plus
+ (not utf8-native)
(string-match-p "\n" query))
(setq query (split-string query "\n")))
(cond
+ (utf8-native
+ ;; RFC 9755 (3): no charset allowed in native UTF-8 mode.
+ (nnimap-command "UID SEARCH %s" query))
((consp query)
;; We're not really streaming, just need to prevent
;; `nnimap-send-command' from waiting for a response.
diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 850061ac1d9..fe5d8e5b00c 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -474,6 +474,23 @@ nnimap-map-port
"993"
port))
+(defcustom nnimap-utf8-native nil
+ "Enable experimental native UTF8 support in IMAP4rev1. See RFC 9755.
+Other IMAP versions are unsupported."
+ :version "32.0"
+ :type 'boolean)
+
+(defun nnimap-utf8-native-mode-p ()
+ ;; Does the server support native UTF8, and we've enabled
+ ;; our own native UTF8?
+ (and nnimap-utf8-native
+ ;; No support for either IMAP4rev2 or
+ ;; IMAP4rev1 + UTF8=ONLY, at present.
+ (nnimap-capability "IMAP4REV1")
+ (nnimap-capability "UTF8=ACCEPT")
+ (not (nnimap-capability "UTF8=ONLY"))
+ (not (nnimap-capability "IMAP4REV2"))))
+
(defun nnimap-open-connection-1 (buffer)
(unless (or nnimap-keepalive-timer
(null nnimap-keepalive-intervals))
@@ -597,6 +614,22 @@ nnimap-open-connection-1
(when nnimap-object
(when (nnimap-capability "QRESYNC")
(nnimap-command "ENABLE QRESYNC"))
+ (when (nnimap-utf8-native-mode-p)
+ ;; RFC 9755 (3): a client MUST use "ENABLE UTF8=ACCEPT"
+ ;; to indicate that the client accepts UTF-8 quoted
+ ;; strings. "ENABLE UTF8=ACCEPT" MUST only be used in
+ ;; the authenticated state.
+ (nnimap-command "ENABLE UTF8=ACCEPT"))
+ (when (and (not nnimap-utf8-native)
+ (nnimap-capability "IMAP4REV1")
+ (nnimap-capability "UTF8=ACCEPT")
+ (not (nnimap-capability "UTF8=ONLY"))
+ (not (nnimap-capability "IMAP4REV2")))
+ (nnheader-message
+ 7
+ "%s: to use non-latin characters in queries, \
+set `nnimap-utf8-native' to non-`nil'"
+ nnimap-address))
(nnheader-message 7 "Opening connection to %s...done"
nnimap-address)
(nnimap-process nnimap-object))))))))
@@ -1960,16 +1993,27 @@ nnimap-find-connection
(defvar nnimap-sequence 5)
(defun nnimap-send-command (&rest args)
- (setf (nnimap-last-command-time nnimap-object) (current-time))
- (process-send-string
- (get-buffer-process (current-buffer))
- (nnimap-log-command
- (format "%d %s%s\n"
- (incf nnimap-sequence)
- (apply #'format args)
- (if (nnimap-newlinep nnimap-object)
- ""
- "\r"))))
+ (let ((cmd
+ (nnimap-log-command
+ (format "%d %s%s\n"
+ (incf nnimap-sequence)
+ (apply #'format args)
+ (if (nnimap-newlinep nnimap-object)
+ ""
+ "\r")))))
+ (when (nnimap-utf8-native-mode-p)
+ ;; RFC 9755 (3): the server MUST reject any octet
+ ;; sequences that fail to comply with the formal
+ ;; syntax requirements of UTF-8.
+ (setq cmd
+ (encode-coding-string
+ cmd
+ (select-safe-coding-system cmd nil '(t utf-8))
+ t)))
+ (setf (nnimap-last-command-time nnimap-object) (current-time))
+ (process-send-string
+ (get-buffer-process (current-buffer))
+ cmd))
;; Some servers apparently can't have many outstanding
;; commands, so throttle them.
(unless nnimap-streaming
--=-=-=--
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 29 Aug 2026 18:25:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Aug 29 14:25:42 2026 Received: from localhost ([127.0.0.1]:58336 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x0NkL-0002l7-MX for submit <at> debbugs.gnu.org; Sat, 29 Aug 2026 14:25:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38108) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1x0NkI-0002jp-0S for 81684 <at> debbugs.gnu.org; Sat, 29 Aug 2026 14:25:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1x0NkC-0001ug-Ix; Sat, 29 Aug 2026 14:25:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=uDSq/SdhtCIwRI1sWQe3O7lVr+2VmK17YjJWnmZOJVk=; b=B/2334vlvErb KZrJ594hl74XetkrChfTXwjPo3Zdn3ajgcUXXFsYYxpRujDwD9ecBwXq0xU1SRDCI1IAmF48Am2w9 1z7tiKINa7iYDofL4NqdQXqg1icg1UeuvZ9Qz4zOYeZUq7je2NN+/RoMN3HVE4mWH6gTzbBRBx3cd Y2txS1kFDWZ277PjmHUpeCJWGbrf0eqFMARyokUpUGRBksYip/6LdAu2e9d57eqG/W1HMBzKbMdxZ VEVRGkXfkHQiv9GwBuTjQvxTsDnz5YCKKIX9x6o5y68eq4pNWVanMPbL0Hqj+ypXJzHZGNtvYRwqX BD0vKsvlyCD6NdRAGa9g5A==; Date: Sat, 29 Aug 2026 21:25:30 +0300 Message-Id: <86mru4ain9.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <87bjakesm0.fsf@HIDDEN> (message from Nickolai Dobrynin on Sat, 29 Aug 2026 19:36:39 +0200) Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail References: <878q5xiz39.fsf@HIDDEN> <86h5kdb4w6.fsf@HIDDEN> <87bjakesm0.fsf@HIDDEN> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Cc: 81684 <at> debbugs.gnu.org > Date: Sat, 29 Aug 2026 19:36:39 +0200 > > Hi Eli, > > >> As the adoption of RFC 9755 grows, some IMAP servers end up supporting > >> non-ASCII queries exclusively through the native UTF8 capability. This > >> patch is the first step in adding this capability. Non-ASCII usernames > >> and passwords have not been tested for. Neither have non-ASCII headers. > >> > >> (setopt nnimap-utf8-native t) > >> > >> enables this extension, while > >> > >> (setopt nnimap-utf8-native nil) > >> > >> (the default) bypasses it entirely. Likewise, with non-UTF8-capable IMAP > >> servers, GNUS acts backward-compatibly, and `nnimap-utf8-native' is ignored. > >> > >> Since enabling the UTF8 capability requires a special command at start > >> up, the start up time increases with each authenticated user. > >> > >> PS Surprisingly, with this extension on, Gmail becomes fully > >> UTF8-searchable. > > > > Thanks. This patch is large enough for us to need a copyright > > assignment from you. If you are okay with signing it, I will send you > > the form to fill and the instructions to go with it. > > I will be sending in a new patch. The new patch will reflect the > discussions we had, as well as my improved grasp of the subject. > > In the meantime, you can send me the form. Form sent off-list.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 29 Aug 2026 17:36:51 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Aug 29 13:36:51 2026 Received: from localhost ([127.0.0.1]:58237 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x0Mz4-0005o5-Kn for submit <at> debbugs.gnu.org; Sat, 29 Aug 2026 13:36:50 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:51665) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1x0Mz1-0005n3-My for 81684 <at> debbugs.gnu.org; Sat, 29 Aug 2026 13:36:48 -0400 Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-49ccfbe062eso2520655e9.3 for <81684 <at> debbugs.gnu.org>; Sat, 29 Aug 2026 10:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788025001; x=1788629801; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=a+v1D5ps2x1/vxRH/F3c4v4jhxw0dpzU8K/aPPDy6jA=; b=FijaaT/o6bfz72XaWG9DZ/Q5DdhWrhSeojL4R5mxDMvnrFXeo2inJc+jAzttbkgMls AkUS0/rfDXH+7zAgiFBrGGyg7W2Y9QJ9gJnUjg3eiCpC9RUsgUAelCeCLrQFU2puNi2d j3/3RoWm2ignh0QNz03cR383CY20QrmnN7lqM+Z76ck4zPyK7FLvEENhalLUK1v5IcIP LK0V0ouEFWnUqgJLKlr2dRtec/yWBC/cwohUU+eom4MGF56p6iilFRE5qrBwqp04N2VP Hoq8X76oNu6JqzteH+tPfO6aVvsaxGpxDeBGbuhOp+S1sIQyw20QpYfIGXeS+Xs8LMgW Me9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788025001; x=1788629801; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=a+v1D5ps2x1/vxRH/F3c4v4jhxw0dpzU8K/aPPDy6jA=; b=S7nesHf7dDKlWD5E+oCrF6iGTGc5SJYp6fWMgPbAXcjrEoR9ufm/kLziO3NVD2R1LL LEEZFJvFr+G2uJL4l7V6VVYfJldFeHScewNIhrqjTwbesTISOsnrEyScamy0UAfS7sOX May84IvFG0gamzqkqdPM+6TqBeYvnOE9X/g6fiE71ZRz9GVVGtrA9giFSNSz8h0SIm+o 8GGvZGYXMgGdCH/lRWFMMQbF/ZsIrQWwYLRnjO9fPMM0xJ+wv4lkuYZazrWrzMn4yBff fU+yJ6sqiaTKKlLhd3xzB/T/Q/KM9HJznJs/NeEMChb4FpP00ANJhCBtKNG+rEZxhk1b 23uQ== X-Gm-Message-State: AFuF++lmbrnIBM8vIRFdNp2PstTeUyk6kDilG1cVqkaBoDRnXMz6dOzc dpcu+1E1KZURFyqyNJ13VfYW9JV8bOKBJTmkH0wBszX+TQ0YPA3C2Pk0s6ccJg== X-Gm-Gg: AR+sD12yxn+k1Y+yKl8SHbvhr+ArsOoCkXUUdCyIrx6PktXBFCART59NlwtiChkdQoe XWOKWmp4qbMAt0B1+dkQAHKXNMPerjRfTGZ28juQhkyTJmfn8BLY0Ew1eIm53qvzys3KeYaGFAX PRfFzXWo63X07KkwHArVjeT4x1nIm6YBNJw4pKJWGnV4stTsqGSU80kiOC1LgKcG2SX6s7VCQeR /D9UtL0mYMUzdNiKg0Rh43Z1hlMFKMu7XAP9DTj8+fJz6am7CvmL8HzLbfDJvPHs3BjHg61A/1x YzzBsDHqQv2s4r7BJmzHlSi3CPUA1PjXQ+cmvlBS8Fl3HMHY2ooCAppwWa1952qdtz9nlwRbpj7 baVCjwoB4xGUzno4o4FmunwaalQrR9b+StSRV3Lsj2b3sqAa2VWf4UGoMYkh4fBu6FStNeIUlcM SVEbDxQYg2E4idthuAlSsQ9OcwsvNSN/pCsnFkkDNx1m+HXsHYLjCDIjnFjZG0LrixTllnAvh0P cwogtNcTlHGuAJpfJ+StNPseqvjstvyajSP5eAfYKmH1SyIwDP8QxXUoNlHB3jJLLpplMdMveFI xQ== X-Received: by 2002:a05:600c:1f82:b0:493:aa0a:45ad with SMTP id 5b1f17b1804b1-49b91c19d37mr228941525e9.2.1788025001219; Sat, 29 Aug 2026 10:36:41 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef1512060562a65d3790d9c1.dip0.t-ipconnect.de. [2003:ea:ef15:1206:562:a65d:3790:d9c1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b963c3c0asm228199815e9.10.2026.08.29.10.36.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 Aug 2026 10:36:40 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail In-Reply-To: <86h5kdb4w6.fsf@HIDDEN> (Eli Zaretskii's message of "Sat, 29 Aug 2026 13:24:57 +0300") References: <878q5xiz39.fsf@HIDDEN> <86h5kdb4w6.fsf@HIDDEN> Content-Language: en-US Date: Sat, 29 Aug 2026 19:36:39 +0200 Message-ID: <87bjakesm0.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Hi Eli, >> As the adoption of RFC 9755 grows, some IMAP servers end up supporting >> non-ASCII queries exclusively through the native UTF8 capability. This >> patch is the first step in adding this capability. Non-ASCII usernames >> and passwords have not been tested for. Neither have non-ASCII headers. >> >> (setopt nnimap-utf8-native t) >> >> enables this extension, while >> >> (setopt nnimap-utf8-native nil) >> >> (the default) bypasses it entirely. Likewise, with non-UTF8-capable IMAP >> servers, GNUS acts backward-compatibly, and `nnimap-utf8-native' is ignored. >> >> Since enabling the UTF8 capability requires a special command at start >> up, the start up time increases with each authenticated user. >> >> PS Surprisingly, with this extension on, Gmail becomes fully >> UTF8-searchable. > > Thanks. This patch is large enough for us to need a copyright > assignment from you. If you are okay with signing it, I will send you > the form to fill and the instructions to go with it. I will be sending in a new patch. The new patch will reflect the discussions we had, as well as my improved grasp of the subject. In the meantime, you can send me the form.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 29 Aug 2026 10:25:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Aug 29 06:25:12 2026 Received: from localhost ([127.0.0.1]:54943 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1x0GFL-00072d-PJ for submit <at> debbugs.gnu.org; Sat, 29 Aug 2026 06:25:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60090) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1x0GFJ-00071p-1U for 81684 <at> debbugs.gnu.org; Sat, 29 Aug 2026 06:25:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1x0GFC-0007hs-W7; Sat, 29 Aug 2026 06:25:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=OauKgo2NWEe7NXbbUxPwVH8hZB+YFHqPQwIIosILlIk=; b=QUOph/ppked3 9ZiJb9xi58iJFivGdUf/hqFTjLVux9kT9yLK9DimUYTvilkNB9eez5Kba/9KdH0MOqtLlBO8NicwH le/C8EBVcw9UPNtyAMIfWP/IVwzMTEMAla9s0SCuPowf3Z9v8IpuB9wtfjj/PtnRvYYiHyAepI5j4 69azfo0J47pu0xHiwTxbiZn/hV3X2Y5/rOdtN7ss/ZKohUg0eQyokM1IN9FJYRxmnNIB5Y1dv7cTo j5gZ7xaBziIqv4QjE6y37IT3DHmtqArVByshWDoa9i2sz/2LC9J2EBJrE1+gAbPc6nACKeSlHNw72 8iUUi3hn3fFltv32Y9OOgw==; Date: Sat, 29 Aug 2026 13:24:57 +0300 Message-Id: <86h5kdb4w6.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <878q5xiz39.fsf@HIDDEN> (message from Nickolai Dobrynin on Sun, 23 Aug 2026 00:12:42 +0200) Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail References: <878q5xiz39.fsf@HIDDEN> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Date: Sun, 23 Aug 2026 00:12:42 +0200 > > As the adoption of RFC 9755 grows, some IMAP servers end up supporting > non-ASCII queries exclusively through the native UTF8 capability. This > patch is the first step in adding this capability. Non-ASCII usernames > and passwords have not been tested for. Neither have non-ASCII headers. > > (setopt nnimap-utf8-native t) > > enables this extension, while > > (setopt nnimap-utf8-native nil) > > (the default) bypasses it entirely. Likewise, with non-UTF8-capable IMAP > servers, GNUS acts backward-compatibly, and `nnimap-utf8-native' is ignored. > > Since enabling the UTF8 capability requires a special command at start > up, the start up time increases with each authenticated user. > > PS Surprisingly, with this extension on, Gmail becomes fully > UTF8-searchable. Thanks. This patch is large enough for us to need a copyright assignment from you. If you are okay with signing it, I will send you the form to fill and the instructions to go with it.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 23 Aug 2026 16:49:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 23 12:49:53 2026 Received: from localhost ([127.0.0.1]:38796 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1wyBOL-0000gk-BK for submit <at> debbugs.gnu.org; Sun, 23 Aug 2026 12:49:53 -0400 Received: from mail-ed1-x52b.google.com ([2a00:1450:4864:20::52b]:43338) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1wyBOJ-0000gO-Oc for 81684 <at> debbugs.gnu.org; Sun, 23 Aug 2026 12:49:52 -0400 Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-6a156627e22so6768934a12.1 for <81684 <at> debbugs.gnu.org>; Sun, 23 Aug 2026 09:49:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787503785; x=1788108585; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ffub4K2ZWt6/B+GylOjXa5Gpz3IP1jBgqvzuW4NAiqY=; b=c6AicsoY5CPeGHMAInA8UFNuLcRXvdtrrninciI8pxSmnLtwitImZEE6FVxD/BSUsL HlTKB5k6W6V8irMMbS6X9qAs/I2VE5YMBh+g0E00EU9mw3O4jUwoOu3ulGoIDdVyMzAL ZEeQandh9onkz5/7IlUHtcR/oUI1a/b1SJIr5GzdQ+TXFyEsB8av20qiz+5BzQtJLb0i SndqwzRhTc6CN0sYn5cA7RdaJgsxOkbb38RHN8dJuHXKkxGyUwyfocZLXxMLrsloRcuW wjMOiPf5p7uj5FO5Yxjzfm46EMPcUCEc6LmpTYApNK46oDa0IJKpzsEqBq9Jsvd+ktU/ bCxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787503785; x=1788108585; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ffub4K2ZWt6/B+GylOjXa5Gpz3IP1jBgqvzuW4NAiqY=; b=sByyOCwKjP3R4yZp4coGt8TQH/JoJath5wDKbi4W4ikcl/RfiHBdA5zZrYtTzteIzU cSSeLwDKNdp+FztDJMuThITHoRoJCHXF1nYs7bR+olOOEORUId9DHGIvOo1x1PnFxk7w TqMAyxpNsdHXCSQXJuOsTIzmcA2cBtnQL6aQv/8tufNw5Jl5qmTjCFPvDiMOcwdAzqMf +S4gEPUh9i3bGuDY6mwE1x7maY3B0V8f3YiOfpNP6EOYSMSOiruF9LKHOSuKV+j8WyYK HxiI3EXzM1eON/PeT0Ve5TRJ34dZF9l6pmub/mIT27iPaayzLnFv8eZB/KCOiNA/9F5P EeKg== X-Gm-Message-State: AFuF++kSxdyT7J9OnvTAzRsOdydF0JcdCJMYIVvqZy3ieuOPCIWNUOnc lbkERrY209a3zTbklnh9ngWgsCi2fcqp4aTia1HhvHmcVGLWwDiuQqBDlDtwvQ== X-Gm-Gg: AR+sD13ZooXhspRGye+7GmMe123y3uZp3BocYlv0IkfF9QwhWGMr16RBVh6vgujjjE9 VoDfr90IYquwvBT4rtPJ/2pB9pcdPMIoH07EQ0ixU9LqwjWvP2BGZ6Dam+tDnOqhR6RhZG6Bwj7 scaschOdUjCg/ZSqWc1Z0AxPV7+aFL0HKH+XBWCdDitunafXUS5YcIER7HxeH20CMuh/umOrPkr SvOV6BMRGsa6tBYPwcIch2CMj7DToCPc8iSWRvYkQnXHYpfoGF+8oQg09GCjd7hCtGCUj2WpQnX wJt8vCPouANUpbgvsLsWLu3+2bKJTxEMJSDZjRPxco8ifGIq2IFMH6RG3Q/sIYSpwQ1H1BbGgAv qvADBVKnRI34j44/4cBTIt9eagtGyFFkFQ6yuamj8ycBkup4qWMvyp1m1um+d2uIaK0/zHCz50L /2/3qZ/znBPoRsKUXYaiddd9jHnlMdrk5TLUePuau1rBv55Xy/8VPw462yCoNBw22A0ljB9M/VX D7z0e1yDdrGqeRswLVIgEsgBH0kz+4P8x45U3hYppy+bGZWbGLTRB5V3Pcn57m0HjbRq08mH/Mr PaQ= X-Received: by 2002:a17:907:8b99:b0:c20:ec9c:441a with SMTP id a640c23a62f3a-c244d76835fmr2652824166b.12.1787503785404; Sun, 23 Aug 2026 09:49:45 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef0baf414e9da8a182ec9f35.dip0.t-ipconnect.de. [2003:ea:ef0b:af41:4e9d:a8a1:82ec:9f35]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c249629a901sm949848566b.25.2026.08.23.09.49.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Aug 2026 09:49:44 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail In-Reply-To: <868q5xw5sz.fsf@HIDDEN> (Eli Zaretskii's message of "Sun, 23 Aug 2026 12:21:32 +0300") References: <878q5xiz39.fsf@HIDDEN> <86ld9xwipb.fsf@HIDDEN> <87pkz9qsdh.fsf@HIDDEN> <86fr05wcst.fsf@HIDDEN> <87y0dxs1pq.fsf@HIDDEN> <868q5xw5sz.fsf@HIDDEN> Content-Language: en-US Date: Sun, 23 Aug 2026 18:49:44 +0200 Message-ID: <87o6es6atz.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) >> > In any case, in what sense is this patch "the first step in adding >> > this capability"? >> >> In its current form, the patch adds support for UTF8-encoded IMAP queries >> against UTF8-capable servers, to replace the previous CHARSET-based >> mechanism. The new mechanism is a part of IMAP4rev2 standard. > > So this is "first step" in the sense that you intend to add more > IMAP4rev2 features, is that right? I may add support for non-ASCII credentials and e-mails addresses later. I have no way of testing for those at the moment due to lack of server support. The good news is that some of the features already in GNUS have been subsumed into IMAP4rev2. No extra work is needed there. In IMAP4rev1, they are separate opt-ins, but in IMAP4rev2, they became part of the standard.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 23 Aug 2026 09:21:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 23 05:21:43 2026 Received: from localhost ([127.0.0.1]:35195 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1wy4Oc-0005u9-On for submit <at> debbugs.gnu.org; Sun, 23 Aug 2026 05:21:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50936) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1wy4Ob-0005tm-8K for 81684 <at> debbugs.gnu.org; Sun, 23 Aug 2026 05:21:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1wy4OV-0001Gq-On; Sun, 23 Aug 2026 05:21:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=8sUgoLDn7P5qC0/VBtvXM0YAd8nYmYs6vwRZmacPE48=; b=qK14M5ocvyqA ezZ+Tyi48ncgz65HO93Teh3D1fjW0X7nywH0367oWmFYTPe7vzomlGz56UBV9RNwS2hnWG7UgVxQQ 9h4P2xQbypLXevQn+hJfv7aKp42E161pEHBxliFZkrxj30pACizhHpx73yyNaFGqitttIzdXM30cZ +ievoTtvxLRv0pu08rIFXxHeFjdIGGq3zXyr1tf2F3435pDTwI1pzK/Qb7Q6vs2TA2V7NkZYsjx+v Ibv9ldy9jRMONw3ubYh/CzgYTuj488g7d+Hhz9eR5Y9tfP2bHj6C2oJ+F2+KVe0QKTYsvD7QGP16e C8NF2hazsgGpY7bI/1rJ5Q==; Date: Sun, 23 Aug 2026 12:21:32 +0300 Message-Id: <868q5xw5sz.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <87y0dxs1pq.fsf@HIDDEN> (message from Nickolai Dobrynin on Sun, 23 Aug 2026 10:03:29 +0200) Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail References: <878q5xiz39.fsf@HIDDEN> <86ld9xwipb.fsf@HIDDEN> <87pkz9qsdh.fsf@HIDDEN> <86fr05wcst.fsf@HIDDEN> <87y0dxs1pq.fsf@HIDDEN> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Cc: 81684 <at> debbugs.gnu.org > Date: Sun, 23 Aug 2026 10:03:29 +0200 > > > Well, previously the UTF-8 encoding of a query was not a requirement; > > now it is. So I'm not sure I agree with you. > > Indeed! I will look into this. Some non-UTF8-capable servers respond to > GNUS' non-ASCII queries properly. This led be to (falsely) believe that > all the queries are UTF8-encoded. Thanks. Please tell what you found, and add any additional patches if needed. Then we can install all of this. > > In any case, in what sense is this patch "the first step in adding > > this capability"? > > In its current form, the patch adds support for UTF8-encoded IMAP queries > against UTF8-capable servers, to replace the previous CHARSET-based > mechanism. The new mechanism is a part of IMAP4rev2 standard. So this is "first step" in the sense that you intend to add more IMAP4rev2 features, is that right?
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 23 Aug 2026 08:03:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 23 04:03:40 2026 Received: from localhost ([127.0.0.1]:34636 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1wy3B5-00066S-Ez for submit <at> debbugs.gnu.org; Sun, 23 Aug 2026 04:03:39 -0400 Received: from mail-ed1-x52e.google.com ([2a00:1450:4864:20::52e]:53445) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1wy3B2-00066B-KG for 81684 <at> debbugs.gnu.org; Sun, 23 Aug 2026 04:03:37 -0400 Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-6a1542cdb53so2993748a12.2 for <81684 <at> debbugs.gnu.org>; Sun, 23 Aug 2026 01:03:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787472210; x=1788077010; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=I0wEiFd/kppPnrvXGrm/VcWfixPdVSkl4CSrnPuyrP0=; b=OQQbBtSGcJ65JwVpJWjDtkXnnmMpPSD/jznA1yj+XlsPAj2CIEF1LHwXX3phrNwqVv xseCdrmyzTOxkJfNG1iv6A8n+jA9kXq0lPaUQPHjITw78aGxPt0BeDSH508U1BEedj8M 6FheiEhLivT+uQ1+ULnFww09X72JHdxJKtloyLnvbBnImrmGxOvJ7m7RGhWPGLFXayj7 m5SfWavHB/y7lqjhrm0KqeSmgTTP0RmpQTl/T/wgo2Nspgy8jaDTXXROJTgHSe5DuSWc pcPybVIC2SGeZpRZeF22TAC2RieGy73RDEN4NxklCxDkMJcXN5Idunma04EOrcIwjnuL KckA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787472210; x=1788077010; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=I0wEiFd/kppPnrvXGrm/VcWfixPdVSkl4CSrnPuyrP0=; b=qxREs4Jl6sRjJ/UL9h6mJE3goIWPj/TZ8U6b57sMJeM0XVUk/XQKE8ylifZAgb+ZmC xPYEQXs+8DiW58aypIpDt6aazVkZVtALW+9aNakPvb45F/34YS5ZwBbHYBmWVCdlB1FT XQNMUgNyerPn66feFAR1HhFRMI4h7R7NFHbmnBFv3zmoILKytKcwaTxyjpdk5NCXP0uA jQh6WUY9GJrc94yc3RC+zeEBA+R+aMtZsCfDJoq9Eru3XFHlsNwFTTCYUkrmYrZoTe86 v890OX/EIc9rB8kDiwlhLMU89TdOiMA82vDFY8u+HMCaFc+7oqk9tB2AEw+M9ylE2Kr9 cD1w== X-Gm-Message-State: AFuF++kqZ8yhZrQFxesNqG2cZMc+t031kJDe8leIA5Bbl4tUQ1tXmhAL w1O0Lp3gGU3DdlF+pmtS/3jL3Yb0H9HWpsTxFygx6dRyk0Qm8SHtsO7MtEC+fw== X-Gm-Gg: AR+sD13d+fFc2DOZQrwCkmhBX07uhYZ2d3Y0HQHNSvbE5Amgwt8/s1jXFirwJZJci7W iHbsecWiOj0FE1sqJ83lCmmy+Oqr6rY4fo+LsulbgKTTlXuB+uVwJ+tgu4iLWVupaIm+mVFOKK3 I2gRsQ9dEa4M2KDh92Uf5qoCPnv0ceCu+p17Zw3oguAHP2aVkyuC/wcV+nOCyCvWKJQrNoNwUT+ EtHrWyYRqXGfxaB4neNHyGjQ/vGWZK+WBUcmmtSNMkecnUmSKtaYkXigXgaAQGCyDknTeDEmHIY kyCI1SLtbzWrVlVdQO3IUDptwRqSx4B34PReDnqalNrhNBC8FV4e1I+eLaMzRKr0l/Qlpx33uLl 0pzlnkQLw5z9ktB6NheRKYzqmbUBtJI+sJjUs9MA7l0Xo7+iVYAvqj6LFBvGbyNONQkzJCcos3c Aq70Zkc5dXRlIUb+Z2yYEMOeAhU67mfJUN5p95uI+gp2k+/OBdqrM0QsFi6AKVSLwJ3Q5XJykmI kfdQ7Vf+kaXt43WM1jGwx5IkvigspxXP1snsBFNyCWNmMMw6Emd4lqZ3CJ81T5vfs3lNw== X-Received: by 2002:a05:6402:434a:b0:6a3:691a:9d82 with SMTP id 4fb4d7f45d1cf-6a582af951cmr12727549a12.3.1787472210217; Sun, 23 Aug 2026 01:03:30 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef0baf414e9da8a182ec9f35.dip0.t-ipconnect.de. [2003:ea:ef0b:af41:4e9d:a8a1:82ec:9f35]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-6a59e025609sm2338619a12.12.2026.08.23.01.03.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Aug 2026 01:03:29 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail In-Reply-To: <86fr05wcst.fsf@HIDDEN> (Eli Zaretskii's message of "Sun, 23 Aug 2026 09:50:26 +0300") References: <878q5xiz39.fsf@HIDDEN> <86ld9xwipb.fsf@HIDDEN> <87pkz9qsdh.fsf@HIDDEN> <86fr05wcst.fsf@HIDDEN> Content-Language: en-US Date: Sun, 23 Aug 2026 10:03:29 +0200 Message-ID: <87y0dxs1pq.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) >> With the new extension on, the query travels the same path it would have >> otherwise traveled, had the extension been off. So, if the query was properly >> UTF8-encoded before, it is properly encoded now. BUT: if the encoding >> mechanism had a bug, that bug, surely, will remain. >> >> In that sense, the encoding issue is orthogonal to this feature. > > Well, previously the UTF-8 encoding of a query was not a requirement; > now it is. So I'm not sure I agree with you. Indeed! I will look into this. Some non-UTF8-capable servers respond to GNUS' non-ASCII queries properly. This led be to (falsely) believe that all the queries are UTF8-encoded. >> >> Non-ASCII usernames and passwords have not been tested for. Neither >> >> have non-ASCII headers. >> > >> > If you are saying that this patch is not intended to enable the >> > feature completely and was not tested as such, then I would suggest to >> > wait until you have a complete series of patches which together enable >> > the feature, and install it then. Installing partial features which >> > aren't supposed to be usable yet (if this is what it is; apologies if >> > I misunderstand) is not necessarily a good idea. >> >> Since the traditional IMAP protocol does not allow non-ASCII >> credentials, GNUS in its present form cannot work with such servers >> anyway. With the new feature installed, this limitation will remain. >> >> The e-mail servers that support UTF-8 _and_ allow non-ASCII usernames, >> passwords and addresses are hard to come by. (Gmail is UTF8-capable but >> only accepts ASCII credentials; all the rest have no UTF8 support in >> IMAP, AFAIK). It would be welcome if somebody could let me access a >> testing account on such a system, so I could do more testing. >> >> In summary, I would expect the extension in its current form to be very much >> usable. I added the disclaimer for transparency only. > > Did you try the patched code with queries that include non-ASCII > characters? I mean the query text itself, not the username or > password. Yes, of course. I ran my tests against Gmail and a different non-UTF8-capable server. > If you tried, please make sure that it also works in a > locale whose codeset is something other than UTF-8, to make sure we > don't have bugs somewhere. I will look into that. > Or are you saying that the actual stuff we send in a query is only > allowed to be ASCII, No. Some old IMAP servers support the CHARSET directive which allows for non-ASCII queries. The problem is that (1) the CHARSET support is not uniform and (2) newer servers occasionally don't support non-ASCII queries at all except through the new standard. > and the "ENABLE UTF8=ACCEPT" command is purely > declarative? It is declarative in the sense that "the IMAP server MUST NOT send UTF-8 in quoted-strings to the client unless the client has indicated support for that syntax by using" this command. > In any case, in what sense is this patch "the first step in adding > this capability"? In its current form, the patch adds support for UTF8-encoded IMAP queries against UTF8-capable servers, to replace the previous CHARSET-based mechanism. The new mechanism is a part of IMAP4rev2 standard.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 23 Aug 2026 06:50:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 23 02:50:36 2026 Received: from localhost ([127.0.0.1]:34049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1wy22O-00019C-AE for submit <at> debbugs.gnu.org; Sun, 23 Aug 2026 02:50:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52356) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1wy22M-00018z-FQ for 81684 <at> debbugs.gnu.org; Sun, 23 Aug 2026 02:50:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1wy22H-0004YU-4J; Sun, 23 Aug 2026 02:50:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=nBKcCTeiMa2qBA24gqkzn4Huxz1gueqbR6hAizy8Phs=; b=opfs/zBTag+B IoVSxNKwRXL6Kto+Jz9/5XWGsBi41YPdudFL6YP/3rPgE5jERFdxBxxktaMgH+OKEMsTHEk2/GxiG NvEBwNIbGp7CNvZLBmy+JWGQYRBYlQUzLUc8pdnzb7EzONsSuZGVVaPIH+A55OI59sBFwhRMSZwvf p64MvCvC0x9MBMU9MNgIZ45wOMriwQzj4BZU5XspPdDF9trglFkRMXZ67VK2m3dWYOn2A+f9VZUny 31MiXTOEENTK2obDIx4EzgzVcfBT49KQbQ13mEATEIT4Tn4XsbMdPje0imCWRU0n0WVIYFd2+54ST 7ZmTrLUAkWD1ylUWZrgZcQ==; Date: Sun, 23 Aug 2026 09:50:26 +0300 Message-Id: <86fr05wcst.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <87pkz9qsdh.fsf@HIDDEN> (message from Nickolai Dobrynin on Sun, 23 Aug 2026 08:10:34 +0200) Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail References: <878q5xiz39.fsf@HIDDEN> <86ld9xwipb.fsf@HIDDEN> <87pkz9qsdh.fsf@HIDDEN> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Cc: 81684 <at> debbugs.gnu.org > Date: Sun, 23 Aug 2026 08:10:34 +0200 > > Hi Eli, > > > Thanks. I know nothing about nnimap, but I expected to see in the > > patch encoding with UTF-8 in some place, or binding of > > coding-system-for-write to 'utf-8', or something similar, to ensure > > the query is UTF-8 encoded. Otherwise, how can we make sure this will > > work reliably regardless of the locale's defaults of the user? > > With the new extension on, the query travels the same path it would have > otherwise traveled, had the extension been off. So, if the query was properly > UTF8-encoded before, it is properly encoded now. BUT: if the encoding > mechanism had a bug, that bug, surely, will remain. > > In that sense, the encoding issue is orthogonal to this feature. Well, previously the UTF-8 encoding of a query was not a requirement; now it is. So I'm not sure I agree with you. > >> Non-ASCII usernames and passwords have not been tested for. Neither > >> have non-ASCII headers. > > > > If you are saying that this patch is not intended to enable the > > feature completely and was not tested as such, then I would suggest to > > wait until you have a complete series of patches which together enable > > the feature, and install it then. Installing partial features which > > aren't supposed to be usable yet (if this is what it is; apologies if > > I misunderstand) is not necessarily a good idea. > > Since the traditional IMAP protocol does not allow non-ASCII > credentials, GNUS in its present form cannot work with such servers > anyway. With the new feature installed, this limitation will remain. > > The e-mail servers that support UTF-8 _and_ allow non-ASCII usernames, > passwords and addresses are hard to come by. (Gmail is UTF8-capable but > only accepts ASCII credentials; all the rest have no UTF8 support in > IMAP, AFAIK). It would be welcome if somebody could let me access a > testing account on such a system, so I could do more testing. > > In summary, I would expect the extension in its current form to be very much > usable. I added the disclaimer for transparency only. Did you try the patched code with queries that include non-ASCII characters? I mean the query text itself, not the username or password. If you tried, please make sure that it also works in a locale whose codeset is something other than UTF-8, to make sure we don't have bugs somewhere. Or are you saying that the actual stuff we send in a query is only allowed to be ASCII, and the "ENABLE UTF8=ACCEPT" command is purely declarative? In any case, in what sense is this patch "the first step in adding this capability"?
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 23 Aug 2026 06:10:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 23 02:10:43 2026 Received: from localhost ([127.0.0.1]:33836 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1wy1Pn-0007IV-5y for submit <at> debbugs.gnu.org; Sun, 23 Aug 2026 02:10:43 -0400 Received: from mail-ej1-x62c.google.com ([2a00:1450:4864:20::62c]:45085) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>) id 1wy1Pl-0007IF-Px for 81684 <at> debbugs.gnu.org; Sun, 23 Aug 2026 02:10:42 -0400 Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-c16794450aeso342782366b.2 for <81684 <at> debbugs.gnu.org>; Sat, 22 Aug 2026 23:10:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787465436; x=1788070236; darn=debbugs.gnu.org; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=XDYiyTfKlzqHhpCKsxlJoooFJD2KLN4IofjudPkkjN4=; b=oJQ6imi1f/a94L5sY8zP/ODzmgsQK+B/0CJbuOrBj0/Y7soFnJW0FnKg61M6YnPe7E TsXhunST7P9Uvlg/9Tm8sVE/tD8/BBwZlwqaeLOG5iq0vOqUDzmH4VP4K65+adJ/U8ms in1G7wwgcIrZy7f1wriIltr7eeQVu1FIeYIIpmp64mXR3TrnUBGqIR5hUw0H2tpqEe8b a3eDjxvaPDCixVefdnyzOGkbJTol1XqMjzXc0Sj8O55O0D9You+nFMNu9EE+tY1Y8lrD 5JbBaBS7pxu7mzTTZubg0k5yAlwxCVIu/AJc0P4jupMUeN4XZWmjnNhymU/El7IzZJ0r /pFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787465436; x=1788070236; h=content-type:mime-version:message-id:date:content-language :references:in-reply-to:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=XDYiyTfKlzqHhpCKsxlJoooFJD2KLN4IofjudPkkjN4=; b=V+WCiUG0XGqRkfbDLeQ+7dKRzgdoKOGmw7/hdc2rrcBjpTCh7i2jxy4I0FastpBA3B E684zVt4mqXFrkYbPvzEsGBifSEQ0udGrSIi1fKXu+O6Dwz7KbtgzZAyuj7EtTu7eVIG Zcsnle+PNZ0FzdKSnngtQ/LrGydPbCgJU3pfA8LLVjQvulid63e2RWEmNKkWYWUhTXz0 LlDFRWS7vG8vmAvxBqDt4dlhpTnl0u9E/9anT1Oblc/3vnYQ/cO6C/gzCy88WqOQFMi3 O9q4IWhsttoL2CF9PitMbBIvlapg8zfW11/hwiGcWxMsvMrVxMsr6Vbym8YDRlaUwzhM tqOA== X-Gm-Message-State: AFuF++kjyDzgkZabDmkhawdtyj2IWbwYBfIsobwxNy5mXA8j1ogjhfme JsHo+MO5JMueVZtrQcAHzOb8e4tmWBEFCStqCylLh7O9zbuE9hXIt4FXpie84Q== X-Gm-Gg: AR+sD13Eh/afX23vUaARpS7ODw6+aZLYZYkktsM5CgEmJzyEKAFOOtaoqtBjvChgJzN KpoxZNzrvUJ4RJkMe14tWXdBLzvQKIn3mGg25wP5fVFUXx6DxXu8pkFlZT2AdiISY23l0/3nezR IgegsklcdEFm9fYogcoAGyynIN7Awin6WAu3FsIVxZWjFEO7qMnnUuDpF7PFZZFa9zyiKhA0V8j netDCERNXMhjUg4JkgiUm0Tj9iNmWelfDlWsWifqjl6UgrkBLEq+0EPpKsJWl6pbvShvH8HGhH3 09eQpTSCVhOMbvHXYOX5fdkNyw51WRzY7CgBcjTpo6PPAGr4tuxVaxFIZaxeyQ7AzUTHf0JYokL QRWAM+iCjoN72RkDhHdPplZmOxHf5UABNlmy8ZQgRf3oBg1HFqYcfMnlrMEkIjygx0A0BA3IBMO BHUZbpFREeHTfVZfUsTyMm0T/8ebljk1FFQeMlNWkJSOmWZklL3y3QAxvuFCsHl7uuHc+7eJSKM Kv3+dEafIwZ1EvHaIXrARze6DJCgmzP7Rk7A6YXngdwGjuUqQ1iv5dxY8CMEpTkraBcgfsAu99r Eg== X-Received: by 2002:a17:907:e14b:10b0:c19:80cf:4c75 with SMTP id a640c23a62f3a-c246a6ab42dmr1666356666b.20.1787465435360; Sat, 22 Aug 2026 23:10:35 -0700 (PDT) Received: from nova-laptop.dobrynin.net (p200300eaef0baf414e9da8a182ec9f35.dip0.t-ipconnect.de. [2003:ea:ef0b:af41:4e9d:a8a1:82ec:9f35]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c249606b161sm674058266b.11.2026.08.22.23.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 22 Aug 2026 23:10:34 -0700 (PDT) From: Nickolai Dobrynin <ndobrynin@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail In-Reply-To: <86ld9xwipb.fsf@HIDDEN> (Eli Zaretskii's message of "Sun, 23 Aug 2026 07:42:56 +0300") References: <878q5xiz39.fsf@HIDDEN> <86ld9xwipb.fsf@HIDDEN> Content-Language: en-US Date: Sun, 23 Aug 2026 08:10:34 +0200 Message-ID: <87pkz9qsdh.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.0 (/) Hi Eli, > Thanks. I know nothing about nnimap, but I expected to see in the > patch encoding with UTF-8 in some place, or binding of > coding-system-for-write to 'utf-8', or something similar, to ensure > the query is UTF-8 encoded. Otherwise, how can we make sure this will > work reliably regardless of the locale's defaults of the user? With the new extension on, the query travels the same path it would have otherwise traveled, had the extension been off. So, if the query was properly UTF8-encoded before, it is properly encoded now. BUT: if the encoding mechanism had a bug, that bug, surely, will remain. In that sense, the encoding issue is orthogonal to this feature. >> Non-ASCII usernames and passwords have not been tested for. Neither >> have non-ASCII headers. > > If you are saying that this patch is not intended to enable the > feature completely and was not tested as such, then I would suggest to > wait until you have a complete series of patches which together enable > the feature, and install it then. Installing partial features which > aren't supposed to be usable yet (if this is what it is; apologies if > I misunderstand) is not necessarily a good idea. Since the traditional IMAP protocol does not allow non-ASCII credentials, GNUS in its present form cannot work with such servers anyway. With the new feature installed, this limitation will remain. The e-mail servers that support UTF-8 _and_ allow non-ASCII usernames, passwords and addresses are hard to come by. (Gmail is UTF8-capable but only accepts ASCII credentials; all the rest have no UTF8 support in IMAP, AFAIK). It would be welcome if somebody could let me access a testing account on such a system, so I could do more testing. In summary, I would expect the extension in its current form to be very much usable. I added the disclaimer for transparency only.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.Received: (at 81684) by debbugs.gnu.org; 23 Aug 2026 04:43:14 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 23 00:43:14 2026 Received: from localhost ([127.0.0.1]:33414 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1wy038-0001UM-9f for submit <at> debbugs.gnu.org; Sun, 23 Aug 2026 00:43:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42014) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1wy035-0001Ty-6x for 81684 <at> debbugs.gnu.org; Sun, 23 Aug 2026 00:43:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1wy02v-0005rA-Bg; Sun, 23 Aug 2026 00:43:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=3RdLjJZsUcWdCA/K+lkEZNegFSI6Jm1WaRgxQE/EHvs=; b=cxSHa6c6fOpl NdUXwwoH1Sqle+2k1BZxRbvysQ5d/QclnaT78NK8MrtePBTW8KzSN8KGt8m1x/TiaZoT+YFWdpxiK B2FLIHMec1e2cURtMNal3+Z1u7tCRPK1MNJ0lLDZVgAxE7NG8a1A+fupMOp+fdheLV9O/DQZFb+Av AG4J+7Ccgj3VZfPRH0lJS2sdPyoNvSGc2fcAg4HerEXPQijVf3f4Ar9YBVNDqMrSSbqbx7ydQqxsC XGZylAekaQQsukeo4hNxnUPnqisr2SgL5rmRBh3FJDoaTw6ZEOUt7ORAxgXGgqwvG8sIehTdfUx2C MwKlY/xk/fITIXdIlUkcZg==; Date: Sun, 23 Aug 2026 07:42:56 +0300 Message-Id: <86ld9xwipb.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Nickolai Dobrynin <ndobrynin@HIDDEN> In-Reply-To: <878q5xiz39.fsf@HIDDEN> (message from Nickolai Dobrynin on Sun, 23 Aug 2026 00:12:42 +0200) Subject: Re: bug#81684: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes non-ASCII IMAP queries made against UTF8-capable servers to fail References: <878q5xiz39.fsf@HIDDEN> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 81684 Cc: 81684 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) > From: Nickolai Dobrynin <ndobrynin@HIDDEN> > Date: Sun, 23 Aug 2026 00:12:42 +0200 > > As the adoption of RFC 9755 grows, some IMAP servers end up supporting > non-ASCII queries exclusively through the native UTF8 capability. This > patch is the first step in adding this capability. Non-ASCII usernames > and passwords have not been tested for. Neither have non-ASCII headers. > > (setopt nnimap-utf8-native t) > > enables this extension, while > > (setopt nnimap-utf8-native nil) > > (the default) bypasses it entirely. Likewise, with non-UTF8-capable IMAP > servers, GNUS acts backward-compatibly, and `nnimap-utf8-native' is ignored. > > Since enabling the UTF8 capability requires a special command at start > up, the start up time increases with each authenticated user. Thanks. I know nothing about nnimap, but I expected to see in the patch encoding with UTF-8 in some place, or binding of coding-system-for-write to 'utf-8', or something similar, to ensure the query is UTF-8 encoded. Otherwise, how can we make sure this will work reliably regardless of the locale's defaults of the user? > Non-ASCII usernames and passwords have not been tested for. Neither > have non-ASCII headers. If you are saying that this patch is not intended to enable the feature completely and was not tested as such, then I would suggest to wait until you have a complete series of patches which together enable the feature, and install it then. Installing partial features which aren't supposed to be usable yet (if this is what it is; apologies if I misunderstand) is not necessarily a good idea.
bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.
Received: (at submit) by debbugs.gnu.org; 22 Aug 2026 22:12:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Aug 22 18:12:57 2026
Received: from localhost ([127.0.0.1]:60323 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1wxtxQ-0003Ma-Lu
for submit <at> debbugs.gnu.org; Sat, 22 Aug 2026 18:12:57 -0400
Received: from lists1p.gnu.org ([2001:470:142::17]:39156)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <ndobrynin@HIDDEN>)
id 1wxtxO-0003Lf-GX
for submit <at> debbugs.gnu.org; Sat, 22 Aug 2026 18:12:55 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ndobrynin@HIDDEN>)
id 1wxtxJ-0004Zj-3Y
for bug-gnu-emacs@HIDDEN; Sat, 22 Aug 2026 18:12:49 -0400
Received: from mail-ed1-x529.google.com ([2a00:1450:4864:20::529])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.90_1) (envelope-from <ndobrynin@HIDDEN>)
id 1wxtxH-0006zj-4v
for bug-gnu-emacs@HIDDEN; Sat, 22 Aug 2026 18:12:48 -0400
Received: by mail-ed1-x529.google.com with SMTP id
4fb4d7f45d1cf-6a36982a875so3383222a12.0
for <bug-gnu-emacs@HIDDEN>; Sat, 22 Aug 2026 15:12:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20251104; t=1787436765; x=1788041565; darn=gnu.org;
h=content-type:mime-version:message-id:date:subject:to:from:from:to
:cc:subject:date:message-id:reply-to:content-type;
bh=8wYSIUUImHsReY0nd/wwElRoO55tHs3CMZ7feIRwkHY=;
b=W5yIuwP4lgfoPbPknakiXdpjHS4QjBUtu7QO9bqJOCyA6e9iqVDm52acEesb9ipBtm
2qwg0kwaEWFkX4dmTst1DeE7FivJQJoNwrQVJPiGbhggCUHJuI5L+nicyAjT4iWo4fVW
yiqjj8zs7P3IyD3953iNKj2HS3iTofn49Hwwjk0F6358FeBCnZubl4gUNb/5jKg8834h
jAYJuCam9GIN/fQ3ggxq6cDa2TvFJVgNDGWSyQgO9oWKmHBf8Fo1gOnd7CfbwPJpWYf2
pvHY2RUfxS/8yWiw9OtT9Z019GsbdJyJTyk6f2q8Yg46vD9fKNBdhbaEzl6KIpqdDroe
gCPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20251104; t=1787436765; x=1788041565;
h=content-type:mime-version:message-id:date:subject:to:from:x-gm-gg
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to
:content-type;
bh=8wYSIUUImHsReY0nd/wwElRoO55tHs3CMZ7feIRwkHY=;
b=sXSVJJf+pQoOLM1Z/s7e1p/5GPSSwGNu4YpkP80Jbb5Vts9YTICIdoHK6CiuLiZbo9
F51fbg4APVfKmhIpH23ca5Qv3sB/iYgDwmmUh0AHPR2ULD2Nj/bQ0cr0uU0ICJ26RK6R
iPc22BuyuCcns4xTETQVkVXE9XpgUFyDI7WDZNK7SvA1c8mXLiI2j7p68fD8jgRQdUXQ
CpHObV2g/CnhWkNH/0LcZyT4JLoc2zHoVbyxgSP6TiKsD6OlAXmahOlpvy5WENxnmBff
jeHqg0A49ZfiQsuXJHUCDALYohWepx9UqIcTB7LXX/wrftc0a+yRAaDMpCZULE2gBWkD
1FbA==
X-Gm-Message-State: AFuF++la1nX21wREx6vHNxf5JCI/biO1OPCzsOb0X3XN/Xf01Vxtglif
wR8SyKOIORX3MSAYCy3saVqXuQja4dMV3RN3QzzDk86pBD3Ez0WT8WGNzgA4OA==
X-Gm-Gg: AR+sD10pAgPJYaQPxd8xJs1vGFdmeYX+AEFd5MAIsugXhvBEaWclOT8gG1GiEDjF/aa
UwLf3p9AbTAe6//1XMJrGc8sFANFyfsm/Hoi6di8chaPDGKAcRM+3hK+oic4FdwwRD5LM+9ufBt
llSfvypCYvlbFiNCPpYiRa2ovH8ubEYDedUBsY+nkq8CEv/mJBxZKriJd2fQjmIfDzzpfF0Lna9
5j2YKHoTVjJT+3LcRZuSJv5OsujFriN7Dkm6YjdkDdkpqJPHFJz5H+i7wKJNUz/VI5LZH5/64uA
tbqXAu3wRs1ogMk07Bij4omyhkI1zhv/7iKZzxBIUI2Xvg1c8Ip8dnkpnALlFy/bpnAeyQoIeGA
8k0xm+4RfCphRTlYt7HQfQyvRHTdZEF1If6G5PP+6woJziTj8Jbao6F3rz35KpN8BiPVrqqFKGV
x742KDAwg1x0pVYXzHurNG7bUMVzofHPzBBxtMvkfHbdlpwB9rCtf1kiWy4GBvZYBcHGRNCmpDC
4ne6mgxpT8l2Bo77f6C9Pv6UKKOPiTjG5Q0QcFutPxIdEwFWKcP5SVV4Y7fyR/JJn0=
X-Received: by 2002:a05:6402:249f:b0:6a3:ebe1:c2c2 with SMTP id
4fb4d7f45d1cf-6a42f14dfacmr19841317a12.2.1787436764821;
Sat, 22 Aug 2026 15:12:44 -0700 (PDT)
Received: from nova-laptop.dobrynin.net
(p200300eaef0baf9ef01214b36494cac9.dip0.t-ipconnect.de.
[2003:ea:ef0b:af9e:f012:14b3:6494:cac9])
by smtp.gmail.com with ESMTPSA id
4fb4d7f45d1cf-6a3feec9e2esm14059172a12.5.2026.08.22.15.12.43
for <bug-gnu-emacs@HIDDEN>
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Sat, 22 Aug 2026 15:12:43 -0700 (PDT)
From: Nickolai Dobrynin <ndobrynin@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 32.0.50; Bug fix/missing feature: lack of UTF8 support causes
non-ASCII IMAP queries made against UTF8-capable servers to fail
X-Debbugs-Package: emacs,gnus
X-Debbugs-Cc:
Date: Sun, 23 Aug 2026 00:12:42 +0200
Message-ID: <878q5xiz39.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Received-SPF: pass client-ip=2a00:1450:4864:20::529;
envelope-from=ndobrynin@HIDDEN; helo=mail-ed1-x529.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: As the adoption of RFC 9755 grows, some IMAP servers end up
supporting non-ASCII queries exclusively through the native UTF8 capability.
This patch is the first step in adding this capability. Non-ASC [...]
Content analysis details: (2.0 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust [2001:470:142:0:0:0:0:17 listed in] [list.dnswl.org]
1.0 FORGED_GMAIL_RCVD 'From' gmail.com does not match 'Received'
headers
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (ndobrynin[at]gmail.com)
1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail)
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)
--=-=-=
Content-Type: text/plain
As the adoption of RFC 9755 grows, some IMAP servers end up supporting
non-ASCII queries exclusively through the native UTF8 capability. This
patch is the first step in adding this capability. Non-ASCII usernames
and passwords have not been tested for. Neither have non-ASCII headers.
(setopt nnimap-utf8-native t)
enables this extension, while
(setopt nnimap-utf8-native nil)
(the default) bypasses it entirely. Likewise, with non-UTF8-capable IMAP
servers, GNUS acts backward-compatibly, and `nnimap-utf8-native' is ignored.
Since enabling the UTF8 capability requires a special command at start
up, the start up time increases with each authenticated user.
PS Surprisingly, with this extension on, Gmail becomes fully
UTF8-searchable.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
filename=gnus-imap-rfc9755-20260822212437.patch
Content-Description: Initial support for UTF8 in IMAP
diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el
index 2e9adeec23d..e28c8b5129f 100644
--- a/lisp/gnus/gnus-search.el
+++ b/lisp/gnus/gnus-search.el
@@ -796,7 +796,18 @@ gnus-search-nnselect
nil)
(defclass gnus-search-imap (gnus-search-engine)
- ((literal-plus
+ ((utf8-native
+ :initarg :utf8-native
+ :initform nil
+ :type boolean
+ :documentation
+ "Can this search engine handle UTF-8-encoded international
+ characters in queries, user names, mail addresses, and
+ message headers? (RFC 9755) This slot is set automatically
+ by the imap server, and can be overridden by `nnimap-utf8-native'.")
+ ;; FIXME: non-ASCII usernames and passwords have not been tested for.
+ ;; Neither have non-ASCII headers and e-mail addresses.
+ (literal-plus
:initarg :literal-plus
:initform nil
:type boolean
@@ -1103,6 +1114,12 @@ gnus-search-run-search
;; `nnimap-open-connection', but it's too frustrating to try to
;; get to the server from the process buffer.
(with-current-buffer (nnimap-buffer)
+ (setf (slot-value engine 'utf8-native)
+ (when (and nnimap-utf8-native
+ (or
+ (nnimap-capability "UTF8=ACCEPT")
+ ;; UTF8=ONLY implies UTF8=ACCEPT
+ (nnimap-capability "UTF8=ONLY"))) t))
(setf (slot-value engine 'literal-plus)
(when (nnimap-capability "LITERAL+") t))
;; MULTISEARCH not yet implemented.
@@ -1160,9 +1177,9 @@ gnus-search-run-search
(cl-defmethod gnus-search-imap-search-command ((engine gnus-search-imap)
(query string))
"Create the IMAP search command for QUERY.
-Currently takes into account support for the LITERAL+ capability.
-Other capabilities could be tested here."
- (with-slots (literal-plus) engine
+Currently takes into account support for the LITERAL+ and UTF8-NATIVE
+capabilities. Other capabilities could be tested here."
+ (with-slots (literal-plus utf8-native) engine
(when (and literal-plus
(string-match-p "\n" query))
(setq query (split-string query "\n")))
@@ -1173,7 +1190,13 @@ gnus-search-imap-search-command
(let* ((nnimap-streaming t)
(call
(nnimap-send-command
- "UID SEARCH CHARSET UTF-8 %s"
+ ;; RFC 9755 (3): once an IMAP client has enabled UTF-8
+ ;; support with the "ENABLE UTF8=ACCEPT" command,
+ ;; the client MUST NOT issue a "SEARCH" command that
+ ;; contains a charset specification.
+ (if utf8-native
+ "UID SEARCH %s"
+ "UID SEARCH CHARSET UTF-8 %s")
(pop query))))
(dolist (l query)
(process-send-string (get-buffer-process (current-buffer)) l)
diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 850061ac1d9..234520b4368 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -474,6 +474,11 @@ nnimap-map-port
"993"
port))
+(defcustom nnimap-utf8-native nil
+ "Enable native UTF8 support in IMAP."
+ :version "32.0"
+ :type 'boolean)
+
(defun nnimap-open-connection-1 (buffer)
(unless (or nnimap-keepalive-timer
(null nnimap-keepalive-intervals))
@@ -597,6 +602,23 @@ nnimap-open-connection-1
(when nnimap-object
(when (nnimap-capability "QRESYNC")
(nnimap-command "ENABLE QRESYNC"))
+ (when (and nnimap-utf8-native
+ (or
+ (nnimap-capability "UTF8=ACCEPT")
+ ;; UTF8=ONLY implies UTF8=ACCEPT
+ (nnimap-capability "UTF8=ONLY")))
+ ;; RFC 9755 (3): a client MUST use "ENABLE UTF8=ACCEPT"
+ ;; to indicate that the client accepts UTF-8 quoted
+ ;; strings. "ENABLE UTF8=ACCEPT" MUST only be used in
+ ;; the authenticated state.
+ (nnimap-command "ENABLE UTF8=ACCEPT"))
+ (when (and (not nnimap-utf8-native)
+ (nnimap-capability "UTF8=ONLY"))
+ (nnheader-message
+ 3
+ "%s won't handle non-ASCII queries unless we enable UTF8 support; \
+see nnimap-utf8-native"
+ nnimap-address))
(nnheader-message 7 "Opening connection to %s...done"
nnimap-address)
(nnimap-process nnimap-object))))))))
@@ -608,6 +630,9 @@ nnimap-login
;; Prefer plain LOGIN if it's enabled (since it requires fewer
;; round trips than CRAM-MD5, and it's less likely to be buggy),
;; and we're using an encrypted connection.
+ ;;
+ ;; FIXME: RFC 9755 (5) prohibits the "LOGIN" command in favor
+ ;; of "AUTHENTICATE" for non-ASCII login credentials.
((and (not (nnimap-capability "LOGINDISABLED"))
(eq (nnimap-stream-type nnimap-object) 'tls)
(or (null nnimap-authenticator)
@@ -792,10 +817,18 @@ nnimap-get-whole-article
t)))
(defun nnimap-capability (capability)
- (member capability (nnimap-capabilities nnimap-object)))
+ (let ((capabilities (nnimap-capabilities nnimap-object))
+ (subsumed-into-IMAP4rev2 '("IDLE" "LITERAL+" "MOVE"
+ "NAMESPACE" "UTF8=ACCEPT"
+ "UTF8=ONLY")))
+ (or (member capability capabilities)
+ (and (member "IMAP4REV2" capabilities)
+ (member capability subsumed-into-IMAP4rev2)))))
(defun nnimap-ver4-p ()
- (nnimap-capability "IMAP4REV1"))
+ (or
+ (nnimap-capability "IMAP4REV1")
+ (nnimap-capability "IMAP4REV2")))
(defun nnimap-get-partial-article (article parts structure)
(let ((result
--=-=-=--
Nickolai Dobrynin <ndobrynin@HIDDEN>:bug-gnu-emacs@HIDDEN, bugs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN, bugs@HIDDEN:bug#81684; Package emacs,gnus.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.