GNU bug report logs - #39506
mm-shr unibyte assumption

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs,gnus; Reported by: dick <dick.r.chiang@HIDDEN>; Keywords: notabug; Done: dick <dick.r.chiang@HIDDEN>; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 39506 <at> debbugs.gnu.org:


Received: (at 39506) by debbugs.gnu.org; 21 Feb 2020 14:34:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 21 09:34:57 2020
Received: from localhost ([127.0.0.1]:46594 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j59OK-0001sm-Ri
	for submit <at> debbugs.gnu.org; Fri, 21 Feb 2020 09:34:57 -0500
Received: from mail-qt1-f179.google.com ([209.85.160.179]:36473)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dick.r.chiang@HIDDEN>) id 1j59OJ-0001sW-3I
 for 39506 <at> debbugs.gnu.org; Fri, 21 Feb 2020 09:34:55 -0500
Received: by mail-qt1-f179.google.com with SMTP id t13so1397583qto.3
 for <39506 <at> debbugs.gnu.org>; Fri, 21 Feb 2020 06:34:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=NbcWNbTjHBbRjos0tf421Khk+blO6Z9duGtCLr9zkBY=;
 b=ku3DrjgeE2PKXierzq4YN/IWLP8ZpWzcI3PFC1dEcwerx22tROOhKp54KT2L80PNuZ
 64InGxZb9OC7sN9sUI2TAWFLkl7me7iRvaCN41/dqPr64AtHTop+BQ+kKzkkBm8z+S7J
 3WKtUdidgdPP4mULVxsZAwcgSeMzpKZqClWgdaBrJapcrhDMxBIEiG31SgqTlpEImEFj
 JJpUmLLywdYnK+u5JeALNVSm2j9G8QaCbgB6D0nj55DKNWmBvjzbKxI0Ky2cvqsEo17p
 +byIl1sSwAzzTLu7+QPgRwtp6RteselP2JNZw/8rRyanjARqnau3yIljdSDDks4d2unu
 qNiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=NbcWNbTjHBbRjos0tf421Khk+blO6Z9duGtCLr9zkBY=;
 b=fPPafr6wdqPY9B6JWxifauS+5ONPKykJ374/eLeWE+1Ly2pAcDO9zSi5FNivmWLfLj
 0t39tbbPD3cgJ3A/08W/BVxkHlIrrjcJvUAR3rhOslE049nU6i1As0QlTRmd3s3SFm5m
 fxCHos9J1whARNviO3bfGDN6ysOTVhKvbsWnfiAKpXZ/CHQrF8KrtSc30VIHq7xy9oKm
 3Xuw0uj7ONR4J86tdY9JUXElT8+Wt8f7kbmYm0cja//uD6Ecl9Rjg66ppynuQkxtdRD9
 ePm3aNeHIp/ieQ5kVpLfeoDoaK6uOdfs3owB7yID43Usa+pGGltUJfn4n0WlXhqPWF6s
 dy6g==
X-Gm-Message-State: APjAAAWjiMkb3Gecsj85U8AzJ3+75EnQu5wtlcztQU3QsKKzj+WELXef
 qGkO26AL6LF+Ep1T2JbR/Ak=
X-Google-Smtp-Source: APXvYqzBZ8UustjKV6W1hzLgvU4Ea3N2Uil6S7tJukRjsCOgmjM8odqD30M/KB3Arb/QJxjUrSjqQA==
X-Received: by 2002:ac8:5208:: with SMTP id r8mr32623786qtn.131.1582295689587; 
 Fri, 21 Feb 2020 06:34:49 -0800 (PST)
Received: from localhost (pool-100-33-98-8.nycmny.fios.verizon.net.
 [100.33.98.8])
 by smtp.gmail.com with ESMTPSA id g23sm311177qke.59.2020.02.21.06.34.48
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 21 Feb 2020 06:34:48 -0800 (PST)
From: dick.r.chiang@HIDDEN
To: Lars Ingebrigtsen <larsi@HIDDEN>
Subject: Re: bug#39506: patch
References: <87tv419ail.fsf@dick> <877e0xf72p.fsf@dick> <87v9ohdr21.fsf@dick>
 <jwvmu9tuee0.fsf-monnier+emacs@HIDDEN> <87o8u8ewdj.fsf@dick>
 <jwvh800vp43.fsf-monnier+emacs@HIDDEN> <87imk15r2w.fsf@HIDDEN>
Date: Fri, 21 Feb 2020 09:34:48 -0500
In-Reply-To: <87imk15r2w.fsf@HIDDEN> (Lars Ingebrigtsen's message of "Thu,
 20 Feb 2020 14:24:55 +0100")
Message-ID: <87k14g9fg7.fsf@dick>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.60 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/html; charset=utf-8
X-Spam-Score: 0.7 (/)
X-Debbugs-Envelope-To: 39506
Cc: 39506 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.3 (/)

Yes, mm handle buffers should contain bytes only.  Thanks for taking the time
to educate.




Information forwarded to bug-gnu-emacs@HIDDEN, bugs@HIDDEN:
bug#39506; Package emacs,gnus. Full text available.
bug closed, send any further explanations to 39506 <at> debbugs.gnu.org and dick <dick.r.chiang@HIDDEN> Request was from dick <dick.r.chiang@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Added tag(s) notabug. Request was from dick <dick.r.chiang@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 39506 <at> debbugs.gnu.org:


Received: (at 39506) by debbugs.gnu.org; 20 Feb 2020 13:25:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 20 08:25:08 2020
Received: from localhost ([127.0.0.1]:44011 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j4lpD-0006Nd-UZ
	for submit <at> debbugs.gnu.org; Thu, 20 Feb 2020 08:25:08 -0500
Received: from quimby.gnus.org ([95.216.78.240]:46856)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1j4lpB-0006Mz-TA
 for 39506 <at> debbugs.gnu.org; Thu, 20 Feb 2020 08:25:06 -0500
Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=marnie)
 by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1j4lp2-00038n-BD; Thu, 20 Feb 2020 14:24:58 +0100
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#39506: patch
References: <87tv419ail.fsf@dick> <877e0xf72p.fsf@dick> <87v9ohdr21.fsf@dick>
 <jwvmu9tuee0.fsf-monnier+emacs@HIDDEN> <87o8u8ewdj.fsf@dick>
 <jwvh800vp43.fsf-monnier+emacs@HIDDEN>
Date: Thu, 20 Feb 2020 14:24:55 +0100
In-Reply-To: <jwvh800vp43.fsf-monnier+emacs@HIDDEN> (Stefan Monnier's message
 of "Sat, 08 Feb 2020 14:51:02 -0500")
Message-ID: <87imk15r2w.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 Content preview: Stefan Monnier <monnier@HIDDEN> writes: > So I
 suspect
 that the source of your problem is earlier, where some code > incorrectly
 decodes some content too early. Hence the need to better > understand where
 those chinese chars come from. 
 Content analysis details:   (-2.9 points, 5.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 0.0 URIBL_BLOCKED          ADMINISTRATOR NOTICE: The query to URIBL was
 blocked.  See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
 for more information. [URIs: ingebrigtsen.no]
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
 [score: 0.0000]
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39506
Cc: 39506 <at> debbugs.gnu.org, dick.r.chiang@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Stefan Monnier <monnier@HIDDEN> writes:

> So I suspect that the source of your problem is earlier, where some code
> incorrectly decodes some content too early.  Hence the need to better
> understand where those chinese chars come from.

Yes, I think that's correct -- the mm handle buffers should contain
bytes only, so it would be interesting to know how an mm handle buffer
ended up containing characters.

Dick, do you have a way to reproduce that bug?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN, bugs@HIDDEN:
bug#39506; Package emacs,gnus. Full text available.

Message received at 39506 <at> debbugs.gnu.org:


Received: (at 39506) by debbugs.gnu.org; 8 Feb 2020 19:51:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 08 14:51:15 2020
Received: from localhost ([127.0.0.1]:51855 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j0W8I-0005JX-UF
	for submit <at> debbugs.gnu.org; Sat, 08 Feb 2020 14:51:15 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:12600)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1j0W8F-0005JG-6L
 for 39506 <at> debbugs.gnu.org; Sat, 08 Feb 2020 14:51:13 -0500
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 7FAF5100EC5;
 Sat,  8 Feb 2020 14:51:05 -0500 (EST)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id D3AB5100E92;
 Sat,  8 Feb 2020 14:51:03 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1581191463;
 bh=sEF6FrfmBoCFHXVWZGCoPZsLxPO0mu1ph/uXNbMmFGo=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=as1rnEJ5glqKpU/xLNZsmEKe2q4yNaus10ME9Eo4dfznl48WcuwvJfOyYxHijklec
 dtEEeyXz3ajNVvucavxYWrtfuJOxHHPjwTu9ZgarpK/na+twEhh6OEMDFc3uLFZLfD
 szOa8a2HW1gXWohRG7Jn7kR83Buz7UEGHDPUccCUdMDwo/ebTSgGTZwWawtYTbz0Kk
 rJ7qqXMC+ZiNYqfzUyUCALyIE1UQ9GAwHoUAS79Pa1wfatA0iyQG1psXB7NkR1e2x/
 DSzWexkrXW0HtW0CEy+gYldjmkmQTxUMJe0/EjPFwfzI6JK/SagWTE15Pwp7N5RNEU
 vsXNiVKEn5f4Q==
Received: from pastel (unknown [157.52.14.222])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6831D120422;
 Sat,  8 Feb 2020 14:51:03 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: dick.r.chiang@HIDDEN
Subject: Re: bug#39506: patch
Message-ID: <jwvh800vp43.fsf-monnier+emacs@HIDDEN>
References: <87tv419ail.fsf@dick> <877e0xf72p.fsf@dick> <87v9ohdr21.fsf@dick>
 <jwvmu9tuee0.fsf-monnier+emacs@HIDDEN> <87o8u8ewdj.fsf@dick>
Date: Sat, 08 Feb 2020 14:51:02 -0500
In-Reply-To: <87o8u8ewdj.fsf@dick> (dick r. chiang's message of "Sat, 08 Feb
 2020 14:01:44 -0500")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL 0.005 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 39506
Cc: 39506 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

>> - Before deciding whether unibyte or multibyte is the right choice, the
>>   main question is whether the buffer contains bytes or chars.
> My buffer contained some Chinese multibytes.

That suggests it contains characters rather than bytes.  How did
that happen?  Where does this buffer ('s contents) come from?

> You can see my unit test in the patch.

In your unit test, you artificially create a multibyte buffer with
chinese chars, so that doesn't answer my question ;-)

AFAIK `mm-with-part` is designed for MIME parts and MIME parts can only
contain bytes at that point.  Only after we extract them as bytes and
apply `mm-decode-content-transfer-encoding` to it can we consider
decoding those bytes into chars.

So I suspect that the source of your problem is earlier, where some code
incorrectly decodes some content too early.  Hence the need to better
understand where those chinese chars come from.


        Stefan





Information forwarded to bug-gnu-emacs@HIDDEN, bugs@HIDDEN:
bug#39506; Package emacs,gnus. Full text available.

Message received at 39506 <at> debbugs.gnu.org:


Received: (at 39506) by debbugs.gnu.org; 8 Feb 2020 19:01:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 08 14:01:53 2020
Received: from localhost ([127.0.0.1]:51802 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j0VMW-00049Y-Vi
	for submit <at> debbugs.gnu.org; Sat, 08 Feb 2020 14:01:53 -0500
Received: from mail-qk1-f173.google.com ([209.85.222.173]:32946)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dick.r.chiang@HIDDEN>) id 1j0VMV-00049L-Ey
 for 39506 <at> debbugs.gnu.org; Sat, 08 Feb 2020 14:01:51 -0500
Received: by mail-qk1-f173.google.com with SMTP id h4so2624621qkm.0
 for <39506 <at> debbugs.gnu.org>; Sat, 08 Feb 2020 11:01:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=HtaJaYDkbjlo4X4oycci0Drn/SgT01ekOhTmM2tL6CE=;
 b=KAHzYiDAED7PV+eAVYnxl0WHJETlxpdQo2NhPFc82QZ1JuEXovIDXwoB5OO5lNGibM
 Ac0CHIhdNp2TVNPXA0phbLgFOKK+Ij61SbyoqdPcxBcpYLEm2AE/X08By29/yRNvN7bW
 syt0GAbgRqtjG2kK07i2MsScOwtBJfJH5Yeiy/1B6xvC7SKIgLMUpu86SPQ/Km4olBnP
 eOtkILFAzCQzvgyl75YOyn77+tj1baSanwCL8FebN+n+9/+JzV52ZxylU+AggPsR3Rak
 fGcVxC1OaJ7gfUnnbo2rzYgzVEee7RjPILpgjwgpXr0UFdE7vTPTqdrIZC4nX5HyZ0Y+
 D6fg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=HtaJaYDkbjlo4X4oycci0Drn/SgT01ekOhTmM2tL6CE=;
 b=DFWXYZyn/dXiqOCxQfGEUJXj6B+Az84kK6/LYxYi3sXHOi31X7zNlZnsH1JY+2ETi0
 Mu/JenCd9bqnOs81N7s62Z6Za/zJcPKc3K58NXBKuJPjqPkiiXOFCqiqcvYonbkfroAn
 77pXxWu+b6GzrrhIF+XjkuLg432O9y0tH6TJRKSPkKLsAadjyU1LQqkLQwnZJPiaBXBO
 ht32G7uEqIrcnFD8RLqYWd7LTadS8aJ7NENo5spBblp/rl6caNxm8e8TYR+fhjEd+Mr3
 pdA+3dhcrPcan6p75KUwjfA0PLA3HQnkzk3T0TSU1GgY/sOZU3ET4/NVL9nfTrA6Osn2
 SnEQ==
X-Gm-Message-State: APjAAAWhfeudLKitiTo2T2Yi+qo7Iukyn5asN9abe1v/p4yo8kUnx8b2
 UFmfCj2s1grlnkalQfunAm0=
X-Google-Smtp-Source: APXvYqzR1T4CL/5MDz3SM8CXXa3rztaVdL/t/DCwKM6kZnHGwtaqWmiK8eySOharaQrztYiqFLwjGw==
X-Received: by 2002:ae9:edc8:: with SMTP id c191mr3541946qkg.393.1581188505775; 
 Sat, 08 Feb 2020 11:01:45 -0800 (PST)
Received: from localhost (pool-100-33-98-8.nycmny.fios.verizon.net.
 [100.33.98.8])
 by smtp.gmail.com with ESMTPSA id o10sm3383272qko.38.2020.02.08.11.01.44
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 08 Feb 2020 11:01:45 -0800 (PST)
From: dick.r.chiang@HIDDEN
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#39506: patch
References: <87tv419ail.fsf@dick> <877e0xf72p.fsf@dick> <87v9ohdr21.fsf@dick>
 <jwvmu9tuee0.fsf-monnier+emacs@HIDDEN>
Date: Sat, 08 Feb 2020 14:01:44 -0500
In-Reply-To: <jwvmu9tuee0.fsf-monnier+emacs@HIDDEN> (Stefan Monnier's message
 of "Sat, 08 Feb 2020 13:32:11 -0500")
Message-ID: <87o8u8ewdj.fsf@dick>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.60 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39506
Cc: 39506 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

> The focus of the sentence is on "before": the previous code already set the
> buffer to unibyte, but it did it afterwards.

Ah, so the buffer in question is merely the working temp-buffer, not the
handle-buffer.

Indeed, the previous code did it afterwards, and therein lies my salvation
because insert-buffer-substring of multibytes to a unibyte buffer corrupts.
I think the Miles Bader code replicated the multibyteness of the
handle-buffer to the temp-buffer, called insert-buffer-substring, and then
converted the temp-buffer to unibyte.

> - How does `mm-with-part` relate to `mm-shr`?

mm-shr calls mm-with-part.

> - Before deciding whether unibyte or multibyte is the right choice, the
>   main question is whether the buffer contains bytes or chars.

My buffer contained some Chinese multibytes.  You can see my unit test in the
patch.

>   AFAIK `mm-with-part` should only ever handle bytes (otherwise calling
>   `mm-decode-content-transfer-encoding` doesn't make much sense).

Okay, maybe mm-decode-content-transfer-encoding is a noop when it doesn't make
sense.

I'm not completely on top of all this, but my individual use case rather
prefers the old Miles Bader order of ops.  I can easily work around it if you feel
your 2008 change genuinely fixed something that was broken (my patch is largely
speculative and inconsiderate of broader ramifications).




Information forwarded to bug-gnu-emacs@HIDDEN, bugs@HIDDEN:
bug#39506; Package emacs,gnus. Full text available.

Message received at 39506 <at> debbugs.gnu.org:


Received: (at 39506) by debbugs.gnu.org; 8 Feb 2020 18:32:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 08 13:32:25 2020
Received: from localhost ([127.0.0.1]:51787 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j0Uu1-0003Tu-0b
	for submit <at> debbugs.gnu.org; Sat, 08 Feb 2020 13:32:25 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:8056)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1j0Uty-0003Tf-4U
 for 39506 <at> debbugs.gnu.org; Sat, 08 Feb 2020 13:32:24 -0500
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 62A3644E389;
 Sat,  8 Feb 2020 13:32:16 -0500 (EST)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C032744E384;
 Sat,  8 Feb 2020 13:32:14 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1581186734;
 bh=9KZAbmI4orluEinLIwszUB8IOgYcEfvrszTQnQyoukI=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=SvTvoQDFrobvf+es42wE30nWoHXk2rf9eNfwPp+roqlftrCGwoXvviRpIGAG2SQ1y
 SD6WaWLY+ol3Ua6vBFvoDFR86kZZTJ/p0JtMQGaqkHojE95Ii8iUTM+mCMVAFZZSfD
 WWrR8us/ObApClWbuxiwxWTIfWn6RnG3Q6tmIfG0t/BoBGJuwb4AuM8jvsUh++4EPs
 q4mmjsiQDY3yPHBzL6efJ056Ecw1YYkW9vuDU4cJvn22hbn1UJ3qf+Fh1hfHq5Y4xl
 erwTJHObCBldzzMMg/8cKSWS2xiNyFiJmu704TW92VajR3tvevOe3B8t+KIzfKDSt0
 W2ZusZjYvzuSw==
Received: from pastel (unknown [157.52.14.222])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 489DC1207A2;
 Sat,  8 Feb 2020 13:32:14 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: dick.r.chiang@HIDDEN
Subject: Re: bug#39506: patch
Message-ID: <jwvmu9tuee0.fsf-monnier+emacs@HIDDEN>
References: <87tv419ail.fsf@dick> <877e0xf72p.fsf@dick> <87v9ohdr21.fsf@dick>
Date: Sat, 08 Feb 2020 13:32:11 -0500
In-Reply-To: <87v9ohdr21.fsf@dick> (dick r. chiang's message of "Sat, 08 Feb
 2020 10:41:58 -0500")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL 0.006 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 39506
Cc: 39506 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> In the changelog message of commit d4eb2b7, you claim to "set the
> buffer to unibyte before inserting" but it's not clear where.

The focus of the sentence is on "before": the previous code already set
the buffer to unibyte, but it did it afterwards.

> Then in a later comment, you say "may happen that the handle-buffer is
> multibyte... in which case now is a good time to adjust it, since we
> know ... it should be unibyte."  The revision before that preserves
> the multibyte-p setting, so I do the same in this patch.

As the motivation for your patch, you write:
> In my Gnus experience, the source buffer that mm-shr acts upon can be
> multibyte.

Two questions:
- How does `mm-with-part` relate to `mm-shr`?
- Before deciding whether unibyte or multibyte is the right choice, the
  main question is whether the buffer contains bytes or chars.
  AFAIK `mm-with-part` should only ever handle bytes (otherwise calling
  `mm-decode-content-transfer-encoding` doesn't make much sense).
  Your patch suggests you have a use case where this is not true.
  Can you give more details about this case?
  Maybe a backtrace showing how we got to this particular call to
  `mm-with-part`?


-- Stefan





Information forwarded to bug-gnu-emacs@HIDDEN, bugs@HIDDEN:
bug#39506; Package emacs,gnus. Full text available.

Message received at 39506 <at> debbugs.gnu.org:


Received: (at 39506) by debbugs.gnu.org; 8 Feb 2020 15:10:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 08 10:10:47 2020
Received: from localhost ([127.0.0.1]:51703 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j0Rkt-0005I1-9u
	for submit <at> debbugs.gnu.org; Sat, 08 Feb 2020 10:10:47 -0500
Received: from mail-qt1-f173.google.com ([209.85.160.173]:32794)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dick.r.chiang@HIDDEN>) id 1j0Rkr-0005Hp-I8
 for 39506 <at> debbugs.gnu.org; Sat, 08 Feb 2020 10:10:46 -0500
Received: by mail-qt1-f173.google.com with SMTP id d5so1810636qto.0
 for <39506 <at> debbugs.gnu.org>; Sat, 08 Feb 2020 07:10:45 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:subject:references:date:in-reply-to:message-id:user-agent
 :mime-version; bh=diw9+RJE3GvRk8aHRj+cHtau3dNUlwXeQ4VEhe1Wd24=;
 b=F5t4AaB9f7J2NwXiKLVN4yOuZqKm9TUjrURkijFG73KbfS5uJoSfbf3jI+9SggUu5+
 PqfiylWSEU+M8ykFI3SiGzpdtWhM/6GR89tJYhXcpKQwcIvVsVmQuPyRoqG8UHcxjlY2
 +Dt8teYCZU5zd48TxLNonRsYOLsi8N3j6B1ibtP0Om+++C9Q4fUeCBR7U9ZLKEryNqYu
 Eoc2UkD0Rk152YCydU/GdSSET01PNi9TYSBnaPvdNpdMSERhbreFig75wHb8BdQ+LQes
 KFJcj12bPB8MCgdzd+24GLzOqdIT8oeIXffkv3jKvG4ivqqNSDiM/85tUl4IprxpQtsy
 eYjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=diw9+RJE3GvRk8aHRj+cHtau3dNUlwXeQ4VEhe1Wd24=;
 b=ZRYn3NZ7RlEb8qVz5fTtUEiTfLTRdMT18RkGUY9wlgXSwwwR8iuJqCQ897GRZgvaS7
 xcVfKCurJ0A/mAd/g8GloqbfrOb3PaQPv9XT+RwVUTzR2V5twkzr14a3FJcYZaFnCLbb
 6NpcbshLZQoeigUGhSfLNuj44Jnz8sv/gNlK/PYLZ2N1XuMn/bTq8UXuVPO6is+5YU0h
 BosqVvZPQ8YC4Ed89JwTOhpHsSyu7BKks05nqINmLe13G7Kqnz/hUdoiPLHW/CnGF0R0
 CdEy4B2CXW30IBGGkXItKTGLjF58es2b+iszdYsRMyJgljts49QK0bkeGjLI36VT5ajZ
 nd1g==
X-Gm-Message-State: APjAAAVePf9CQwPTtt8tZueu4yhSNYUZTF3ixMS+YmbO29xeDEn8unjY
 KIOHVXo0ON/QVJ0EB4+MQxcxzY0q
X-Google-Smtp-Source: APXvYqwcJ4j3i7pbZlCjAQDjBnb/LuEOQaRJohhsjOew6hx/XYg/ycoTVxn3pNKg+wBgyzgzwvo3jQ==
X-Received: by 2002:ac8:1866:: with SMTP id n35mr3121204qtk.28.1581174639912; 
 Sat, 08 Feb 2020 07:10:39 -0800 (PST)
Received: from localhost (pool-100-33-98-8.nycmny.fios.verizon.net.
 [100.33.98.8])
 by smtp.gmail.com with ESMTPSA id d25sm2954503qkk.77.2020.02.08.07.10.39
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 08 Feb 2020 07:10:39 -0800 (PST)
From: dick.r.chiang@HIDDEN
X-Google-Original-From: 39506 <at> debbugs.gnu.org
To: 39506 <at> debbugs.gnu.org, smonnier@HIDDEN
Subject: Re: bug#39506: patch
References: <87tv419ail.fsf@dick>
Date: Sat, 08 Feb 2020 10:10:38 -0500
In-Reply-To: <87tv419ail.fsf@dick> (dick's message of "Fri, 07 Feb 2020
 19:40:34 -0500")
Message-ID: <877e0xf72p.fsf@dick>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.60 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39506
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

In the changelog message of commit d4eb2b7, you claim to "set the buffer to unibyte
before inserting" but it's not clear where.  Then in a later comment, you say
"may happen that the handle-buffer is multibyte... in which case now is a good
time to adjust it, since we know ... it should be unibyte."  The revision before that
preserves the multibyte-p setting, so I do the same in this patch.




Information forwarded to bug-gnu-emacs@HIDDEN, bugs@HIDDEN:
bug#39506; Package emacs,gnus. Full text available.
bug reassigned from package 'emacs' to 'emacs,gnus'. Request was from dick <dick.r.chiang@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Changed bug title to 'mm-shr unibyte assumption' from 'patch' Request was from dick <dick.r.chiang@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 8 Feb 2020 00:40:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 07 19:40:43 2020
Received: from localhost ([127.0.0.1]:50494 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1j0EAs-0007PH-Pn
	for submit <at> debbugs.gnu.org; Fri, 07 Feb 2020 19:40:43 -0500
Received: from lists.gnu.org ([209.51.188.17]:43035)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dick.r.chiang@HIDDEN>) id 1j0EAq-0007P9-Ew
 for submit <at> debbugs.gnu.org; Fri, 07 Feb 2020 19:40:40 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:44112)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <dick.r.chiang@HIDDEN>) id 1j0EAp-0008MB-1z
 for bug-gnu-emacs@HIDDEN; Fri, 07 Feb 2020 19:40:40 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_20,FREEMAIL_FROM,
 URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <dick.r.chiang@HIDDEN>) id 1j0EAn-0003XI-Gi
 for bug-gnu-emacs@HIDDEN; Fri, 07 Feb 2020 19:40:38 -0500
Received: from mail-qt1-x833.google.com ([2607:f8b0:4864:20::833]:36128)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <dick.r.chiang@HIDDEN>)
 id 1j0EAn-0003ON-8y
 for bug-gnu-emacs@HIDDEN; Fri, 07 Feb 2020 19:40:37 -0500
Received: by mail-qt1-x833.google.com with SMTP id t13so838106qto.3
 for <bug-gnu-emacs@HIDDEN>; Fri, 07 Feb 2020 16:40:36 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:subject:date:message-id:user-agent:mime-version
 :content-disposition:content-transfer-encoding:content-description;
 bh=xS/EMWITDmLJFAUheVzQJaqzLvaOQmEGFNNvTCrFuZI=;
 b=PrM4aD5oEGmkwuG/4+smH9Ene65Sdheo7p+58qJ6bJ4VBOd7HRgCJaVHDAsK3XGTx4
 7PcKgk7ots10M6IKijhhrJsw0YlDjYEJuO0wTLhL/ZjAUwaOAXmGpwkQ0XMUctSUn2sO
 2kVh/gaTYK/JoxdhuwDeDLlae6rNgzeL1bClaiouQVYLlYVhfLNoZiY3Eh4lUrvHJy6i
 XdKK74VCen3zwb5+b/IZM6QHgyJvYkkN8dEALJb489nEaYn1cIt+K3NPDOM9/c5UvYXL
 f3PRXiymIWhBdbgO3Ry/MxNSJZblPXfbNABh8FEtmnFYe6CLBZJqxXEVKgEyntMLOr1c
 7pzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:subject:date:message-id:user-agent
 :mime-version:content-disposition:content-transfer-encoding
 :content-description;
 bh=xS/EMWITDmLJFAUheVzQJaqzLvaOQmEGFNNvTCrFuZI=;
 b=W/jbpOresZ+ATxYJP+U3XSApFBq+6fl+W+i+Ey7lKi1eIe+dbNdY/DENZGrixsXgzB
 P/wS2rDAdsVCHo8GKbmyOg2uWkvF07dydTLtKgVEC1x9f/u4/uEN4z3rsvcuUQ16jzrK
 17CHmX93SDtwBppek4IBYBr2lVkNNouc3SrYTKOSzIPfH5UQEGmY9yCwfz1xY/OYWYpl
 Jna2AC3Du221oEouOvCY83jMhxo/O2/M4w5KIR2vsjEaN/CGSZeUBmAjqh4Xj8LYHXOH
 qxnqVKyCA/0y2W1vEdWepiTfHTVnBJkP0siDM/7D1/y3CFhYHb3lFGbogWmelD9536Dh
 1Jeg==
X-Gm-Message-State: APjAAAXYogABFm5XhLTjfgALIjBUpugi+y+0bLWZ+eFea3A4AFSJUgHW
 XyLxild2+VtBu5wSypaPNlQRIG3X
X-Google-Smtp-Source: APXvYqxIc2vlpu7ofHfuGSMJTsShlfyUfuO51HflYfCXrziHfDuwemzXr7b4FOqOPzeOTO/z1xefmQ==
X-Received: by 2002:ac8:745a:: with SMTP id h26mr915951qtr.320.1581122435658; 
 Fri, 07 Feb 2020 16:40:35 -0800 (PST)
Received: from localhost (pool-100-33-98-8.nycmny.fios.verizon.net.
 [100.33.98.8])
 by smtp.gmail.com with ESMTPSA id 135sm2193779qkl.68.2020.02.07.16.40.34
 for <bug-gnu-emacs@HIDDEN>
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 07 Feb 2020 16:40:34 -0800 (PST)
From: dick <dick.r.chiang@HIDDEN>
To: bug-gnu-emacs <bug-gnu-emacs@HIDDEN>
Subject: patch
Date: Fri, 07 Feb 2020 19:40:34 -0500
Message-ID: <87tv419ail.fsf@dick>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: inline;
 filename=0001-Question-assumption-that-mm-source-buffer-is-unibyte.patch
Content-Transfer-Encoding: quoted-printable
Content-Description: patch
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 2607:f8b0:4864:20::833
X-Spam-Score: 2.3 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: From 2674b6a08a90b9a97d3adf2b3e4497b61880e173 Mon Sep 17
 00:00:00
 2001 From: dickmao <none> Date: Fri, 7 Feb 2020 19:33:13 -0500 Subject: [PATCH]
 Question assumption that mm source buffer is unibyte In my Gnus experience,
 the source buffer that mm-shr acts upon can be multibyte. 
 Content analysis details:   (2.3 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 0.0 URIBL_BLOCKED          ADMINISTRATOR NOTICE: The query to URIBL was
 blocked.  See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
 for more information. [URIs: gnu.org]
 -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at https://www.dnswl.org/,
 low trust [209.51.188.17 listed in list.dnswl.org]
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail
 provider (dick.r.chiang[at]gmail.com)
 2.0 SPOOFED_FREEMAIL       No description available.
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

From 2674b6a08a90b9a97d3adf2b3e4497b61880e173 Mon Sep 17 00:00:00 2001
From: dickmao <none>
Date: Fri, 7 Feb 2020 19:33:13 -0500
Subject: [PATCH] Question assumption that mm source buffer is unibyte

In my Gnus experience, the source buffer that mm-shr acts upon can be
multibyte.

* lisp/gnus/mm-decode.el (mm-with-part): propagate multibyte/unibyte setting
from source buffer to target.
* test/lisp/gnus/mm-decode-tests.el (test-mm-decode-multibyte): add test
---
 lisp/gnus/mm-decode.el            |  8 ++-----
 test/lisp/gnus/mm-decode-tests.el | 39 +++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 6 deletions(-)
 create mode 100644 test/lisp/gnus/mm-decode-tests.el

diff --git a/lisp/gnus/mm-decode.el b/lisp/gnus/mm-decode.el
index d33bb56dc9..19a18b4f45 100644
--- a/lisp/gnus/mm-decode.el
+++ b/lisp/gnus/mm-decode.el
@@ -1255,16 +1255,12 @@ mm-handle-displayed-p
=20
 (defmacro mm-with-part (handle &rest forms)
   "Run FORMS in the temp buffer containing the contents of HANDLE."
-  ;; The handle-buffer's content is a sequence of bytes, not a sequence of
-  ;; chars, so the buffer should be unibyte.  It may happen that the
-  ;; handle-buffer is multibyte for some reason, in which case now is a go=
od
-  ;; time to adjust it, since we know at this point that it should
-  ;; be unibyte.
   `(let* ((handle ,handle))
      (when (and (mm-handle-buffer handle)
 		(buffer-name (mm-handle-buffer handle)))
        (with-temp-buffer
-	 (mm-disable-multibyte)
+         (set-buffer-multibyte (buffer-local-value 'enable-multibyte-chara=
cters
+                                                   (mm-handle-buffer handl=
e)))
 	 (insert-buffer-substring (mm-handle-buffer handle))
 	 (mm-decode-content-transfer-encoding
 	  (mm-handle-encoding handle)
diff --git a/test/lisp/gnus/mm-decode-tests.el b/test/lisp/gnus/mm-decode-t=
ests.el
new file mode 100644
index 0000000000..8a9d471b74
--- /dev/null
+++ b/test/lisp/gnus/mm-decode-tests.el
@@ -0,0 +1,39 @@
+;;; mm-decode-tests.el --- tests for gnus/mm-decode.el    -*- lexical-bind=
ing:t -*-
+
+;; Copyright (C) 2019-2020 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Code:
+
+(require 'ert)
+(require 'mm-decode)
+
+(ert-deftest test-mm-decode-multibyte ()
+  (should
+   (or (not (fboundp 'libxml-parse-html-region))
+       (with-temp-buffer
+         (set-buffer-multibyte t)
+         (save-excursion
+           (insert "<p>=E6=9C=80=E8=BF=91=E4=B9=9F=E6=83=B3=E5=B0=9D=E8=AF=
=95,=E4=BD=86=E6=98=AF=E6=84=9F=E8=A7=89=E8=9B=AE=E9=9A=BE=E7=9A=84,=E6=AF=
=94=E5=A6=82=E4=B8=8D=E7=9F=A5=E9=81=93=E5=A6=82=E4=BD=95=E5=9C=A8"))
+         (let ((handle (mm-make-handle
+                        (current-buffer)
+                        (rfc2231-parse-qp-string
+                         "Content-Type: text/html; charset=3DUTF-8"))))
+           (not (zerop (length (with-temp-buffer (mm-shr handle)
+                                                 (buffer-string))))))))))
+
+;;; mm-decode-tests.el ends here
--=20
2.24.1





Acknowledgement sent to dick <dick.r.chiang@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#39506; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 21 Feb 2020 14:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.