GNU bug report logs - #29918
26.0.90; serial-term error in process filter

Previous Next

Package: emacs;

Reported by: Gemini Lasswell <gazally <at> runbox.com>

Date: Sun, 31 Dec 2017 20:58:01 UTC

Severity: normal

Tags: fixed

Found in version 26.0.90

Fixed in version 26.3

Done: Noam Postavsky <npostavs <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29918 in the body.
You can then email your comments to 29918 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 31 Dec 2017 20:58:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Gemini Lasswell <gazally <at> runbox.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Sun, 31 Dec 2017 20:58:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.0.90; serial-term error in process filter
Date: Sun, 31 Dec 2017 12:57:19 -0800
When a serial-term buffer is connected to a device with bad wiring or
hardware problems, it spams the minibuffer with messages like this:

error in process filter: Args out of range: "\361", -1 [2 times]
error in process filter: Args out of range: "\377\375", -1 [2 times]
error in process filter: Args out of range: "\370\340\374\374", -1 [2 times]
error in process filter: Args out of range: "\376\340\360\370", -1 [2 times]
error in process filter: Args out of range: "\377\377\374\376\371\376\301\377", -1 [2 times]
error in process filter: Args out of range: "\376", -1 [2 times]

Emacs 25.3 is silent under the same conditions.

I'm using an FTDI USB to serial converter and a Raspberry pi. I
discovered the bug after I had wired things up with a flaky
breadboard, but it's quick to reproduce by disconnecting ground.

With emacs -Q:

Connect USB from your Emacs machine to the FTDI device.
Connect GND, TX and RX from the FTDI to the Raspberry Pi.
Turn on the Pi.
M-x serial-term RET /dev/your-device-name-here RET 115200 RET
Now disconnect GND.

Result: The serial-term buffer starts to fill with gibberish, which is
expected and also happens in 25.3, and the process filter errors start
showing up at irregular intervals.


In GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
 of 2017-12-31 built on chinook
Repository revision: 312c5655669a882186884626f0cf361de70e679d
Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/home/gem/src/emacs/emacs-26/bin --with-modules
 --with-x-toolkit=gtk3 --with-xft --config-cache'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS NOTIFY LIBSELINUX
GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort map mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair
time-date mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet
lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese composite charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 95859 7544)
 (symbols 48 20453 1)
 (miscs 40 44 104)
 (strings 32 28532 1259)
 (string-bytes 1 795727)
 (vectors 16 14820)
 (vector-slots 8 502732 10570)
 (floats 8 49 68)
 (intervals 56 205 0)
 (buffers 992 11))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 14 Jul 2019 17:53:01 GMT) Full text and rfc822 format available.

Message #8 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Gemini Lasswell <gazally <at> runbox.com>
Cc: 29918 <at> debbugs.gnu.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sun, 14 Jul 2019 19:52:50 +0200
Gemini Lasswell <gazally <at> runbox.com> writes:

> When a serial-term buffer is connected to a device with bad wiring or
> hardware problems, it spams the minibuffer with messages like this:
>
> error in process filter: Args out of range: "\361", -1 [2 times]
> error in process filter: Args out of range: "\377\375", -1 [2 times]
> error in process filter: Args out of range: "\370\340\374\374", -1 [2 times]
> error in process filter: Args out of range: "\376\340\360\370", -1 [2 times]
> error in process filter: Args out of range: "\377\377\374\376\371\376\301\377", -1 [2 times]
> error in process filter: Args out of range: "\376", -1 [2 times]
>
> Emacs 25.3 is silent under the same conditions.

Does setting (setq debug-on-error t) give you a backtrace for these
errors?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Mon, 15 Jul 2019 21:47:02 GMT) Full text and rfc822 format available.

Message #11 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 29918 <at> debbugs.gnu.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Mon, 15 Jul 2019 14:45:50 -0700
[Message part 1 (text/plain, inline)]
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Does setting (setq debug-on-error t) give you a backtrace for these
> errors?

No, but I just made one happen by removing the internal_condition_case_1
wrapper on the read_process_output_call in read_and_dispose_of_process_output.
Then after I got the first backtrace showing that the error was in
term-emulate-terminal I eval-defun'd it to make a more useful backtrace:

[term-emulate-terminal.txt (text/plain, attachment)]
[Message part 3 (text/plain, inline)]
This is from master.

Thanks for taking a look at this.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Wed, 17 Jul 2019 10:59:01 GMT) Full text and rfc822 format available.

Message #14 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Gemini Lasswell <gazally <at> runbox.com>
Cc: 29918 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> gmail.com>
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Wed, 17 Jul 2019 12:58:51 +0200
Gemini Lasswell <gazally <at> runbox.com> writes:

> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
>
>> Does setting (setq debug-on-error t) give you a backtrace for these
>> errors?
>
> No, but I just made one happen by removing the internal_condition_case_1
> wrapper on the read_process_output_call in read_and_dispose_of_process_output.
> Then after I got the first backtrace showing that the error was in
> term-emulate-terminal I eval-defun'd it to make a more useful backtrace:
>
> Debugger entered--Lisp error: (args-out-of-range "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>   aref("\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>   (char-charset (aref decoded-substring (- count 1 partial)))

I'm guessing this bug was introduced by:

commit 47019a521f774fbd13441e178a6a82c9989b9912
Author: Noam Postavsky <npostavs <at> gmail.com>
Date:   Thu Jan 18 08:22:47 2018 -0500

    Switch term.el to lexical binding, and clean up code a bit
    
It's a huge patch, though, and it's difficult to follow the changes in
logic.

The old code had:

-			  (setq count (length decoded-substring))
-                          ;; Check for multibyte characters that ends
-                          ;; before end of string, and save it for
-                          ;; next time.
-                          (when (= funny str-length)
-                            (let ((partial 0))
-                              (while (eq (char-charset (aref decoded-substring
-                                                             (- count 1 partial)))
-                                         'eight-bit)
-                                (cl-incf partial))

(And it's that aref that's failing with (- count 1 partial) being -1,
and the new one is

+                (when (= funny str-length)
+                  (let ((partial 0)
+                        (count (length decoded-substring)))
+                    (while (eq (char-charset (aref decoded-substring
+                                                   (- count 1 partial)))
+                               'eight-bit)
+                      (cl-incf partial))

with that length being recalculated that way...

Noam?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Wed, 17 Jul 2019 11:37:02 GMT) Full text and rfc822 format available.

Message #17 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 29918 <at> debbugs.gnu.org, Gemini Lasswell <gazally <at> runbox.com>
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Wed, 17 Jul 2019 07:36:34 -0400
[Message part 1 (text/plain, inline)]
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Gemini Lasswell <gazally <at> runbox.com> writes:
>
>> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
>>
>>> Does setting (setq debug-on-error t) give you a backtrace for these
>>> errors?
>>
>> No, but I just made one happen by removing the internal_condition_case_1
>> wrapper on the read_process_output_call in read_and_dispose_of_process_output.
>> Then after I got the first backtrace showing that the error was in
>> term-emulate-terminal I eval-defun'd it to make a more useful backtrace:
>>
>> Debugger entered--Lisp error: (args-out-of-range "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>>   aref("\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>>   (char-charset (aref decoded-substring (- count 1 partial)))
>
> I'm guessing this bug was introduced by:
>
> commit 47019a521f774fbd13441e178a6a82c9989b9912
> Author: Noam Postavsky <npostavs <at> gmail.com>
> Date:   Thu Jan 18 08:22:47 2018 -0500
>
>     Switch term.el to lexical binding, and clean up code a bit

It can't be that patch, because it's only in master, not emacs-26.  I
think it's rather [1: 134e86b360] (also mine).

[1: 134e86b360]: 2017-01-03 08:58:40 -0500
  Handle multibyte chars spanning chunks in term.el
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=134e86b360cab0d0a5cb634b71a4b06ec26c5f1f

If I understand the backtrace correctly, then the problem is that I
didn't think of the case where the whole string fails to decode.  I
think the patch below should fix it; there is a possibility it prevents
correct decoding in case the terminal receives a utf-8 character in
single byte chunks (though I'm not sure if it's even valid for a program
to emit single bytes like that).

[0001-Handle-completely-undecoded-input-in-term-Bug-29918.patch (text/x-diff, inline)]
From 3e2663f8366ea82f52f47019b63fca243e3f88d9 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs <at> gmail.com>
Date: Wed, 17 Jul 2019 07:20:20 -0400
Subject: [PATCH] Handle completely undecoded input in term (Bug#29918)

* lisp/term.el (term-emulate-terminal): Avoid errors if the whole
decoded string is eight-bit characters.
---
 lisp/term.el | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/lisp/term.el b/lisp/term.el
index cbef68dc0a..9785ce3024 100644
--- a/lisp/term.el
+++ b/lisp/term.el
@@ -2900,11 +2900,12 @@ term-emulate-terminal
                           ;; next time.
                           (when (= funny str-length)
                             (let ((partial 0))
-                              (while (eq (char-charset (aref decoded-substring
-                                                             (- count 1 partial)))
-                                         'eight-bit)
+                              (while (and (< partial count)
+                                          (eq (char-charset (aref decoded-substring
+                                                                  (- count 1 partial)))
+                                              'eight-bit))
                                 (cl-incf partial))
-                              (when (> partial 0)
+                              (when (> count partial 0)
                                 (setq term-terminal-undecoded-bytes
                                       (substring decoded-substring (- partial)))
                                 (setq decoded-substring
-- 
2.11.0


Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Thu, 18 Jul 2019 08:46:02 GMT) Full text and rfc822 format available.

Message #20 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Thu, 18 Jul 2019 11:45:28 +0300
> From: Noam Postavsky <npostavs <at> gmail.com>
> Date: Wed, 17 Jul 2019 07:36:34 -0400
> Cc: 29918 <at> debbugs.gnu.org, Gemini Lasswell <gazally <at> runbox.com>
> 
> If I understand the backtrace correctly, then the problem is that I
> didn't think of the case where the whole string fails to decode.  I
> think the patch below should fix it

Yes, I think you are right.

> there is a possibility it prevents correct decoding in case the
> terminal receives a utf-8 character in single byte chunks (though
> I'm not sure if it's even valid for a program to emit single bytes
> like that).

I agree.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Thu, 18 Jul 2019 11:23:02 GMT) Full text and rfc822 format available.

Message #23 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Thu, 18 Jul 2019 07:22:10 -0400
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:

>> there is a possibility it prevents correct decoding in case the
>> terminal receives a utf-8 character in single byte chunks (though
>> I'm not sure if it's even valid for a program to emit single bytes
>> like that).
>
> I agree.

I thought of limiting partial to 4, to avoid this possibility, since
utf-8 characters can be 4 bytes at most (not sure about other encodings
though).  

[0001-Handle-completely-undecoded-input-in-term-Bug-29918.patch (text/plain, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Thu, 18 Jul 2019 11:35:01 GMT) Full text and rfc822 format available.

Message #26 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Thu, 18 Jul 2019 14:33:57 +0300
> From: Noam Postavsky <npostavs <at> gmail.com>
> Cc: 29918 <at> debbugs.gnu.org,  gazally <at> runbox.com,  larsi <at> gnus.org
> Date: Thu, 18 Jul 2019 07:22:10 -0400
> 
> >> there is a possibility it prevents correct decoding in case the
> >> terminal receives a utf-8 character in single byte chunks (though
> >> I'm not sure if it's even valid for a program to emit single bytes
> >> like that).
> >
> > I agree.
> 
> I thought of limiting partial to 4, to avoid this possibility, since
> utf-8 characters can be 4 bytes at most (not sure about other encodings
> though).  

I don't understand why limit, if we assume that no program will write
UTF-8 encoded sequences byte-byte.  What am I missing?

In general, I'd prefer the simplest fix, as this is going to be pushed
to the release branch, yes?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Fri, 19 Jul 2019 00:17:02 GMT) Full text and rfc822 format available.

Message #29 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Thu, 18 Jul 2019 20:15:58 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

> I don't understand why limit, if we assume that no program will write
> UTF-8 encoded sequences byte-byte.  What am I missing?

I wasn't assuming that, I was wondering whether it was true or not.

> In general, I'd prefer the simplest fix, as this is going to be pushed
> to the release branch, yes?

I wasn't sure about pushing to the release branch either, but yes, I
think my first patch is safer for the release branch (since it only
affects cases where we are currently signaling an error).

Gemini, could you try it and confirm that it fixes your problem please?





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sat, 20 Jul 2019 12:50:02 GMT) Full text and rfc822 format available.

Message #32 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 29918 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sat, 20 Jul 2019 05:49:28 -0700
Noam Postavsky <npostavs <at> gmail.com> writes:

> Gemini, could you try it and confirm that it fixes your problem please?

I tried both patches and both solve the problem.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 21 Jul 2019 02:33:01 GMT) Full text and rfc822 format available.

Message #35 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Gemini Lasswell <gazally <at> runbox.com>
Cc: 29918 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sat, 20 Jul 2019 22:32:02 -0400
tags 29918 fixed
close 29918 26.3
quit

Gemini Lasswell <gazally <at> runbox.com> writes:

> Noam Postavsky <npostavs <at> gmail.com> writes:
>
>> Gemini, could you try it and confirm that it fixes your problem please?
>
> I tried both patches and both solve the problem.

Thanks, I added a test and pushed the first patch to emacs-26.

150bdfe43a 2019-07-20T21:35:21-04:00 "Handle completely undecoded input in term (Bug#29918)"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=150bdfe43acde8423612cbff4eafbbb88878b497





Added tag(s) fixed. Request was from Noam Postavsky <npostavs <at> gmail.com> to control <at> debbugs.gnu.org. (Sun, 21 Jul 2019 02:33:02 GMT) Full text and rfc822 format available.

bug marked as fixed in version 26.3, send any further explanations to 29918 <at> debbugs.gnu.org and Gemini Lasswell <gazally <at> runbox.com> Request was from Noam Postavsky <npostavs <at> gmail.com> to control <at> debbugs.gnu.org. (Sun, 21 Jul 2019 02:33:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 21 Jul 2019 13:50:01 GMT) Full text and rfc822 format available.

Message #42 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, larsi <at> gnus.org,
 Noam Postavsky <npostavs <at> gmail.com>
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sun, 21 Jul 2019 09:49:02 -0400
> I don't understand why limit, if we assume that no program will write
> UTF-8 encoded sequences byte-byte.

I don't think we can assume that, in general.


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 21 Jul 2019 13:54:01 GMT) Full text and rfc822 format available.

Message #45 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, Eli Zaretskii <eliz <at> gnu.org>,
 larsi <at> gnus.org
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sun, 21 Jul 2019 09:52:39 -0400
>>> there is a possibility it prevents correct decoding in case the
>>> terminal receives a utf-8 character in single byte chunks (though
>>> I'm not sure if it's even valid for a program to emit single bytes
>>> like that).
>> I agree.
> I thought of limiting partial to 4, to avoid this possibility, since
> utf-8 characters can be 4 bytes at most (not sure about other encodings
> though).

That sounds like a good idea.

Just a side note to mention that there are other multibyte encodings
than utf-8 and I have no reason to believe that they're never used for
terminals, so we should look for the maximum over all coding-systems
rather than only utf-8.


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 21 Jul 2019 14:39:02 GMT) Full text and rfc822 format available.

Message #48 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, larsi <at> gnus.org,
 npostavs <at> gmail.com
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sun, 21 Jul 2019 17:38:29 +0300
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Noam Postavsky <npostavs <at> gmail.com>,  29918 <at> debbugs.gnu.org,
>   gazally <at> runbox.com,  larsi <at> gnus.org
> Date: Sun, 21 Jul 2019 09:49:02 -0400
> 
> > I don't understand why limit, if we assume that no program will write
> > UTF-8 encoded sequences byte-byte.
> 
> I don't think we can assume that, in general.

What would be a situation where it is false?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 21 Jul 2019 14:46:03 GMT) Full text and rfc822 format available.

Message #51 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, npostavs <at> gmail.com
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sun, 21 Jul 2019 16:45:33 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
>> Cc: Noam Postavsky <npostavs <at> gmail.com>,  29918 <at> debbugs.gnu.org,
>>   gazally <at> runbox.com,  larsi <at> gnus.org
>> Date: Sun, 21 Jul 2019 09:49:02 -0400
>> 
>> > I don't understand why limit, if we assume that no program will write
>> > UTF-8 encoded sequences byte-byte.
>> 
>> I don't think we can assume that, in general.
>
> What would be a situation where it is false?

Isn't sending bytes one-by-one a pretty common thing for (primitive)
programs to do?  I'd assume so, especially for serial communication
programs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#29918; Package emacs. (Sun, 21 Jul 2019 14:49:02 GMT) Full text and rfc822 format available.

Message #54 received at 29918 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 29918 <at> debbugs.gnu.org, gazally <at> runbox.com, monnier <at> iro.umontreal.ca,
 npostavs <at> gmail.com
Subject: Re: bug#29918: 26.0.90; serial-term error in process filter
Date: Sun, 21 Jul 2019 17:47:50 +0300
> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>,  npostavs <at> gmail.com,
>   29918 <at> debbugs.gnu.org,  gazally <at> runbox.com
> Date: Sun, 21 Jul 2019 16:45:33 +0200
> 
> >> > I don't understand why limit, if we assume that no program will write
> >> > UTF-8 encoded sequences byte-byte.
> >> 
> >> I don't think we can assume that, in general.
> >
> > What would be a situation where it is false?
> 
> Isn't sending bytes one-by-one a pretty common thing for (primitive)
> programs to do?  I'd assume so, especially for serial communication
> programs.

We are talking about term.el, don't we?  So the programs relevant here
are those which run in a terminal, whether serial or not, and they are
generally expected to talk in characters, not bytes.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 19 Aug 2019 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 250 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.