GNU bug report logs -
#7117
23.2.2 mangles terminal escape sequences
Previous Next
Reported by: Ryan Johnson <ryanjohn <at> ece.cmu.edu>
Date: Mon, 27 Sep 2010 12:08:01 UTC
Severity: normal
Tags: moreinfo
Found in version 23.2.2
Done: Lars Ingebrigtsen <larsi <at> gnus.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 7117 in the body.
You can then email your comments to 7117 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Mon, 27 Sep 2010 12:08:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Ryan Johnson <ryanjohn <at> ece.cmu.edu>
:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org
.
(Mon, 27 Sep 2010 12:08:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi all,
Emacs-23.2 occasionally loses parts of terminal escape sequences,
confusing xt-mouse.el and dumping garbage text in the user's buffer.
Steps to reproduce:
1. ssh to a machine with emacs-23 installed (I suspect slower
networks would expose this more, but the bug bites me even over
intranet)
2. compile tee-input.c (see below) into a shared library (on solaris:
`cc -g -G -xcode=pic13 -ldl -hlibtee-input.so tee-input.c -g -o
libtee-input.so')
3. invoke `LD_PRELOAD=libtee-input.so emacs -nw -Q 2>input.txt'
4. M-x xterm-mouse-mode
5. flick the mouse scroll wheel hard, so it generates many ticks in
quick succession (note the garbage that results)
6. M-x show-lossage (note the mangled escape sequences)
7. C-x C-c
8. Examine input.txt (note the intact escape sequences)
Doing the above with emacs-22.3.1 always works fine, but emacs-23.2.2
mangles one or more escape sequences (input.txt is not corrupted). I
suspect this and bug #6920 are symptoms of the same problem.
The number of characters missing ranges from 1 (only ESC missing) to all
but one (only [ present); I don't think I've yet seen an entire sequence
disappear. The harder the flick the more numerous the mangled escape
sequences. It's not an OS issue because I can reproduce it on three very
different machines (32-bit cygwin, 64-bit redhat, 32-bit solaris). It's
also not a problem with the terminal because the intercepted input is
correct. Finally, it's not a problem with xt-mouse.el because
show-lossage is incorrect.
To give one example I generated using the above steps, emacs *scratch*
buffer shows:
`b[[M`b[
The output of show-lossage is (note the orphaned ` b [, and later the
orphaned [, and finally the orphaned M ` b [):
ESC [ > 7 7 ; 9 0 0 ; 0 c ESC x x t e r m - m o u s
e TAB RET ESC [ M ` b [ ` b [ ESC [ M ` b [ ESC [ M
` b [ ESC [ M ` b [ ESC [ M ` b [ ESC [ M ` b [ [ M
` b [ ESC [ M ` b [ ESC O P l
The relevant snippet of input.txt is:
;;;;;ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESC[M`b[ESCOPl
Thoughts?
Ryan
===== tee-input.c ====
#include <dlfcn.h>
#include <unistd.h>
typedef ssize_t (*read_func)(int fildes, void *buf, size_t nbyte);
ssize_t read(int fildes, void *buf, size_t nbyte) {
static read_func old_read = 0;
if(!old_read)
old_read = (read_func) dlsym(RTLD_NEXT, "read");
ssize_t rval = old_read(fildes, buf, nbyte);
if(rval > 0)
write(2, buf, rval);
return rval;
}
=================
[Message part 2 (text/html, inline)]
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Mon, 27 Sep 2010 20:50:03 GMT)
Full text and
rfc822 format available.
Message #8 received at 7117 <at> debbugs.gnu.org (full text, mbox):
> Steps to reproduce:
> 1. ssh to a machine with emacs-23 installed (I suspect slower
> networks would expose this more, but the bug bites me even over
> intranet)
> 2. compile tee-input.c (see below) into a shared library (on solaris:
> `cc -g -G -xcode=pic13 -ldl -hlibtee-input.so tee-input.c -g -o
> libtee-input.so')
> 3. invoke `LD_PRELOAD=libtee-input.so emacs -nw -Q 2>input.txt'
> 4. M-x xterm-mouse-mode
> 5. flick the mouse scroll wheel hard, so it generates many ticks in
> quick succession (note the garbage that results)
> 6. M-x show-lossage (note the mangled escape sequences)
> 7. C-x C-c
> 8. Examine input.txt (note the intact escape sequences)
Does (set-keyboard-coding-system 'binary) circumvent the problem?
Stefan
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Tue, 28 Sep 2010 04:52:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 7117 <at> debbugs.gnu.org (full text, mbox):
On 9/27/2010 10:52 PM, Stefan Monnier wrote:
>> Steps to reproduce:
>> 1. ssh to a machine with emacs-23 installed (I suspect slower
>> networks would expose this more, but the bug bites me even over
>> intranet)
>> 2. compile tee-input.c (see below) into a shared library (on solaris:
>> `cc -g -G -xcode=pic13 -ldl -hlibtee-input.so tee-input.c -g -o
>> libtee-input.so')
>> 3. invoke `LD_PRELOAD=libtee-input.so emacs -nw -Q 2>input.txt'
>> 4. M-x xterm-mouse-mode
>> 5. flick the mouse scroll wheel hard, so it generates many ticks in
>> quick succession (note the garbage that results)
>> 6. M-x show-lossage (note the mangled escape sequences)
>> 7. C-x C-c
>> 8. Examine input.txt (note the intact escape sequences)
> Does (set-keyboard-coding-system 'binary) circumvent the problem?
No. In fact, the solaris machine which I ran the example on defaults to
`no-conversion' (an alias of binary iirc) for some reason. Sorry, I
forgot to mention before.
This matches my expectations, since bug #6920 arises before any coding
system touches the input.
Ryan
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Thu, 30 Sep 2010 11:35:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 7117 <at> debbugs.gnu.org (full text, mbox):
On 9/28/2010 6:53 AM, Ryan Johnson wrote:
> On 9/27/2010 10:52 PM, Stefan Monnier wrote:
>>> Steps to reproduce:
>>> 1. ssh to a machine with emacs-23 installed (I suspect slower
>>> networks would expose this more, but the bug bites me even over
>>> intranet)
>>> 2. compile tee-input.c (see below) into a shared library (on
>>> solaris:
>>> `cc -g -G -xcode=pic13 -ldl -hlibtee-input.so tee-input.c -g -o
>>> libtee-input.so')
>>> 3. invoke `LD_PRELOAD=libtee-input.so emacs -nw -Q 2>input.txt'
>>> 4. M-x xterm-mouse-mode
>>> 5. flick the mouse scroll wheel hard, so it generates many ticks in
>>> quick succession (note the garbage that results)
>>> 6. M-x show-lossage (note the mangled escape sequences)
>>> 7. C-x C-c
>>> 8. Examine input.txt (note the intact escape sequences)
>> Does (set-keyboard-coding-system 'binary) circumvent the problem?
> No. In fact, the solaris machine which I ran the example on defaults
> to `no-conversion' (an alias of binary iirc) for some reason. Sorry, I
> forgot to mention before.
>
> This matches my expectations, since bug #6920 arises before any coding
> system touches the input.
Any hints on where the problem might lurk? I poked around a little but
without luck -- the functions that power read-char are many hundreds of
lines long and seem to weave between lisp and C at regular intervals.
I'm willing to go source diving but it's a bit daunting to wade into the
C code again without a starting reference.
Thanks,
Ryan
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Thu, 30 Sep 2010 14:37:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 7117 <at> debbugs.gnu.org (full text, mbox):
On 9/30/2010 1:37 PM, Ryan Johnson wrote:
> On 9/28/2010 6:53 AM, Ryan Johnson wrote:
>> On 9/27/2010 10:52 PM, Stefan Monnier wrote:
>>>> Steps to reproduce:
>>>> 1. ssh to a machine with emacs-23 installed (I suspect slower
>>>> networks would expose this more, but the bug bites me even over
>>>> intranet)
>>>> 2. compile tee-input.c (see below) into a shared library (on
>>>> solaris:
>>>> `cc -g -G -xcode=pic13 -ldl -hlibtee-input.so tee-input.c -g -o
>>>> libtee-input.so')
>>>> 3. invoke `LD_PRELOAD=libtee-input.so emacs -nw -Q 2>input.txt'
>>>> 4. M-x xterm-mouse-mode
>>>> 5. flick the mouse scroll wheel hard, so it generates many ticks in
>>>> quick succession (note the garbage that results)
>>>> 6. M-x show-lossage (note the mangled escape sequences)
>>>> 7. C-x C-c
>>>> 8. Examine input.txt (note the intact escape sequences)
>>> Does (set-keyboard-coding-system 'binary) circumvent the problem?
>> No. In fact, the solaris machine which I ran the example on defaults
>> to `no-conversion' (an alias of binary iirc) for some reason. Sorry,
>> I forgot to mention before.
>>
>> This matches my expectations, since bug #6920 arises before any
>> coding system touches the input.
> Any hints on where the problem might lurk? I poked around a little but
> without luck -- the functions that power read-char are many hundreds
> of lines long and seem to weave between lisp and C at regular
> intervals. I'm willing to go source diving but it's a bit daunting to
> wade into the C code again without a starting reference.
OK, I bit the bullet and fired up a debugger. I've narrowed down the
problem some.
In keyboard.c, all keyboard input is buffered in kbd_buffer, which is a
circular buffer with tail kbd_fetch_ptr. In all cases, when a control
sequence gets mangled it's because kbd_fetch_ptr was incremented by more
than one slot between iterations of the command loop (often by several
dozen). However, it turns out that incrementing by more than one slot
happens quite often and only sometimes exhibits the bug.
More digging shows that multiple increments are due to Fdiscard_input
being called with the following stack trace:
=>[1] Fdiscard_input(), line 10947 in "keyboard.c"
[2] cmd_error_internal(data = -1059860928, context = 0xffbfee40 ""),
line 1301 in "keyboard.c"
[3] cmd_error(data = -1059860928), line 1234 in "keyboard.c"
[4] internal_condition_case(bfun = 0x111578 = &command_loop_1(),
handlers = 1077839640, hfun = 0x109b8c = &cmd_error(register Lisp_Object
data)), line 1480 in "eval.c"
[5] command_loop_2(), line 1360 in "keyboard.c"
[6] internal_catch(tag = 1077836792, func = 0x10283c =
&command_loop_2(), arg = 1077782856), line 1226 in "eval.c"
[7] command_loop(), line 1339 in "keyboard.c"
[8] recursive_edit_1(), line 954 in "keyboard.c"
[9] Frecursive_edit(), line 1016 in "keyboard.c"
[10] main(argc = 3, argv = 0xffbff374), line 1833 in "emacs.c"
At this point the symptom is easy enough to understand: sometimes
discard-input gets unlucky and chops the front off of a partly-arrived
escape sequence.
The real question is, why would the command loop encounter errors and
need to discard input at all?
[1] _longjmp()
=>[2] unwind_to_catch()
[3] Fsignal()
[4] xsignal()
[5] xsignal0()
[6] window_scroll()
[7] scroll_command()
[8] Fscroll_down()
[9] Ffuncall()
[10] Feval()
[11] internal_lisp_condition_case()
[12] Fbyte_code()
[13] funcall_lambda()
[14] Ffuncall()
[15] Fapply()
[16] apply1()
[17] Fcall_interactively()
[18] Ffuncall()
[19] call3()
[20] Fcommand_execute()
[21] command_loop_1()
[22] internal_condition_case()
[23] command_loop_2()
[24] internal_catch()
[25] command_loop()
[26] recursive_edit_1()
[27] Frecursive_edit()
[28] main()
From the above, it seems that scrolling past beginning or end of buffer
triggers an error, which I guess is somewhat justifiable. However, on a
very slow connection this morning mouse-clicking between or even within
buffers caused problems. Unfortunately, setting debug-on-error doesn't
seem to catch errors generated from C code.
I could try to break on longjmp over another slow connection, but with
the number of times it gets called for other reasons this would be slow
going...
Ideas?
Ryan
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Thu, 30 Sep 2010 14:58:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 7117 <at> debbugs.gnu.org (full text, mbox):
> From: Ryan Johnson <ryanjohn <at> ece.cmu.edu>
> Cc: 7117 <at> debbugs.gnu.org
>
> At this point the symptom is easy enough to understand: sometimes
> discard-input gets unlucky and chops the front off of a partly-arrived
> escape sequence.
>
> The real question is, why would the command loop encounter errors and
> need to discard input at all?
> [1] _longjmp()
> =>[2] unwind_to_catch()
> [3] Fsignal()
> [4] xsignal()
> [5] xsignal0()
> [6] window_scroll()
The only calls to xsignal0 I see in window_scroll (or, rather, in its
subroutines) is to signal Qbeginning_of_buffer or Qend_of_buffer.
> From the above, it seems that scrolling past beginning or end of buffer
> triggers an error, which I guess is somewhat justifiable.
Indeed. Perhaps we need some infrastructure to ignore errors in this
case (I assume `ignore-errors' won't help). Or maybe we should allow
not to discard input when we signal an error. Or maybe discard-input
should be smarter, and not discard partial escape sequences?
> However, on a
> very slow connection this morning mouse-clicking between or even within
> buffers caused problems. Unfortunately, setting debug-on-error doesn't
> seem to catch errors generated from C code.
>
> I could try to break on longjmp over another slow connection, but with
> the number of times it gets called for other reasons this would be slow
> going...
>
> Ideas?
You could put a breakpoint in Fsignal, and see if the problems with
mouse-clicking within buffers is also caused by signaling an error.
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Thu, 30 Sep 2010 15:12:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 7117 <at> debbugs.gnu.org (full text, mbox):
On 9/30/2010 4:59 PM, Eli Zaretskii wrote:
>> From: Ryan Johnson<ryanjohn <at> ece.cmu.edu>
>> Cc: 7117 <at> debbugs.gnu.org
>>
>> At this point the symptom is easy enough to understand: sometimes
>> discard-input gets unlucky and chops the front off of a partly-arrived
>> escape sequence.
>>
>> The real question is, why would the command loop encounter errors and
>> need to discard input at all?
>> [1] _longjmp()
>> =>[2] unwind_to_catch()
>> [3] Fsignal()
>> [4] xsignal()
>> [5] xsignal0()
>> [6] window_scroll()
> The only calls to xsignal0 I see in window_scroll (or, rather, in its
> subroutines) is to signal Qbeginning_of_buffer or Qend_of_buffer.
>
>> From the above, it seems that scrolling past beginning or end of buffer
>> triggers an error, which I guess is somewhat justifiable.
> Indeed. Perhaps we need some infrastructure to ignore errors in this
> case (I assume `ignore-errors' won't help). Or maybe we should allow
> not to discard input when we signal an error. Or maybe discard-input
> should be smarter, and not discard partial escape sequences?
One thing I don't get is, I've been using emacs over painfully slow ssh
connections for literally years -- sometimes slow enough that keystrokes
take visible time to echo. This was never really an issue before. In
retrospect, it did pop up occasionally all along, but it was infrequent
enough to hide in the shadow of xterm mouse escape sequences confusing
emacs for x>95. Now it interferes with normal work. Is there some change
that would have done this? I'm using the same bad internet connection as
when I had emacs-22...
>> However, on a
>> very slow connection this morning mouse-clicking between or even within
>> buffers caused problems. Unfortunately, setting debug-on-error doesn't
>> seem to catch errors generated from C code.
>>
>> I could try to break on longjmp over another slow connection, but with
>> the number of times it gets called for other reasons this would be slow
>> going...
>>
>> Ideas?
> You could put a breakpoint in Fsignal, and see if the problems with
> mouse-clicking within buffers is also caused by signaling an error.
I'll try that next time it becomes annoying (it's hit-and-miss...)
Ryan
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Thu, 30 Sep 2010 15:42:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 7117 <at> debbugs.gnu.org (full text, mbox):
> Date: Thu, 30 Sep 2010 17:14:16 +0200
> From: Ryan Johnson <ryanjohn <at> ece.cmu.edu>
> CC: monnier <at> iro.umontreal.ca, 7117 <at> debbugs.gnu.org
>
> >> From the above, it seems that scrolling past beginning or end of buffer
> >> triggers an error, which I guess is somewhat justifiable.
> > Indeed. Perhaps we need some infrastructure to ignore errors in this
> > case (I assume `ignore-errors' won't help). Or maybe we should allow
> > not to discard input when we signal an error. Or maybe discard-input
> > should be smarter, and not discard partial escape sequences?
> One thing I don't get is, I've been using emacs over painfully slow ssh
> connections for literally years -- sometimes slow enough that keystrokes
> take visible time to echo. This was never really an issue before.
What would you expect to see, that would cause you think it was "an
issue"? When Emacs is keyboard-driven, typing text and scrolling
commands seldom happen in such a quick succession that hitting end of
buffer while scrolling would discard text you typed meanwhile. And
even if it did, how would you know for sure you actually typed it?
Information forwarded
to
owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Thu, 30 Sep 2010 16:10:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 7117 <at> debbugs.gnu.org (full text, mbox):
On 9/30/2010 5:44 PM, Eli Zaretskii wrote:
>> Date: Thu, 30 Sep 2010 17:14:16 +0200
>> From: Ryan Johnson<ryanjohn <at> ece.cmu.edu>
>> CC: monnier <at> iro.umontreal.ca, 7117 <at> debbugs.gnu.org
>>
>>>> From the above, it seems that scrolling past beginning or end of buffer
>>>> triggers an error, which I guess is somewhat justifiable.
>>> Indeed. Perhaps we need some infrastructure to ignore errors in this
>>> case (I assume `ignore-errors' won't help). Or maybe we should allow
>>> not to discard input when we signal an error. Or maybe discard-input
>>> should be smarter, and not discard partial escape sequences?
>> One thing I don't get is, I've been using emacs over painfully slow ssh
>> connections for literally years -- sometimes slow enough that keystrokes
>> take visible time to echo. This was never really an issue before.
> What would you expect to see, that would cause you think it was "an
> issue"? When Emacs is keyboard-driven, typing text and scrolling
> commands seldom happen in such a quick succession that hitting end of
> buffer while scrolling would discard text you typed meanwhile. And
> even if it did, how would you know for sure you actually typed it?
It's not just keyboard driven. It's a mouse-enabled terminal and the
mouse click/wheel actions cause the issues (each one generates a burst
of 6-12 characters). As you say, I don't think I've ever seen this
happen with keyboard input; pasting commands from a remote X clipboard
might have a similar effect, but I've never tried.
In the past, scrolling once or twice past buffer begin/end wasn't
guaranteed to dump garbage. It would just beep/flash most of the time,
and the slow network connection didn't seem to make it worse. Now, if
I'm not on intranet, virtually every scroll past-end will dump garbage
somewhere.
Also, the problem I hit this morning was not related to fast
typing/clicking over a slow connection. I was doing an ediff-files
session, and about every other time I clicked on a buffer to edit a
conflict, emacs would reward me with a mangled escape code... sometimes
where point had been and sometimes where it ended up. I wasn't typing
fast or clicking madly -- it was more like "a n n n n b n n <stare a
while> <click> <grumble> <clean up mess>"
The latter problem went away once I got to work and had a fast network;
and the former happens much less frequently here (unless I give the
wheel a really good spin, but that's understandable).
Ryan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Mon, 07 Feb 2022 00:34:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 7117 <at> debbugs.gnu.org (full text, mbox):
Ryan Johnson <ryanjohn <at> ece.cmu.edu> writes:
> Emacs-23.2 occasionally loses parts of terminal escape sequences, confusing
> xt-mouse.el and dumping garbage text in the user's buffer.
(I'm going through old bug reports that unfortunately weren't resolved
at the time.)
There's been many changes in the terminal escape sequence handling since
this was reported over a decade ago -- are you still seeing these issues
in recent Emacs versions?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
Added tag(s) moreinfo.
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org
.
(Mon, 07 Feb 2022 00:34:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#7117
; Package
emacs
.
(Mon, 07 Mar 2022 02:38:03 GMT)
Full text and
rfc822 format available.
Message #37 received at 7117 <at> debbugs.gnu.org (full text, mbox):
Lars Ingebrigtsen <larsi <at> gnus.org> writes:
> There's been many changes in the terminal escape sequence handling since
> this was reported over a decade ago -- are you still seeing these issues
> in recent Emacs versions?
More information was requested, but no response was given within a
month, so I'm closing this bug report. If the problem still exists,
please respond to this email and we'll reopen the bug report.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
bug closed, send any further explanations to
7117 <at> debbugs.gnu.org and Ryan Johnson <ryanjohn <at> ece.cmu.edu>
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org
.
(Mon, 07 Mar 2022 02:38:04 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Mon, 04 Apr 2022 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 3 years and 98 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.