GNU bug report logs - #17497
24.4.50; TTY menu glitches

Previous Next

Package: emacs;

Reported by: Dmitry Antipov <dmantipov <at> yandex.ru>

Date: Thu, 15 May 2014 12:28:01 UTC

Severity: normal

Tags: patch

Found in version 24.4.50

Done: Lars Ingebrigtsen <larsi <at> gnus.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 17497 in the body.
You can then email your comments to 17497 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 15 May 2014 12:28:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Dmitry Antipov <dmantipov <at> yandex.ru>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Thu, 15 May 2014 12:28:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Antipov <dmantipov <at> yandex.ru>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.4.50; TTY menu glitches
Date: Thu, 15 May 2014 16:26:25 +0400
[Message part 1 (text/plain, inline)]
Now I'm seeing two TTY menu glitches:

0) Double-"selected" item (actually selected item on this screenshot
   is "New Window Below").

1) Out-of-menu "--" draw combined with incorrect help text in echo area.

$TERM is rxvt-unicode (version 9.20), if that matters.

Dmitry

In GNU Emacs 24.4.50.3 (x86_64-unknown-linux-gnu)
 of 2014-05-15 on localhost.localdomain
Repository revision: 117110 dmantipov <at> yandex.ru-20140515100645-4wktg3eo5f0wpbob
System Description:     Fedora release 20 (Heisenbug)

Configured using:
 `configure --prefix=/not/exists --enable-gcc-warnings
 --enable-check-lisp-object-type --enable-checking --without-x
 --without-all --disable-acl 'CFLAGS=-O0 -g3''

Configured features:

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
[ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B
ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC
[ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B
ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC
[ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B
ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC
[ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B
ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC
[ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B
ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC
[ B ESC [ B ESC [ A ESC [ A ESC [ A ESC [ B ESC [ B
ESC [ B ESC [ B ESC [ A ESC [ B ESC [ B ESC [ B ESC
[ A ESC [ A ESC [ A ESC [ A ESC [ B ESC [ A ESC [ A
ESC [ A ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B C-g
ESC [ B ESC [ B ESC x e m a c s DEL DEL DEL DEL DEL
DEL r e p o TAB r TAB RET

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
End of buffer [2 times]
delete-backward-char: Text is read-only
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message dired format-spec
rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail regexp-opt rfc2047
rfc2045 ietf-drums mm-util help-fns mail-prsvr mail-utils time-date
tooltip electric uniquify ediff-hook vc-hooks lisp-float-type
tabulated-list newcomment lisp-mode prog-mode register page menu-bar
rfn-eshadow timer select mouse jit-lock font-lock syntax facemenu
font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process multi-tty emacs)

Memory information:
((conses 16 73001 4484)
 (symbols 48 16870 0)
 (miscs 40 33 117)
 (strings 32 10242 5816)
 (string-bytes 1 279721)
 (vectors 16 7679)
 (vector-slots 8 331222 25610)
 (floats 8 60 583)
 (intervals 56 203 4)
 (buffers 960 12)
 (heap 1024 23973 1892))
[glitch0.png (image/png, attachment)]
[glitch1.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 15 May 2014 17:38:02 GMT) Full text and rfc822 format available.

Message #8 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 15 May 2014 20:37:48 +0300
> Date: Thu, 15 May 2014 16:26:25 +0400
> From: Dmitry Antipov <dmantipov <at> yandex.ru>
> 
> Now I'm seeing two TTY menu glitches:
> 
> 0) Double-"selected" item (actually selected item on this screenshot
>     is "New Window Below").
> 
> 1) Out-of-menu "--" draw combined with incorrect help text in echo area.
> 
> $TERM is rxvt-unicode (version 9.20), if that matters.

Any hope of a reproducible recipe?  I don't see this on the systems to
which I have access.

FWIW, this looks like a deja-vu of the phenomenon described in
http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00546.html.
As concluded (after a long discussion) in
http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00667.html,
we never actually understood why this happens.  Maybe try to play with
the size of your TTY window, and see if that changes anything.  Or
switch to xterm instead of rxvt.

Also, you could look into a termscript file to see whether Emacs
indeed wrote 2 lines with the red background in the first snapshot, or
wrote the "--" string at incorrect coordinates in the second.  The
discussion mentioned above concluded that perfectly correct terminal
commands caused strange unexplained effects on the screen.

> System Description:     Fedora release 20 (Heisenbug)
                                             ^^^^^^^^^
On second thought, a system with that description is supposed to do
this, no?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 06:37:02 GMT) Full text and rfc822 format available.

Message #11 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Antipov <dmantipov <at> yandex.ru>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 02:36:44 -0400
[Message part 1 (text/plain, inline)]
Using "xfce4-terminal 0.6.3 (Xfce 4.10)", I did

   emacs-24.3.91 -Q -nw
   M-x menu-bar-open RET

then simply held down the down array, and very quickly got it to look
like the first attached image. By holding down the down arrow for a bit,
then the up arrow for a bit, then the down arrow, etc, I got the second
image. It's not 100% reproducible, but seemed to happen fairly often.
I don't expect to hold down the arrow keys in normal usage. :)

[1.png (image/png, attachment)]
[2.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 06:39:02 GMT) Full text and rfc822 format available.

Message #14 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Antipov <dmantipov <at> yandex.ru>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 02:38:22 -0400
Glenn Morris wrote:

> then simply held down the down array, and very quickly got it to look

s/array/arrow




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 06:55:02 GMT) Full text and rfc822 format available.

Message #17 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Antipov <dmantipov <at> yandex.ru>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 02:53:59 -0400
[Message part 1 (text/plain, inline)]
And with the same strategy under "XTerm(303)", I quickly got the attached.
I don't normally use xterm, though, and it must be misconfigured
somehow, because pressing "M-x" produces LATIN SMALL LETTER O WITH
STROKE... :)

[3.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 08:49:02 GMT) Full text and rfc822 format available.

Message #20 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 11:48:42 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Dmitry Antipov <dmantipov <at> yandex.ru>,  17497 <at> debbugs.gnu.org
> Date: Fri, 16 May 2014 02:53:59 -0400
> 
> And with the same strategy under "XTerm(303)", I quickly got the attached.
> I don't normally use xterm, though, and it must be misconfigured
> somehow, because pressing "M-x" produces LATIN SMALL LETTER O WITH
> STROKE... :)

I have no idea what causes this.  I cannot reproduce this, neither on
Windows, nor when running Emacs on GNU/Linux via PuTTY (which emulates
xterm).

Does this problem go away if you enlarge the terminal window such that
the entire File and Tools menus can be displayed without overlaying
the mode line?  If the problems disappear then, perhaps we could solve
them by limiting the number of displayed menu items some more.

If enlarging the window doesn't help, then I don't know what to do,
unless someone shows me a recipe I can use to reproduce the problem on
some machine where I can debug them.  Failing that, perhaps some
terminfo expert could analyze the termscript and show what we do
wrongly.  Last time I looked at termscript produced in a similar case,
it looked entirely legitimate, i.e. at no time did Emacs send a
terminal command to write to the portions of display where a
screenshot showed incorrectly displayed text, and every time a
red-background menu item was incorrectly left behind, there was a
clear command in the termscript that told the terminal to redraw that
very item with the normal (blue) background.  IOW, it looked like for
some reason the terminal was not obeying the commands it was sent.
Doers anyone know why would this happen?

It could also be some timing problem, specific to X, in which case
someone will have to explain how to avoid that, because I have no
idea.

Does this happen if you use the menu "reasonably", i.e. without
leaning on arrow keys?  Does it happen if you use C-p instead of the
down arrow?

I'm at a loss here...




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 09:44:02 GMT) Full text and rfc822 format available.

Message #23 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Antipov <dmantipov <at> yandex.ru>
To: Glenn Morris <rgm <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 13:42:54 +0400
[Message part 1 (text/plain, inline)]
On 05/16/2014 10:53 AM, Glenn Morris wrote:

> And with the same strategy under "XTerm(303)", I quickly got the attached.

I found this issue XTerm-compatible too.

Just for the record, TTY menus are even more broken with Eterm.

Dmitry

[eterm.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 10:28:02 GMT) Full text and rfc822 format available.

Message #26 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: rgm <at> gnu.org, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 13:26:59 +0300
> Date: Fri, 16 May 2014 13:42:54 +0400
> From: Dmitry Antipov <dmantipov <at> yandex.ru>
> CC: 17497 <at> debbugs.gnu.org
> 
> Just for the record, TTY menus are even more broken with Eterm.

How is this more broken?  All I see is the same problem with artifacts
left behind where they shouldn't be.  Like in your original
screenshots.  The number of artifacts is not important; even one of
them is a sign of some problem.

Do you see in the termscript any commands to write these artifacts?
E.g., the red-background "Close" was presumably the selected menu item
at some previous time; do you see a command to overwrite that with the
blue-background "Close" (or something else) in insert mode?

Is there any difference in what you see if you do that in a buffer
which is large enough to fill the entire window with text, like
xdisp.c in its first portion, where there's a large commentary?  IOW,
do these problems depend on what was on the screen before the menu was
dropped down?

The TTY menus work by overwriting portions of the glyph matrix with
the text derived from the menu.  The screen is updated by the normal
Emacs code, which was not touched at all.  So I don't understand why
these artifacts appear when menus are displayed, but not with normal
buffer text display...

Could this be a buffering issue?  Maybe adding some fflush calls will
make a difference?  (Not that I understand how buffering could change
the final result.)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 10:47:02 GMT) Full text and rfc822 format available.

Message #29 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: rgm <at> gnu.org, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 13:46:02 +0300
> Date: Fri, 16 May 2014 13:42:54 +0400
> From: Dmitry Antipov <dmantipov <at> yandex.ru>
> CC: 17497 <at> debbugs.gnu.org
> 
> Just for the record, TTY menus are even more broken with Eterm.

Does the change below help in any way?

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2014-04-18 08:35:09 +0000
+++ src/xdisp.c	2014-05-16 10:35:55 +0000
@@ -21307,6 +21307,8 @@ display_tty_menu_item (const char *item_
 		    width, 0, FRAME_COLS (f) - 1, -1);
 
   row->used[TEXT_AREA] = max (saved_used, row->used[TEXT_AREA]);
+  if (row->used[TEXT_AREA] > 0)
+    row->displays_text_p = 1;
   row->truncated_on_right_p = saved_truncated;
   row->hash = row_hash (row);
   row->full_width_p = saved_width;





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 15:01:01 GMT) Full text and rfc822 format available.

Message #32 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Antipov <dmantipov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 18:59:58 +0400
On 05/16/2014 02:46 PM, Eli Zaretskii wrote:

> Does the change below help in any way?

Unfortunately no, AFAICS.

Dmitry





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 15:27:02 GMT) Full text and rfc822 format available.

Message #35 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 18:26:10 +0300
> Date: Fri, 16 May 2014 18:59:58 +0400
> From: Dmitry Antipov <dmantipov <at> yandex.ru>
> CC: 17497 <at> debbugs.gnu.org
> 
> On 05/16/2014 02:46 PM, Eli Zaretskii wrote:
> 
> > Does the change below help in any way?
> 
> Unfortunately no, AFAICS.

If you record in a termscript while doing what it takes to reproduce
the problem, and the dump that termscript to the screen, do you see
the same artifacts as you saw in Emacs?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 15:48:01 GMT) Full text and rfc822 format available.

Message #38 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 11:47:27 -0400
[Message part 1 (text/plain, inline)]
Eli Zaretskii wrote:

> Does this problem go away if you enlarge the terminal window such that
> the entire File and Tools menus can be displayed without overlaying
> the mode line? 

I maximized the xterm before starting Emacs, and still very quickly got
into the state where there is a duplicate "--" off to the right of the
real menu (same as my "1.png" from earlier). It does seem less common
with a bigger terminal though.

> Does this happen if you use the menu "reasonably", i.e. without
> leaning on arrow keys? 

So far no, IME.

> Does it happen if you use C-p instead of the down arrow?

Leaning on C-p produces glitches, yes. Here's a fun one:

[4.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 16 May 2014 20:23:02 GMT) Full text and rfc822 format available.

Message #41 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 16 May 2014 23:21:53 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Fri, 16 May 2014 11:47:27 -0400
> 
> > Does this happen if you use the menu "reasonably", i.e. without
> > leaning on arrow keys? 
> 
> So far no, IME.

Which means what? that the terminal somehow "optimizes out" some of
the commands it receives when they arrive at a high rate??




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 17 May 2014 09:57:01 GMT) Full text and rfc822 format available.

Message #44 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sat, 17 May 2014 12:56:04 +0300
[Message part 1 (text/plain, inline)]
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Fri, 16 May 2014 11:47:27 -0400
> 
> > Does this problem go away if you enlarge the terminal window such that
> > the entire File and Tools menus can be displayed without overlaying
> > the mode line? 
> 
> I maximized the xterm before starting Emacs, and still very quickly got
> into the state where there is a duplicate "--" off to the right of the
> real menu (same as my "1.png" from earlier). It does seem less common
> with a bigger terminal though.
> 
> > Does this happen if you use the menu "reasonably", i.e. without
> > leaning on arrow keys? 
> 
> So far no, IME.
> 
> > Does it happen if you use C-p instead of the down arrow?
> 
> Leaning on C-p produces glitches, yes. Here's a fun one:

Could both of you please record the terminal commands issued while you
reproduce the problem in a termscript, and then replay that termscript
to the same terminal (outside Emacs) with the attached shell script?
I'm interested to see whether the problem is reproduced by sending the
same commands at a low rate.

If sleeping for 2 seconds doesn't reproduce the problem, maybe try
decreasing the sleep time until it does.

TIA

[script (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 02:50:02 GMT) Full text and rfc822 format available.

Message #47 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rgm <at> gnu.org, dmantipov <at> yandex.ru
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 05:49:05 +0300
Ping!  Could you please do what I asked below?  Thanks.

> Date: Sat, 17 May 2014 12:56:04 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
> 
> Could both of you please record the terminal commands issued while you
> reproduce the problem in a termscript, and then replay that termscript
> to the same terminal (outside Emacs) with the attached shell script?
> I'm interested to see whether the problem is reproduced by sending the
> same commands at a low rate.
> 
> If sleeping for 2 seconds doesn't reproduce the problem, maybe try
> decreasing the sleep time until it does.
> 
> TIA




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 05:45:02 GMT) Full text and rfc822 format available.

Message #50 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 01:44:05 -0400
[Message part 1 (text/plain, inline)]
Replaying the termscript at full speed does not produce the same
glitches, if that makes any sense.

See attached image and assoicated termscript.

BTW, your script reduces to:

while read line; do
  echo "$line"
  sleep 2
done < "$1"

[1.png (image/png, attachment)]
[tscript3.xz (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 07:52:01 GMT) Full text and rfc822 format available.

Message #53 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 09:51:04 +0200
Glenn Morris <rgm <at> gnu.org> writes:

> Replaying the termscript at full speed does not produce the same
> glitches, if that makes any sense.

Then it can only be a bug in the terminal emulator.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 15:59:02 GMT) Full text and rfc822 format available.

Message #56 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 11:58:21 -0400
Andreas Schwab wrote:

> Glenn Morris <rgm <at> gnu.org> writes:
>
>> Replaying the termscript at full speed does not produce the same
>> glitches, if that makes any sense.
>
> Then it can only be a bug in the terminal emulator.

Then it's common to several of them, as has already been said.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 16:20:02 GMT) Full text and rfc822 format available.

Message #59 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 19:19:07 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Thu, 22 May 2014 01:44:05 -0400
> 
> Replaying the termscript at full speed does not produce the same
> glitches, if that makes any sense.

That's weird.  Earlier you said that if you don't lean on the arrow
keys, but press them at normal typing speed, the problem doesn't
happen as well, is that right?  If so, perhaps you could produce a
termscript for the same sequence of keypresses, just at that "normal"
speed, so we could compare them.  (If one of the termscripts has fewer
keypresses than the other, it doesn't matter, as long as you press the
same keys.)

> BTW, your script reduces to:
> 
> while read line; do
>   echo "$line"
>   sleep 2
> done < "$1"

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 16:28:01 GMT) Full text and rfc822 format available.

Message #62 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 12:26:47 -0400
Eli Zaretskii wrote:

> That's weird.  Earlier you said that if you don't lean on the arrow
> keys, but press them at normal typing speed, the problem doesn't
> happen as well, is that right? 

So far I have not seen it to occur without me leaning on keys, but it
could be that it only happens on say 1/100 key presses, as opposed to
when the key rate is high. I don't use text-mode menus normally.

> If so, perhaps you could produce a termscript for the same sequence of
> keypresses, just at that "normal" speed, so we could compare them. (If
> one of the termscripts has fewer keypresses than the other, it doesn't
> matter, as long as you press the same keys.)

You mean I can replace "lean on the down key, lean on the up key, lean
on the down key" with just "down, up, down"?
If so, I'll give it a go later on.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 16:44:02 GMT) Full text and rfc822 format available.

Message #65 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rgm <at> gnu.org, dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 19:43:32 +0300
> Date: Thu, 22 May 2014 19:19:07 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
> 
> > From: Glenn Morris <rgm <at> gnu.org>
> > Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> > Date: Thu, 22 May 2014 01:44:05 -0400
> > 
> > Replaying the termscript at full speed does not produce the same
> > glitches, if that makes any sense.
> 
> That's weird.

Wait, maybe I misunderstood you.  Did you mean that replaying at full
speed (i.e. without the 'sleep' line, I presume?) makes the glitches
go away, or did you mean there are still glitches, but they look
different on the screen?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 16:47:02 GMT) Full text and rfc822 format available.

Message #68 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 19:46:34 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Thu, 22 May 2014 12:26:47 -0400
> 
> > If so, perhaps you could produce a termscript for the same sequence of
> > keypresses, just at that "normal" speed, so we could compare them. (If
> > one of the termscripts has fewer keypresses than the other, it doesn't
> > matter, as long as you press the same keys.)
> 
> You mean I can replace "lean on the down key, lean on the up key, lean
> on the down key" with just "down, up, down"?

Actually, something like 20 times down, then 20 times up would be
better, I think.

Also, if you revert revision 117033 on emacs-24 branch (or try with a
binary built before Apr 29, if you keep them), does the problem still
exist?

TIA




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 16:55:02 GMT) Full text and rfc822 format available.

Message #71 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 12:54:26 -0400
Eli Zaretskii wrote:

> Wait, maybe I misunderstood you.  Did you mean that replaying at full
> speed (i.e. without the 'sleep' line, I presume?) makes the glitches
> go away, or did you mean there are still glitches, but they look
> different on the screen?

With no sleep, there are no glitches (that I could see).
I certainly did not end up with that stray "--" off to the right.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 17:08:01 GMT) Full text and rfc822 format available.

Message #74 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 20:07:40 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Thu, 22 May 2014 12:54:26 -0400
> 
> With no sleep, there are no glitches (that I could see).
> I certainly did not end up with that stray "--" off to the right.

IOW, sending commands at fast rate by leaning on the keys does produce
the glitches, but replaying the same commands at fast rate doesn't.
Curiouser and curiouser...




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 17:20:02 GMT) Full text and rfc822 format available.

Message #77 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 19:19:46 +0200
Glenn Morris <rgm <at> gnu.org> writes:

> Andreas Schwab wrote:
>
>> Glenn Morris <rgm <at> gnu.org> writes:
>>
>>> Replaying the termscript at full speed does not produce the same
>>> glitches, if that makes any sense.
>>
>> Then it can only be a bug in the terminal emulator.
>
> Then it's common to several of them, as has already been said.

That's very well possible, since they are often just forks of each
other.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 17:30:05 GMT) Full text and rfc822 format available.

Message #80 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 13:29:40 -0400
Andreas Schwab wrote:

> That's very well possible, since they are often just forks of each
> other.

OK, so far people have seen such issues with

xterm 303
rxvt-unicode 9.20
xfce4-terminal 0.6.3
eterm
whatever martin was using in
   http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00462.html




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 22 May 2014 17:54:01 GMT) Full text and rfc822 format available.

Message #83 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: schwab <at> suse.de, dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 22 May 2014 20:53:16 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Date: Thu, 22 May 2014 13:29:40 -0400
> Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
> 
> whatever martin was using in
>    http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00462.html

Not sure that was the same problem, as it disappeared when replayed
with a script like the one you used.  It also disappeared in xterm on
the same machine, and when the size of the terminal window was
enlarged.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Fri, 30 May 2014 09:23:02 GMT) Full text and rfc822 format available.

Message #86 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rgm <at> gnu.org, dmantipov <at> yandex.ru
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 30 May 2014 12:22:34 +0300
> Date: Thu, 22 May 2014 19:46:34 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
> 
> > From: Glenn Morris <rgm <at> gnu.org>
> > Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> > Date: Thu, 22 May 2014 12:26:47 -0400
> > 
> > > If so, perhaps you could produce a termscript for the same sequence of
> > > keypresses, just at that "normal" speed, so we could compare them. (If
> > > one of the termscripts has fewer keypresses than the other, it doesn't
> > > matter, as long as you press the same keys.)
> > 
> > You mean I can replace "lean on the down key, lean on the up key, lean
> > on the down key" with just "down, up, down"?
> 
> Actually, something like 20 times down, then 20 times up would be
> better, I think.
> 
> Also, if you revert revision 117033 on emacs-24 branch (or try with a
> binary built before Apr 29, if you keep them), does the problem still
> exist?

Ping!  Any more info on this?

TIA




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 31 May 2014 02:23:01 GMT) Full text and rfc822 format available.

Message #89 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Fri, 30 May 2014 22:22:29 -0400
[Message part 1 (text/plain, inline)]
Eli Zaretskii wrote:

>> Also, if you revert revision 117033 on emacs-24 branch (or try with a
>> binary built before Apr 29, if you keep them), does the problem still
>> exist?

Reverting 117033: still glitchy.

> > > If so, perhaps you could produce a termscript for the same sequence of
> > > keypresses, just at that "normal" speed, so we could compare them. (If
> > > one of the termscripts has fewer keypresses than the other, it doesn't
> > > matter, as long as you press the same keys.)
> > 
> > You mean I can replace "lean on the down key, lean on the up key, lean
> > on the down key" with just "down, up, down"?
> 
> Actually, something like 20 times down, then 20 times up would be
> better, I think.

OK, so in trying to do this, I have noticed it glitching when I press
the keys at normal speed. So I presume it's not useful to send you that
comparison after all.

See attached image and associated typescript.
In this case I think I just opened the menu-bar and pressed the down
arrow three times at normal speed.

FWIW, I could not make it happen on a RHEL 6.5 system at all.
But on a Debian testing system, it happens with multiple terminal emulators.

(I suspect this is going to be one of those things you need to be able
to reproduce to fix...)

[2.png (image/png, attachment)]
[t2.txt.xz (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 31 May 2014 08:21:01 GMT) Full text and rfc822 format available.

Message #92 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sat, 31 May 2014 11:20:59 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Fri, 30 May 2014 22:22:29 -0400
> 
> >> Also, if you revert revision 117033 on emacs-24 branch (or try with a
> >> binary built before Apr 29, if you keep them), does the problem still
> >> exist?
> 
> Reverting 117033: still glitchy.

OK, one suspect down.

> OK, so in trying to do this, I have noticed it glitching when I press
> the keys at normal speed. So I presume it's not useful to send you that
> comparison after all.

No, it's not useful.  But I wonder why earlier you thought that this
didn't happen at normal speed.  Is it more rare at normal speed?

> See attached image and associated typescript.
> In this case I think I just opened the menu-bar and pressed the down
> arrow three times at normal speed.

When I replay that typescript on fencepost.gnu.org (logging into it
via PuTTY, which emulates xterm), I see no glitches at all, FWIW.

Interestingly, there's a cursor shown in the image to the left of
"Visit New File" menu item; it shouldn't be there.

Moreover, if I login to fencepost, set my terminal's height to be 24
lines, like in your screenshot, and then record the termscript with
the same 3 keystrokes as you did, I get an identical script!  So Emacs
behaves identically on these 2 systems, it's something in the terminal
emulators and/or the libraries they use that causes the differences in
behavior.

> FWIW, I could not make it happen on a RHEL 6.5 system at all.
> But on a Debian testing system, it happens with multiple terminal emulators.
> 
> (I suspect this is going to be one of those things you need to be able
> to reproduce to fix...)

Can you reproduce the problem when you log into that Debian testing
system remotely, via some xterm emulator?  If so, I'd appreciate an
SSH login to that system, and a directory with an Emacs tree to play
in.  Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 31 May 2014 17:36:02 GMT) Full text and rfc822 format available.

Message #95 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sat, 31 May 2014 13:35:26 -0400
Eli Zaretskii wrote:

> Can you reproduce the problem when you log into that Debian testing
> system remotely, via some xterm emulator?  If so, I'd appreciate an
> SSH login to that system, and a directory with an Emacs tree to play
> in.  Thanks.

Sorry, that is my laptop, and I don't have incoming ssh enabled on it,
and as a general rule don't want to. Nothing against you personally! :)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 15:12:02 GMT) Full text and rfc822 format available.

Message #98 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rgm <at> gnu.org
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 18:11:08 +0300
One more idea: can you try decreasing the number 900 in this snippet
from dispnew.c:

	  if (FRAME_TERMCAP_P (f))
	    {
	      /* Flush out every so many lines.
		 Also flush out if likely to have more than 1k buffered
		 otherwise.   I'm told that some telnet connections get
		 really screwed by more than 1k output at once.  */
	      FILE *display_output = FRAME_TTY (f)->output;
	      if (display_output)
		{
		  ptrdiff_t outq = __fpending (display_output);
		  if (outq > 900
		      || (outq > 20 && ((i - 1) % preempt_count == 0)))
		    fflush (display_output);
		}
	    }

Or maybe even make display_output line-buffered.  I wonder if that has
any effect on the problem.

TIA




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 16:26:01 GMT) Full text and rfc822 format available.

Message #101 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org
Subject: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 19:25:29 +0300
[I asked Thomas for his expert opinion on this strange issue, and he
graciously agreed.]

> Date: Sun, 01 Jun 2014 11:26:57 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: dickey <at> his.com
> 
> On Sun, Jun 01, 2014 at 06:07:33PM +0300, Eli Zaretskii wrote:
> > > Date: Sat, 31 May 2014 16:09:47 -0400
> > > From: Thomas Dickey <dickey <at> his.com>
> > > Cc: dickey <at> his.com
> > > 
> > > The discussion sounds like a common problem: terminals send cursor- and
> > > function-keys generally as a sequence of bytes, requiring an application
> > > to wait for each sequence to complete.  If each keystroke causes the
> > > application to do something that writes a lot of text to the screen (such
> > > as scrolling), it's possible to have some of those sequences timeout.
> > > Once that happens, the application doesn't see cursor-keys, it sees the
> > > individual bytes - which can be interpreted differently (including echoing
> > > and further messing up the display).
> > > 
> > > While I'm aware that Emacs doesn't use ncurses for screen optimization,
> > > which is the same issue implied in the description of ESCDELAY in the
> > > ncurses manpage, which begins
> > > 
> > >        ESCDELAY
> > >             Specifies the total time, in milliseconds, for  which  ncurses
> > >             will  await  a  character sequence, e.g., a function key.  The
> > >             default value, 1000 milliseconds, is  enough  for  most  uses.
> > >             However, it is made a variable to accommodate unusual applica‐
> > >             tions.
> > > 
> > >             The most common instance where you may  wish  to  change  this
> > >             value  is to work with slow hosts, e.g., running on a network.
> > >             If the host cannot read characters  rapidly  enough,  it  will
> > >             have  the  same effect as if the terminal did not send charac‐
> > >             ters rapidly enough.  The library will still see a timeout.
> > 
> > Thanks.
> > 
> > However, what puzzles me is that the code which implements text-mode
> > menus doesn't change at all how Emacs handles display on a TTY.  We
> > just overwrite portions of the display with the menu text in some
> > internal data structure, then hand out that structure to the code
> > which updates the screen, as if the buffer text has changed.  So how
> > come we never saw similar problems with normal Emacs display on a TTY,
> > e.g. when the user presses PgDn to scroll the text?
> 
> Scrolling one line at a time versus scrolling the whole page differs
> most noticeably if the application doesn't (or can't) use a terminal's
> scrolling features.  If it doesn't scroll, then the whole screen has to
> be repainted for each update.

Emacs optimizes redrawing by comparing the previous and the desired
display, and then only repainting the changed parts.

None of that changes when the menu is displayed, we just overwrite
portions of the desired display data structure, then call the normal
redrawing routines.  That's why it puzzles me why the problems are
being reported only for the menus.

> Repainting the screen after scrolling one line is going to be much slower
> than repainting the screen after scrolling one page.

Yes, I understand.  But repainting the menu when the user presses,
e.g., the down arrow, is no different: Emacs just repaints the
previously-selected menu item in the normal color, then repaints the
one below it in the "selected" color.  (Actually, since the menu item
is usually shorter than the terminal width, only part of each of those
2 lines is redrawn.)

> > Btw, is it OK to CC the Emacs bug tracker, so that this discussion is
> > recorded there?
> 
> yes

Thanks.  The 17497 <at> debbugs.gnu.org address above will achieve that.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 17:14:01 GMT) Full text and rfc822 format available.

Message #104 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 13:12:44 -0400
[Message part 1 (text/plain, inline)]
On Sun, Jun 01, 2014 at 07:25:29PM +0300, Eli Zaretskii wrote:
> Emacs optimizes redrawing by comparing the previous and the desired
> display, and then only repainting the changed parts.

That's the intent - but
> 
> None of that changes when the menu is displayed, we just overwrite
> portions of the desired display data structure, then call the normal
> redrawing routines.  That's why it puzzles me why the problems are
> being reported only for the menus.

I downloaded tscript3.xz from message #50, and am cat'ing it (slowly)
to a 24x80 xterm.  It very clearly is not repainting just changed portions.
At the very beginning of the script, the screen updates just a couple of
lines (old/new) as the cursor moves across menu items.  But as soon as
the cursor goes past the menu items which are first shown, the program
is repainting the entire menu.  Thereafter, even after it has returned
to the original view, there are occasional repaints of the entire menu,
as well as cursor movement to/from the "Close" entry.

Most of the file, however, consists of repainting the menu.

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 17:19:01 GMT) Full text and rfc822 format available.

Message #107 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 13:18:17 -0400
[Message part 1 (text/plain, inline)]
On Sun, Jun 01, 2014 at 01:12:44PM -0400, Thomas Dickey wrote:
> Most of the file, however, consists of repainting the menu.

By the way, since the menu covers only half the screen, there aren't
any general-purpose scrolling optimizations that would help.  (xterm
does support left/right margins, which would be interesting to explore
in this area).  What ncurses does when it's getting behind is to drop
updates - the typeahead feature:

       The  curses  library does ``line-breakout optimization'' by looking for
       typeahead periodically while updating the screen.  If input  is  found,
       and  it is coming from a tty, the current update is postponed until re-
       fresh or doupdate is called again.  This allows faster response to com-
       mands  typed  in  advance.   Normally, the input FILE pointer passed to
       newterm, or stdin in the case that initscr was used, will be used to do
       this typeahead checking.  The typeahead routine specifies that the file
       descriptor fd is to be used to check for typeahead instead.  If  fd  is
       -1, then no typeahead checking is done.

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 18:41:03 GMT) Full text and rfc822 format available.

Message #110 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 21:39:47 +0300
> Date: Sun, 01 Jun 2014 13:12:44 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: dickey <at> his.com, 17497 <at> debbugs.gnu.org
> 
> I downloaded tscript3.xz from message #50, and am cat'ing it (slowly)
> to a 24x80 xterm.  It very clearly is not repainting just changed portions.
> At the very beginning of the script, the screen updates just a couple of
> lines (old/new) as the cursor moves across menu items.  But as soon as
> the cursor goes past the menu items which are first shown, the program
> is repainting the entire menu.  Thereafter, even after it has returned
> to the original view, there are occasional repaints of the entire menu,
> as well as cursor movement to/from the "Close" entry.

I will look into this, thanks.  I think I actually force full redraws
under some conditions.  I will see if they are necessary.

But even these full repaints shouldn't be causing any artifacts as
shown in the screenshots, should they?  Emacs sometimes redraws a full
screen (e.g., when you switch a frame on a TTY), but I never heard any
complaints about the results.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 18:46:01 GMT) Full text and rfc822 format available.

Message #113 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 21:45:07 +0300
> Date: Sun, 01 Jun 2014 13:18:17 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
> 
> What ncurses does when it's getting behind is to drop updates - the
> typeahead feature:
> 
>        The  curses  library does ``line-breakout optimization'' by looking for
>        typeahead periodically while updating the screen.  If input  is  found,
>        and  it is coming from a tty, the current update is postponed until re-
>        fresh or doupdate is called again.  This allows faster response to com-
>        mands  typed  in  advance.   Normally, the input FILE pointer passed to
>        newterm, or stdin in the case that initscr was used, will be used to do
>        this typeahead checking.  The typeahead routine specifies that the file
>        descriptor fd is to be used to check for typeahead instead.  If  fd  is
>        -1, then no typeahead checking is done.

So buffering output more aggressively could help, is that what you are
saying?

We currently fflush the stream every 900 bytes and also every 10
screen lines or so.  Does that sound reasonable?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sun, 01 Jun 2014 19:47:02 GMT) Full text and rfc822 format available.

Message #116 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Sun, 01 Jun 2014 15:46:00 -0400
[Message part 1 (text/plain, inline)]
On Sun, Jun 01, 2014 at 09:45:07PM +0300, Eli Zaretskii wrote:
> > Date: Sun, 01 Jun 2014 13:18:17 -0400
> > From: Thomas Dickey <dickey <at> his.com>
> > Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
> > 
> > What ncurses does when it's getting behind is to drop updates - the
> > typeahead feature:
> > 
> >        The  curses  library does ``line-breakout optimization'' by looking for
> >        typeahead periodically while updating the screen.  If input  is  found,
> >        and  it is coming from a tty, the current update is postponed until re-
> >        fresh or doupdate is called again.  This allows faster response to com-
> >        mands  typed  in  advance.   Normally, the input FILE pointer passed to
> >        newterm, or stdin in the case that initscr was used, will be used to do
> >        this typeahead checking.  The typeahead routine specifies that the file
> >        descriptor fd is to be used to check for typeahead instead.  If  fd  is
> >        -1, then no typeahead checking is done.
> 
> So buffering output more aggressively could help, is that what you are
> saying?
> 
> We currently fflush the stream every 900 bytes and also every 10
> screen lines or so.  Does that sound reasonable?

I don't think that will be enough: the output stream simply is not fast
enough to keep up.  The typeahead feature is crude, but works.  A more
elegant approach (perhaps complicated) would be to keep track of the
changes since the beginning of a repaint, and store _those_ into a
buffer which could be discarded if it is not written before the next
input character is detected.

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Mon, 02 Jun 2014 15:18:02 GMT) Full text and rfc822 format available.

Message #119 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Thomas Dickey <dickey <at> his.com>, Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Mon, 02 Jun 2014 18:17:08 +0300
> Date: Sun, 01 Jun 2014 15:46:00 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: dickey <at> his.com, 17497 <at> debbugs.gnu.org
> 
> > So buffering output more aggressively could help, is that what you are
> > saying?
> > 
> > We currently fflush the stream every 900 bytes and also every 10
> > screen lines or so.  Does that sound reasonable?
> 
> I don't think that will be enough: the output stream simply is not fast
> enough to keep up.

That'd be strange, since it evidently does succeed to keep up when we
redraw the display "normally", i.e. not due to dropping down or
updating a menu.

Anyway, I took another look at all the screenshots posted in this bug
report, and my conclusion is that all of the artifacts seem to be
caused by 2 root causes:

  . Incorrect position of the cursor when a menu item is redrawn.

    This explains why some menu items are "duplicated" elsewhere in
    the menu, and also why items that needed to be redrawn with blue
    background (i.e. in the "non-selected" face) are left with the red
    background of the "selected" face.

    In all of these cases, it looks like the cursor was positioned at
    the EOB of the buffer (*scratch*) displayed beneath the menu.
    This is the "normal" cursor position, as Emacs perceives it, and
    that's where redisplay leaves the cursor after updating the frame
    display.  We override that in the menu-display code, by sending a
    cursor motion command after the frame is completely displayed; it
    looks like in at least some of the cases this cursor motion
    command was not obeyed.

  . The insert mode is not turned off before some string is written to
    the display.

    This explains why we see menu items to the right of the menu: they
    were "pushed" by writing some other text to their left, while in
    insert mode.  There are similar problems in some of the
    screenshots with the help-echo displayed in the echo area.

    Alternatively, this type of artifacts can also be explained by
    incorrect cursor position in the horizontal axis.  To decide which
    explanation is correct, I'd need to see the artifacts when the
    underlying window is full of some buffer text, not almost empty as
    in *scratch*.  Glenn, could I persuade you to try that and show
    the screenshots?

Does the above ring any bells?

Please also note that Emacs tries to be "clever" about cursor motion
on a TTY: it chooses out of several possible methods of moving the
cursor, comparing their "costs" (see cm.c:cmgoto for details).  So
it's possible that different cursor movements emit different commands
to the terminal driver, and somehow trigger these strange effects.

Finally, one difference between the "normal" screen update and the one
we use to display and update TTY menus is that the latter seems to
cause significantly more cursor motion.  One possible way to cut down
on that is to set show-help-function to nil.  Glenn, can I ask you to
try that and see if it helps in any way?

TIA




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Mon, 02 Jun 2014 16:15:02 GMT) Full text and rfc822 format available.

Message #122 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, Thomas Dickey <dickey <at> his.com>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Mon, 02 Jun 2014 12:14:05 -0400
[Message part 1 (text/plain, inline)]
Eli Zaretskii wrote:

>     explanation is correct, I'd need to see the artifacts when the
>     underlying window is full of some buffer text, not almost empty as
>     in *scratch*.  Glenn, could I persuade you to try that and show
>     the screenshots?

See 1.png.

> cause significantly more cursor motion.  One possible way to cut down
> on that is to set show-help-function to nil.  Glenn, can I ask you to
> try that and see if it helps in any way?

It does not help.
See 2.png. It leaves my screen like 3.png after closing the menu.

[1.png (image/png, attachment)]
[2.png (image/png, attachment)]
[3.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Mon, 02 Jun 2014 16:44:01 GMT) Full text and rfc822 format available.

Message #125 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Mon, 02 Jun 2014 19:43:34 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Thomas Dickey <dickey <at> his.com>,  17497 <at> debbugs.gnu.org
> Date: Mon, 02 Jun 2014 12:14:05 -0400
> 
> >     explanation is correct, I'd need to see the artifacts when the
> >     underlying window is full of some buffer text, not almost empty as
> >     in *scratch*.  Glenn, could I persuade you to try that and show
> >     the screenshots?
> 
> See 1.png.

Thanks.  Inconclusive.

> > cause significantly more cursor motion.  One possible way to cut down
> > on that is to set show-help-function to nil.  Glenn, can I ask you to
> > try that and see if it helps in any way?
> 
> It does not help.
> See 2.png. It leaves my screen like 3.png after closing the menu.

Not sure what that means, except that the terminal scrolled for some
reason.

Btw, did you use menu-bar-open (as opposed to F10) in all your
experiments till now, or just this time?

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Mon, 02 Jun 2014 16:48:02 GMT) Full text and rfc822 format available.

Message #128 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Mon, 02 Jun 2014 12:46:53 -0400
Eli Zaretskii wrote:

> Btw, did you use menu-bar-open (as opposed to F10) in all your
> experiments till now, or just this time?

I generally (but not I think always) used menu-bar-open.
Some terminal emulators use F10 to open their own menus, and I could not
be bothered to figure out how to disable that. :)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Mon, 02 Jun 2014 16:57:02 GMT) Full text and rfc822 format available.

Message #131 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Mon, 02 Jun 2014 19:56:18 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dickey <at> his.com,  17497 <at> debbugs.gnu.org
> Date: Mon, 02 Jun 2014 12:46:53 -0400
> 
> I generally (but not I think always) used menu-bar-open.

Just FYI: doing so exposes a subtle bug, due to the fact that "M-x"
adds an item to the menu bar, which confuses the menu code if you use
horizontal arrows or C-f/C-b.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Mon, 02 Jun 2014 17:06:01 GMT) Full text and rfc822 format available.

Message #134 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Mon, 02 Jun 2014 13:05:36 -0400
Eli Zaretskii wrote:

>> I generally (but not I think always) used menu-bar-open.
>
> Just FYI: doing so exposes a subtle bug, due to the fact that "M-x"
> adds an item to the menu bar, which confuses the menu code if you use
> horizontal arrows or C-f/C-b.

OK, well I did say that I used M-x menu-bar-open in my first post to
this report (which isn't even my bug report BTW... :) )

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17497#11




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 03 Jun 2014 04:52:02 GMT) Full text and rfc822 format available.

Message #137 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Tue, 03 Jun 2014 00:51:50 -0400
Eli Zaretskii wrote:

> One more idea: can you try decreasing the number 900 in this snippet
> from dispnew.c:

I set it to 450: still glitchy.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 03 Jun 2014 07:04:02 GMT) Full text and rfc822 format available.

Message #140 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: dmantipov <at> yandex.ru, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Tue, 03 Jun 2014 10:03:01 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dmantipov <at> yandex.ru,  17497 <at> debbugs.gnu.org
> Date: Tue, 03 Jun 2014 00:51:50 -0400
> 
> Eli Zaretskii wrote:
> 
> > One more idea: can you try decreasing the number 900 in this snippet
> > from dispnew.c:
> 
> I set it to 450: still glitchy.

Thanks for trying.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 03 Jun 2014 13:44:02 GMT) Full text and rfc822 format available.

Message #143 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Tue, 03 Jun 2014 09:43:04 -0400
Hi Thomas,

>> Most of the file, however, consists of repainting the menu.
> By the way, since the menu covers only half the screen, there aren't
> any general-purpose scrolling optimizations that would help.  (xterm
> does support left/right margins, which would be interesting to explore
> in this area).  What ncurses does when it's getting behind is to drop
> updates - the typeahead feature:

Currently, we're not concerned about optimization, just about tracking
down a display glitch.  The tricky part of this glitch is that if we
record&replay Emacs's output, the "replay" does not suffer from the
same glitch.

So apparently the terminal emulator behaves differently in the "live"
case than in the "replay" case for some reason.  We tried to replay at
different speeds to see if it was related to timing, but to no avail.

To me, the next logical explanation is that the terminal emulator's
behavior is influenced y the relative timing of *input* and output.

For that reasons, your ncurses explanation is very interesting, yet
I can't imagine how it can be directly related since our "record&replay"
is done "after" ncurses, directly in the stream between Emacs's process
and the terminal emulator.

Do you have some other insight that could explain why the terminal
emulator would react differently in the "live" case than in the
"replay" case?


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 03 Jun 2014 18:49:01 GMT) Full text and rfc822 format available.

Message #146 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org,
 Thomas Dickey <dickey <at> his.com>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Tue, 03 Jun 2014 14:47:49 -0400
[Message part 1 (text/plain, inline)]
On Tue, Jun 03, 2014 at 09:43:04AM -0400, Stefan Monnier wrote:
> Hi Thomas,
> 
> >> Most of the file, however, consists of repainting the menu.
> > By the way, since the menu covers only half the screen, there aren't
> > any general-purpose scrolling optimizations that would help.  (xterm
> > does support left/right margins, which would be interesting to explore
> > in this area).  What ncurses does when it's getting behind is to drop
> > updates - the typeahead feature:
> 
> Currently, we're not concerned about optimization, just about tracking
> down a display glitch.  The tricky part of this glitch is that if we
> record&replay Emacs's output, the "replay" does not suffer from the
> same glitch.

sure - but I gave my best answer at the outset.  The behavior which
you are describing won't show up by doing replay's, because it would
occur when you have input (from the keyboard) interfering with the
output.
 
> So apparently the terminal emulator behaves differently in the "live"
> case than in the "replay" case for some reason.  We tried to replay at
> different speeds to see if it was related to timing, but to no avail.
> 
> To me, the next logical explanation is that the terminal emulator's
> behavior is influenced y the relative timing of *input* and output.

:-)
 
> For that reasons, your ncurses explanation is very interesting, yet
> I can't imagine how it can be directly related since our "record&replay"
> is done "after" ncurses, directly in the stream between Emacs's process
> and the terminal emulator.
> 
> Do you have some other insight that could explain why the terminal
> emulator would react differently in the "live" case than in the
> "replay" case?

repeating: if the cursor-key sequence of characters is misinterpreted
(for example due to timeouts), then fragments of the sequences will
echo as unexpected characters.

Incidentally, if an *output* splits up an escape sequence across
buffers in fflush's, then that can also open up holes in timing.
But I think that's less of concern...

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 03 Jun 2014 21:08:01 GMT) Full text and rfc822 format available.

Message #149 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 00:07:24 +0300
> Date: Tue, 03 Jun 2014 14:47:49 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: Thomas Dickey <dickey <at> his.com>, 17497 <at> debbugs.gnu.org,
>  Eli Zaretskii <eliz <at> gnu.org>
> 
> > Currently, we're not concerned about optimization, just about tracking
> > down a display glitch.  The tricky part of this glitch is that if we
> > record&replay Emacs's output, the "replay" does not suffer from the
> > same glitch.
> 
> sure - but I gave my best answer at the outset.  The behavior which
> you are describing won't show up by doing replay's, because it would
> occur when you have input (from the keyboard) interfering with the
> output.
>  
> > So apparently the terminal emulator behaves differently in the "live"
> > case than in the "replay" case for some reason.  We tried to replay at
> > different speeds to see if it was related to timing, but to no avail.
> > 
> > To me, the next logical explanation is that the terminal emulator's
> > behavior is influenced y the relative timing of *input* and output.
> 
> :-)
>  
> > For that reasons, your ncurses explanation is very interesting, yet
> > I can't imagine how it can be directly related since our "record&replay"
> > is done "after" ncurses, directly in the stream between Emacs's process
> > and the terminal emulator.
> > 
> > Do you have some other insight that could explain why the terminal
> > emulator would react differently in the "live" case than in the
> > "replay" case?
> 
> repeating: if the cursor-key sequence of characters is misinterpreted
> (for example due to timeouts), then fragments of the sequences will
> echo as unexpected characters.
> 
> Incidentally, if an *output* splits up an escape sequence across
> buffers in fflush's, then that can also open up holes in timing.
> But I think that's less of concern...

All this, including input interfering with output, happens during
normal Emacs redisplay, but we have never heard any complaints about
corrupted display like the ones we get with the menus.

The Emacs code which outputs commands and text to the terminal does
not know whether a menu has been dropped or not, it just compares the
previous display with the desired one, and sends commands to update
the regions of the screen that has been changed.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 03 Jun 2014 22:23:01 GMT) Full text and rfc822 format available.

Message #152 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Tue, 03 Jun 2014 18:21:55 -0400
[Message part 1 (text/plain, inline)]
On Wed, Jun 04, 2014 at 12:07:24AM +0300, Eli Zaretskii wrote:
> > Date: Tue, 03 Jun 2014 14:47:49 -0400
> > From: Thomas Dickey <dickey <at> his.com>
> > Cc: Thomas Dickey <dickey <at> his.com>, 17497 <at> debbugs.gnu.org,
> >  Eli Zaretskii <eliz <at> gnu.org>
> > 
> > > Currently, we're not concerned about optimization, just about tracking
> > > down a display glitch.  The tricky part of this glitch is that if we
> > > record&replay Emacs's output, the "replay" does not suffer from the
> > > same glitch.
> > 
> > sure - but I gave my best answer at the outset.  The behavior which
> > you are describing won't show up by doing replay's, because it would
> > occur when you have input (from the keyboard) interfering with the
> > output.
> >  
> > > So apparently the terminal emulator behaves differently in the "live"
> > > case than in the "replay" case for some reason.  We tried to replay at
> > > different speeds to see if it was related to timing, but to no avail.
> > > 
> > > To me, the next logical explanation is that the terminal emulator's
> > > behavior is influenced y the relative timing of *input* and output.
> > 
> > :-)
> >  
> > > For that reasons, your ncurses explanation is very interesting, yet
> > > I can't imagine how it can be directly related since our "record&replay"
> > > is done "after" ncurses, directly in the stream between Emacs's process
> > > and the terminal emulator.
> > > 
> > > Do you have some other insight that could explain why the terminal
> > > emulator would react differently in the "live" case than in the
> > > "replay" case?
> > 
> > repeating: if the cursor-key sequence of characters is misinterpreted
> > (for example due to timeouts), then fragments of the sequences will
> > echo as unexpected characters.
> > 
> > Incidentally, if an *output* splits up an escape sequence across
> > buffers in fflush's, then that can also open up holes in timing.
> > But I think that's less of concern...
> 
> All this, including input interfering with output, happens during
> normal Emacs redisplay, but we have never heard any complaints about
> corrupted display like the ones we get with the menus.

The menus cover _half_ of the screen, defeating any attempt to speed up
scrolling by using the terminal's indexing/scrolling features.  In the
view without menus, the scrolled text (unless you're considering scrolling
a pane of a vertically split window), is full-width, and works with
the terminal's scrolling features.

(If I were debugging something of the sort we're discussing, I'd log
the input-stream in terms of what the application is seeing, to look
for broken cursor-up/down sequences).
 
> The Emacs code which outputs commands and text to the terminal does
> not know whether a menu has been dropped or not, it just compares the
> previous display with the desired one, and sends commands to update
> the regions of the screen that has been changed.

sure - we're talking about characters.

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 03:04:01 GMT) Full text and rfc822 format available.

Message #155 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Tue, 03 Jun 2014 23:03:08 -0400
> repeating: if the cursor-key sequence of characters is misinterpreted
> (for example due to timeouts), then fragments of the sequences will
> echo as unexpected characters.

I do not understand what that means.  By "cursor-key sequence of
characters" do you mean the escape sequence sent by the terminal to
Emacs or the cursor-motion escape sequences sent from Emacs to the
text terminal?
What timeouts are you referring to?  Are you saying that the terminal
emulator has internal timeouts?  If so, can you describe when and how
they come into play?

It seems you're saying that the escape sequences Emacs sends to the
terminal need to be within a single "packet", because if such an escape
sequence is split among several "fflush" the terminal emulator won't
wait indefinitely for the second chunk, and instead will timeout after
a fairly short wait and consider the escape sequence as "unexpected
characters".


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 06:55:02 GMT) Full text and rfc822 format available.

Message #158 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 09:54:24 +0300
> Date: Tue, 03 Jun 2014 18:21:55 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: dickey <at> his.com, monnier <at> iro.umontreal.ca, 17497 <at> debbugs.gnu.org
> 
> > All this, including input interfering with output, happens during
> > normal Emacs redisplay, but we have never heard any complaints about
> > corrupted display like the ones we get with the menus.
> 
> The menus cover _half_ of the screen, defeating any attempt to speed up
> scrolling by using the terminal's indexing/scrolling features.  In the
> view without menus, the scrolled text (unless you're considering scrolling
> a pane of a vertically split window), is full-width, and works with
> the terminal's scrolling features.

You can easily make the "normal" display do the same: just divide the
window in 2 halves by typing "C-x 3", then scroll one of them with
C-v or the arrow keys.  If you actually lean on the scrolling key, you
will make sure output and input are intermingled.

If this situation were inherently conducive to display bugs, we would
have heard about it long time ago.

So I think there's something about the screen update due to menus that
triggers these problems.  I just have run out of ideas about what that
factor(s) could be, and I don't have access to any system where this
happens to debug that.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 08:32:02 GMT) Full text and rfc822 format available.

Message #161 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org,
 Thomas Dickey <dickey <at> his.com>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 04:31:02 -0400
[Message part 1 (text/plain, inline)]
On Tue, Jun 03, 2014 at 11:03:08PM -0400, Stefan Monnier wrote:
> > repeating: if the cursor-key sequence of characters is misinterpreted
> > (for example due to timeouts), then fragments of the sequences will
> > echo as unexpected characters.
> 
> I do not understand what that means.  By "cursor-key sequence of
> characters" do you mean the escape sequence sent by the terminal to
> Emacs or the cursor-motion escape sequences sent from Emacs to the
> text terminal?

From the terminal to Emacs.

Cursor up typically is

	ESC [ A

three characters.  Curses applications behave differently if you type those
three characters, one-by-one, because of timing.

> What timeouts are you referring to?  Are you saying that the terminal
> emulator has internal timeouts?  If so, can you describe when and how
> they come into play?

Typically the _application_ (Emacs) waits for completion of a control sequence.
 
> It seems you're saying that the escape sequences Emacs sends to the
> terminal need to be within a single "packet", because if such an escape
> sequence is split among several "fflush" the terminal emulator won't
> wait indefinitely for the second chunk, and instead will timeout after
> a fairly short wait and consider the escape sequence as "unexpected
> characters".

That's part of it.  Usually the input (from the terminal to the application)
is more important.

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 09:11:01 GMT) Full text and rfc822 format available.

Message #164 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 05:10:17 -0400
[Message part 1 (text/plain, inline)]
On Wed, Jun 04, 2014 at 04:31:02AM -0400, Thomas Dickey wrote:
> On Tue, Jun 03, 2014 at 11:03:08PM -0400, Stefan Monnier wrote:
> > > repeating: if the cursor-key sequence of characters is misinterpreted
> > > (for example due to timeouts), then fragments of the sequences will
> > > echo as unexpected characters.
> > 
> > I do not understand what that means.  By "cursor-key sequence of
> > characters" do you mean the escape sequence sent by the terminal to
> > Emacs or the cursor-motion escape sequences sent from Emacs to the
> > text terminal?
> 
> From the terminal to Emacs.
> 
> Cursor up typically is
> 
> 	ESC [ A

...normal mode - application mode is still three:
  	ESC O A

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 09:39:01 GMT) Full text and rfc822 format available.

Message #167 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Thomas Dickey <dickey <at> his.com>, Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 12:38:04 +0300
On the hunch that the problems discussed here are caused by excess
movement of the cursor, I've made changes in revision 117203 on the
emacs-24 branch to avoid some of them.  Please try that.

If the problems are still there, one last idea I have is to ifdef away
the code in tty_menu_activate that invokes the help-echo display:

      /* Display the help-echo message for the currently-selected menu
	 item.  */
      if ((menu_help_message || prev_menu_help_message)
	  && menu_help_message != prev_menu_help_message)
	{
	  help_callback (menu_help_message,
			 menu_help_paneno, menu_help_itemno);
	  /* Move the cursor to the beginning of the current menu
	     item, so that screen readers and other accessibility aids
	     know where the active region is.  */
	  cursor_to (sf, row, col);
	  prev_menu_help_message = menu_help_message;
	}

This code invokes another update of the frame, in addition to the one
we already did when we called tty_menu_display immediately before
that.  It also moves the cursor (which is positioned by the frame
update to where point is in the selected window).  Maybe avoiding that
will avoid the problems.

If even that doesn't help, then try to ifdef away the calls to
tty_hide_cursor made by tty_menu_activate.

These two last ideas are the only differences I know of between a
"normal" redisplay and redisplay due to update of a menu.  If even
they don't make any difference, I guess this bug will have to remain
open until someone debugs it on a system where the problem can be
reproduced.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 10:17:02 GMT) Full text and rfc822 format available.

Message #170 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com, rgm <at> gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 13:16:21 +0300
Here's a simple recipe to generate screen updates very similar to the
ones caused by a menu update.  Do they cause similar artifacts on the
offending terminal emulators?

 emacs -Q -nw

 Insert and evaluate this function:
 (defun my-help (old new)
  ""
  (message "We are at %s" (point)))

 C-x 3

 C-x C-f src/xdisp.c RET (any sufficiently large file will do)

 M-x hl-line-mode RET

 M-: (progn
      (beginning-of-buffer)
      (dotimes (i 40)
       (put-text-property
        (line-beginning-position)
        (line-end-position)
        'point-entered
        'my-help)
       (forward-line))) RET

Now move cursor down and up through the first 40 lines of the buffer:
you should see the background of the current line change as you move,
and also help-echo-like messages in the echo area for each non-empty
line in the buffer that follows an empty line.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 13:07:02 GMT) Full text and rfc822 format available.

Message #173 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 09:06:11 -0400
>> > repeating: if the cursor-key sequence of characters is misinterpreted
>> > (for example due to timeouts), then fragments of the sequences will
>> > echo as unexpected characters.
>> I do not understand what that means.  By "cursor-key sequence of
>> characters" do you mean the escape sequence sent by the terminal to
>> Emacs or the cursor-motion escape sequences sent from Emacs to the
>> text terminal?
> From the terminal to Emacs.
> Cursor up typically is
> 	ESC [ A
> three characters.

But that's a completely different problem from the one
we're investigating.  The problem is about how the terminal emulator
reacts to Emacs's output, not how Emacs reacts to the terminal
emulator's output.

> Curses applications behave differently if you type those
> three characters, one-by-one, because of timing.

We try pretty hard to avoid using such timeouts in Emacs, so by and
large this doesn't apply to Emacs at all.

> That's part of it.  Usually the input (from the terminal to the application)
> is more important.

But the present bug-report is about the other case.
To repeat the problem:
- a particular run of Emacs sends bytes <typescript> to the terminal
  emulator, and the result doesn't look right.
- yet when we replay the same <typescript> directly to the terminal, the
  result looks correct.
Note also that (other than the display), the state of Emacs at the end
of the test is correct (and forcing a redisplay indeed fixes the
display), so the problem is not in Emacs misinterpreting input because
of timing issues.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 16:10:02 GMT) Full text and rfc822 format available.

Message #176 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 12:08:47 -0400
Eli Zaretskii wrote:

> Here's a simple recipe to generate screen updates very similar to the
> ones caused by a menu update.  Do they cause similar artifacts on the
> offending terminal emulators?

Not for me, no.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 16:10:04 GMT) Full text and rfc822 format available.

Message #179 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca,
 Thomas Dickey <dickey <at> his.com>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 12:09:26 -0400
[Message part 1 (text/plain, inline)]
Eli Zaretskii wrote:

> On the hunch that the problems discussed here are caused by excess
> movement of the cursor, I've made changes in revision 117203 on the
> emacs-24 branch to avoid some of them.  Please try that.

Sorry, still glitches. :(

[1.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 16:16:02 GMT) Full text and rfc822 format available.

Message #182 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 19:15:03 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: dickey <at> his.com,  Stefan Monnier <monnier <at> iro.umontreal.ca>,  17497 <at> debbugs.gnu.org
> Date: Wed, 04 Jun 2014 12:08:47 -0400
> 
> Eli Zaretskii wrote:
> 
> > Here's a simple recipe to generate screen updates very similar to the
> > ones caused by a menu update.  Do they cause similar artifacts on the
> > offending terminal emulators?
> 
> Not for me, no.

As expected.  Thanks for testing.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 16:24:01 GMT) Full text and rfc822 format available.

Message #185 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 19:23:03 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Thomas Dickey <dickey <at> his.com>,  monnier <at> iro.umontreal.ca,  17497 <at> debbugs.gnu.org
> Date: Wed, 04 Jun 2014 12:09:26 -0400
> 
> > On the hunch that the problems discussed here are caused by excess
> > movement of the cursor, I've made changes in revision 117203 on the
> > emacs-24 branch to avoid some of them.  Please try that.
> 
> Sorry, still glitches. :(

Not surprisingly.

It still happens because these 3 lines:

  --
  Quit                  C-x C-c
 Save unsaved buffers, then exit

were written starting from cursor position at the EOB of *scratch*,
instead of way below, on the line below the "Delete Frame" item.  IOW,
some cursor motion commands emitted by Emacs are not obeyed for some
reason.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 17:12:02 GMT) Full text and rfc822 format available.

Message #188 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 20:10:50 +0300
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Thomas Dickey <dickey <at> his.com>,  monnier <at> iro.umontreal.ca,  17497 <at> debbugs.gnu.org
> Date: Wed, 04 Jun 2014 12:09:26 -0400
> 
> Eli Zaretskii wrote:
> 
> > On the hunch that the problems discussed here are caused by excess
> > movement of the cursor, I've made changes in revision 117203 on the
> > emacs-24 branch to avoid some of them.  Please try that.
> 
> Sorry, still glitches. :(

If you put a breakpoint in update_frame and in update_frame_with_menu,
like shown below, how many calls do you see when you press an arrow
key (or C-n/C-p) that produces a glitch?  I expect 1 call to
update_frame_with_menu followed by one call to update_frame.  Here's a
GDB session on fencepost to show what I see:

  (gdb) break update_frame
  Breakpoint 1 at 0x41b605: file dispnew.c, line 3006.
  (gdb) commands
  Type commands for breakpoint(s) 3, one per line.
  End with a line saying just "end".
  >continue
  >end
  (gdb) break update_frame_with_menu
  Breakpoint 2 at 0x41b81b: file dispnew.c, line 3112.
  (gdb) commands
  Type commands for breakpoint(s) 4, one per line.
  End with a line saying just "end".
  >continue
  >end
  (gdb) c
  Continuing.

  Breakpoint 1, update_frame (f=0xd3e808, force_p=false, 
      inhibit_hairy_id_p=false) at dispnew.c:3006
  3006	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 2, update_frame_with_menu (f=0xd3e808, row=-1, col=-1)
      at dispnew.c:3112
  3112	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 2, update_frame_with_menu (f=0xd3e808, row=0, col=0)
      at dispnew.c:3112
  3112	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 2, update_frame_with_menu (f=0xd3e808, row=1, col=0)
      at dispnew.c:3112
  3112	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 1, update_frame (f=0xd3e808, force_p=true, inhibit_hairy_id_p=true)
      at dispnew.c:3006
  3006	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 2, update_frame_with_menu (f=0xd3e808, row=2, col=0)
      at dispnew.c:3112
  3112	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 1, update_frame (f=0xd3e808, force_p=true, inhibit_hairy_id_p=true)
      at dispnew.c:3006
  3006	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 2, update_frame_with_menu (f=0xd3e808, row=3, col=0)
      at dispnew.c:3112
  3112	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 1, update_frame (f=0xd3e808, force_p=true, inhibit_hairy_id_p=true)
      at dispnew.c:3006
  3006	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 2, update_frame_with_menu (f=0xd3e808, row=2, col=0)
      at dispnew.c:3112
  3112	  struct window *root_window = XWINDOW (f->root_window);

  Breakpoint 1, update_frame (f=0xd3e808, force_p=true, inhibit_hairy_id_p=true)
      at dispnew.c:3006
  3006	  struct window *root_window = XWINDOW (f->root_window);

Explanations:

 . first, a call to update_frame is because menu-bar-open calls
   'redisplay'

 . next, a call to update_frame_with_menu which comes from
   the beginning of tty_menu_activate, line 3230 of term.c

 . another call to update_frame_with_menu, to display the "File >"
   header of the File menu.

 . yet another call to update_frame_with_menu to display the first
   menu item (Visit New File) in red background

 . finally, a call to update_frame to display the help-echo for "Visit
   New File"

Thereafter, for each keypress of an arrow key, there's always 1 call
to update_frame_with_menu, to display a different menu item in the red
background (see the values of row and col arguments, which identify
the menu item), followed by a single call to update_frame to display
the help echo.

Do you see any calls to update_frame beyond what I described?  If so,
perhaps they are the reason the cursor gets back to the EOB of the
underlying buffer.

TIA




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 04 Jun 2014 20:27:02 GMT) Full text and rfc822 format available.

Message #191 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org,
 Thomas Dickey <dickey <at> his.com>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 16:26:17 -0400
[Message part 1 (text/plain, inline)]
On Wed, Jun 04, 2014 at 09:06:11AM -0400, Stefan Monnier wrote:
> > Curses applications behave differently if you type those
> > three characters, one-by-one, because of timing.
> 
> We try pretty hard to avoid using such timeouts in Emacs, so by and
> large this doesn't apply to Emacs at all.

As it happens, that rule of thumb doesn't apply to terminals:

https://github.com/mirrors/emacs/blob/master/src/keyboard.c

(it's implemented there, at any rate)

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 00:48:02 GMT) Full text and rfc822 format available.

Message #194 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Wed, 04 Jun 2014 20:47:42 -0400
>> > Curses applications behave differently if you type those
>> > three characters, one-by-one, because of timing.
>> We try pretty hard to avoid using such timeouts in Emacs, so by and
>> large this doesn't apply to Emacs at all.
> As it happens, that rule of thumb doesn't apply to terminals:
> https://github.com/mirrors/emacs/blob/master/src/keyboard.c
> (it's implemented there, at any rate)

Where do you see the timeouts?

Or rather, no, don't bother, because even there might be problem in how
we process the input escape sequences, these are unrelated to the
display glitches we see.  So let's focus on the display glitches.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 08:23:02 GMT) Full text and rfc822 format available.

Message #197 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Dickey <dickey <at> his.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org,
 Thomas Dickey <dickey <at> his.com>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 05 Jun 2014 04:21:51 -0400
[Message part 1 (text/plain, inline)]
On Wed, Jun 04, 2014 at 08:47:42PM -0400, Stefan Monnier wrote:
> >> > Curses applications behave differently if you type those
> >> > three characters, one-by-one, because of timing.
> >> We try pretty hard to avoid using such timeouts in Emacs, so by and
> >> large this doesn't apply to Emacs at all.
> > As it happens, that rule of thumb doesn't apply to terminals:
> > https://github.com/mirrors/emacs/blob/master/src/keyboard.c
> > (it's implemented there, at any rate)
> 
> Where do you see the timeouts?

start with
	static bool get_input_pending (int);

The code's waiting for a while to ensure that it gets all of the bytes in
a cursor-key/function-key/mouse-event sequence.
 
> Or rather, no, don't bother, because even there might be problem in how
> we process the input escape sequences, these are unrelated to the
> display glitches we see.  So let's focus on the display glitches.

You could cut the discussion short by making the check that I suggested:
logging the decoded character/special-key values to look for instances
where the decoding returns individual bytes.

-- 
Thomas E. Dickey <dickey <at> invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 08:30:02 GMT) Full text and rfc822 format available.

Message #200 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 05 Jun 2014 10:29:14 +0200
Thomas Dickey <dickey <at> his.com> writes:

> start with
> 	static bool get_input_pending (int);
>
> The code's waiting for a while to ensure that it gets all of the bytes in
> a cursor-key/function-key/mouse-event sequence.

No, it doesn't.  It just reads whatever is pending, without paying
attention to key boundaries.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 13:45:02 GMT) Full text and rfc822 format available.

Message #203 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 05 Jun 2014 09:44:45 -0400
> You could cut the discussion short by making the check that I suggested:
> logging the decoded character/special-key values to look for instances
> where the decoding returns individual bytes.

No we cut it short by noticing that Emacs's internal state is the one we
expect and the display is completely wrong, which is something that
cannot happen due to some weird input sequence.

Please believe us, we *really* know that the problem is not in Emacs's
misunderstanding/misdecoding of the input stream (yes, it does have
bugs there, of course, but they're unrelated).

To repeat: replaying the termscript results in a different display (not
only that, it results in the display we want)!
This can't be due to Emacs's input processing.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 13:48:01 GMT) Full text and rfc822 format available.

Message #206 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Thomas Dickey <dickey <at> his.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 05 Jun 2014 09:47:17 -0400
> The code's waiting for a while to ensure that it gets all of the bytes in
> a cursor-key/function-key/mouse-event sequence.
 
No, it's not.  Try it:

    emacs -Q -nw
    then hit ESC O A on your keyboard.

Feel free to type those three keys real slow, or real fast.  It makes
little difference.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 15:01:01 GMT) Full text and rfc822 format available.

Message #209 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dickey <at> his.com
Cc: 17497 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 05 Jun 2014 18:00:15 +0300
> Date: Thu, 05 Jun 2014 04:21:51 -0400
> From: Thomas Dickey <dickey <at> his.com>
> Cc: Thomas Dickey <dickey <at> his.com>, 17497 <at> debbugs.gnu.org,
>  Eli Zaretskii <eliz <at> gnu.org>
> 
> > Or rather, no, don't bother, because even there might be problem in how
> > we process the input escape sequences, these are unrelated to the
> > display glitches we see.  So let's focus on the display glitches.
> 
> You could cut the discussion short by making the check that I suggested:
> logging the decoded character/special-key values to look for instances
> where the decoding returns individual bytes.

I don't think this is the problem in this case.  The arrow keys are
clearly decoded correctly and obeyed, as we see the reaction to them,
which is to redraw certain portions of the screen.  It's not like an
arrow key we get from the keyboard is sent verbatim to the terminal;
rather, Emacs interprets that key as a command to change the
background of two screen lines, and then sends the related commands,
including cursor motion, to the terminal.  IOW, the cursor motion
commands sent to the terminal are not what we receive from the
keyboard, they are generated by Emacs using a non-trivial logic in
cmgoto (which could decide that it is better to send a single newline
character, if it needs to move down just one line, or move to the
upper-left corner of the screen, for example).

Moreover, using C-n and C-p, which are single bytes, doesn't make the
problem go away.

So some other factor is at work here.  Keyboard input is unrelated,
at least as far as Emacs's code is concerned.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 05 Jun 2014 15:03:02 GMT) Full text and rfc822 format available.

Message #212 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 17497 <at> debbugs.gnu.org, dickey <at> his.com
Subject: Re: bug#17497: 24.4.50; TTY menu glitches
Date: Thu, 05 Jun 2014 18:02:16 +0300
> From: Andreas Schwab <schwab <at> suse.de>
> Date: Thu, 05 Jun 2014 10:29:14 +0200
> Cc: 17497 <at> debbugs.gnu.org
> 
> Thomas Dickey <dickey <at> his.com> writes:
> 
> > start with
> > 	static bool get_input_pending (int);
> >
> > The code's waiting for a while to ensure that it gets all of the bytes in
> > a cursor-key/function-key/mouse-event sequence.
> 
> No, it doesn't.  It just reads whatever is pending, without paying
> attention to key boundaries.

In addition, the function which reads keys isn't called during menu
updates, because we call update_frame_1 with its argument force_p
non-zero, so update_frame_1 doesn't even pay attention whether there's
input available.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Jun 2014 14:35:02 GMT) Full text and rfc822 format available.

Message #215 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: mlang <at> delysid.org, 17497 <at> debbugs.gnu.org
Subject: Re: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Jun 2014 17:33:46 +0300
> Date: Wed, 18 Jun 2014 13:57:45 +0400
> From: Dmitry Antipov <dmantipov <at> yandex.ru>
> Cc: emacs-devel <at> gnu.org
> 
> On 06/18/2014 01:50 PM, Mario Lang wrote:
> 
> > Just tried the text-mode menu in emacs-24 branch, and it is rather
> > broken.  It is kind of hard for me to describe the bug in detail, but
> > scrolling through the menu items of the "Edit" menu gives me randomly
> > wrong aligned menu items.  "Undo" is not aligned with the rest of the
> > menu, but aligned to the left of the screen.  Quitting the menu gives me
> > left-over characters from these wrong-aligned menu items.
> >
> > Can someone experienced with the new text mode menu take a look at this
> > please?  I am seeing it on Linux in tty mode, with a "--without-x"
> > complied emacs.
> 
> Check screenshots from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17497,
> most probably you hit the same known issue.

Since I cannot reproduce this on any system I have access to, I'm
basically waiting for someone to step forward to either give me a
login on a system where this happens, or work closely with me on
debugging this on their machine.  The two last ideas I had and asked
to try didn't get any followup at all, so I guess no one is interested
enough to work on this, or let me do it on their machine.

If no one steps forward, I'm afraid this will remain "a known issue",
at least on some systems and with some terminal emulators.

(I've redirected this discussion to the bug address, btw.)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Jun 2014 14:38:02 GMT) Full text and rfc822 format available.

Message #218 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mario Lang <mlang <at> delysid.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Jun 2014 17:37:11 +0300
> From: Mario Lang <mlang <at> delysid.org>
> Date: Wed, 18 Jun 2014 13:14:51 +0200
> 
> > Check screenshots from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17497,
> > most probably you hit the same known issue.
> 
> I can't see plain image file screenshots, but the (terse) description in
> the bug report sort of matches what I see, so likely this is the same
> issue.  Thanks for pointing this out.

Can you state your system name/version and the terminal emulator you
are using, for the record?  Past experience indicates that perhaps
this problem affects only some systems/emulators.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 19 Jun 2014 14:13:02 GMT) Full text and rfc822 format available.

Message #221 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: gnu-emacs-bug <at> moderators.isc.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Thu, 19 Jun 2014 14:11:41 +0000 (UTC)
Hi, Eli.

Eli Zaretskii <eliz <at> gnu.org> wrote:
>> From: Mario Lang <mlang <at> delysid.org>
>> Date: Wed, 18 Jun 2014 13:14:51 +0200
 
>> > Check screenshots from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17497,
>> > most probably you hit the same known issue.
 
>> I can't see plain image file screenshots, but the (terse) description in
>> the bug report sort of matches what I see, so likely this is the same
>> issue.  Thanks for pointing this out.

> Can you state your system name/version and the terminal emulator you
> are using, for the record?  Past experience indicates that perhaps
> this problem affects only some systems/emulators.

I think this is the case.  I've started an up to date system from the
Emacs-24 branch, running on a GNU system on a Linux tty, with -Q.  I
configured the build with:

   ./configure --with-gif=no --with-tiff=no --with-gpm

.  The menus seem to work fine, except for one glitch.  When I click the
(GPM) mouse on a menu, the menu drops down beginning horizontally where
the mouse was, rather than where the menu label was.  I.e., I see this:

    File Ed Edit > ns Buffers Tools Lisp-Interaction Help
            Undo                    C-x u
            Cut                     C-w
    ...

, when I would rather expect to see this:

    File Edit > tions Buffers Tools Lisp-Interaction Help
         Undo                    C-x u
         Cut                     C-w
    ....

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 19 Jun 2014 15:08:02 GMT) Full text and rfc822 format available.

Message #224 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alan Mackenzie <acm <at> muc.de>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Thu, 19 Jun 2014 18:07:08 +0300
> From: Alan Mackenzie <acm <at> muc.de>
> Date: Thu, 19 Jun 2014 14:11:41 +0000 (UTC)
> 
> .  The menus seem to work fine, except for one glitch.  When I click the
> (GPM) mouse on a menu, the menu drops down beginning horizontally where
> the mouse was, rather than where the menu label was.  I.e., I see this:
> 
>     File Ed Edit > ns Buffers Tools Lisp-Interaction Help
>             Undo                    C-x u
>             Cut                     C-w
>     ...
> 
> , when I would rather expect to see this:
> 
>     File Edit > tions Buffers Tools Lisp-Interaction Help
>          Undo                    C-x u
>          Cut                     C-w
>     ....

This is not a glitch, this is intended behavior when your text
terminal has a mouse.  Clicking anywhere inside "Edit" will drop down
the "Edit" menu starting at the click position.

Can you drop the menu with F10, try navigating through menus with
arrow keys, both horizontal and vertical, and report whether you see
any problems?  If you see any problems, I'd appreciate screenshots
showing them.

Whatever the results, I'd like you know what are your OS and terminal
emulator, please.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Thu, 19 Jun 2014 16:38:02 GMT) Full text and rfc822 format available.

Message #227 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Thu, 19 Jun 2014 16:33:10 +0000
Hi, Eli.

On Thu, Jun 19, 2014 at 06:07:08PM +0300, Eli Zaretskii wrote:
> > From: Alan Mackenzie <acm <at> muc.de>
> > Date: Thu, 19 Jun 2014 14:11:41 +0000 (UTC)

> > .  The menus seem to work fine, except for one glitch.  When I click the
> > (GPM) mouse on a menu, the menu drops down beginning horizontally where
> > the mouse was, rather than where the menu label was.  I.e., I see this:

> >     File Ed Edit > ns Buffers Tools Lisp-Interaction Help
> >             Undo                    C-x u
> >             Cut                     C-w
> >     ...

> > , when I would rather expect to see this:

> >     File Edit > tions Buffers Tools Lisp-Interaction Help
> >          Undo                    C-x u
> >          Cut                     C-w
> >     ....

> This is not a glitch, this is intended behavior when your text
> terminal has a mouse.  Clicking anywhere inside "Edit" will drop down
> the "Edit" menu starting at the click position.

Ah.  OK.  This seems at bit strange to me, but if it's intended
behaviour, then it's clearly not a glitch.

> Can you drop the menu with F10, try navigating through menus with
> arrow keys, both horizontal and vertical, and report whether you see
> any problems?  If you see any problems, I'd appreciate screenshots
> showing them.

I see no problems whatsoever.

> Whatever the results, I'd like you know what are your OS and terminal
> emulator, please.

Gentoo GNU/Linux, on a Linux virtual terminal (i.e. not in a terminal
emulator, not on X).

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 21 Jun 2014 10:15:02 GMT) Full text and rfc822 format available.

Message #230 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Mario Lang <mlang <at> delysid.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: The text-mode menu looks very broken in emacs-24
Date: Sat, 21 Jun 2014 12:14:44 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Mario Lang <mlang <at> delysid.org>
>> Date: Wed, 18 Jun 2014 13:14:51 +0200
>> 
>> > Check screenshots from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17497,
>> > most probably you hit the same known issue.
>> 
>> I can't see plain image file screenshots, but the (terse) description in
>> the bug report sort of matches what I see, so likely this is the same
>> issue.  Thanks for pointing this out.
>
> Can you state your system name/version and the terminal emulator you
> are using, for the record?  Past experience indicates that perhaps
> this problem affects only some systems/emulators.

I am running in GNU/Screen on GNU/Linux (kernel 3.14).

This bug only happens when scrolling down anything else than the first
toplevel entry, because the first toplevel menu is already
left-justified...  And it does not happen all the time.  In fact,
while it is reproducible, it doesnt reproduce the same way everytime I
try it.  Sometimes I need to go down 10 or 15 elements until one gets}
drawn in the wrong position, and sometimes I just go down two or three
items.  I am not sure, but it *feels* like the issue is easier to
reproduce if I hit down-arrow rapidly in succession.

This *feels* like memory corruption, as if the x coordinate is randomly
set to 0 in some situations.

-- 
CYa,
  ⡍⠁⠗⠊⠕




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 21 Jun 2014 11:27:02 GMT) Full text and rfc822 format available.

Message #233 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mario Lang <mlang <at> delysid.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: The text-mode menu looks very broken in emacs-24
Date: Sat, 21 Jun 2014 14:26:15 +0300
> From: Mario Lang <mlang <at> delysid.org>
> Cc: 17497 <at> debbugs.gnu.org
> Date: Sat, 21 Jun 2014 12:14:44 +0200
> 
> I am running in GNU/Screen on GNU/Linux (kernel 3.14).

Thanks.  Which brand of GNU/Linux is that?

> This bug only happens when scrolling down anything else than the first
> toplevel entry, because the first toplevel menu is already
> left-justified...  And it does not happen all the time.  In fact,
> while it is reproducible, it doesnt reproduce the same way everytime I
> try it.  Sometimes I need to go down 10 or 15 elements until one gets}
> drawn in the wrong position, and sometimes I just go down two or three
> items.  I am not sure, but it *feels* like the issue is easier to
> reproduce if I hit down-arrow rapidly in succession.
> 
> This *feels* like memory corruption, as if the x coordinate is randomly
> set to 0 in some situations.

I don't think it's memory corruption.  I suspect the problem is
related to the different ways cmgoto chooses to get to the specified
screen coordinates.  If someone on the affected systems could tweak
cmgoto such that only the direct move is used, i.e. this code:

      p = (dcm == tty->Wcm->cm_habs
           ? tgoto (dcm, row, col)
           : tgoto (dcm, col, row));
      emacs_tputs (tty, p, 1, cmputc);
      curY (tty) = row, curX (tty) = col;

then perhaps we could see if my suspicions are in the right direction.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Mar 2015 15:46:02 GMT) Full text and rfc822 format available.

Message #236 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Martin Trojer <martin.trojer <at> gmail.com>
To: 17497 <at> debbugs.gnu.org
Subject: Re: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Mar 2015 11:45:17 +0000
[Message part 1 (text/plain, inline)]
> I don't think it's memory corruption.  I suspect the problem is
> related to the different ways cmgoto chooses to get to the specified
> screen coordinates.  If someone on the affected systems could tweak
> cmgoto such that only the direct move is used, i.e. this code:
>
>      p = (dcm == tty->Wcm->cm_habs
>           ? tgoto (dcm, row, col)
>           : tgoto (dcm, col, row));
>      emacs_tputs (tty, p, 1, cmputc);
>      curY (tty) = row, curX (tty) = col;
>
> then perhaps we could see if my suspicions are in the right direction.


Hi Eli,

I just applied this patch trying to solve a different but related
issue (https://www.virtualbox.org/ticket/13687).

I am happy to report that applying this patch completely solved the
redrawing issues I was experiencing!

One option for you to recreate this issue locally seems to be to
install virtual box, config a virtual machine with multiple cores, ssh
into it and rung emacs.

I've put my diff here
(https://github.com/martintrojer/emacs/commit/bdff1ff98d02f4307659c052d0b35a40a36f0706),
and I understand it can't be merged as-is hopefully this can help you
find a solution that go into mainline emacs.
[Message part 2 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Mar 2015 16:41:01 GMT) Full text and rfc822 format available.

Message #239 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Martin Trojer <martin.trojer <at> gmail.com>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Mar 2015 18:40:02 +0200
> Date: Wed, 18 Mar 2015 11:45:17 +0000
> From: Martin Trojer <martin.trojer <at> gmail.com>
> 
> > I don't think it's memory corruption.  I suspect the problem is
> > related to the different ways cmgoto chooses to get to the specified
> > screen coordinates.  If someone on the affected systems could tweak
> > cmgoto such that only the direct move is used, i.e. this code:
> >
> >      p = (dcm == tty->Wcm->cm_habs
> >           ? tgoto (dcm, row, col)
> >           : tgoto (dcm, col, row));
> >      emacs_tputs (tty, p, 1, cmputc);
> >      curY (tty) = row, curX (tty) = col;
> >
> > then perhaps we could see if my suspicions are in the right direction.
> 
> Hi Eli,
> I just applied this patch trying to solve a different but related issue (https://www.virtualbox.org/ticket/13687).
> I am happy to report that applying this patch completely solved the redrawing issues I was experiencing!

You don't quote the person who wrote the above, so I need to ask you
who wrote the code.  We need to know to whom to attribute it.

Also, I'd be happier if someone could explain why this is the the
right thing to do in that place.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Mar 2015 16:52:02 GMT) Full text and rfc822 format available.

Message #242 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Martin Trojer <martin.trojer <at> gmail.com>
To: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Mar 2015 16:51:48 +0000
[Message part 1 (text/plain, inline)]
Hello,

I believe that snippet of code was from you :) I simply put that quoted
snipped in the cmgoto function as per your instructions. I'm sure this
isn't the 'right thing to do' since it completely bypasses the switch
statement that tries to do the cheapest cursor move.

For what I can tell from the long discussion this is very hairy bug to
observe and debug. But for some reason that I don't fully understand this
patch solves my re-draw issues.

Also, if you are interested in solving this issue properly, running emacs
inside a VM (in this case Virtualbox) seems to make re-draw issues a lot
more likely and thus perhaps easier to diagnose.

Cheers...

On Wed, Mar 18, 2015 at 4:40 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Date: Wed, 18 Mar 2015 11:45:17 +0000
> > From: Martin Trojer <martin.trojer <at> gmail.com>
> >
> > > I don't think it's memory corruption.  I suspect the problem is
> > > related to the different ways cmgoto chooses to get to the specified
> > > screen coordinates.  If someone on the affected systems could tweak
> > > cmgoto such that only the direct move is used, i.e. this code:
> > >
> > >      p = (dcm == tty->Wcm->cm_habs
> > >           ? tgoto (dcm, row, col)
> > >           : tgoto (dcm, col, row));
> > >      emacs_tputs (tty, p, 1, cmputc);
> > >      curY (tty) = row, curX (tty) = col;
> > >
> > > then perhaps we could see if my suspicions are in the right direction.
> >
> > Hi Eli,
> > I just applied this patch trying to solve a different but related issue (
> https://www.virtualbox.org/ticket/13687).
> > I am happy to report that applying this patch completely solved the
> redrawing issues I was experiencing!
>
> You don't quote the person who wrote the above, so I need to ask you
> who wrote the code.  We need to know to whom to attribute it.
>
> Also, I'd be happier if someone could explain why this is the the
> right thing to do in that place.
>
> Thanks.
>
[Message part 2 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Mar 2015 17:12:02 GMT) Full text and rfc822 format available.

Message #245 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Martin Trojer <martin.trojer <at> gmail.com>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Mar 2015 19:11:27 +0200
> Date: Wed, 18 Mar 2015 16:51:48 +0000
> From: Martin Trojer <martin.trojer <at> gmail.com>
> 
> I believe that snippet of code was from you :) I simply put that quoted snipped
> in the cmgoto function as per your instructions. I'm sure this isn't the 'right
> thing to do' since it completely bypasses the switch statement that tries to do
> the cheapest cursor move. 

So what patch did you install, exactly, that fixed the problem?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 18 Mar 2015 17:14:01 GMT) Full text and rfc822 format available.

Message #248 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Martin Trojer <martin.trojer <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Wed, 18 Mar 2015 17:13:13 +0000
[Message part 1 (text/plain, inline)]
From bdff1ff98d02f4307659c052d0b35a40a36f0706 Mon Sep 17 00:00:00 2001
From: Martin Trojer <martin.trojer <at> gmail.com>
Date: Wed, 18 Mar 2015 11:44:02 +0000
Subject: [PATCH] emacs bug #17497 fix

---
 src/cm.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/cm.c b/src/cm.c
index 474f280..ed17447 100644
--- a/src/cm.c
+++ b/src/cm.c
@@ -371,6 +371,16 @@ cmgoto (struct tty_display_info *tty, int row, int col)
       dcm = tty->Wcm->cm_abs;
     }

+    /* only use direct moves */
+    cost = 0;
+    p = (dcm == tty->Wcm->cm_habs
+         ? tgoto (dcm, row, col)
+         : tgoto (dcm, col, row));
+    emacs_tputs (tty, p, 1, evalcost);
+    emacs_tputs (tty, p, 1, cmputc);
+    curY (tty) = row, curX (tty) = col;
+    return;
+
   /*
    * In the following comparison, the = in <= is because when the costs
    * are the same, it looks nicer (I think) to move directly there.


On Wed, Mar 18, 2015 at 5:11 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Date: Wed, 18 Mar 2015 16:51:48 +0000
> > From: Martin Trojer <martin.trojer <at> gmail.com>
> >
> > I believe that snippet of code was from you :) I simply put that quoted
> snipped
> > in the cmgoto function as per your instructions. I'm sure this isn't the
> 'right
> > thing to do' since it completely bypasses the switch statement that
> tries to do
> > the cheapest cursor move.
>
> So what patch did you install, exactly, that fixed the problem?
>
[Message part 2 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 07 Oct 2017 09:40:01 GMT) Full text and rfc822 format available.

Message #251 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Martin Trojer <martin.trojer <at> gmail.com>
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Sat, 07 Oct 2017 12:38:58 +0300
> Date: Wed, 18 Mar 2015 17:13:13 +0000
> From: Martin Trojer <martin.trojer <at> gmail.com>
> Cc: 17497 <at> debbugs.gnu.org
> 
> From bdff1ff98d02f4307659c052d0b35a40a36f0706 Mon Sep 17 00:00:00 2001
> From: Martin Trojer <martin.trojer <at> gmail.com>
> Date: Wed, 18 Mar 2015 11:44:02 +0000
> Subject: [PATCH] emacs bug #17497 fix
> 
> ---
>  src/cm.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/src/cm.c b/src/cm.c
> index 474f280..ed17447 100644
> --- a/src/cm.c
> +++ b/src/cm.c
> @@ -371,6 +371,16 @@ cmgoto (struct tty_display_info *tty, int row, int col)
>        dcm = tty->Wcm->cm_abs;
>      }
>  
> +    /* only use direct moves */
> +    cost = 0;
> +    p = (dcm == tty->Wcm->cm_habs
> +         ? tgoto (dcm, row, col)
> +         : tgoto (dcm, col, row));
> +    emacs_tputs (tty, p, 1, evalcost);
> +    emacs_tputs (tty, p, 1, cmputc);
> +    curY (tty) = row, curX (tty) = col;
> +    return;
> +
>    /*
>     * In the following comparison, the = in <= is because when the costs
>     * are the same, it looks nicer (I think) to move directly there.

I now do see this problem on a GNU/Linux system to which I have
access, but applying this patch didn't solve the problem...




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Sat, 07 Oct 2017 11:42:01 GMT) Full text and rfc822 format available.

Message #254 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: martin.trojer <at> gmail.com
Cc: 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Sat, 07 Oct 2017 14:41:08 +0300
> Date: Sat, 07 Oct 2017 12:38:58 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 17497 <at> debbugs.gnu.org
> 
> > Date: Wed, 18 Mar 2015 17:13:13 +0000
> > From: Martin Trojer <martin.trojer <at> gmail.com>
> > Cc: 17497 <at> debbugs.gnu.org
> > 
> > From bdff1ff98d02f4307659c052d0b35a40a36f0706 Mon Sep 17 00:00:00 2001
> > From: Martin Trojer <martin.trojer <at> gmail.com>
> > Date: Wed, 18 Mar 2015 11:44:02 +0000
> > Subject: [PATCH] emacs bug #17497 fix
> > 
> > ---
> >  src/cm.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/src/cm.c b/src/cm.c
> > index 474f280..ed17447 100644
> > --- a/src/cm.c
> > +++ b/src/cm.c
> > @@ -371,6 +371,16 @@ cmgoto (struct tty_display_info *tty, int row, int col)
> >        dcm = tty->Wcm->cm_abs;
> >      }
> >  
> > +    /* only use direct moves */
> > +    cost = 0;
> > +    p = (dcm == tty->Wcm->cm_habs
> > +         ? tgoto (dcm, row, col)
> > +         : tgoto (dcm, col, row));
> > +    emacs_tputs (tty, p, 1, evalcost);
> > +    emacs_tputs (tty, p, 1, cmputc);
> > +    curY (tty) = row, curX (tty) = col;
> > +    return;
> > +
> >    /*
> >     * In the following comparison, the = in <= is because when the costs
> >     * are the same, it looks nicer (I think) to move directly there.
> 
> I now do see this problem on a GNU/Linux system to which I have
> access, but applying this patch didn't solve the problem...

Some debugging indicates that somehow the cursor is sometimes moved
behind our backs, when a frame with a TTY menu is being updated.  So I
installed a semi-kludgey workaround that seems to fix 99% of the
glitches.  Since I don't really understand why this change is needed,
and because of the 1% of glitches that still happen, I'm not closing
the bug, and will continue looking into this.

If someone knows which display ops could move the cursor without going
through cursor_to, please tell.

I hope the solution is not limited to the one system where I see this
problem.  People who experienced the problem are encouraged to try the
latest release branch and report the results here.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Tue, 25 Jun 2019 23:33:02 GMT) Full text and rfc822 format available.

Message #257 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: martin.trojer <at> gmail.com, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Wed, 26 Jun 2019 01:32:52 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

> Some debugging indicates that somehow the cursor is sometimes moved
> behind our backs, when a frame with a TTY menu is being updated.  So I
> installed a semi-kludgey workaround that seems to fix 99% of the
> glitches.  Since I don't really understand why this change is needed,
> and because of the 1% of glitches that still happen, I'm not closing
> the bug, and will continue looking into this.

This was a year and a half ago.  I tried the recipe for reproducing the
bug for a couple of minutes, and I couldn't get it to glitch here, so is
it time to close this bug report?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#17497; Package emacs. (Wed, 26 Jun 2019 02:41:02 GMT) Full text and rfc822 format available.

Message #260 received at 17497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: martin.trojer <at> gmail.com, 17497 <at> debbugs.gnu.org
Subject: Re: bug#17497: The text-mode menu looks very broken in emacs-24
Date: Wed, 26 Jun 2019 05:40:13 +0300
> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: martin.trojer <at> gmail.com,  17497 <at> debbugs.gnu.org
> Date: Wed, 26 Jun 2019 01:32:52 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > Some debugging indicates that somehow the cursor is sometimes moved
> > behind our backs, when a frame with a TTY menu is being updated.  So I
> > installed a semi-kludgey workaround that seems to fix 99% of the
> > glitches.  Since I don't really understand why this change is needed,
> > and because of the 1% of glitches that still happen, I'm not closing
> > the bug, and will continue looking into this.
> 
> This was a year and a half ago.  I tried the recipe for reproducing the
> bug for a couple of minutes, and I couldn't get it to glitch here, so is
> it time to close this bug report?

Yes, I think so.  Thanks.




bug closed, send any further explanations to 17497 <at> debbugs.gnu.org and Dmitry Antipov <dmantipov <at> yandex.ru> Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Wed, 26 Jun 2019 07:36:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Wed, 24 Jul 2019 11:24:11 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 300 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.