GNU bug report logs - #37782
Emacs 27 client adding [I] to the beginning of the buffer

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37782 in the body.
You can then email your comments to 37782 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Emacs 27 client adding [I] to the beginning of the buffer
Date: Wed, 16 Oct 2019 19:45:12 -0300

I noticed that when running emacs 27 in client/server mode I often
open a file to find out that the string "[I]" (it's an upper-case i,
not a lower-case L) has been added to the beginning of the buffer. I
don't have this problem with 26.3.

Best regards
--
Carlos

Message #10 received at 37782-close <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: 37782-close <at> debbugs.gnu.org
Subject: Emacs 27 client adding [I] to the beginning of the buffer
Date: Wed, 16 Oct 2019 23:18:08 -0300

I'm closing this because it seems to be related to the interaction of
emacs-27 with the doom-themes package, and not to emacs-27 per se. If
it turns out to be a problem in emacs I will report it again.

Message #15 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: 37782 <at> debbugs.gnu.org
Subject: Re: Emacs 27 client adding [I] to the beginning of the buffer
Date: Wed, 16 Oct 2019 23:33:50 -0300

> If it turns out to be a problem in emacs I will report it again.

Ok, it turned out to be a problem in emacs :).

A minimal configuration to reproduce it would be: (load-theme 'wombat).

Then launch `emacs --daemon` and finally `emacsclient -t`.

Btw, it only happens in the terminal.

Message #18 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 11:03:05 +0300

> From: Carlos Pita <carlosjosepita <at> gmail.com>
> Date: Wed, 16 Oct 2019 23:33:50 -0300
> 
> > If it turns out to be a problem in emacs I will report it again.
> 
> Ok, it turned out to be a problem in emacs :).
> 
> A minimal configuration to reproduce it would be: (load-theme 'wombat).
> 
> Then launch `emacs --daemon` and finally `emacsclient -t`.
> 
> Btw, it only happens in the terminal.

Most probably comes from escape sequences Emacs sends to the terminal
to detect support of advanced features.  This particular one seems to
come from detecting focus-in support, see xterm.el:

    ;; These translation functions actually call the focus handlers
    ;; internally and return an empty sequence, causing us to go on to
    ;; read the next event.
    (define-key map "\e[I" #'xterm-translate-focus-in)
    (define-key map "\e[O" #'xterm-translate-focus-out)

Please debug to find out why that particular theme causes this
failure.

Thanks.

Message #21 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 09:08:40 -0300

The problem doesn't show when TERM=xterm, but only when TERM=xterm-24bit.

I created a terminfo definition as described in
https://www.gnu.org/software/emacs/manual/html_node/efaq/Colors-on-a-TTY.html
and was using it without any problem in emacs26.

Besides the aforementioned issue, I get true 24-bit color and
everything else works just fine.

The "\[I" sequence is defined for rxvt in xterm.el and it seems to me
weird that it's being used since my terminal is gnome-terminal and, as
I said, TERM=xterm-24bit.

Any idea?

Thanks!

Message #24 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 09:26:26 -0300

Moreover, in emacs 26 those two focus-in/out sequences are not defined at all.

Message #27 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 10:01:45 -0300

I don't have time to further debug this right now but I believe it
might be related to

(defun xterm--init-focus-tracking ()
  "Terminal initialization for focus tracking mode."
  (send-string-to-terminal "\e[?1004h")

somehow not being correctly or timely initialized.

See this similar issue
https://superuser.com/questions/931873/o-and-i-appearing-on-iterm2-when-focus-lost

Message #30 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 16:12:06 +0300

> From: Carlos Pita <carlosjosepita <at> gmail.com>
> Date: Thu, 17 Oct 2019 09:08:40 -0300
> Cc: 37782 <at> debbugs.gnu.org
> 
> Any idea?

Well, since the difference is between xterm and xterm-24bit, I'd start
by looking at what these two settings change, and why.

Message #33 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 16:16:45 +0300

> From: Carlos Pita <carlosjosepita <at> gmail.com>
> Date: Thu, 17 Oct 2019 09:26:26 -0300
> Cc: 37782 <at> debbugs.gnu.org
> 
> Moreover, in emacs 26 those two focus-in/out sequences are not defined at all.

Which probably explains why you didn't see the problem in Emacs 26,
right?

Message #36 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Thu, 17 Oct 2019 16:20:02 +0300

> From: Carlos Pita <carlosjosepita <at> gmail.com>
> Date: Thu, 17 Oct 2019 10:01:45 -0300
> Cc: 37782 <at> debbugs.gnu.org
> 
> I don't have time to further debug this right now but I believe it
> might be related to
> 
> (defun xterm--init-focus-tracking ()
>   "Terminal initialization for focus tracking mode."
>   (send-string-to-terminal "\e[?1004h")
> 
> somehow not being correctly or timely initialized.

Most probably (which is why I pointed to the corresponding escape
sequences), but the question is exactly why does this not work as
expected?  Is the terminal you are using too fast or too slow to
respond?  Is something else wrong?  We need to answer these questions
to make some progress with this issue.

Maybe someone else can reproduce and debug this.

Message #39 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Fri, 18 Oct 2019 01:29:41 -0300

[Message part 1 (text/plain, inline)]

> Most probably (which is why I pointed to the corresponding escape
> sequences), but the question is exactly why does this not work as
> expected?  Is the terminal you are using too fast or too slow to
> respond?  Is something else wrong?  We need to answer these questions
> to make some progress with this issue.
>
> Maybe someone else can reproduce and debug this.

I've had a hard time debugging this issue but the cause is still unclear to me.

I can easily reproduce the problem with an empty configuration, see
the attached screencast. There I'm just running

emacs --daemon
emacsclient -t

with an empty configuration. Since the behavior is not very
"deterministic" I was many times mislead to believe that the cause was
this or that theme or package, and maybe some packages that make heavy
use of color or other escape sequences do might tend to produce the
artifact more often, I don't know.

Things that I can say for (99%) sure:

1. It only happens in a terminal.
2. It only happens in client/server mode.
3. It only happens the first time I open a client in the terminal.
4. Commenting the  `(send-string-to-terminal "\e[?1004h")` line in
xterm.el suppresses the `[I` sequence (but this of course by disabling
focus tracking).
5. After the initial artifact, focus is indeed tracked correctly, i.e.
xterm-translate-focus-in/out are called as expected for `[I` and `[O`
sequences. But the first time xterm-translate-focus-in is NOT called.
6. When focus tracking is activated xterm-function-map was already
pushed to input-decode-map so AFAICS focus handlers should already be
properly installed.
7. Sometimes it is `[I`, sometimes it is `[I]`.

My hunch is that the escape sequence is not correctly parsed, that the
`[I` part is taken as normal input from the keyboard because the
escape prefix got lost or was interspersed with another sequence.
Probably the answer is in some part of read_key_sequence, but the
combination of client/server mode and low-level tty input is beyond my
debugging abilities.

Best regards
--
Carlos

[Screencast from 18-10-19 01:01:46.webm (video/webm, attachment)]

Message #42 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Sat, 19 Oct 2019 00:20:05 -0300

[Message part 1 (text/plain, inline)]

After hours and hours (and hours) of life sucking debugging I found
out the offending code (in read_key_sequence):

    if ((FIXNUMP (key) && XFIXNUM (key) == -2) /* wrong_kboard_jmpbuf */
/* When switching to a new tty (with a new keyboard),
   read_char returns the new buffer, rather than -2
   (Bug#5095).  This is because `terminal-init-xterm'
   calls read-char, which eats the wrong_kboard_jmpbuf
   return.  Any better way to fix this? -- cyd  */
|| (interrupted_kboard != current_kboard))
  {
   ....
   mock_input = 0;
   goto replay_entire_sequence;
  }

The problem with the fix for #5095 is that it's dropping the current
event, which is a valid char event and not -2. Thus, the first esc
char sent by the terminal after turning on focus tracking is lost and
we get `[`, `I` events separately inserted into the buffer afterwards.

Attached is a patch that replays key as mock input when it's not -2.

Can I ask for some of my hidpi fringe or colorized comint/python
history patches to be merged in exchange ;) ?

Best regards
--
Carlos

[0001-Replay-key-if-kboard-is-interrupted-while-initializi.patch (text/x-patch, attachment)]

Message #47 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Sat, 19 Oct 2019 10:06:13 +0300

> From: Carlos Pita <carlosjosepita <at> gmail.com>
> Date: Sat, 19 Oct 2019 00:20:05 -0300
> Cc: 37782 <at> debbugs.gnu.org
> 
> Can I ask for some of my hidpi fringe or colorized comint/python
> history patches to be merged in exchange ;) ?

They are in my queue; sorry for not yet having time to do a thorough
review.  Thank you for your patience.

Message #52 received at 37782-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Carlos Pita <carlosjosepita <at> gmail.com>
Cc: 37782-done <at> debbugs.gnu.org
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
 buffer
Date: Sat, 26 Oct 2019 13:30:54 +0300

> From: Carlos Pita <carlosjosepita <at> gmail.com>
> Date: Sat, 19 Oct 2019 00:20:05 -0300
> Cc: 37782 <at> debbugs.gnu.org
> 
> The problem with the fix for #5095 is that it's dropping the current
> event, which is a valid char event and not -2. Thus, the first esc
> char sent by the terminal after turning on focus tracking is lost and
> we get `[`, `I` events separately inserted into the buffer afterwards.
> 
> Attached is a patch that replays key as mock input when it's not -2.

Thanks, pushed, and closing the bug.

Message #55 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: 37782 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, Carlos Pita
 <carlosjosepita <at> gmail.com>
Cc: emacs <at> FreeBSD.org, emacs-devel <emacs-devel <at> gnu.org>
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
Date: Fri, 08 Nov 2019 01:09:27 -0400

[Message part 1 (text/plain, inline)]

Hello Carlos and Eli,

The fix for bug#37782 in commit e3cebbb (2019-10-19) introduced an error
when starting emacsclient.

    undefined: KEY must be an integer, cons, symbol, or string

I think the message is from keymap.c around line 2279.

This error is reproducible and is happening on FreeBSD (versions 12.0
and 12.1 confirmed).

Regards,

Joseph

[signature.asc (application/pgp-signature, inline)]

Message #64 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Joseph Mingrone <jrm <at> ftfl.ca>
Cc: 37782 <at> debbugs.gnu.org, carlosjosepita <at> gmail.com
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
Date: Sat, 07 Dec 2019 11:33:55 +0200

> From: Joseph Mingrone <jrm <at> ftfl.ca>
> Cc: emacs-devel <emacs-devel <at> gnu.org>, emacs <at> FreeBSD.org
> Date: Fri, 08 Nov 2019 01:09:27 -0400
> 
> Hello Carlos and Eli,
> 
> The fix for bug#37782 in commit e3cebbb (2019-10-19) introduced an error
> when starting emacsclient.
> 
>     undefined: KEY must be an integer, cons, symbol, or string
> 
> I think the message is from keymap.c around line 2279.
> 
> This error is reproducible and is happening on FreeBSD (versions 12.0
> and 12.1 confirmed).

No response from Carlos in a month, so I tried to fix this bug as best
as I could.  Joseph, could you please see if your problem is solved by
the latest master branch?

Carlos, if you are still here, please test if your original problem is
still solved, or did I re-introduce it by the fix.

Thanks.

Message #67 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Joseph Mingrone <jrm <at> ftfl.ca>, 37782 <at> debbugs.gnu.org,
 carlosjosepita <at> gmail.com
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
Date: Tue, 14 Apr 2020 21:29:35 -0400

tags 37782 fixed
close 37782 27.1
quit

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Joseph Mingrone <jrm <at> ftfl.ca>
>> Cc: emacs-devel <emacs-devel <at> gnu.org>, emacs <at> FreeBSD.org
>> Date: Fri, 08 Nov 2019 01:09:27 -0400
>> 
>> Hello Carlos and Eli,
>> 
>> The fix for bug#37782 in commit e3cebbb (2019-10-19) introduced an error
>> when starting emacsclient.
>> 
>>     undefined: KEY must be an integer, cons, symbol, or string
>> 
>> I think the message is from keymap.c around line 2279.
>> 
>> This error is reproducible and is happening on FreeBSD (versions 12.0
>> and 12.1 confirmed).
>
> No response from Carlos in a month, so I tried to fix this bug as best
> as I could.  Joseph, could you please see if your problem is solved by
> the latest master branch?
>
> Carlos, if you are still here, please test if your original problem is
> still solved, or did I re-introduce it by the fix.

I think by now we can assume no news is good news.  Closing.

[1: 1161c7d8ec]: 2019-12-07 11:28:05 +0200
  Fix fallout from fixing bug#37782
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1161c7d8ec388dc0698e7a398798a4f0e98e0403

Message #74 received at 37782 <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 37782 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 carlosjosepita <at> gmail.com
Subject: Re: bug#37782: Emacs 27 client adding [I] to the beginning of the
Date: Tue, 14 Apr 2020 22:36:10 -0300

Noam Postavsky <npostavs <at> gmail.com> writes:

> tags 37782 fixed
> close 37782 27.1
> quit

> Eli Zaretskii <eliz <at> gnu.org> writes:

>>> From: Joseph Mingrone <jrm <at> ftfl.ca>
>>> Cc: emacs-devel <emacs-devel <at> gnu.org>, emacs <at> FreeBSD.org
>>> Date: Fri, 08 Nov 2019 01:09:27 -0400

>>> Hello Carlos and Eli,

>>> The fix for bug#37782 in commit e3cebbb (2019-10-19) introduced an error
>>> when starting emacsclient.

>>>     undefined: KEY must be an integer, cons, symbol, or string

>>> I think the message is from keymap.c around line 2279.

>>> This error is reproducible and is happening on FreeBSD (versions 12.0
>>> and 12.1 confirmed).

>> No response from Carlos in a month, so I tried to fix this bug as best
>> as I could.  Joseph, could you please see if your problem is solved by
>> the latest master branch?

>> Carlos, if you are still here, please test if your original problem is
>> still solved, or did I re-introduce it by the fix.

> I think by now we can assume no news is good news.  Closing.

> [1: 1161c7d8ec]: 2019-12-07 11:28:05 +0200
>   Fix fallout from fixing bug#37782
>   https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1161c7d8ec388dc0698e7a398798a4f0e98e0403

Hi all,

For some reason my response from last December didn't show up even though I CCed 37782 <at> debbugs.gnu.org.

  > Hello Eli,

  > The error message on emacsclient start up is gone after your fix.

  > Thank you,

  > Joseph

In any case, thank you all.

Joseph

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.