GNU bug report logs - #51253
28.0.60; Meta keys broken when viper is active

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 51253 in the body.
You can then email your comments to 51253 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Cummings <john <at> rootabega.net>
To: "bug-gnu-emacs <at> gnu.org" <bug-gnu-emacs <at> gnu.org>
Subject: 28.0.60; Meta keys broken when viper is active
Date: Sun, 17 Oct 2021 20:44:40 +0000

This is a break introduced in 2020 after the 27.2 release.

emacs -Q recipe:

  (setq viper-inhibit-startup-message 't)
  (setq viper-expert-level '5)
  (setq viper-mode t)
  (require 'viper)
  ;only if/when testing this variant
  ;default is t
  ;(setq viper-no-multiple-ESC nil);

  In vi state or vi insert state, press M-x, M-s, M-:, etc.
  In terminal sessions, it will be like you pressed ESC and x
  separately and slowly.
  In graphical sessions, you will see a message like
  "M-x is undefined".


SUMMARY
This is is the latest wrinkle in a series of changes and bugs related
to viper. Let me know if you want me to update a previous bug report
instead of having this new one. I also have not included the
report-bug diag info for the Emacs versions and builds I tested with,
since this is pretty clearly fundamental to Emacs key handling and
viper functionality.  More detail is below, but I'll try to summarize
it very briefly first: Keys like M-x, M-:, etc., are broken in viper
vi/insert state since bug
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18182 was fixed in
5d522b430bd5ecfb8f082906cd634883dbb68f3e. This changed viper-ESC-key
from <escape> to ESC, which resulted in a direct key binding of (27
. 'viper-intercept-ESC-key) while in the viper vi/insert states.

In a terminal, the ESC and x events encoding M-x are processed like
two separate key sequences. This is because that direct ESC binding
makes viper-intercept-ESC-key consume the ESC event greedily, and then
the x is processed after that.

On a graphical display, the direct ESC binding prevents Emacs from
finding ESC prefix mappings that provide bindings for M- sequences
when it gets events like #x8000078 for M-x.

Since the latest change that causes this behavior was meant to make
ESC, aka ^[, work to exit vi insert state on graphical displays (as
Eli pointed out, it already worked on terminals at the time), would an
acceptable compromise in the short term be to let ^[ remain broken for
this purpose so that M- keys may work? In that bug, the reporter was
OK with the workaround Stefan suggested:
  (define-key input-decode-map [?\e] [escape])
Maybe that could also be added to all graphical sessions?



HISTORY
Rough timeline of changes that contribute to this viper behavior:

f5e1b6804dc2307983e4c55d4d6530549ddccbb7
Emacs ~24, Feb 2013
This is the starting point, where everything seemed to work in harmony.


git: 99d0d6dc23f0fd2ee6d64f0f18a33f2b791c642d
bzr?: 11521f1228e447bb68ff7a48a30148b99323c04e
Emacs ~24, Feb 2013
Changes were made to keymap handling functions in C. Prior to this
change, Even if there was direct ESC keybinding, Emacs could still
properly find the ESC prefix mappings that define M- bindings. i.e.,
you could map a command to ESC and use M- keys at the same time. After
this commit, the direct ESC binding meant that M- keys would be
undefined in graphical Emacs. So graphical Emacs could not use M-x in
viper vi/insert state, instead seeing 'M-x is undefined'. Terminal
emacs still worked OK, since that ESC binding was the viper ESC key
handler, and it could correctly identify ESC-as-meta event sequences
from their timing.

A recipe to see this specific change in behavior in graphical sessions:
   (local-set-key [27] (lambda ()
       (interactive) (insert " command ")))
Prior to this change, you could run this and still be able to run M-x.
After this change, you will get "M-x is undefined"


https://debbugs.gnu.org/cgi/bugreport.cgi?bug=13793
24.3.50; M-x broken in viper and X
Feb 2013
Frank Fischer noticed this problem and reported it, and provided some
analysis that helped me confirm the change to the keymap behavior I
mentioned in the previous paragraph. The fix for this bug was the next
change:


f1e6674bb32058c7ef683d5f8f8ac67f99e96dd2
Emacs ~24, Jul 2013
The key element of this change was to change viper-ESC-key from ESC to
<escape>. Since the (27 . 'intercept-viper-ESC-key) mapping changed to
(escape . 'intercept-viper-ESC-key), Graphical Emacs was no longer
blocked from finding the ESC prefix mappings for M- keys.  But this is
where ^[ stopped working to exit vi insert state in graphical
Emacs. Since viper--tty-ESC-filter doesn't run in graphical sessions,
and the viper mappings were now on <escape>, ^[ was just treated as a
pending ESC- prefix. In termainal sessions, viper--tty-ESC-filter
translates the ^[ to an <escape> after the timeout, and viper handles
it as designed.


https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18182 reported
(to be fixed later)
24.3.92; C-[ does not work as ESC in viper-mode
iquiw noticed this break and reported it


Emacs 27.2
This release behaves the same as the previous Emacs version in this
timeline.


5d522b430bd5ecfb8f082906cd634883dbb68f3e
Emacs 28.0.50
Fixes bug#18182 by reverting a part of
f1e6674bb32058c7ef683d5f8f8ac67f99e96dd2: viper-ESC-key gets set back
to ESC, which puts the direct ESC mapping for viper-intercept-ESC-key
back in the viper minor maps.  This is where the current broken
behavior comes from with keys like M-x.  This is the first time we see
this bug in terminal sessions (AFAIK).  (I believe this is because the
changes in f1e6674 changed the points at which manual ESC events were
distinguished from ESC-as-meta events, from the original
"envelop-style" to the newer style.) In the terminal, ESC is processed
immediately by viper-intercept-ESC-key, which either exits insert
state or does nothing if not in insert state, and then the x event is
processed.  (There is one exception: when viper-no-multiple-ESC is nil
and you are in vi state, the viper-ESC function does successfully
interpret ESC x in Emacs-style as an M-x.  This appears to be the only
case when that happens.) In graphical sessions, the direct ESC binding
means M-x is undefined like it was in
99d0d6dc23f0fd2ee6d64f0f18a33f2b791c642d



ANALYSIS
It seems like the behavior in v24/git f1e6674 and Emacs 27.2 was the
most stable in recent history. As suggested earlier, since the only
viper bug (that I know of) was that ^[ wouldn't exit insert state in
graphical sessions, would tolerating that bug be an acceptable
worst-case scenario for now, as opposed to all M- keys being broken in
viper terminal/graphical?

And if you do want to fix that behavior, would it be as simple as the
workaround Stefan offered:
  (define-key input-decode-map [?\e] [escape])
for all graphical sessions?

It doesn't seem like there are many other options, since having a
binding for ^[ will break M- keys. But on that note, as Frank Fischer
wondered, wouldn't fixing that original 2013 change to the keymap C
code be the best fix? Since ESC keybindings conflict with M-
keybindings in the keymap formats, it seems right that graphical Emacs
would have had a way to process them separately. Was that behavior
deliberately removed, or could it return some day?  I'll note that, in
the timeline I looked at here, it's only graphical Emacs that ever had
the inherent ability to resolve M- bindings even when there were
direct ESC bindings.  Terminal Emacs would always invoke whatever the
ESC binding was, which is why the viper interceptors had to do the
timeout checks. I doubt that would change.

Message #8 received at 51253 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: John Cummings <john <at> rootabega.net>
Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, 51253 <at> debbugs.gnu.org
Subject: Re: bug#51253: 28.0.60; Meta keys broken when viper is active
Date: Tue, 19 Oct 2021 16:13:39 -0700

John Cummings <john <at> rootabega.net> writes:

> Keys like M-x, M-:, etc., are broken in viper
> vi/insert state since bug
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18182 was fixed in
> 5d522b430bd5ecfb8f082906cd634883dbb68f3e. This changed viper-ESC-key
> from <escape> to ESC, which resulted in a direct key binding of (27
> . 'viper-intercept-ESC-key) while in the viper vi/insert states.

Maybe that commit should just be reverted?

Message #11 received at 51253 <at> debbugs.gnu.org (full text, mbox):

From: John Cummings <john <at> rootabega.net>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, 51253 <at> debbugs.gnu.org
Subject: bug#51253: 28.0.60; Meta keys broken when viper is active
Date: Tue, 19 Oct 2021 23:32:09 +0000

Stefan Kangas <stefan <at> marxist.se> wrote:

> John Cummings john <at> rootabega.net writes:
>
> > Keys like M-x, M-:, etc., are broken in viper
> > vi/insert state since bug
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18182 was fixed in
> > 5d522b430bd5ecfb8f082906cd634883dbb68f3e. This changed viper-ESC-key
> > from <escape> to ESC, which resulted in a direct key binding of (27
> > . 'viper-intercept-ESC-key) while in the viper vi/insert states.
>
> Maybe that commit should just be reverted?

I've looked into this some more, and I think reverting it is the
way that the entirety of the viper ESC-handling stack was designed
to function. At least on terminals. i.e., viper--tty-escape-filter
is meant to translate a solitary ESC into [escape], which
viper-intercept-ESC-key is meant to handle. And on graphical terminals,
we have the prefix vs. direct binding conflict that seems to (in my
limited but growing understanding) completely rule out binding any
commands to ESC.

But if this does get reverted, and it turns out that allowing ESC to
exit vi insert is still important, it doesn't seem like it would be
too difficult to ignore direct ESC command bindings when looking up
meta prefix bindings here:
https://git.savannah.gnu.org/cgit/emacs.git/tree/src/keymap.c?id=f3aa648093a70c8ed15e764863a16fdf7126cdc4#n343
I've been playing with it to help me get more familiar with it; maybe
I'll have a proof of concept one day.

Message #14 received at 51253 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: John Cummings <john <at> rootabega.net>
Cc: 51253 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#51253: 28.0.60; Meta keys broken when viper is active
Date: Wed, 20 Oct 2021 10:08:48 +0200

John Cummings <john <at> rootabega.net> writes:

>> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18182 was fixed in
>> > 5d522b430bd5ecfb8f082906cd634883dbb68f3e. This changed viper-ESC-key
>> > from <escape> to ESC, which resulted in a direct key binding of (27
>> > . 'viper-intercept-ESC-key) while in the viper vi/insert states.
>>
>> Maybe that commit should just be reverted?
>
> I've looked into this some more, and I think reverting it is the
> way that the entirety of the viper ESC-handling stack was designed
> to function.

I've now reverted it in emacs-28, and I'm reopening bug#18182.

> But if this does get reverted, and it turns out that allowing ESC to
> exit vi insert is still important, it doesn't seem like it would be
> too difficult to ignore direct ESC command bindings when looking up
> meta prefix bindings here:
> https://git.savannah.gnu.org/cgit/emacs.git/tree/src/keymap.c?id=f3aa648093a70c8ed15e764863a16fdf7126cdc4#n343
> I've been playing with it to help me get more familiar with it; maybe
> I'll have a proof of concept one day.

It would be nice to have it working, so if you can come up with a
solution, that'd be great.  (Post the solution to 18182, please.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #19 received at 51253 <at> debbugs.gnu.org (full text, mbox):

From: John Cummings <john <at> rootabega.net>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 51253 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: bug#51253: 28.0.60; Meta keys broken when viper is active
Date: Wed, 20 Oct 2021 08:24:54 +0000

Lars Ingebrigtsen <larsi <at> gnus.org> wrote:

> John Cummings john <at> rootabega.net writes:
>
> > > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18182 was fixed in
> > > > 5d522b430bd5ecfb8f082906cd634883dbb68f3e. This changed viper-ESC-key
> > > > from <escape> to ESC, which resulted in a direct key binding of (27
> > > > . 'viper-intercept-ESC-key) while in the viper vi/insert states.
> > >
> > > Maybe that commit should just be reverted?
> >
> > I've looked into this some more, and I think reverting it is the
> > way that the entirety of the viper ESC-handling stack was designed
> > to function.
>
> I've now reverted it in emacs-28, and I'm reopening bug#18182.

Thanks, it's working again on my end! By the way, it had broken both
terminal and gui Emacs, just in different ways (re: the commit message).

I'll see what more I can do about the keymap logic.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.