GNU bug report logs - #48042
26.3; Macros don't work with french-postfix input method

Previous Next

Package: emacs;

Reported by: harven <at> free.fr

Date: Mon, 26 Apr 2021 18:07:02 UTC

Severity: normal

Tags: confirmed

Found in versions 28.0.50, 26.3

Done: Eli Zaretskii <eliz <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 48042 in the body.
You can then email your comments to 48042 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Mon, 26 Apr 2021 18:07:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to harven <at> free.fr:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Mon, 26 Apr 2021 18:07:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: harven <at> free.fr
To: bug-gnu-emacs <at> gnu.org
Subject: 26.3; Macros don't work with french-postfix input method
Date: Mon, 26 Apr 2021 20:05:54 +0200
Starting with emacs -Q in the *scratch* buffer, choose 
the french-postfix input method for the buffer, for example by typing

    M-x set-input-method RET french-postfix RET

Then register a simple macro inputing characters in buffer, for example

    C-x ( yes RET C-x )

Finally execute the macro

    C-x e

The word inserted at point is not "yes" but "yeess"
Characters insertion in macros mysteriously prints some characters twice.
This is not specific to the *scratch* buffer and can be demonstrated
in any buffer with the "french-postfix" method enabled. 
Other than that, macros seem to work fine.

In GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.14)
 of 2020-03-26, modified by Debian built on lcy01-amd64-020
Windowing system distributor 'The X.Org Foundation', version 11.0.12009000
System Description:	Ubuntu 20.04.2 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading quail/latin-post...done
You can run the command ‘set-input-method’ with C-x RET C-\
Defining kbd macro...
Keyboard macro defined
(Type e to repeat macro)

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-mEZBk7/emacs-26.3+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: fr_FR.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
mml-sec password-cache epa derived epg epg-config gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils cl-seq cl-extra edmacro kmacro cus-start
cus-load quail help-mode easymenu cl-loaddefs cl-lib elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 112963 9527)
 (symbols 48 21805 1)
 (miscs 40 55 154)
 (strings 32 33142 1268)
 (string-bytes 1 820564)
 (vectors 16 16447)
 (vector-slots 8 512262 11228)
 (floats 8 51 104)
 (intervals 56 269 0)
 (buffers 992 12))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Mon, 26 Apr 2021 18:24:02 GMT) Full text and rfc822 format available.

Message #8 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: harven <at> free.fr
Cc: 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3;
 Macros don't work with french-postfix input method
Date: Mon, 26 Apr 2021 21:22:57 +0300
> From: harven <at> free.fr
> Date: Mon, 26 Apr 2021 20:05:54 +0200
> 
> Starting with emacs -Q in the *scratch* buffer, choose 
> the french-postfix input method for the buffer, for example by typing
> 
>     M-x set-input-method RET french-postfix RET
> 
> Then register a simple macro inputing characters in buffer, for example
> 
>     C-x ( yes RET C-x )
> 
> Finally execute the macro
> 
>     C-x e
> 
> The word inserted at point is not "yes" but "yeess"

In Emacs 27 and later, I get "yess".  So the situation is better, but
not entirely fixed yet.




Added tag(s) confirmed. Request was from Stefan Kangas <stefan <at> marxist.se> to control <at> debbugs.gnu.org. (Wed, 28 Apr 2021 10:56:02 GMT) Full text and rfc822 format available.

bug Marked as found in versions 28.0.50. Request was from Stefan Kangas <stefan <at> marxist.se> to control <at> debbugs.gnu.org. (Wed, 28 Apr 2021 10:56:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Wed, 28 Apr 2021 18:25:02 GMT) Full text and rfc822 format available.

Message #15 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: harven <at> free.fr
To: 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3;
 Macros don't work with french-postfix input method
Date: Wed, 28 Apr 2021 20:24:30 +0200
The bug can probably be reproduced with all latin postfix input methods.
I tried the following methods from latin-post.el 
and got the same behavior as the french-postfix method.

french-alt-postfix german-postfix croatian-postfix greek-postfix
latin-postfix latin-1-postfix esperanto-postfix  spanish-postfix 
latin-alt-postfix swedish-postfix scandinavian-postfix
norwegian-postfix italian-postfix icelandic-postfix 

The doubling happens for letters that admit accented versions.

So the bug should probably be renamed 
"Macros don't work with postfix input methods."




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 09:30:02 GMT) Full text and rfc822 format available.

Message #18 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: harven <at> free.fr
Cc: 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 09:29:08 +0000
[Message part 1 (text/plain, inline)]
This was an interesting bug.  Let's first define a recipe to make it more 
apparent:

C-x C-m C-\ french-postfix RET C-x ( pa^te'_a`_l'Unicode RET C-x ) C-x e

This should insert "pâté_à_l'Unicode".  It worked as expected with Emacs 
21-24.

After commit 30a6b1f814, which was the result of the discussion in 
http://lists.gnu.org/archive/html/emacs-devel/2015-08/msg00193.html , and 
before commit 03e3440dbb, it inserts "paâtteé__aà__l'UUnniiccocododee". 
This is what one gets in Emacs 25 and 26.  This is not surprising, as the 
recipe in that discussion and quail.el both use unread-command-events, but 
expect an opposite effect.

After commit 03e3440dbb, which was the result of bug#32108, it inserts 
"pâtté__à__l'Unnicocdode".  This is what one gets in Emacs 27 (and 28 till 
now).

I attach a patch to fix that bug.  I checked that the recipes that led to 
the two above commits still work correctly, and that it does not introduce 
regressions with make check.

Note that the inhibit--record-char variable, which was introduced by 
commit 03e3440dbb, is, after applying that patch, only used once, namely 
in lisp/term/xterm.el, as a result of bug#44908.  It is not used in ELPA 
or MELPA.  I'm not convinced that bug#44908 is a sufficient reason to keep 
that variable.
[Fix-input-method-bug-when-recording-keyboard-macros.patch (text/x-diff, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 09:56:01 GMT) Full text and rfc822 format available.

Message #21 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 10:55:12 +0100
Gregory Heytings <gregory <at> heytings.org> writes:

> +      ! (! NILP (KVAR (current_kboard, defining_kbd_macro)) &&
> +         ! NILP (Fsymbol_value (Qcurrent_input_method))))

Nit: Can we De Morgan?

  (NILP (KVAR (current_kboard, defining_kbd_macro)) ||
   NILP (Fsymbol_value (Qcurrent_input_method)))

Thanks,

-- 
Basil




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 10:04:02 GMT) Full text and rfc822 format available.

Message #24 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 10:03:49 +0000
>> +      ! (! NILP (KVAR (current_kboard, defining_kbd_macro)) &&
>> +         ! NILP (Fsymbol_value (Qcurrent_input_method))))
>
> Nit: Can we De Morgan?
>
>  (NILP (KVAR (current_kboard, defining_kbd_macro)) ||
>   NILP (Fsymbol_value (Qcurrent_input_method)))
>

I tried this in an earlier version, but find the De Morgan'd version less 
clear.  Perhaps it's just me.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 11:10:01 GMT) Full text and rfc822 format available.

Message #27 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3;
 Macros don't work with french-postfix input method
Date: Fri, 14 May 2021 14:09:34 +0300
> Date: Fri, 14 May 2021 09:29:08 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> Cc: 48042 <at> debbugs.gnu.org
> 
> --- a/src/keyboard.c
> +++ b/src/keyboard.c
> @@ -3098,7 +3098,11 @@ read_char (int commandflag, Lisp_Object map,
>    /* When we consume events from the various unread-*-events lists, we
>       bypass the code that records input, so record these events now if
>       they were not recorded already.  */
> -  if (!recorded)
> +  if (!recorded &&
> +      /* However, don't record events when a keyboard macro is being
> +         defined and an input method is activated (Bug#48042).  */
> +      ! (! NILP (KVAR (current_kboard, defining_kbd_macro)) &&
> +         ! NILP (Fsymbol_value (Qcurrent_input_method))))

Bother: AFAIK, current-input-method non-nil means the user activated
an input method, it doesn't mean we are in the middle of typing a key
sequence that will yield a character via the input method.  That is, a
user could activate an input method, but still keep typing just ASCII
characters.  So why is this condition correct here to avoid recording
input more than once?

This is why I tried to have a variable that quail.el binds while
actually processing keys.  I'd appreciate some explanation for why
that didn't work 100% in the case in point (it still avoided recording
twice some of the keys, so it isn't entirely wrong).

Stefan, any comments?

Also, please avoid quoting a bug number in comments as a replacement
for explaining the code, unless the explanation would need a very long
and convoluted text.  We should strive to make our comments
self-explanatory.  (That some code was added/modified to fix a certain
bug is very easy to find out using "git log -L" and similar commands,
so a bug number is almost always redundant in comments.)

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 13:39:02 GMT) Full text and rfc822 format available.

Message #30 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, harven <at> free.fr,
 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 13:38:03 +0000
[Message part 1 (text/plain, inline)]
>
> Bother: AFAIK, current-input-method non-nil means the user activated an 
> input method, it doesn't mean we are in the middle of typing a key 
> sequence that will yield a character via the input method.  That is, a 
> user could activate an input method, but still keep typing just ASCII 
> characters.  So why is this condition correct here to avoid recording 
> input more than once?
>

I'm not sure I understand your question.  With an input method is 
activated and characters that are not "reprocessed" by the input method 
are typed, they are not added to the unread-command-events list, so this 
part of the code (which is entered with the goto reread_for_input_method 
above) is simply not executed.  When they are "reprocessed", they are 
added to unread-command-events, but the keys typed in by the user have 
already been recorded.

Until commit 30a6b1f814, that part of the code (if (!recorded)...) did not 
exist, and my patch simply restores the behavior that quail.el expects by 
skipping it.  Note that in 2015 you yourself described that commit as a 
"naïve attempt at fixing" the problem.

I attach an even better patch, which removes the condition that a keyboard 
macro is being defined.  Now the keys displayed in C-h l are also correct.

>
> This is why I tried to have a variable that quail.el binds while 
> actually processing keys.  I'd appreciate some explanation for why that 
> didn't work 100% in the case in point (it still avoided recording twice 
> some of the keys, so it isn't entirely wrong).
>

I don't claim to understand everything, but what I see is that 
unread-command-events is also used (set) in quail-update-translation, 
quail-next-translation, quail-prev-translation, 
quail-next-translation-block, quail-prev-translation-block and 
quail-minibuffer-message.
[Fix-key-recording-bug-when-an-input-method-is-activa.patch (text/x-diff, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 13:55:01 GMT) Full text and rfc822 format available.

Message #33 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 09:54:41 -0400
> I'm not sure I understand your question.  With an input method is activated
> and characters that are not "reprocessed" by the input method are typed,
> they are not added to the unread-command-events list, so this part of the

Not by quail, indeed, but `unread-command-events` may be set for other
reasons and some of those should be recorded.


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 14:06:02 GMT) Full text and rfc822 format available.

Message #36 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 17:04:59 +0300
> Date: Fri, 14 May 2021 13:38:03 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, harven <at> free.fr, 
>     48042 <at> debbugs.gnu.org
> 
> > Bother: AFAIK, current-input-method non-nil means the user activated an 
> > input method, it doesn't mean we are in the middle of typing a key 
> > sequence that will yield a character via the input method.  That is, a 
> > user could activate an input method, but still keep typing just ASCII 
> > characters.  So why is this condition correct here to avoid recording 
> > input more than once?
> 
> I'm not sure I understand your question.  With an input method is 
> activated and characters that are not "reprocessed" by the input method 
> are typed, they are not added to the unread-command-events list, so this 
> part of the code (which is entered with the goto reread_for_input_method 
> above) is simply not executed.

If you search the Lisp files for unread-command-events, you will see
how many features unrelated to input methods do add stuff to
unread-command-events.  What if an input method were active while one
of those unrelated features does that?

> Until commit 30a6b1f814, that part of the code (if (!recorded)...) did not 
> exist, and my patch simply restores the behavior that quail.el expects by 
> skipping it.

That part didn't exist because once upon a time we recorded everything
immediately as it happened.  When that changed (for a good reason, as
described in the discussion cited by the commit log message), we
needed to keep track of what was and what wasn't recorded.  So
restoring the behavior back to what it was then doesn't help me to
gain confidence in the correctness of the patch, sorry.  Going back to
what we had there is certainly not something we should do.

> Note that in 2015 you yourself described that commit as a "naïve
> attempt at fixing" the problem.

Every change in keyboard.c is "naïve" nowadays, since no one among us
really understands all of its intricacies.  Of course, if you do have
a clear understanding of what's going on there, it'd be splendid, but
I hope you will then be able to convince in the correctness of what
you propose.

> I attach an even better patch, which removes the condition that a keyboard 
> macro is being defined.  Now the keys displayed in C-h l are also correct.

Hmm... but the from_macro label seems to indicate that this code runs
for macros as well, even when input method is not active?

> > This is why I tried to have a variable that quail.el binds while 
> > actually processing keys.  I'd appreciate some explanation for why that 
> > didn't work 100% in the case in point (it still avoided recording twice 
> > some of the keys, so it isn't entirely wrong).
> 
> I don't claim to understand everything, but what I see is that 
> unread-command-events is also used (set) in quail-update-translation, 
> quail-next-translation, quail-prev-translation, 
> quail-next-translation-block, quail-prev-translation-block and 
> quail-minibuffer-message.

So you are saying we should bind the variable there as well?  If that
works, I'd be much happier, since in that case we will be sure we only
bypass recording exactly where that's needed.  However, ISTR that at
the time I tried adding the binding at least in some of those places,
and it didn't work well, or maybe didn't have any effect.  I think I
then convinced myself that only the first event needs this protection
(but I might be misremembering).  So it would be good to have a clear
understanding why this particular use case evades the current
protection against duplicate recording.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 14:09:02 GMT) Full text and rfc822 format available.

Message #39 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 14:08:06 +0000
>> I'm not sure I understand your question.  With an input method is 
>> activated and characters that are not "reprocessed" by the input method 
>> are typed, they are not added to the unread-command-events list, so 
>> this part of the
>
> Not by quail, indeed, but `unread-command-events` may be set for other 
> reasons and some of those should be recorded.
>

They will be recorded, unless an input method is activated.  What could 
happen now is that when an input method is activated and 
unread-command-events is set, these events will not be recorded.  Let's 
perhaps wait for another bug report to see whether this happens and how it 
can be fixed?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 14:13:02 GMT) Full text and rfc822 format available.

Message #42 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 17:12:43 +0300
> Date: Fri, 14 May 2021 14:08:06 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: Eli Zaretskii <eliz <at> gnu.org>, 48042 <at> debbugs.gnu.org, harven <at> free.fr
> 
> > Not by quail, indeed, but `unread-command-events` may be set for other 
> > reasons and some of those should be recorded.
> 
> They will be recorded, unless an input method is activated.  What could 
> happen now is that when an input method is activated and 
> unread-command-events is set, these events will not be recorded.  Let's 
> perhaps wait for another bug report to see whether this happens and how it 
> can be fixed?

I don't think this is wise.  We may be breaking features much more
important than postfix input methods used in macros.

I suggest that we first try to fully understand why binding
inhibit--record-char doesn't work in this case, and take it from
there.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 14:17:01 GMT) Full text and rfc822 format available.

Message #45 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 14:16:26 +0000
>
> If you search the Lisp files for unread-command-events,
>

Of course I did this.

>
> you will see how many features unrelated to input methods do add stuff 
> to unread-command-events.  What if an input method were active while one 
> of those unrelated features does that?
>

As I just said to Stefan, we could perhaps wait for an actual bug report 
to see whether this happens?  I tried to fix this bug, it still seems to 
me that what I proposed is TRT because it fixes the problem at its "root", 
but I cannot fix yet unknown bugs.

>
> That part didn't exist because once upon a time we recorded everything 
> immediately as it happened.  When that changed (for a good reason, as 
> described in the discussion cited by the commit log message), we needed 
> to keep track of what was and what wasn't recorded.  So restoring the 
> behavior back to what it was then doesn't help me to gain confidence in 
> the correctness of the patch, sorry.  Going back to what we had there is 
> certainly not something we should do.
>

We're not going back to what we had earlier, the patch goes back to what 
we had only when an input method is activated.

>
> Hmm... but the from_macro label seems to indicate that this code runs 
> for macros as well, even when input method is not active?
>

And in that case, given that the condition checks whether an input method 
is active, the event will be recorded.

>
> So you are saying we should bind the variable there as well?
>

I don't know, and didn't test that.  As I said above, it seems to me that 
fixing a bug at its "root" is much better than trying to add code 
elsewhere to circumvent the bug.

>
> Thanks.
>

What do you expect me to do now?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 14:37:02 GMT) Full text and rfc822 format available.

Message #48 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 17:36:00 +0300
> Date: Fri, 14 May 2021 14:16:26 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
> 
> What do you expect me to do now?

If you are asking how I'd like to proceed with this bug, then, as I
wrote in another message, I'd like us to have a full understanding of
why binding inhibit--record-char didn't work well enough in the case
in point.  It did solve some of the problem we had before that change,
but didn't solve all of it -- why?

Once we understand that, I think we will have a much better basis for
discussing the possible solutions.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 15:01:02 GMT) Full text and rfc822 format available.

Message #51 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 15:00:28 +0000
>> What do you expect me to do now?
>
> If you are asking how I'd like to proceed with this bug, then, as I 
> wrote in another message, I'd like us to have a full understanding of 
> why binding inhibit--record-char didn't work well enough in the case in 
> point.  It did solve some of the problem we had before that change, but 
> didn't solve all of it -- why?
>
> Once we understand that, I think we will have a much better basis for 
> discussing the possible solutions.
>

Okay, I'll have a look.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 15:12:02 GMT) Full text and rfc822 format available.

Message #54 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 18:11:14 +0300
> Date: Fri, 14 May 2021 15:00:28 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
> 
> > If you are asking how I'd like to proceed with this bug, then, as I 
> > wrote in another message, I'd like us to have a full understanding of 
> > why binding inhibit--record-char didn't work well enough in the case in 
> > point.  It did solve some of the problem we had before that change, but 
> > didn't solve all of it -- why?
> >
> > Once we understand that, I think we will have a much better basis for 
> > discussing the possible solutions.
> 
> Okay, I'll have a look.

Thank you.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 15:52:02 GMT) Full text and rfc822 format available.

Message #57 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Gregory Heytings <gregory <at> heytings.org>, 48042 <at> debbugs.gnu.org,
 harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 11:51:41 -0400
> If you are asking how I'd like to proceed with this bug, then, as I
> wrote in another message, I'd like us to have a full understanding of
> why binding inhibit--record-char didn't work well enough in the case
> in point.  It did solve some of the problem we had before that change,
> but didn't solve all of it -- why?

BTW, another possible way to attack the problem is to arrange for the
code that pushes to `unread-command-events` to use events of the form
`(t . EVENT)` so as to tell explicitly that we don't want them recorded.

I haven't re-read enough of the thread and corresponding code to know
which approach might be more applicable here; I'm just mentioning it in
case someone had forgotten about that option (been there, done that ;-)


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 16:00:02 GMT) Full text and rfc822 format available.

Message #60 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: gregory <at> heytings.org, 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 18:59:00 +0300
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Gregory Heytings <gregory <at> heytings.org>,  harven <at> free.fr,
>   48042 <at> debbugs.gnu.org
> Date: Fri, 14 May 2021 11:51:41 -0400
> 
> BTW, another possible way to attack the problem is to arrange for the
> code that pushes to `unread-command-events` to use events of the form
> `(t . EVENT)` so as to tell explicitly that we don't want them recorded.

Maybe we will do something like that, or maybe we'll decide something
else.  But before we do that, I'd like to have a good understanding
how the current code malfunctions and why.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 17:08:02 GMT) Full text and rfc822 format available.

Message #63 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 17:07:41 +0000
>> If you are asking how I'd like to proceed with this bug, then, as I 
>> wrote in another message, I'd like us to have a full understanding of 
>> why binding inhibit--record-char didn't work well enough in the case in 
>> point.  It did solve some of the problem we had before that change, but 
>> didn't solve all of it -- why?
>
> BTW, another possible way to attack the problem is to arrange for the 
> code that pushes to `unread-command-events` to use events of the form 
> `(t . EVENT)` so as to tell explicitly that we don't want them recorded.
>

I guess you mean '(no-record . EVENT)'?  The docstring indicates that '(t 
. EVENT)' means that EVENT is added to 'this-command-keys'.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Fri, 14 May 2021 17:14:02 GMT) Full text and rfc822 format available.

Message #66 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Fri, 14 May 2021 13:13:45 -0400
>>> If you are asking how I'd like to proceed with this bug, then, as I wrote
>>> in another message, I'd like us to have a full understanding of why
>>> binding inhibit--record-char didn't work well enough in the case in
>>> point.  It did solve some of the problem we had before that change, but
>>> didn't solve all of it -- why?
>>
>> BTW, another possible way to attack the problem is to arrange for the code
>> that pushes to `unread-command-events` to use events of the form `(t
>> . EVENT)` so as to tell explicitly that we don't want them recorded.
>>
>
> I guess you mean '(no-record . EVENT)'?  The docstring indicates that '(t
> . EVENT)' means that EVENT is added to 'this-command-keys'.

Oops, indeed,


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Sat, 15 May 2021 09:47:02 GMT) Full text and rfc822 format available.

Message #69 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 48042 <at> debbugs.gnu.org, harven <at> free.fr
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Sat, 15 May 2021 09:46:02 +0000
[Message part 1 (text/plain, inline)]
Here is an updated patch, which uses the '(no-record . EVENT)' approach, 
and whose changes are therefore limited to quail.el.
[Fix-key-recording-bug-when-an-input-method-is-activa.patch (text/x-diff, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Sat, 15 May 2021 10:22:02 GMT) Full text and rfc822 format available.

Message #72 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Sat, 15 May 2021 13:21:04 +0300
> Date: Sat, 15 May 2021 09:46:02 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: Eli Zaretskii <eliz <at> gnu.org>, harven <at> free.fr, 48042 <at> debbugs.gnu.org
> 
> Here is an updated patch, which uses the '(no-record . EVENT)' approach, 
> and whose changes are therefore limited to quail.el.

Thanks.

To tell you the truth, I'm a bit worried that you inhibit recording
everywhere in quail.el, which seems to contradict my analysis from
back when I made the inhibit--record-char.  How many different input
methods did you try with macros and "C-h l" to make sure this change
indeed causes Emacs to record each produced character just once, and
nothing is either omitted or recorded more than once?  Did you try
some CJK input methods, for example, which offer several alternatives
for each key sequence the user typed?

Other than this main issue, this LGTM, modulo some minor comments
below.

> +(defun quail-add-unread-command-events (key &optional reset)
> +  "Add KEY to `unread-command-events'.

This summary line should mention that the function arranges for the
events not to be recorded, and perhaps also explain in the rest of the
doc string (or in a comment) why is that needed here.

> +When KEY is a character, it is prepended to `unread-command-events' as a
> +cons with a no-record car.
> +When KEY is a vector, its elements are prepended to `unread-command-events'
> +as conses with a no-record car.

The last sentence above is not clear enough, IMO; I originally
interpreted it incorrectly.  I would suggest to reword:

  If KEY is a vector of events, the events are prepended
  to `unread-command-events', after converting each event
  to a cons cell of the form (no-record . EVENT).

> +When RESET is non-nil, the events in `unread-command-events' are first
> +discarded."

I'm not sure we need this as part of the function: it makes the
function more complicated for no good reason.  Why not reset
unread-command-events "by hand" in the one place where that is needed?
Or maybe even explicitly use (no-record . 7) in that one place, and
then you can avoid calling this function, since that one place does
something very different from the rest.

Please also make a single changeset out of this one and the one where
you remove inhibit--record-char; see there for more comments.  I see
no need to separate them into two commits.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Sat, 15 May 2021 18:48:01 GMT) Full text and rfc822 format available.

Message #75 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Sat, 15 May 2021 18:47:00 +0000
[Message part 1 (text/plain, inline)]
I attach the updated patch.

Some comments below:

>
> To tell you the truth, I'm a bit worried that you inhibit recording 
> everywhere in quail.el, which seems to contradict my analysis from back 
> when I made the inhibit--record-char.  How many different input methods 
> did you try with macros and "C-h l" to make sure this change indeed 
> causes Emacs to record each produced character just once, and nothing is 
> either omitted or recorded more than once?  Did you try some CJK input 
> methods, for example, which offer several alternatives for each key 
> sequence the user typed?
>

I tried french-postfix, russian-computer, ucs, TeX, arabic, japanese and 
chinese-py.  From what I can see, everything works as expected, but I 
don't speak or read these languages, and there are more than 200 input 
methods, I cannot test them all.

>> +When RESET is non-nil, the events in `unread-command-events' are first
>> +discarded."
>
> I'm not sure we need this as part of the function: it makes the function 
> more complicated for no good reason.  Why not reset 
> unread-command-events "by hand" in the one place where that is needed? 
> Or maybe even explicitly use (no-record . 7) in that one place, and then 
> you can avoid calling this function, since that one place does something 
> very different from the rest.
>

I did this to simplify future debugging sessions.  Having Quail interact 
with unread-command-events in a single place will make that easier.  And 
the added complexity is really small.

>> @@ -883,6 +882,9 @@ terminal-init-xterm
>>    (xterm--init-bracketed-paste-mode)
>>    ;; We likewise unconditionally enable support for focus tracking.
>>    (xterm--init-focus-tracking)
>> +  ;; Clear the lossage to discard the responses of the terminal emulator
>> +  ;; during initialization (Bug#44908)
>> +  (clear-this-command-keys)
>
> Btw, what happens if while this code runs, the user types something? We 
> don't want those events to be cleared.
>

It's quite hard to type something at that moment AFAICS, the whole 
terminal initialization typically takes at most 50 ms.  I tried to type 
something as fast as possible during the terminal initialization, and what 
happens is this (visible in *Messages*):

q is undefined
M-] is undefined
; is undefined
r is undefined
g is undefined
b is undefined
: is undefined
f is undefined [4 times]
/ is undefined
f is undefined [4 times]
/ is undefined
f is undefined [4 times]

or this:

q is undefined
M-[ > is undefined
; is undefined [2 times]
c is undefined

The first key is the pressed key ("q"), the other keys are some of the 
keys of the terminal initialization process (the digits are bound to 
digit-argument in the *GNU Emacs* buffer and are therefore not undefined):

ESC [ > ESC [ > 4 1 ; 3 6 6 ; 0 c ESC ] 1 1 ; r g b : f f f f / f f f f / 
f f f f ESC \

IOW, if the user presses something at that moment, the terminal 
initialization process does not work as expected.

(FWIW, I would have closed bug#44908 as a wontfix, seeing a few unwanted 
keys in the lossage after starting Emacs is a really minor annoyance.)
[Fix-key-recording-bug-when-an-input-method-is-activa.patch (text/x-diff, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Sat, 15 May 2021 18:53:01 GMT) Full text and rfc822 format available.

Message #78 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Sat, 15 May 2021 21:52:24 +0300
> Date: Sat, 15 May 2021 18:47:00 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
> 
> > To tell you the truth, I'm a bit worried that you inhibit recording 
> > everywhere in quail.el, which seems to contradict my analysis from back 
> > when I made the inhibit--record-char.  How many different input methods 
> > did you try with macros and "C-h l" to make sure this change indeed 
> > causes Emacs to record each produced character just once, and nothing is 
> > either omitted or recorded more than once?  Did you try some CJK input 
> > methods, for example, which offer several alternatives for each key 
> > sequence the user typed?
> 
> I tried french-postfix, russian-computer, ucs, TeX, arabic, japanese and 
> chinese-py.  From what I can see, everything works as expected, but I 
> don't speak or read these languages, and there are more than 200 input 
> methods, I cannot test them all.

No one can test all of them, that's not what I meant.

> > Btw, what happens if while this code runs, the user types something? We 
> > don't want those events to be cleared.
> >
> 
> It's quite hard to type something at that moment AFAICS, the whole 
> terminal initialization typically takes at most 50 ms.  I tried to type 
> something as fast as possible during the terminal initialization, and what 
> happens is this (visible in *Messages*):
> 
> q is undefined
> M-] is undefined
> ; is undefined
> r is undefined
> g is undefined
> b is undefined
> : is undefined
> f is undefined [4 times]
> / is undefined
> f is undefined [4 times]
> / is undefined
> f is undefined [4 times]
> 
> or this:
> 
> q is undefined
> M-[ > is undefined
> ; is undefined [2 times]
> c is undefined
> 
> The first key is the pressed key ("q"), the other keys are some of the 
> keys of the terminal initialization process (the digits are bound to 
> digit-argument in the *GNU Emacs* buffer and are therefore not undefined):
> 
> ESC [ > ESC [ > 4 1 ; 3 6 6 ; 0 c ESC ] 1 1 ; r g b : f f f f / f f f f / 
> f f f f ESC \
> 
> IOW, if the user presses something at that moment, the terminal 
> initialization process does not work as expected.

So maybe we shouldn't discard the input if the initialization fails?
Otherwise we leave no traces that could allow understanding what
happened.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48042; Package emacs. (Sat, 15 May 2021 20:18:01 GMT) Full text and rfc822 format available.

Message #81 received at 48042 <at> debbugs.gnu.org (full text, mbox):

From: Gregory Heytings <gregory <at> heytings.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Sat, 15 May 2021 20:17:31 +0000
[Message part 1 (text/plain, inline)]
>> IOW, if the user presses something at that moment, the terminal 
>> initialization process does not work as expected.
>
> So maybe we shouldn't discard the input if the initialization fails? 
> Otherwise we leave no traces that could allow understanding what 
> happened.
>

Okay, updated patch attached.
[Fix-key-recording-bug-when-an-input-method-is-activa.patch (text/x-diff, attachment)]

Reply sent to Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility. (Sat, 29 May 2021 08:21:02 GMT) Full text and rfc822 format available.

Notification sent to harven <at> free.fr:
bug acknowledged by developer. (Sat, 29 May 2021 08:21:02 GMT) Full text and rfc822 format available.

Message #86 received at 48042-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gregory Heytings <gregory <at> heytings.org>
Cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042-done <at> debbugs.gnu.org
Subject: Re: bug#48042: 26.3; Macros don't work with french-postfix input
 method
Date: Sat, 29 May 2021 11:20:25 +0300
> Date: Sat, 15 May 2021 20:17:31 +0000
> From: Gregory Heytings <gregory <at> heytings.org>
> cc: monnier <at> iro.umontreal.ca, harven <at> free.fr, 48042 <at> debbugs.gnu.org
> 
> Okay, updated patch attached.

Thanks, installed.

Please note that I needed to make a couple of minor fixes in the
commit log message, both in form and in contents.  Please in the
future try following our conventions there.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 26 Jun 2021 11:24:07 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 276 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.