GNU bug report logs - #48136
28.0.50; [feature/pgtk] ISO_Level5_Shift broken

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 48136 in the body.
You can then email your comments to 48136 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Sat, 01 May 2021 11:34:42 +0200

[Message part 1 (text/plain, inline)]

- Use pgtk feature branch GNU Emacs 
- Use a keyboard layout that utilizes ISO_Level5_Shift (e.g. neoqwertz, bone, …) 
- Launch with `emacs -Q`
- Try to use ISO_Level5_Shift to access arrow keys or other shifted key
- `H-<down> is undefined` appears in echo area instead of sending down 
arrow and invoking `next-line` (This works in GNU Emacs without pgtk) 

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.16.0) 
Repository revision: 8b51761dfda1c85c1e0b2344b005bb435117937a 
Repository branch: master 
Windowing system distributor 'System Description: NixOS 21.05 (Okapi) 

Configured using: 
'configure 
--prefix=/nix/store/nkq5x0gbzjalnq7xq78abfsaxfg00yx7-emacs-pgtkgcc-20210425.0 
--disable-build-details --with-modules --with-x-toolkit=gtk3 
--with-cairo --with-native-compilation --with-pgtk' 

Configured features: 
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON 
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY 
PDUMPER PGTK PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XIM 
GTK3 ZLIB 

Important settings: 
value of $LC_CTYPE: zh_CN.UTF-8 
value of $LC_TIME: en_DK.UTF-8 
value of $LANG: en_US.UTF-8 
locale-coding-system: utf-8-unix 

Major mode: Lisp Interaction 

Minor modes in effect: 
tooltip-mode: t 
global-eldoc-mode: t 
eldoc-mode: t 
electric-indent-mode: t 
mouse-wheel-mode: t 
tool-bar-mode: t 
menu-bar-mode: t 
file-name-shadow-mode: t 
global-font-lock-mode: t 
font-lock-mode: t 
blink-cursor-mode: t 
auto-composition-mode: t 
auto-encryption-mode: t 
auto-compression-mode: t 
line-number-mode: t 
transient-mark-mode: t 

Load-path shadows: 
None found. 

Features: 
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs 
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail 
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs 
eieio-loaddefs password-cache json map text-property-search time-date 
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies 
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs 
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils 
china-util iso-transl tooltip eldoc electric uniquify ediff-hook 
vc-hooks lisp-float-type mwheel term/pgtk-win pgtk-win term/common-win 
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace 
newcomment text-mode elisp-mode lisp-mode prog-mode register page 
tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar 
mouse jit-lock font-lock syntax font-core term/tty-colors frame 
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese 
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek 
romanian slovak czech european ethiopic indian cyrillic chinese 
composite charscript charprop case-table epa-hook jka-cmpr-hook help 
simple abbrev obarray cl-preloaded nadvice button loaddefs faces 
cus-face macroexp files window text-properties overlay sha1 md5 base64 
format env code-pages mule custom widget hashtable-print-readable 
backquote threads dbusbind inotify dynamic-setting system-font-setting 
font-render-setting cairo move-toolbar gtk x-toolkit pgtk multi-tty 
make-network-process nativecomp emacs) 

Memory information: 
((conses 16 67501 7027) 
(symbols 48 6638 0) 
(strings 32 19211 1528) 
(string-bytes 1 687662) 
(vectors 16 13850) 
(vector-slots 8 339430 10291) 
(floats 8 22 31) 
(intervals 56 205 0) 
(buffers 992 10)) 

[Message part 2 (text/html, inline)]

Message #8 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: 48136 <at> debbugs.gnu.org
Subject: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Sat, 08 May 2021 13:50:28 +0200

[Message part 1 (text/plain, inline)]

Addendum:

I just tried if the same applied to terminal Emacs by launching it with `emacs -nw` and it does not (presumably because Super and Hyper are not passed from the terminal emulator to Emacs itself).

I was able to work around the bug by using `(define-key key-translation-map (kbd "H-<down>") (kbd "<down>"))` etc. for all ISO_Level5 keys but that is a bit too cluncky and static for my tastes. Is there a way to just translate every "H-" binding to just not have the modifier applied? Is there a list of all possible keys Emacs accepts and that I can use to just `map` over the `define-key` statement.

xmodmap output:

```
xmodmap:  up to 3 keys per modifier, (keycodes in parentheses):

shift       Shift_L (0x32),  Shift_R (0x3e)
lock        Shift_L (0x32)
control     Control_L (0x40),  Control_R (0x69)
mod1        Alt_L (0x85),  Meta_L (0xcd)
mod2        BadKey (0xcf)
mod3        ISO_Level5_Shift (0xcb)
mod4        Super_L (0x25),  Super_R (0x86),  Super_L (0xce)
mod5        ISO_Level3_Shift (0x5c)
```

[Message part 2 (text/html, inline)]

Message #11 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Yuuki Harano <masm+emacs <at> masm11.me>
To: gnu-emacs <at> lycheese.org
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Sun, 09 May 2021 19:17:18 +0900 (JST)

On Sat, 08 May 2021 13:50:28 +0200,
	"Elia Nolz" <gnu-emacs <at> lycheese.org> wrote:
> I was able to work around the bug by using `(define-key key-translation-map (kbd "H-<down>") (kbd "<down>"))` etc. for all ISO_Level5 keys but that is a bit too cluncky and static for my tastes. Is there a way to just translate every "H-" binding to just not have the modifier applied? Is there a list of all possible keys Emacs accepts and that I can use to just `map` over the `define-key` statement.
> 
> xmodmap output:
> 
> ```
> xmodmap:  up to 3 keys per modifier, (keycodes in parentheses):
> 
> shift       Shift_L (0x32),  Shift_R (0x3e)
> lock        Shift_L (0x32)
> control     Control_L (0x40),  Control_R (0x69)
> mod1        Alt_L (0x85),  Meta_L (0xcd)
> mod2        BadKey (0xcf)
> mod3        ISO_Level5_Shift (0xcb)
> mod4        Super_L (0x25),  Super_R (0x86),  Super_L (0xce)
> mod5        ISO_Level3_Shift (0x5c)
> ```

Thanks for xmodmap output.
mod3 is used as hyper in pgtk emacs.

Does this work for you?

(put 'none 'modifier-value 0)
(setq x-hyper-keysym 'none)

-- 
Yuuki Harano

Message #14 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Sun, 09 May 2021 12:47:38 +0200

[Message part 1 (text/plain, inline)]

On Sun, May 9, 2021, at 12:17, Yuuki Harano wrote:
> Does this work for you?
> 
> (put 'none 'modifier-value 0)
> (setq x-hyper-keysym 'none)

Yes this works perfectly for ISO_Level5_Shift. Thank you very much.

Sadly I stumbled upon another problem:

ISO_Level3_Shift combined with other modifiers is also partly broken. E.g. when using C-ISO_Level3_Shift-w (ISO_Level3_Shift-w is & in my layout) it just sends '&' instead of 'C-&'. Strangely enough the same is not true for 'C-ISO_Level3_Shift-a' which sends 'C-]' as it should. And 'C-IL3-u' sends 'ESC' instead of 'C-['. I am thoroughly confused.

[Message part 2 (text/html, inline)]

Message #17 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Thu, 20 May 2021 11:30:46 +0200

[Message part 1 (text/plain, inline)]

On Sun, May 9, 2021, at 12:47, Elia Nolz wrote:
> On Sun, May 9, 2021, at 12:17, Yuuki Harano wrote:
>> Does this work for you?
>> 
>> (put 'none 'modifier-value 0)
>> (setq x-hyper-keysym 'none)
> 
> Yes this works perfectly for ISO_Level5_Shift. Thank you very much.
> 
> Sadly I stumbled upon another problem:
> 
> ISO_Level3_Shift combined with other modifiers is also partly broken. E.g. when using C-ISO_Level3_Shift-w (ISO_Level3_Shift-w is & in my layout) it just sends '&' instead of 'C-&'. Strangely enough the same is not true for 'C-ISO_Level3_Shift-a' which sends 'C-]' as it should. And 'C-IL3-u' sends 'ESC' instead of 'C-['. I am thoroughly confused.

I tested it a bit more and ended up with the following list:

Working key combinations (Ctrl and ISO_Level3_Shift held):
C-_
C-[ (I have since realized that it is expected behaviour that this sends ESC)
C-]
C-^
C-\
C-/
C-{
C-}
C-|
C-~
C-`
C-@

Non-working combinations (Ctrl held):
C-ü
C-ä
C-ö

Non-working combinantions (Ctrl and ISO_Level3_Shift held):
C-…
C-*
C-#
C-$
C-!
C-<
C->
C-=
C-&
C-?
C-(
C-)
C--
C-:
C-+
C-%
C-"
C-'
C-;
C-ſ

I am not really sure how to debug this further since I do not know where the processing of interpreting a key press starts in Emacs. Any tips on this?

[Message part 2 (text/html, inline)]

Message #20 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Yuuki Harano <masm+emacs <at> masm11.me>
To: gnu-emacs <at> lycheese.org
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Thu, 20 May 2021 20:02:27 +0900 (JST)

On Thu, 20 May 2021 11:30:46 +0200,
	"Elia Nolz" <gnu-emacs <at> lycheese.org> wrote:
> I am not really sure how to debug this further since I do not know
> where the processing of interpreting a key press starts in
> Emacs. Any tips on this?

Key events are handled in key_press_event() in pgtkterm.c.
I think `keysym` contains a character code, which is already
interpreted.

-- 
Yuuki Harano

Message #23 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Thu, 20 May 2021 15:28:49 +0200

[Message part 1 (text/plain, inline)]

On Thu, May 20, 2021, at 13:02, Yuuki Harano wrote:
> Key events are handled in key_press_event() in pgtkterm.c.
> I think `keysym` contains a character code, which is already
> interpreted.

I stepped through the code with gdb and watched the keysym values.

I observed the following values for keysym in this order when pressing "C-L3-i" (sends "C-{") (working case):
65507 (state goes to 4 --> state to 0 and modifiers to 4)
65027 (state becomes 132 --> modifiers becomes 132 and state 128)
123 (no changes to state and modifiers)
65515 (should be me using Super+o to switch workspaces in xmonad)

And these for the non working case "C-L3-n" (which sends "(" and should send "C-("):
65507 (state to 4 --> state to 0 and modifiers to 4)
65027 (both state and modifiers become 0)
65515

I was unable to get the keysym of the key itself in the non-working cases.

Used breakpoints in pgtkterm.c:
5669
5687
5701
5710
5731
5809
5860
5876

I sadly do not know enough C to understand why this happens. Does this help you at all? Any further information I could provide that would be helpful?

[Message part 2 (text/html, inline)]

Message #26 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Yuuki Harano <masm+emacs <at> masm11.me>
To: gnu-emacs <at> lycheese.org
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 21 May 2021 01:02:50 +0900 (JST)

Thank you for the detail.

On Thu, 20 May 2021 15:28:49 +0200,
	"Elia Nolz" <gnu-emacs <at> lycheese.org> wrote:
> I was unable to get the keysym of the key itself in the non-working cases.

1. Start pgtk emacs.
2. Execute:
   (pgtk-use-im-context nil)
3. Test once more.

-- 
Yuuki Harano

Message #29 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Thu, 20 May 2021 18:58:13 +0200

[Message part 1 (text/plain, inline)]

On Thu, May 20, 2021, at 18:02, Yuuki Harano wrote:
>    (pgtk-use-im-context nil)
With that the keybindings all work. But now I can no longer activate fcitx within Emacs…is it possible to get fcitx working another way or do I need to switch to ddskk or similar?

[Message part 2 (text/html, inline)]

Message #32 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Yuuki Harano <masm+emacs <at> masm11.me>
To: gnu-emacs <at> lycheese.org
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 21 May 2021 15:57:57 +0900 (JST)

On Thu, 20 May 2021 18:58:13 +0200,
	"Elia Nolz" <gnu-emacs <at> lycheese.org> wrote:
>>    (pgtk-use-im-context nil)
> With that the keybindings all work. But now I can no longer activate
> fcitx within Emacs…is it possible to get fcitx working another way
> or do I need to switch to ddskk or similar?

It disables GtkIM, so you can't use fcitx from emacs.

> do I need to switch to ddskk or similar?

Yes.

-- 
Yuuki Harano

Message #35 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 21 May 2021 10:22:42 +0200

[Message part 1 (text/plain, inline)]

On Fri, May 21, 2021, at 08:57, Yuuki Harano wrote:
> > do I need to switch to ddskk or similar?
> 
> Yes.
Ok, I will do that then.

I have documented the configuration needed to get back to Emacs' default behaviour on my side but I have seen quite a few people who use a similar setup to mine struggle with the changes to the behaviour and some drop pgtk altogether.

Will there be a disclaimer or similar when pgtk is merged into master or when that version of Emacs is released? Maybe it would be advisable to make these changes the default since that is how Emacs behaved to date?

[Message part 2 (text/html, inline)]

Message #38 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Yuuki Harano <masm+emacs <at> masm11.me>
To: gnu-emacs <at> lycheese.org
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 21 May 2021 18:56:34 +0900 (JST)

On Fri, 21 May 2021 10:22:42 +0200,
	"Elia Nolz" <gnu-emacs <at> lycheese.org> wrote:
> Maybe it would be advisable to make these changes the default since that is how Emacs behaved to date?

Does "these changes" mean this?

>    (pgtk-use-im-context nil)

Previously nil is the default.
But some people couldn't input correctly, so I reverted it to t,
also because I thought t was what the gtk app should be.
See:
https://github.com/masm11/emacs/issues/43

-- 
Yuuki Harano

Message #41 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Wed, 09 Jun 2021 11:21:45 +0200

[Message part 1 (text/plain, inline)]

On Fri, May 21, 2021, at 11:56, Yuuki Harano wrote:
> Does "these changes" mean this?
> 
> >    (pgtk-use-im-context nil)

No, I was referring to the new default behaviour of the hyper key and how it changes how Emacs handles ISO_Level5_Shift:

(put 'none 'modifier-value 0)
(setq x-hyper-keysym 'none)


> Previously nil is the default.
> But some people couldn't input correctly, so I reverted it to t,
> also because I thought t was what the gtk app should be.
> See:
> https://github.com/masm11/emacs/issues/43

Regarding the behaviour with input methods, I honestly think that this is a bug that should be fixed since it straight up makes working Emacs setups impossible by disabling fcitx support (I have so far been unable to get ddskk working on NixOS).

[Message part 2 (text/html, inline)]

Message #44 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Thu, 24 Jun 2021 10:52:30 +0200

[Message part 1 (text/plain, inline)]

On Wed, Jun 9, 2021, at 11:21, Elia Nolz wrote:
> Regarding the behaviour with input methods, I honestly think that this is a bug that should be fixed since it straight up makes working Emacs setups impossible by disabling fcitx support (I have so far been unable to get ddskk working on NixOS).
After updating Emacs (and without changing my config) both fcitx and the ISO_Level5 and 3 keybindings work now.

emacs-repository-version: c6ada6e9f1f82d36a05ecc49844e9400e0da8fdd

[Message part 2 (text/html, inline)]

Message #47 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Yuuki Harano <masm+emacs <at> masm11.me>
To: gnu-emacs <at> lycheese.org
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 25 Jun 2021 21:05:53 +0900 (JST)

On Thu, 24 Jun 2021 10:52:30 +0200,
	"Elia Nolz" <gnu-emacs <at> lycheese.org> wrote:
> After updating Emacs (and without changing my config) both fcitx and the ISO_Level5 and 3 keybindings work now.
> 
> emacs-repository-version: c6ada6e9f1f82d36a05ecc49844e9400e0da8fdd

Can you investigate what commit fixed your issue?
Maybe 7085fecf53d9506b1f626e3cab1dd90e33aa9ed0?

-- 
Yuuki Harano

Message #50 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: "Elia Nolz" <gnu-emacs <at> lycheese.org>
To: "Yuuki Harano" <masm+emacs <at> masm11.me>
Cc: 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 01 Oct 2021 17:09:28 +0200

[Message part 1 (text/plain, inline)]

> Can you investigate what commit fixed your issue?
> Maybe 7085fecf53d9506b1f626e3cab1dd90e33aa9ed0?
Sorry for the late reply. I had health problems and am still not back to normal. I tried the commits below and the behaviour wasn't any different:
7085fecf53d9506b1f626e3cab1dd90e33aa9ed0
67a4c8b5ad9ebfb3fa2c7fae433aa6b6b5e92154
486e77befd2ea7fd750b1f24eb5dfaf2b91d70b3
66a36f1e5a323aed3d39db1044a1b71373123832
 I sadly do not have the time to any more testing than this.

And maybe something interesting: It only works when `(pgtk-use-im-context nil)` is called during startup in init.el or similar. When evaluating it by hand within Emacs fcitx stops working.

[Message part 2 (text/html, inline)]

Message #53 received at 48136 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Elia Nolz" <gnu-emacs <at> lycheese.org>
Cc: Yuuki Harano <masm+emacs <at> masm11.me>, 48136 <at> debbugs.gnu.org
Subject: Re: bug#48136: 28.0.50; [feature/pgtk] ISO_Level5_Shift broken
Date: Fri, 01 Jul 2022 13:13:47 +0200

"Elia Nolz" <gnu-emacs <at> lycheese.org> writes:

> After updating Emacs (and without changing my config) both fcitx and
> the ISO_Level5 and 3 keybindings work now.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

Skimming this bug report, it seems like the reported issue was fixed,
and I'm therefore closing this bug report.  If that's mistaken, please
respond to the debbugs address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.