GNU bug report logs - #77754
31.0.50; completion-table-with-quoting and completion-lazy-hilit don't work together

Previous Next

To reply to this bug, email your comments to 77754 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; completion-table-with-quoting and completion-lazy-hilit
 don't work together
Date: Fri, 11 Apr 2025 17:47:09 -0400

1. touch 'foo bxx' 'foo byy'
2. emacs -Q
3. M-x shell
4. Type "echo foo" and hit TAB; it will complete to "echo foo\ b"
5. Hit ? to bring up *Completions*
6. Note that the highlighting is incorrect; the next character should
   be highlighted in pink as the next difference, but it's not.

If you modify minibuffer-completion-help to not bind
completion-lazy-hilit to t (which I recently added), the highlighting
will be correct.

This is because completion-lazy-hilit=t makes the highlighting function
run on the *quoted* string (with the \), whereas
completion-lazy-hilit=nil runs highlighting on the *unquoted* string,
which is then highlighted correctly.

IMO, we should fix this by changing the API exposed for
completion-table-with-quoting.  As mentioned in a comment in
completion--twq-all:

   ;; The better solution is to not quote the *Completions* display,
   ;; which nicely circumvents the problem.

If we just stopped quoting *Completions*, we would avoid this problem.

This would necessitate adding some new way to quote the completion
before inserting it.  We could add some new piece of completion metadata
which provides a function which gets called when a completion candidate
is about to be inserted, and returns a string that should be inserted
instead - it can perform the quoting.  (Or it could even be a new
completion table action?)

I'm happy to make that change if that sounds reasonable to others.

(While adding this, I'd also like to make this new function return a
cons whose cdr is the new location of point after inserting the
candidate - that would be useful for various other fixes, including
making the emacs22 style behave better)


In GNU Emacs 31.0.50 (build 9, x86_64-pc-linux-gnu, GTK+ Version
 3.22.30, cairo version 1.15.12) of 2025-04-08 built on igm-qws-u22796a
Repository revision: fb4ae239053b5e03ade2af28573fe6608458a8b7
Repository branch: HEAD
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Rocky Linux 8.10 (Green Obsidian)

Configured using:
 'configure -C --with-gif=no'

Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX
LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM
XINERAMA XINPUT2 XPM XRANDR GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Eshell

Minor modes in effect:
  eshell-prompt-mode: t
  eshell-pred-mode: t
  eshell-hist-mode: t
  eshell-cmpl-mode: t
  eshell-proc-mode: t
  eshell-arg-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug lisp-mnt message mailcap yank-media puny
dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg
rfc6068 epg-config gnus-util time-date mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils pcmpl-unix cl-extra
help-mode em-unix em-term term disp-table ehelp em-script em-prompt
em-pred em-ls em-hist em-glob em-extpipe em-cmpl em-dirs em-basic
em-banner em-alias esh-mode esh-var eshell esh-cmd generator esh-ext
esh-proc esh-opt esh-io esh-arg esh-module esh-module-loaddefs esh-util
pp cl-seq project byte-opt gv sh-script smie treesit executable compile
text-property-search files-x shell pcomplete comint subr-x ansi-osc
ansi-color ring comp-run bytecomp byte-compile comp-common rx warnings
icons cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk
x-toolkit xinput2 x multi-tty move-toolbar make-network-process
tty-child-frames native-compile emacs)

Memory information:
((conses 16 136537 11841) (symbols 48 9968 0) (strings 32 57881 2879)
 (string-bytes 1 2842600) (vectors 16 25782)
 (vector-slots 8 321965 10866) (floats 8 62 66) (intervals 56 493 7)
 (buffers 984 16))

Message #8 received at 77754 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: 77754 <at> debbugs.gnu.org
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#77754: 31.0.50; completion-table-with-quoting and
 completion-lazy-hilit don't work together
Date: Mon, 14 Apr 2025 13:41:30 -0400

[Message part 1 (text/plain, inline)]

Spencer Baugh <sbaugh <at> janestreet.com> writes:
> 1. touch 'foo bxx' 'foo byy'
> 2. emacs -Q
> 3. M-x shell
> 4. Type "echo foo" and hit TAB; it will complete to "echo foo\ b"
> 5. Hit ? to bring up *Completions*
> 6. Note that the highlighting is incorrect; the next character should
>    be highlighted in pink as the next difference, but it's not.
>
> If you modify minibuffer-completion-help to not bind
> completion-lazy-hilit to t (which I recently added), the highlighting
> will be correct.
>
> This is because completion-lazy-hilit=t makes the highlighting function
> run on the *quoted* string (with the \), whereas
> completion-lazy-hilit=nil runs highlighting on the *unquoted* string,
> which is then highlighted correctly.
>
> IMO, we should fix this by changing the API exposed for
> completion-table-with-quoting.  As mentioned in a comment in
> completion--twq-all:
>
>    ;; The better solution is to not quote the *Completions* display,
>    ;; which nicely circumvents the problem.
>
> If we just stopped quoting *Completions*, we would avoid this problem.
>
> This would necessitate adding some new way to quote the completion
> before inserting it.  We could add some new piece of completion metadata
> which provides a function which gets called when a completion candidate
> is about to be inserted, and returns a string that should be inserted
> instead - it can perform the quoting.  (Or it could even be a new
> completion table action?)

Fortunately it actually turns out to be much, much simpler than this.
We continue to return the quoted completions from
completion-all-completions, and just use completion--unquoted for
display instead of the quoted completion.

Completion UIs should start calling the new completion-for-display
function, but even if they don't, nothing new breaks.

[0001-Display-unquoted-candidates-in-Completions.patch (text/x-patch, inline)]

From a42a44b996096a8c80bc972089b0e6e1f0bbd1f8 Mon Sep 17 00:00:00 2001
From: Spencer Baugh <sbaugh <at> janestreet.com>
Date: Mon, 14 Apr 2025 13:37:11 -0400
Subject: [PATCH] Display unquoted candidates in *Completions*

To fix bug#77754 and generally improve *Completions* behavior,
start displaying the unquoted versions of completion candidates
in *Completions* and in icomplete.

External completion UIs can call completion-for-display to
obtain the same behavior (and fix similar bugs in their UI).

* lisp/minibuffer.el (completion-for-display): Add.
(completion--twq-all): Add FIXME to simplify this function.
(completion--insert): Call completion-for-display. (bug#77754)
* lisp/icomplete.el (icomplete--render-vertical)
(icomplete-completions): Call completion-for-display.
---
 lisp/icomplete.el  |  4 ++--
 lisp/minibuffer.el | 10 +++++++++-
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/lisp/icomplete.el b/lisp/icomplete.el
index 35842b53e6b..bc6a8f82ab3 100644
--- a/lisp/icomplete.el
+++ b/lisp/icomplete.el
@@ -1024,7 +1024,7 @@ icomplete--render-vertical
                                 'icomplete-selected-match 'append comp)
      collect (concat prefix
                      (make-string (max 0 (- max-prefix-len (length prefix))) ? )
-                     (completion-lazy-hilit comp)
+                     (completion-for-display comp)
                      (make-string (max 0 (- max-comp-len (length comp))) ? )
                      suffix)
      into lines-aux
@@ -1188,7 +1188,7 @@ icomplete-completions
                   (setq determ (concat open-bracket "" close-bracket)))
                 (while (and comps (not limit))
                   (setq comp
-                        (let ((cur (completion-lazy-hilit (car comps))))
+                        (let ((cur (completion-for-display (car comps))))
                           (if prefix-len (substring cur prefix-len) cur))
                         comps (cdr comps))
                   (setq prospects-len
diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 09a91534322..2bb5b551430 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -696,6 +696,8 @@ completion--twq-all
       ;; would be to return unquoted elements together with a function to
       ;; requote them, so that *Completions* can show nicer unquoted values
       ;; which only get quoted when needed by choose-completion.
+      ;; FIXME: *Completions* now shows unquoted values by using
+      ;; completion--unquoted, so this function can be greatly simplified.
       (nconc
        (mapcar (lambda (completion)
                  (cl-assert (string-prefix-p prefix completion 'ignore-case) t)
@@ -2432,7 +2434,7 @@ completion--insert
   (if (not (consp str))
       (add-text-properties
        (point)
-       (let ((str (completion-lazy-hilit str)))
+       (let ((str (completion-for-display str)))
          (insert
           (if group-fun
               (funcall group-fun str 'transform)
@@ -4322,6 +4324,12 @@ completion-lazy-hilit
       (funcall completion-lazy-hilit-fn (copy-sequence str))
     str))
 
+(defun completion-for-display (str)
+  "Return the string that should be displayed for completion candidate STR.
+
+This will be `face'-propertized as appropriate."
+  (completion-lazy-hilit (or (get-text-property 0 'completion--unquoted str) str)))
+
 (defun completion--hilit-from-re (string regexp &optional point-idx)
   "Fontify STRING using REGEXP POINT-IDX.
 Uses `completions-common-part' and `completions-first-difference'
-- 
2.39.3

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.