GNU bug report logs - #77718
31.0.50; completion styles substring and flex are broken

Previous Next

To reply to this bug, email your comments to 77718 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stephen Berman <stephen.berman <at> gmx.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; completion styles substring and flex are broken
Date: Fri, 11 Apr 2025 00:22:01 +0200

0. Start Emacs like this:

   emacs -Q --eval "(custom-set-variables '(completion-category-overrides '((file (styles substring)))))"

   or like this:

   emacs -Q --eval "(custom-set-variables '(completion-category-overrides '((file (styles flex)))))"

1. Type `C-x d' and at the prompt enter: `/usr/ TAB'.
=> Now the text following the prompt is this: "/usr//" and "/usr/" is
fontified with face file-name-shadow.

2. Type `TAB' again.
=> Now the text following the prompt is this: "//" and the first forward
slash is fontified with face file-name-shadow.

3. Type `TAB' again.
=> Now a *Completions* buffer pops up listing all the directories
directly under root (dev/, etc/, home/ and so on).

This only happens with the substring and flex completion styles; with
any of the other completion styles (basic, partial-completion, emacs22,
initials), or if Emacs is started just with -Q, then after step 1 as
expected a *Completions* buffer pops up listing the directories under
/usr (bin/, include/, lib/ and so on).

The broken behavior with the substring and flex styles is due to the
change in completion-pcm--merge-completions in this commit:

commit 0fbba16387513e7692b46885833e4a9c218251f0
Author:     Spencer Baugh <sbaugh <at> janestreet.com>
Commit:     Stefan Monnier <monnier <at> iro.umontreal.ca>
CommitDate: Tue Apr 8 14:36:30 2025 -0400

  Preserve an explicit * in pcm-try-completion
  [...]
  * lisp/minibuffer.el (completion-pcm--merge-completions): Don't drop
  important wildcards. (bug#74420)


In GNU Emacs 31.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
 3.24.49, cairo version 1.18.4) of 2025-04-10 built on strobelfs2
Repository revision: c0ea954d0f650227dc518f02a292daeb27cf0c37
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101016
System Description: Linux From Scratch r12.3-20

Configured using:
 'configure -C 'CFLAGS=-Og -g3' PKG_CONFIG_PATH=/opt/qt6/lib/pkgconfig'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG
RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER
WEBP X11 XDBE XIM XINERAMA XINPUT2 XPM XRANDR GTK3 ZLIB

Message #8 received at 77718 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stephen Berman <stephen.berman <at> gmx.net>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Spencer Baugh <sbaugh <at> janestreet.com>
Cc: 77718 <at> debbugs.gnu.org
Subject: Re: bug#77718: 31.0.50;
 completion styles substring and flex are broken
Date: Fri, 11 Apr 2025 10:22:46 +0300

> Date: Fri, 11 Apr 2025 00:22:01 +0200
> From:  Stephen Berman via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> 0. Start Emacs like this:
> 
>    emacs -Q --eval "(custom-set-variables '(completion-category-overrides '((file (styles substring)))))"
> 
>    or like this:
> 
>    emacs -Q --eval "(custom-set-variables '(completion-category-overrides '((file (styles flex)))))"
> 
> 1. Type `C-x d' and at the prompt enter: `/usr/ TAB'.
> => Now the text following the prompt is this: "/usr//" and "/usr/" is
> fontified with face file-name-shadow.
> 
> 2. Type `TAB' again.
> => Now the text following the prompt is this: "//" and the first forward
> slash is fontified with face file-name-shadow.
> 
> 3. Type `TAB' again.
> => Now a *Completions* buffer pops up listing all the directories
> directly under root (dev/, etc/, home/ and so on).
> 
> This only happens with the substring and flex completion styles; with
> any of the other completion styles (basic, partial-completion, emacs22,
> initials), or if Emacs is started just with -Q, then after step 1 as
> expected a *Completions* buffer pops up listing the directories under
> /usr (bin/, include/, lib/ and so on).
> 
> The broken behavior with the substring and flex styles is due to the
> change in completion-pcm--merge-completions in this commit:
> 
> commit 0fbba16387513e7692b46885833e4a9c218251f0
> Author:     Spencer Baugh <sbaugh <at> janestreet.com>
> Commit:     Stefan Monnier <monnier <at> iro.umontreal.ca>
> CommitDate: Tue Apr 8 14:36:30 2025 -0400
> 
>   Preserve an explicit * in pcm-try-completion
>   [...]
>   * lisp/minibuffer.el (completion-pcm--merge-completions): Don't drop
>   important wildcards. (bug#74420)

Thanks, adding the guilty parties to the discussion.

Message #11 received at 77718 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77718 <at> debbugs.gnu.org, Stephen Berman <stephen.berman <at> gmx.net>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#77718: 31.0.50; completion styles substring and flex are
 broken
Date: Fri, 11 Apr 2025 09:03:12 -0400

Eli Zaretskii <eliz <at> gnu.org> writes:
>> Date: Fri, 11 Apr 2025 00:22:01 +0200
>> From:  Stephen Berman via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>> 
>> 0. Start Emacs like this:
>> 
>>    emacs -Q --eval "(custom-set-variables '(completion-category-overrides '((file (styles substring)))))"
>> 
>>    or like this:
>> 
>>    emacs -Q --eval "(custom-set-variables '(completion-category-overrides '((file (styles flex)))))"
>> 
>> 1. Type `C-x d' and at the prompt enter: `/usr/ TAB'.
>> => Now the text following the prompt is this: "/usr//" and "/usr/" is
>> fontified with face file-name-shadow.
>> 
>> 2. Type `TAB' again.
>> => Now the text following the prompt is this: "//" and the first forward
>> slash is fontified with face file-name-shadow.
>> 
>> 3. Type `TAB' again.
>> => Now a *Completions* buffer pops up listing all the directories
>> directly under root (dev/, etc/, home/ and so on).
>> 
>> This only happens with the substring and flex completion styles; with
>> any of the other completion styles (basic, partial-completion, emacs22,
>> initials), or if Emacs is started just with -Q, then after step 1 as
>> expected a *Completions* buffer pops up listing the directories under
>> /usr (bin/, include/, lib/ and so on).
>> 
>> The broken behavior with the substring and flex styles is due to the
>> change in completion-pcm--merge-completions in this commit:
>> 
>> commit 0fbba16387513e7692b46885833e4a9c218251f0
>> Author:     Spencer Baugh <sbaugh <at> janestreet.com>
>> Commit:     Stefan Monnier <monnier <at> iro.umontreal.ca>
>> CommitDate: Tue Apr 8 14:36:30 2025 -0400
>> 
>>   Preserve an explicit * in pcm-try-completion
>>   [...]
>>   * lisp/minibuffer.el (completion-pcm--merge-completions): Don't drop
>>   important wildcards. (bug#74420)

Thanks for the detailed bug report.

Reduced slightly, the issue is that
  (completion-pcm--merge-completions '("x/" "y/") '(prefix))
now returns 
  ("/" any prefix "" "")
when it used to return
  (any "" "")

I think this may be related to the implict "any" added by
completion-pcm--merge-completions.  I'll look into it and find a fix
(and add a test covering this case).

Message #14 received at 77718 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77718 <at> debbugs.gnu.org, Stephen Berman <stephen.berman <at> gmx.net>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#77718: 31.0.50; completion styles substring and flex are
 broken
Date: Fri, 11 Apr 2025 15:43:18 -0400

Spencer Baugh <sbaugh <at> janestreet.com> writes:
> Reduced slightly, the issue is that
>   (completion-pcm--merge-completions '("x/" "y/") '(prefix))
> now returns 
>   ("/" any prefix "" "")
> when it used to return
>   (any "" "")
>
> I think this may be related to the implict "any" added by
> completion-pcm--merge-completions.  I'll look into it and find a fix
> (and add a test covering this case).

Update: That's not the issue; this new return value for
completion-pcm--merge-completions is actually correct, it's a desirable
consequence of my bugfix patch.

The issue is actually in completion-table-with-quoting, with filename
completion and substitute-in-file-name; it doesn't handle // properly.

I'm trying to fix it; it may be a somewhat large change to
completion-table-with-quoting.

Stefan, a related question: do you think it would be reasonable to split
completion-table-with-quoting into two functions, separating the
substitute-in-file-name and comint-quote-filename cases?  It seems to me
that the comint-quote-filename case is way simpler than the
substitute-in-file-name case, and it would be a lot easier to understand
if the two cases were done separately, without the abstraction of
requote and unquote functions.  I can do that if that seems reasonable.
It would make fixing this issue a lot easier.

(I don't think we need to maintain the same interface for
completion-table-with-quoting - I see no usage in the 200 packages used
at my site - I strongly suspect, given the complexity, that the only
users are in Emacs core)

Message #17 received at 77718 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: 77718 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Stephen Berman <stephen.berman <at> gmx.net>
Subject: Re: bug#77718: 31.0.50; completion styles substring and flex are
 broken
Date: Fri, 11 Apr 2025 17:37:52 -0400

> Stefan, a related question: do you think it would be reasonable to split
> completion-table-with-quoting into two functions, separating the
> substitute-in-file-name and comint-quote-filename cases?  It seems to me
> that the comint-quote-filename case is way simpler than the
> substitute-in-file-name case, and it would be a lot easier to understand
> if the two cases were done separately, without the abstraction of
> requote and unquote functions.  I can do that if that seems reasonable.
> It would make fixing this issue a lot easier.

It's worth a try.  The system we have is the best I could come up with
back then but is hideous, so I'm interested in a simpler solution.

> (I don't think we need to maintain the same interface for
> completion-table-with-quoting - I see no usage in the 200 packages used
> at my site - I strongly suspect, given the complexity, that the only
> users are in Emacs core)

It's definitely not worth worrying about that while designing
a replacement.  Maybe we'll want to preserve backward compatibility, but
let's first get a good replacement.


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.