GNU bug report logs - #80070
31.0.50; Setting to keep recent duplicates in minibuffer history

Previous Next

To reply to this bug, email your comments to 80070 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Daniel Mendler <mail <at> daniel-mendler.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; Setting to keep recent duplicates in minibuffer history
Date: Wed, 24 Dec 2025 11:04:37 +0100

The variable `history-delete-duplicates' supports two settings, nil to
keep all duplicates and t to delete all duplicates.

I propose to support a third value, possibly an integer. This setting
will have the effect that `add-to-minibuffer' keeps recent duplicates,
but deletes duplicates in the older tail of the history list. The amount
of history to keep could be configured by the integer value of
`history-delete-duplicates'.

In my configuration I experiment with this advice to keep duplicates in
the first 100 history entries (I use long histories).

(defun +history-delete-old-duplicates (var elt &rest _)
  (when-let ((hist (symbol-value var))
             ((and (listp hist) (integerp history-length)))
             (old (nthcdr 100 hist)))
    (setcdr old (delete elt (cdr old)))))
(advice-add #'add-to-history :before #'+history-delete-old-duplicates)

By preserving duplicates in the history we can obtain a ranking to sort
recent candidates by frequency. For example `minibuffer-sort-by-history`
could profit from this additional information similar to the Prescient
package, which sorts by "frecency" (frequency and recency). The
advantage of still deleting duplicates in the older tail of the history
is that rarely used candidates do not vanish so quickly and stay around
for longer.

A more sophisticated alternative to keeping duplicates in the younger
part of the history would be to keep exponentially decaying candidate
usage statistics as done by Prescient. This information would have to be
stored elsewhere, for example in a property `history-usage-statistics'
of the history variable symbol. Unfortunately this would also complicate
`savehist', since this additional property information would have to be
saved too.

In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.49, cairo version 1.18.4) of 2025-12-23
Windowing system distributor 'The X.Org Foundation', version 11.0.12101016
System Description: Debian GNU/Linux 13 (trixie)

Configured using:
 'configure --prefix=$HOME/.local/share/emacs
 --without-compress-install --with-tree-sitter --with-native-compilation
 --with-dbus --without-selinux --without-threads --disable-gc-mark-trace
 --without-gsettings --without-gpm --with-cairo --with-cairo-xcb
 --with-xinput2 --with-x-toolkit=gtk3 --without-toolkit-scroll-bars
 'CFLAGS=-O3 -mtune=native -march=native''

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ JPEG LIBOTF LIBSYSTEMD
LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP
SOUND SQLITE3 TIFF TREE_SITTER WEBP X11 XDBE XIM XINERAMA XINPUT2 XPM
XRANDR GTK3 ZLIB

Message #8 received at 80070 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 80070 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#80070: 31.0.50;
 Setting to keep recent duplicates in minibuffer history
Date: Wed, 24 Dec 2025 16:19:28 +0200

> Cc: JD Smith <jdtsmith <at> gmail.com>
> Date: Wed, 24 Dec 2025 11:04:37 +0100
> From:  Daniel Mendler via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> By preserving duplicates in the history we can obtain a ranking to sort
> recent candidates by frequency. For example `minibuffer-sort-by-history`
> could profit from this additional information similar to the Prescient
> package, which sorts by "frecency" (frequency and recency). The
> advantage of still deleting duplicates in the older tail of the history
> is that rarely used candidates do not vanish so quickly and stay around
> for longer.
> 
> A more sophisticated alternative to keeping duplicates in the younger
> part of the history would be to keep exponentially decaying candidate
> usage statistics as done by Prescient. This information would have to be
> stored elsewhere, for example in a property `history-usage-statistics'
> of the history variable symbol. Unfortunately this would also complicate
> `savehist', since this additional property information would have to be
> saved too.

I think the latter alternative is cleaner.  Keeping duplicates in the
history has a negative effect on usability, and so doing that because
some other feature wants to have access to duplicates is basically
punishing users on behalf of features.  It is IMO cleaner to have the
feature create or use infrastructure it needs without adversely
affecting users.

Thanks.

Message #11 received at 80070 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Mendler <mail <at> daniel-mendler.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 80070 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#80070: 31.0.50; Setting to keep recent duplicates in
 minibuffer history
Date: Wed, 24 Dec 2025 23:36:38 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Cc: JD Smith <jdtsmith <at> gmail.com>
>> Date: Wed, 24 Dec 2025 11:04:37 +0100
>> From:  Daniel Mendler via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>> 
>> By preserving duplicates in the history we can obtain a ranking to sort
>> recent candidates by frequency. For example `minibuffer-sort-by-history`
>> could profit from this additional information similar to the Prescient
>> package, which sorts by "frecency" (frequency and recency). The
>> advantage of still deleting duplicates in the older tail of the history
>> is that rarely used candidates do not vanish so quickly and stay around
>> for longer.
>> 
>> A more sophisticated alternative to keeping duplicates in the younger
>> part of the history would be to keep exponentially decaying candidate
>> usage statistics as done by Prescient. This information would have to be
>> stored elsewhere, for example in a property `history-usage-statistics'
>> of the history variable symbol. Unfortunately this would also complicate
>> `savehist', since this additional property information would have to be
>> saved too.
>
> I think the latter alternative is cleaner.  Keeping duplicates in the
> history has a negative effect on usability, and so doing that because
> some other feature wants to have access to duplicates is basically
> punishing users on behalf of features.  It is IMO cleaner to have the
> feature create or use infrastructure it needs without adversely
> affecting users.

Okay, I also find it okay to keep the statistics separate. The question
is how to handle saving the data. Do you recommend to extend savehist to
save such special variable properties? Maybe there could be a variable
`savehist-additional-properties' which configures a list of properties
of history variables to save?

I had discussed about this with JD before - so I would be interested if
JD has some thoughts about this.

> Thanks.

Message #14 received at 80070 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 80070 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#80070: 31.0.50; Setting to keep recent duplicates in
 minibuffer history
Date: Thu, 25 Dec 2025 08:04:02 +0200

> From: Daniel Mendler <mail <at> daniel-mendler.de>
> Cc: 80070 <at> debbugs.gnu.org,  jdtsmith <at> gmail.com
> Date: Wed, 24 Dec 2025 23:36:38 +0100
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> A more sophisticated alternative to keeping duplicates in the younger
> >> part of the history would be to keep exponentially decaying candidate
> >> usage statistics as done by Prescient. This information would have to be
> >> stored elsewhere, for example in a property `history-usage-statistics'
> >> of the history variable symbol. Unfortunately this would also complicate
> >> `savehist', since this additional property information would have to be
> >> saved too.
> >
> > I think the latter alternative is cleaner.  Keeping duplicates in the
> > history has a negative effect on usability, and so doing that because
> > some other feature wants to have access to duplicates is basically
> > punishing users on behalf of features.  It is IMO cleaner to have the
> > feature create or use infrastructure it needs without adversely
> > affecting users.
> 
> Okay, I also find it okay to keep the statistics separate. The question
> is how to handle saving the data. Do you recommend to extend savehist to
> save such special variable properties? Maybe there could be a variable
> `savehist-additional-properties' which configures a list of properties
> of history variables to save?

I don't think I understand the issues with savehist: can you
elaborate?  Why would savehist bother about how we keep duplicate
entries when the user said she is not interested in duplicates?  It
seems to me that in those cases savehist should work as if duplicates
didn't exist, and only the features which need the information about
duplicates should care about how this information is recorded.  Or
what am I missing?

Message #17 received at 80070 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Mendler <mail <at> daniel-mendler.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 80070 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#80070: 31.0.50; Setting to keep recent duplicates in
 minibuffer history
Date: Thu, 25 Dec 2025 11:10:40 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Daniel Mendler <mail <at> daniel-mendler.de>
>> Cc: 80070 <at> debbugs.gnu.org,  jdtsmith <at> gmail.com
>> Date: Wed, 24 Dec 2025 23:36:38 +0100
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> >> A more sophisticated alternative to keeping duplicates in the younger
>> >> part of the history would be to keep exponentially decaying candidate
>> >> usage statistics as done by Prescient. This information would have to be
>> >> stored elsewhere, for example in a property `history-usage-statistics'
>> >> of the history variable symbol. Unfortunately this would also complicate
>> >> `savehist', since this additional property information would have to be
>> >> saved too.
>> >
>> > I think the latter alternative is cleaner.  Keeping duplicates in the
>> > history has a negative effect on usability, and so doing that because
>> > some other feature wants to have access to duplicates is basically
>> > punishing users on behalf of features.  It is IMO cleaner to have the
>> > feature create or use infrastructure it needs without adversely
>> > affecting users.
>> 
>> Okay, I also find it okay to keep the statistics separate. The question
>> is how to handle saving the data. Do you recommend to extend savehist to
>> save such special variable properties? Maybe there could be a variable
>> `savehist-additional-properties' which configures a list of properties
>> of history variables to save?
>
> I don't think I understand the issues with savehist: can you
> elaborate?  Why would savehist bother about how we keep duplicate
> entries when the user said she is not interested in duplicates?  It
> seems to me that in those cases savehist should work as if duplicates
> didn't exist, and only the features which need the information about
> duplicates should care about how this information is recorded.  Or
> what am I missing?

I only wonder how the additional data is saved to disk to preserve it
across Emacs restarts, if the data is kept separately, in a symbol
property. savehist only stores the variable value, but not property
values. If we want to keep frequency statistics it must be saved to disk
too, in order to not lose it.

Daniel

Message #20 received at 80070 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 80070 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#80070: 31.0.50; Setting to keep recent duplicates in
 minibuffer history
Date: Thu, 25 Dec 2025 13:52:53 +0200

> From: Daniel Mendler <mail <at> daniel-mendler.de>
> Cc: 80070 <at> debbugs.gnu.org,  jdtsmith <at> gmail.com
> Date: Thu, 25 Dec 2025 11:10:40 +0100
> 
> >> Okay, I also find it okay to keep the statistics separate. The question
> >> is how to handle saving the data. Do you recommend to extend savehist to
> >> save such special variable properties? Maybe there could be a variable
> >> `savehist-additional-properties' which configures a list of properties
> >> of history variables to save?
> >
> > I don't think I understand the issues with savehist: can you
> > elaborate?  Why would savehist bother about how we keep duplicate
> > entries when the user said she is not interested in duplicates?  It
> > seems to me that in those cases savehist should work as if duplicates
> > didn't exist, and only the features which need the information about
> > duplicates should care about how this information is recorded.  Or
> > what am I missing?
> 
> I only wonder how the additional data is saved to disk to preserve it
> across Emacs restarts, if the data is kept separately, in a symbol
> property.

Are we sure we need to preserve this across sessions?

> savehist only stores the variable value, but not property
> values. If we want to keep frequency statistics it must be saved to disk
> too, in order to not lose it.

But we could invent a new variable, and add its saving, no?

Message #23 received at 80070 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 80070 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#80070: 31.0.50; Setting to keep recent duplicates in
 minibuffer history
Date: Thu, 25 Dec 2025 13:10:15 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Daniel Mendler <mail <at> daniel-mendler.de>
>> Cc: 80070 <at> debbugs.gnu.org,  jdtsmith <at> gmail.com
>> Date: Thu, 25 Dec 2025 11:10:40 +0100
>> 
>> >> Okay, I also find it okay to keep the statistics separate. The question
>> >> is how to handle saving the data. Do you recommend to extend savehist to
>> >> save such special variable properties? Maybe there could be a variable
>> >> `savehist-additional-properties' which configures a list of properties
>> >> of history variables to save?
>> >
>> > I don't think I understand the issues with savehist: can you
>> > elaborate?  Why would savehist bother about how we keep duplicate
>> > entries when the user said she is not interested in duplicates?  It
>> > seems to me that in those cases savehist should work as if duplicates
>> > didn't exist, and only the features which need the information about
>> > duplicates should care about how this information is recorded.  Or
>> > what am I missing?
>> 
>> I only wonder how the additional data is saved to disk to preserve it
>> across Emacs restarts, if the data is kept separately, in a symbol
>> property.
>
> Are we sure we need to preserve this across sessions?

Yes, that's important. The point is to preserve the usage statistics
over a long time to give better sorting during completion. The function
`minibuffer-sort-by-history' could take advantage of the data.

>> savehist only stores the variable value, but not property
>> values. If we want to keep frequency statistics it must be saved to disk
>> too, in order to not lose it.
>
> But we could invent a new variable, and add its saving, no?

Either we would invent a new variable for every minibuffer history
variable, e.g., `command-history--statistics' for the variable
`command-history', or introduce a variable
`minibuffer-history-statistics' which holds all the data as an alist or
hash table for each variable. I guess that would work well, except that
the data would be slightly less convenient to inspect.

Daniel

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.