GNU bug report logs - #35689
Customizable char-fold

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35689 in the body.
You can then email your comments to 35689 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: bug-gnu-emacs <at> gnu.org
Subject: Customizable char-fold
Date: Sun, 12 May 2019 00:22:53 +0300

[Message part 1 (text/plain, inline)]

Tags: patch

This patch adds long-awaited customization to char-fold.el:

[char-fold-defcustom.patch (text/x-diff, inline)]

diff --git a/lisp/char-fold.el b/lisp/char-fold.el
index e61bc3edc6..24882008a4 100644
--- a/lisp/char-fold.el
+++ b/lisp/char-fold.el
@@ -24,6 +24,30 @@
 
 (eval-and-compile (put 'char-fold-table 'char-table-extra-slots 1))
 
+(defcustom char-fold-include-base nil
+  "Include mappings from composite character to base letter."
+  :type 'boolean
+  :group 'matching
+  :version "27.1")
+
+(defcustom char-fold-include-alist
+  '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
+    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
+    (?` "❛" "‘" "‛" "󠀢" "❮" "‹"))
+  "Additional character mappings to include."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (repeat (string :tag "To")))
+  :group 'lisp
+  :version "27.1")
+
+(defcustom char-fold-exclude-alist nil
+  "Character mappings to exclude from default setting."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (character :tag "To"))
+  :group 'lisp
+  :version "27.1")
+
+
 (defconst char-fold-table
   (eval-when-compile
     (let ((equiv (make-char-table 'char-fold-table))
@@ -76,7 +106,11 @@ char-fold-table
                                       (aref equiv-multi (car decomp))))
                         (aset equiv (car decomp)
                               (cons (char-to-string char)
-                                    (aref equiv (car decomp))))))))
+                                    (aref equiv (car decomp))))
+                        (when char-fold-include-base
+                          (aset equiv char
+                                (cons (char-to-string (car decomp))
+                                      (aref equiv (car decomp)))))))))
                (funcall make-decomp-match-char decomp char)
                ;; Do it again, without the non-spacing characters.
                ;; This allows 'a' to match 'ä'.
@@ -98,13 +132,18 @@ char-fold-table
        table)
 
       ;; Add some manual entries.
-      (dolist (it '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
-                    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
-                    (?` "❛" "‘" "‛" "󠀢" "❮" "‹")))
+      (dolist (it char-fold-include-alist)
         (let ((idx (car it))
               (chars (cdr it)))
           (aset equiv idx (append chars (aref equiv idx)))))
 
+      ;; Remove some entries.
+      (dolist (it char-fold-exclude-alist)
+        (let ((idx (car it))
+              (char (cdr it)))
+          (when (aref equiv idx)
+            (aset equiv idx (remove (char-to-string char) (aref equiv idx))))))
+
       ;; Convert the lists of characters we compiled into regexps.
       (map-char-table
        (lambda (char dec-list)

Message #8 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Sun, 12 May 2019 22:12:01 +0300

[Message part 1 (text/plain, inline)]

> This patch adds long-awaited customization to char-fold.el:

I noticed that its compilation fails with:

    ELC      char-fold.elc
  In toplevel form:
  char-fold.el:153:31:Error: Symbol's value as variable is void: char-fold-include-base
  Makefile:296: recipe for target 'char-fold.elc' failed
  make[2]: *** [char-fold.elc] Error 1

So a new patch added 'eval-and-compile' to all defcustoms:

[char-fold-defcustom.2.patch (text/x-diff, inline)]

diff --git a/lisp/char-fold.el b/lisp/char-fold.el
index e61bc3edc6..a60d49dd8e 100644
--- a/lisp/char-fold.el
+++ b/lisp/char-fold.el
@@ -24,6 +24,30 @@
 
 (eval-and-compile (put 'char-fold-table 'char-table-extra-slots 1))
 
+(eval-and-compile (defcustom char-fold-include-base nil
+  "Include mappings from composite character to base letter."
+  :type 'boolean
+  :group 'matching
+  :version "27.1"))
+
+(eval-and-compile (defcustom char-fold-include-alist
+  '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
+    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
+    (?` "❛" "‘" "‛" "󠀢" "❮" "‹"))
+  "Additional character mappings to include."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (repeat (string :tag "To")))
+  :group 'lisp
+  :version "27.1"))
+
+(eval-and-compile (defcustom char-fold-exclude-alist nil
+  "Character mappings to exclude from default setting."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (character :tag "To"))
+  :group 'lisp
+  :version "27.1"))
+
+
 (defconst char-fold-table
   (eval-when-compile
     (let ((equiv (make-char-table 'char-fold-table))
@@ -76,7 +109,11 @@ char-fold-table
                                       (aref equiv-multi (car decomp))))
                         (aset equiv (car decomp)
                               (cons (char-to-string char)
-                                    (aref equiv (car decomp))))))))
+                                    (aref equiv (car decomp))))
+                        (when char-fold-include-base
+                          (aset equiv char
+                                (cons (char-to-string (car decomp))
+                                      (aref equiv (car decomp)))))))))
                (funcall make-decomp-match-char decomp char)
                ;; Do it again, without the non-spacing characters.
                ;; This allows 'a' to match 'ä'.
@@ -98,13 +135,18 @@ char-fold-table
        table)
 
       ;; Add some manual entries.
-      (dolist (it '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
-                    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
-                    (?` "❛" "‘" "‛" "󠀢" "❮" "‹")))
+      (dolist (it char-fold-include-alist)
         (let ((idx (car it))
               (chars (cdr it)))
           (aset equiv idx (append chars (aref equiv idx)))))
 
+      ;; Remove some entries.
+      (dolist (it char-fold-exclude-alist)
+        (let ((idx (car it))
+              (char (cdr it)))
+          (when (aref equiv idx)
+            (aset equiv idx (remove (char-to-string char) (aref equiv idx))))))
+
       ;; Convert the lists of characters we compiled into regexps.
       (map-char-table
        (lambda (char dec-list)

Message #11 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Sun, 12 May 2019 16:30:32 -0400

Juri Linkov <juri <at> linkov.net> writes:

>> This patch adds long-awaited customization to char-fold.el:
>
> I noticed that its compilation fails with:
>
>     ELC      char-fold.elc
>   In toplevel form:
>   char-fold.el:153:31:Error: Symbol's value as variable is void: char-fold-include-base
>   Makefile:296: recipe for target 'char-fold.elc' failed
>   make[2]: *** [char-fold.elc] Error 1
>
> So a new patch added 'eval-and-compile' to all defcustoms:

Will it actually work for a user to customize this?  It looks like the
values are used during compile time, so after changing the value the
user would have to recompile char-fold.el?

Message #14 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Mon, 13 May 2019 23:31:58 +0300

[Message part 1 (text/plain, inline)]

>>> This patch adds long-awaited customization to char-fold.el:
>>
>> I noticed that its compilation fails with:
>>
>>     ELC      char-fold.elc
>>   In toplevel form:
>>   char-fold.el:153:31:Error: Symbol's value as variable is void: char-fold-include-base
>>   Makefile:296: recipe for target 'char-fold.elc' failed
>>   make[2]: *** [char-fold.elc] Error 1
>>
>> So a new patch added 'eval-and-compile' to all defcustoms:
>
> Will it actually work for a user to customize this?  It looks like the
> values are used during compile time, so after changing the value the
> user would have to recompile char-fold.el?

Oh, right.  Do you see a problem with a better patch:

[char-fold-defcustom.3.patch (text/x-diff, inline)]

diff --git a/lisp/char-fold.el b/lisp/char-fold.el
index e61bc3edc6..6c3f809c2b 100644
--- a/lisp/char-fold.el
+++ b/lisp/char-fold.el
@@ -24,8 +24,43 @@
 
 (eval-and-compile (put 'char-fold-table 'char-table-extra-slots 1))
 
-(defconst char-fold-table
-  (eval-when-compile
+(eval-and-compile (defcustom char-fold-include-base nil
+  "Include mappings from composite character to base letter."
+  :type 'boolean
+  :set (lambda (sym val)
+         (set sym val)
+         (when (boundp 'char-fold-table)
+           (setq char-fold-table (char-fold-make-table))))
+  :group 'matching
+  :version "27.1"))
+
+(eval-and-compile (defcustom char-fold-include-alist
+  '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
+    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
+    (?` "❛" "‘" "‛" "󠀢" "❮" "‹"))
+  "Additional character mappings to include."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (repeat (string :tag "To")))
+  :set (lambda (sym val)
+         (set sym val)
+         (when (boundp 'char-fold-table)
+           (setq char-fold-table (char-fold-make-table))))
+  :group 'lisp
+  :version "27.1"))
+
+(eval-and-compile (defcustom char-fold-exclude-alist nil
+  "Character mappings to exclude from default setting."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (character :tag "To"))
+  :set (lambda (sym val)
+         (set sym val)
+         (when (boundp 'char-fold-table)
+           (setq char-fold-table (char-fold-make-table))))
+  :group 'lisp
+  :version "27.1"))
+
+(eval-and-compile
+  (defun char-fold-make-table ()
     (let ((equiv (make-char-table 'char-fold-table))
           (equiv-multi (make-char-table 'char-fold-table))
           (table (unicode-property-table-internal 'decomposition)))
@@ -76,7 +123,11 @@ char-fold-table
                                       (aref equiv-multi (car decomp))))
                         (aset equiv (car decomp)
                               (cons (char-to-string char)
-                                    (aref equiv (car decomp))))))))
+                                    (aref equiv (car decomp))))
+                        (when char-fold-include-base
+                          (aset equiv char
+                                (cons (char-to-string (car decomp))
+                                      (aref equiv (car decomp)))))))))
                (funcall make-decomp-match-char decomp char)
                ;; Do it again, without the non-spacing characters.
                ;; This allows 'a' to match 'ä'.
@@ -98,13 +149,18 @@ char-fold-table
        table)
 
       ;; Add some manual entries.
-      (dolist (it '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
-                    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
-                    (?` "❛" "‘" "‛" "󠀢" "❮" "‹")))
+      (dolist (it char-fold-include-alist)
         (let ((idx (car it))
               (chars (cdr it)))
           (aset equiv idx (append chars (aref equiv idx)))))
 
+      ;; Remove some entries.
+      (dolist (it char-fold-exclude-alist)
+        (let ((idx (car it))
+              (char (cdr it)))
+          (when (aref equiv idx)
+            (aset equiv idx (remove (char-to-string char) (aref equiv idx))))))
+
       ;; Convert the lists of characters we compiled into regexps.
       (map-char-table
        (lambda (char dec-list)
@@ -113,7 +169,11 @@ char-fold-table
                (set-char-table-range equiv char re)
              (aset equiv char re))))
        equiv)
-      equiv))
+      equiv)))
+
+(defconst char-fold-table
+  (eval-when-compile
+    (char-fold-make-table))
   "Used for folding characters of the same group during search.
 This is a char-table with the `char-fold-table' subtype.
 

Message #17 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Mon, 13 May 2019 18:18:18 -0400

Juri Linkov <juri <at> linkov.net> writes:

> Oh, right.  Do you see a problem with a better patch:

> +(eval-and-compile (defcustom char-fold-include-base nil
> +  "Include mappings from composite character to base letter."
> +  :type 'boolean
> +  :set (lambda (sym val)
> +         (set sym val)
> +         (when (boundp 'char-fold-table)
> +           (setq char-fold-table (char-fold-make-table))))

Looks like it could work (though I haven't tested).  The docstrings are
too terse for me easily follow, and they should probably include
something along the lines of "Setting this variable directly does not
take effect; either use M-x customize or ..."

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: bug-gnu-emacs <at> gnu.org, Noam Postavsky <npostavs <at> gmail.com>,
 Juri Linkov <juri <at> linkov.net>
Cc: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Tue, 14 May 2019 09:37:14 +0300

On May 14, 2019 1:18:18 AM GMT+03:00, Noam Postavsky <npostavs <at> gmail.com> wrote:
> Juri Linkov <juri <at> linkov.net> writes:
> 
> > Oh, right.  Do you see a problem with a better patch:
> 
> > +(eval-and-compile (defcustom char-fold-include-base nil
> > +  "Include mappings from composite character to base letter."
> > +  :type 'boolean
> > +  :set (lambda (sym val)
> > +         (set sym val)
> > +         (when (boundp 'char-fold-table)
> > +           (setq char-fold-table (char-fold-make-table))))
> 
> Looks like it could work (though I haven't tested).  The docstrings
> are
> too terse for me easily follow, and they should probably include
> something along the lines of "Setting this variable directly does not
> take effect; either use M-x customize or ..."

We don't gave defcustoms inside eval-and/when-compile anywhere else.  Do we really need this?  For starters, it would defeat cus-dep.el, I think.

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bug-gnu-emacs <at> gnu.org, 35689 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> gmail.com>
Subject: Re: bug#35689: Customizable char-fold
Date: Tue, 14 May 2019 23:14:12 +0300

>> > Oh, right.  Do you see a problem with a better patch:
>>
>> > +(eval-and-compile (defcustom char-fold-include-base nil
>> > +  "Include mappings from composite character to base letter."
>> > +  :type 'boolean
>> > +  :set (lambda (sym val)
>> > +         (set sym val)
>> > +         (when (boundp 'char-fold-table)
>> > +           (setq char-fold-table (char-fold-make-table))))
>>
>> Looks like it could work (though I haven't tested).  The docstrings
>> are
>> too terse for me easily follow, and they should probably include
>> something along the lines of "Setting this variable directly does not
>> take effect; either use M-x customize or ..."
>
> We don't gave defcustoms inside eval-and/when-compile anywhere else.
> Do we really need this?  For starters, it would defeat cus-dep.el, I think.

Indeed, better to try and simplify this.  The goal is to pre-compile
the default char-table because its calculation is compute-intensive,
and to recalculate a new value of char-table only in case
when customized values differ from the default values.

I can't find a standard way of doing this.  So instead of using eval-and-compile
I'll try to recalculate the value explicitly when variables are customized:

  (when (or (get 'char-fold-include-base  'customized-value)
            (get 'char-fold-include-alist 'customized-value)
            (get 'char-fold-exclude-alist 'customized-value))
    (setq char-fold-table (char-fold-make-table)))

Message #34 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> gmail.com
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 35689 <at> debbugs.gnu.org, npostavs <at> gmail.com
Subject: Re: bug#35689: Customizable char-fold
Date: Thu, 16 May 2019 10:47:47 -0400

Juri Linkov <juri <at> linkov.net> writes:

>> We don't gave defcustoms inside eval-and/when-compile anywhere else.
>> Do we really need this?  For starters, it would defeat cus-dep.el, I think.
>
> Indeed, better to try and simplify this.  The goal is to pre-compile
> the default char-table because its calculation is compute-intensive,
> and to recalculate a new value of char-table only in case
> when customized values differ from the default values.
>
> I can't find a standard way of doing this.  So instead of using eval-and-compile
> I'll try to recalculate the value explicitly when variables are customized:
>
>   (when (or (get 'char-fold-include-base  'customized-value)
>             (get 'char-fold-include-alist 'customized-value)
>             (get 'char-fold-exclude-alist 'customized-value))
>     (setq char-fold-table (char-fold-make-table)))

Instead of looking at symbol property values, which can make for a
confusing time when setting variables outside of customize, I think it
would be nicer to do something like this:

    (eval-and-compile (defconst char-fold--include-base-default ...))

    (defcustom char-fold-include-base char-fold--include-base-default
      :initialize #'custom-initialize-changed
      :set (lambda (sym val)
             (set-default sym val)
             ;; FIXME: Maybe delay this until after-init-time,
             ;; to avoid redundant calls to char-fold-make-table.
             (setq char-fold-table (char-fold-make-table)))
      ...)

    (eval-and-compile
      (defun char-fold-make-table ()
         ...
         (or (bound-and-true-p 'char-fold-include-base)
             char-fold--include-base-default)
         ...))

Message #37 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: npostavs <at> gmail.com
Cc: Eli Zaretskii <eliz <at> gnu.org>, 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Thu, 16 May 2019 23:13:17 +0300

>> I can't find a standard way of doing this.  So instead of using eval-and-compile
>> I'll try to recalculate the value explicitly when variables are customized:
>>
>>   (when (or (get 'char-fold-include-base  'customized-value)
>>             (get 'char-fold-include-alist 'customized-value)
>>             (get 'char-fold-exclude-alist 'customized-value))
>>     (setq char-fold-table (char-fold-make-table)))
>
> Instead of looking at symbol property values, which can make for a
> confusing time when setting variables outside of customize, I think it
> would be nicer to do something like this:
>
>     (eval-and-compile (defconst char-fold--include-base-default ...))
>
>     (defcustom char-fold-include-base char-fold--include-base-default
>       :initialize #'custom-initialize-changed

I tried different possible values of :initialize,
but not custom-initialize-changed.  I'll try this now.

The problem I encountered with the previous solution it that
calling `(setq char-fold-table (char-fold-make-table))' above
while loading char-fold.el by autoload, garbled data returned
from `(unicode-property-table-internal 'decomposition)',
it just returned garbage, maybe due to a broken coding.
I was busy debugging unidata-get-decomposition to
understand where this data corruption occurs.

Message #40 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: npostavs <at> gmail.com
Cc: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Tue, 21 May 2019 23:34:20 +0300

[Message part 1 (text/plain, inline)]

>>> I can't find a standard way of doing this.  So instead of using eval-and-compile
>>> I'll try to recalculate the value explicitly when variables are customized:
>>>
>>>   (when (or (get 'char-fold-include-base  'customized-value)
>>>             (get 'char-fold-include-alist 'customized-value)
>>>             (get 'char-fold-exclude-alist 'customized-value))
>>>     (setq char-fold-table (char-fold-make-table)))
>>
>> Instead of looking at symbol property values, which can make for a
>> confusing time when setting variables outside of customize, I think it
>> would be nicer to do something like this:
>>
>>     (defcustom char-fold-include-base char-fold--include-base-default
>>       :initialize #'custom-initialize-changed
>>       :set (lambda (sym val)
>>              (set-default sym val)
>>              ;; FIXME: Maybe delay this until after-init-time,
>>              ;; to avoid redundant calls to char-fold-make-table.
>
> I tried different possible values of :initialize,
> but not custom-initialize-changed.  I'll try this now.

I see no problems other than redundant calls to char-fold-make-table
when more than one variable is customized.  char-fold.el is autoloaded
when isearch calls char-fold-to-regexp for the first time, so I'm not
sure how after-init-hook could help in this case.

[char-fold-defcustom.4.patch (text/x-diff, inline)]

diff --git a/lisp/char-fold.el b/lisp/char-fold.el
index 426b1a9f84..16d6d484f0 100644
--- a/lisp/char-fold.el
+++ b/lisp/char-fold.el
@@ -21,13 +21,22 @@
 
 ;;; Code:
 
-(eval-and-compile (put 'char-fold-table 'char-table-extra-slots 1))
+(eval-and-compile
+  (put 'char-fold-table 'char-table-extra-slots 1)
+  (defconst char-fold--symmetric-default nil)
+  (defconst char-fold--include-alist-default
+    '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
+      (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
+      (?` "❛" "‘" "‛" "󠀢" "❮" "‹")))
+  (defconst char-fold--exclude-alist-default nil))
+
 
-(defconst char-fold-table
-  (eval-when-compile
-    (let ((equiv (make-char-table 'char-fold-table))
-          (equiv-multi (make-char-table 'char-fold-table))
-          (table (unicode-property-table-internal 'decomposition)))
+(eval-and-compile
+  (defun char-fold-make-table ()
+    (let* ((equiv (make-char-table 'char-fold-table))
+           (equiv-multi (make-char-table 'char-fold-table))
+           (search-spaces-regexp nil) ; bug#35802
+           (table (unicode-property-table-internal 'decomposition)))
       (set-char-table-extra-slot equiv 0 equiv-multi)
 
       ;; Ensure the table is populated.
@@ -75,7 +84,12 @@ char-fold-table
                                       (aref equiv-multi (car decomp))))
                         (aset equiv (car decomp)
                               (cons (char-to-string char)
-                                    (aref equiv (car decomp))))))))
+                                    (aref equiv (car decomp))))
+                        (when (or (bound-and-true-p char-fold-symmetric)
+                                  char-fold--symmetric-default)
+                          (aset equiv char
+                                (cons (char-to-string (car decomp))
+                                      (aref equiv (car decomp)))))))))
                (funcall make-decomp-match-char decomp char)
                ;; Do it again, without the non-spacing characters.
                ;; This allows 'a' to match 'ä'.
@@ -97,13 +111,20 @@ char-fold-table
        table)
 
       ;; Add some manual entries.
-      (dolist (it '((?\" "＂" "“" "”" "”" "„" "⹂" "〞" "‟" "‟" "❞" "❝" "❠" "“" "„" "〝" "〟" "🙷" "🙶" "🙸" "«" "»")
-                    (?' "❟" "❛" "❜" "‘" "’" "‚" "‛" "‚" "󠀢" "❮" "❯" "‹" "›")
-                    (?` "❛" "‘" "‛" "󠀢" "❮" "‹")))
+      (dolist (it (or (bound-and-true-p char-fold-include-alist)
+                      char-fold--include-alist-default))
         (let ((idx (car it))
               (chars (cdr it)))
           (aset equiv idx (append chars (aref equiv idx)))))
 
+      ;; Remove some entries.
+      (dolist (it (or (bound-and-true-p char-fold-exclude-alist)
+                      char-fold--exclude-alist-default))
+        (let ((idx (car it))
+              (char (cdr it)))
+          (when (aref equiv idx)
+            (aset equiv idx (remove (char-to-string char) (aref equiv idx))))))
+
       ;; Convert the lists of characters we compiled into regexps.
       (map-char-table
        (lambda (char dec-list)
@@ -112,7 +133,11 @@ char-fold-table
                (set-char-table-range equiv char re)
              (aset equiv char re))))
        equiv)
-      equiv))
+      equiv)))
+
+(defconst char-fold-table
+  (eval-when-compile
+    (char-fold-make-table))
   "Used for folding characters of the same group during search.
 This is a char-table with the `char-fold-table' subtype.
 
@@ -135,6 +160,40 @@ char-fold-table
 
 Exceptionally for the space character (32), ALIST is ignored.")
 
+(defcustom char-fold-symmetric char-fold--symmetric-default
+  "Include symmetric mappings from composite character back to base letter."
+  :type 'boolean
+  :initialize #'custom-initialize-changed
+  :set (lambda (sym val)
+         (set-default sym val)
+         ;; FIXME: Maybe delay this until after-init-hook,
+         ;; to avoid redundant calls to char-fold-make-table.
+         (setq char-fold-table (char-fold-make-table)))
+  :group 'matching
+  :version "27.1")
+
+(defcustom char-fold-include-alist char-fold--include-alist-default
+  "Additional character mappings to include."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (repeat (string :tag "To")))
+  :initialize #'custom-initialize-changed
+  :set (lambda (sym val)
+         (set-default sym val)
+         (setq char-fold-table (char-fold-make-table)))
+  :group 'lisp
+  :version "27.1")
+
+(defcustom char-fold-exclude-alist char-fold--exclude-alist-default
+  "Character mappings to exclude from default setting."
+  :type '(alist :key-type (character :tag "From")
+                :value-type (character :tag "To"))
+  :initialize #'custom-initialize-changed
+  :set (lambda (sym val)
+         (set-default sym val)
+         (setq char-fold-table (char-fold-make-table)))
+  :group 'lisp
+  :version "27.1")
+
 (defun char-fold--make-space-string (n)
   "Return a string that matches N spaces."
   (format "\\(?:%s\\|%s\\)"

Message #43 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> gmail.com
To: Juri Linkov <juri <at> linkov.net>
Cc: 35689 <at> debbugs.gnu.org, npostavs <at> gmail.com
Subject: Re: bug#35689: Customizable char-fold
Date: Tue, 21 May 2019 17:45:55 -0400

Juri Linkov <juri <at> linkov.net> writes:

>>>       :set (lambda (sym val)
>>>              (set-default sym val)
>>>              ;; FIXME: Maybe delay this until after-init-time,
>>>              ;; to avoid redundant calls to char-fold-make-table.
>>
>> I tried different possible values of :initialize,
>> but not custom-initialize-changed.  I'll try this now.
>
> I see no problems other than redundant calls to char-fold-make-table
> when more than one variable is customized.  char-fold.el is autoloaded
> when isearch calls char-fold-to-regexp for the first time, so I'm not
> sure how after-init-hook could help in this case.

Ah, sorry, I had somehow got it into my head that char-fold.el is
preloaded.

Message #46 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: npostavs <at> gmail.com
Cc: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Thu, 06 Jun 2019 23:49:45 +0300

>>>>       :set (lambda (sym val)
>>>>              (set-default sym val)
>>>>              ;; FIXME: Maybe delay this until after-init-time,
>>>>              ;; to avoid redundant calls to char-fold-make-table.
>>>
>>> I tried different possible values of :initialize,
>>> but not custom-initialize-changed.  I'll try this now.
>>
>> I see no problems other than redundant calls to char-fold-make-table
>> when more than one variable is customized.  char-fold.el is autoloaded
>> when isearch calls char-fold-to-regexp for the first time, so I'm not
>> sure how after-init-hook could help in this case.
>
> Ah, sorry, I had somehow got it into my head that char-fold.el is
> preloaded.

I still haven't found a way to avoid calling char-fold-make-table
3 times when 3 variables are customized.  Maybe this is not possible
because currently customizable variables have no dependency on each other.

Message #49 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 35689 <at> debbugs.gnu.org, npostavs <at> gmail.com
Subject: Re: bug#35689: Customizable char-fold
Date: Mon, 24 Jun 2019 19:33:45 +0200

Juri Linkov <juri <at> linkov.net> writes:

> -(defconst char-fold-table

[...]

> +(defcustom char-fold-symmetric char-fold--symmetric-default
> +  "Include symmetric mappings from composite character back to base letter."
> +  :type 'boolean
> +  :initialize #'custom-initialize-changed
> +  :set (lambda (sym val)
> +         (set-default sym val)
> +         ;; FIXME: Maybe delay this until after-init-hook,
> +         ;; to avoid redundant calls to char-fold-make-table.
> +         (setq char-fold-table (char-fold-make-table)))

OK, here's an uninformed suggestion -- instead of doing all this to
ensure that we have an updated char-fold-table when the user changes
these defaults, why not call char-fold-make-table when needed?

That is, instead of the char-fold-table variable, you'd have a
char-fold-data, which could be, say, something like this:

(defvar char-fold-data :uninitialized)

`char-fold-to-regexp' (and others) would then start with a call to
char-fold-update which would 

be

(let ((new (list :additional char-fold--include-alist-default
                 :exclude char-fold--exclude-alist-default
                 ...)))
  (unless (equal char-fold-data new)
    (setq char-fold-table (char-fold-make-table)
          char-fold-data new)))

or something along those lines.  That is, check whether anything has
changed, and if not, do nothing, but if it has, recompute?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #52 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 35689 <at> debbugs.gnu.org, npostavs <at> gmail.com
Subject: Re: bug#35689: Customizable char-fold
Date: Mon, 24 Jun 2019 23:40:17 +0300

>> +(defcustom char-fold-symmetric char-fold--symmetric-default
>> +  "Include symmetric mappings from composite character back to base letter."
>> +  :type 'boolean
>> +  :initialize #'custom-initialize-changed
>> +  :set (lambda (sym val)
>> +         (set-default sym val)
>> +         ;; FIXME: Maybe delay this until after-init-hook,
>> +         ;; to avoid redundant calls to char-fold-make-table.
>> +         (setq char-fold-table (char-fold-make-table)))
>
> OK, here's an uninformed suggestion -- instead of doing all this to
> ensure that we have an updated char-fold-table when the user changes
> these defaults, why not call char-fold-make-table when needed?
>
> That is, instead of the char-fold-table variable, you'd have a
> char-fold-data, which could be, say, something like this:
>
> (defvar char-fold-data :uninitialized)
>
> `char-fold-to-regexp' (and others) would then start with a call to
> char-fold-update which would
>
> be
>
> (let ((new (list :additional char-fold--include-alist-default
>                  :exclude char-fold--exclude-alist-default
>                  ...)))
>   (unless (equal char-fold-data new)
>     (setq char-fold-table (char-fold-make-table)
>           char-fold-data new)))
>
> or something along those lines.  That is, check whether anything has
> changed, and if not, do nothing, but if it has, recompute?

Thanks for the idea, I'll try to use such composite value to detect
updates in one of three variables.

Message #57 received at 35689-done <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: 35689-done <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Tue, 23 Jul 2019 23:28:09 +0300

Thanks to everyone who helped!  Pushed to master as 376f5df3cc.

Message #60 received at 35689 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: 35689 <at> debbugs.gnu.org
Subject: Re: bug#35689: Customizable char-fold
Date: Mon, 29 Jul 2019 01:46:55 +0300

[Message part 1 (text/plain, inline)]

tags 35689 + fixed
close 35689 27.0.50
quit

I'm closing this customization feature request.

More default values for char-folding could be added depending on user's locale.
But this is another feature that could be implemented within i18n efforts.

PS: As a demonstration what the current customization feature can achieve,
I'm attaching a short snippet that allows searching using Cyrillic translit:

[char-fold-translit.el (application/emacs-lisp, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.