GNU bug report logs - #21028
Performance regression in revision af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21028 in the body.
You can then email your comments to 21028 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel
 <clement.pitclaudel <at> live.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 10 Jul 2015 03:34:23 -0700

[Message part 1 (text/plain, inline)]

Hi all,

I noticed a performance regression between 24.3 and 25.0.50, connected to CJK fonts. The following steps reproduce the issue in master:

1. emacs -Q, and open the scratch buffer
2. Run the following code:
    (progn
     (dotimes (_ 3)
       (set-fontset-font t 'unicode (font-spec :name "Arial") nil 'append))
     (dotimes (_ 50)  
       (insert "日本国\n")))
3. Move the cursor around. Notice that motion is extremely slow (in particular C-p and C-n). So is shift-selection. (if it is not, try increasing the 3 above, maybe to 5 or 10).

This test case is the smallest I could come up with; there might be other ways to trigger the problem. In my particular case, it is caused by having 10 or so distinct fontset rules, covering different blocks of characters. This problem was mentioned elsewhere online too (for example https://github.com/purcell/emacs.d/issues/273).

I tried to pinpoint the commit that introduced the bug, and found revision af1a69f4d17a482c359d98c00ef86fac835b5fac by bisecting between 24.3 and master. Here's the commit in question:

  af1a69f4d17a482c359d98c00ef86fac835b5fac is the first bad commit
  commit af1a69f4d17a482c359d98c00ef86fac835b5fac
  Author: Dmitry Antipov <dmantipov <at> yandex.ru>
  Date:   Wed Apr 2 17:24:19 2014 +0400

      * font.c (font_list_entities): Do not add empty vector to font cache.
      (font_matching_entity): Likewise.  If matching entity is found, insert
      1-item vector with this entity instead of entity itself (Bug#17125).

  :040000 040000 2065e6bd6470194ed976fc343a0fd8932cc88a6c 73da4b1e64a36b066c965e70b9a0ba79c091336d M	src

(by the way, the process was a bit slow; is there a way to make builds faster, maybe by building only the emacs binary? `make src/emacs` does not work, and `make src` spends some time loading el files and generating autoloads)

I added the full bisect log below. Is there any extra info that I should provide?
Thanks!
Clément.

---

git bisect start
# bad: [53cf3cfec12250d89790482d031f95a3cd5c484a] Don't check the exit status, it can be misleading
git bisect bad 53cf3cfec12250d89790482d031f95a3cd5c484a
# good: [24958590a00900371b6b3b154fc1df5c980d056c] Add 24.3 release to ChangeLogs
git bisect good 24958590a00900371b6b3b154fc1df5c980d056c
# good: [d16ae6228826c561bdb3701bd65d7517bd9e30d5] * fns.c (Frandom): Fix rare bug where the result isn't random.
git bisect good d16ae6228826c561bdb3701bd65d7517bd9e30d5
# bad: [935fa6151b3e411c5308d62338744ff25f1a8ddb] * lisp/textmodes/css-mode.el (scss-mode): Fix typo.
git bisect bad 935fa6151b3e411c5308d62338744ff25f1a8ddb
# skip: [e6a4c15ff1a2dad24eb695d0f9f1e63398aadf9f] * lisp/emacs-lisp/easy-mmode.el (define-minor-mode): Use mode function name instead of variable name in hook docstring.  (Bug#18349)
git bisect skip e6a4c15ff1a2dad24eb695d0f9f1e63398aadf9f
# bad: [774da90d8c23f3b27194fb5c82e8b1ccdb77aae2] Mention new whitespace-mode option: big-indent.
git bisect bad 774da90d8c23f3b27194fb5c82e8b1ccdb77aae2
# bad: [18b345686d6a6f00eadb866b541b4f458fcfdb02] ps-samp.el: Make it slightly less awful
git bisect bad 18b345686d6a6f00eadb866b541b4f458fcfdb02
# bad: [aabbbc45793415e1cf41b61abb1737e296d60409] Merge from emacs-24; up to 2014-05-08T06:58:46Z!rgm <at> gnu.org
git bisect bad aabbbc45793415e1cf41b61abb1737e296d60409
# skip: [551a89e12d1ac5198a4de25ce24d98358bb1f8db] Standardize case of "Front-Cover Texts" in texi file permissions notices.
git bisect skip 551a89e12d1ac5198a4de25ce24d98358bb1f8db
# skip: [7b207d6d4f2de3ee07ffc8b02464c2b186372970] Improve usage of AC_INIT
git bisect skip 7b207d6d4f2de3ee07ffc8b02464c2b186372970
# skip: [1e1a3a324856cf7480030b6dbd7f4476c5130ad9] Correct merge
git bisect skip 1e1a3a324856cf7480030b6dbd7f4476c5130ad9
# skip: [1730d9634acba46a173ae8ef6afc853602424dd6] ede autoload tweak
git bisect skip 1730d9634acba46a173ae8ef6afc853602424dd6
# skip: [76377e461836419770c548872e5d88c6e111439c] * internals.texi (C Dialect): New section.
git bisect skip 76377e461836419770c548872e5d88c6e111439c
# skip: [4f3a895b33178230e27d5ad2fb81d5a3f5aa5d9e] * lisp/simple.el (cycle-spacing--context, cycle-spacing): Doc tweaks.
git bisect skip 4f3a895b33178230e27d5ad2fb81d5a3f5aa5d9e
# good: [d1ff9ee403221755ccc259e86e2112959f881047] * minibuf.c (read_minibuf): Avoid C99ism in previous patch.
git bisect good d1ff9ee403221755ccc259e86e2112959f881047
# bad: [1b058e42524353c9ff133ea330876ed2d39b6515] ChangeLog fix
git bisect bad 1b058e42524353c9ff133ea330876ed2d39b6515
# good: [0b4fe0787b957624ebffa5d123c69d5c0a5d69e2] Doc tweaks related to file locking
git bisect good 0b4fe0787b957624ebffa5d123c69d5c0a5d69e2
# good: [5c30ab7a71cbd9ca50e2baa2baad595005f5e5f3] vhdl-mode.el small fixup
git bisect good 5c30ab7a71cbd9ca50e2baa2baad595005f5e5f3
# good: [200c532bd04a67a89db602462d74706051c61178] Inhibit quote autopairing more frequently
git bisect good 200c532bd04a67a89db602462d74706051c61178
# bad: [3a9e7a49deea49088a773c321e52185e922748d6] * make-dist: Further update AC_INIT regexp.
git bisect bad 3a9e7a49deea49088a773c321e52185e922748d6
# bad: [4fd68bf6cc1dce6c95001fbbac95cef32c86359d] Revert subr.el workaround for GC bug.
git bisect bad 4fd68bf6cc1dce6c95001fbbac95cef32c86359d
# bad: [09aba8153a8297e415477dbfa9a8c7e999fb3457] Merge from emacs-24; up to 2014-03-28T01:39:30Z!rgm <at> gnu.org
git bisect bad 09aba8153a8297e415477dbfa9a8c7e999fb3457
# bad: [af1a69f4d17a482c359d98c00ef86fac835b5fac] * font.c (font_list_entities): Do not add empty vector to font cache. (font_matching_entity): Likewise.  If matching entity is found, insert 1-item vector with this entity instead of entity itself (Bug#17125).
git bisect bad af1a69f4d17a482c359d98c00ef86fac835b5fac
# first bad commit: [af1a69f4d17a482c359d98c00ef86fac835b5fac] * font.c (font_list_entities): Do not add empty vector to font cache. (font_matching_entity): Likewise.  If matching entity is found, insert 1-item vector with this entity instead of entity itself (Bug#17125).

[signature.asc (application/pgp-signature, attachment)]

Message #8 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in
 revision	af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 10 Jul 2015 15:30:45 +0300

> Date: Fri, 10 Jul 2015 03:34:23 -0700
> From: Clément Pit--Claudel
>  <clement.pitclaudel <at> live.com>
> 
> 2. Run the following code:
>     (progn
>      (dotimes (_ 3)
>        (set-fontset-font t 'unicode (font-spec :name "Arial") nil 'append))
>      (dotimes (_ 50)  
>        (insert "日本国\n")))
> 3. Move the cursor around. Notice that motion is extremely slow (in particular C-p and C-n). So is shift-selection. (if it is not, try increasing the 3 above, maybe to 5 or 10).
> 
> This test case is the smallest I could come up with; there might be other ways to trigger the problem. In my particular case, it is caused by having 10 or so distinct fontset rules, covering different blocks of characters. This problem was mentioned elsewhere online too (for example https://github.com/purcell/emacs.d/issues/273).

What kind of complex fontset setup do you have?  FWIW, I don't see any
perceptible slowdown with your recipe and the default fontsets.

Message #11 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in
 revision	af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 10 Jul 2015 15:41:43 +0300

> Date: Fri, 10 Jul 2015 03:34:23 -0700
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> 
> I tried to pinpoint the commit that introduced the bug, and found revision af1a69f4d17a482c359d98c00ef86fac835b5fac by bisecting between 24.3 and master. Here's the commit in question:
> 
>   af1a69f4d17a482c359d98c00ef86fac835b5fac is the first bad commit
>   commit af1a69f4d17a482c359d98c00ef86fac835b5fac
>   Author: Dmitry Antipov <dmantipov <at> yandex.ru>
>   Date:   Wed Apr 2 17:24:19 2014 +0400
> 
>       * font.c (font_list_entities): Do not add empty vector to font cache.
>       (font_matching_entity): Likewise.  If matching entity is found, insert
>       1-item vector with this entity instead of entity itself (Bug#17125).

Maybe I'm blind, but I don't see anything in that changeset that could
explain a performance hit.  The modified code seems to do
approximately the same amount of work, and create the same data
structures, as the original one.  Are you sure that backing out those
changes fixes your problem?

Message #14 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 10 Jul 2015 11:42:13 -0400

Clément Pit--Claudel wrote:

> (by the way, the process was a bit slow; is there a way to make builds
> faster, maybe by building only the emacs binary? `make src/emacs` does
> not work, and `make src` spends some time loading el files and
> generating autoloads)

I posted the recipe I use at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19983#62

_Perhaps_ you can stop at the bootstrap-emacs stage in this case.

(BTW, "make -C src emacs" should work now, but obviously that doesn't
help when you go to older revisions.)

Message #17 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel
 <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in
 revision	af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 10 Jul 2015 09:02:30 -0700

[Message part 1 (text/plain, inline)]

On 07/10/2015 05:30 AM, Eli Zaretskii wrote:
>> Date: Fri, 10 Jul 2015 03:34:23 -0700
>> From: Clément Pit--Claudel
>>  <clement.pitclaudel <at> live.com>
>>
>> 2. Run the following code:
>>     (progn
>>      (dotimes (_ 3)
>>        (set-fontset-font t 'unicode (font-spec :name "Arial") nil 'append))
>>      (dotimes (_ 50)  
>>        (insert "日本国\n")))
>> 3. Move the cursor around. Notice that motion is extremely slow (in particular C-p and C-n). So is shift-selection. (if it is not, try increasing the 3 above, maybe to 5 or 10).
>>
>> This test case is the smallest I could come up with; there might be other ways to trigger the problem. In my particular case, it is caused by having 10 or so distinct fontset rules, covering different blocks of characters. This problem was mentioned elsewhere online too (for example https://github.com/purcell/emacs.d/issues/273).
> 
> What kind of complex fontset setup do you have?  FWIW, I don't see any
> perceptible slowdown with your recipe and the default fontsets.

The relevant snippet looks like this:

(let ((cjk-font "WenQuanYi Micro Hei Mono")
      (fontset t)) ; "fontset-default"
  (set-fontset-font t (cons ?≔ ?≔) "FreeSerif" nil 'prepend)
  (set-fontset-font fontset 'unicode (font-spec :name "Consolas") nil 'append)
  (set-fontset-font fontset 'unicode (font-spec :name "Symbola") nil 'append)
  (set-fontset-font fontset '(#x4E00 . #x9FFF) (font-spec :name cjk-font))
  (set-fontset-font fontset '(#x3400 . #x4DFF) (font-spec :name cjk-font))
  (set-fontset-font fontset '(#x20000 . #x2A6DF) (font-spec :name cjk-font))
  (set-fontset-font fontset '(#xF900 . #xFAFF) (font-spec :name cjk-font))
  (set-fontset-font fontset '(#x2F800 . #x2FA1F) (font-spec :name cjk-font)))

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel
 <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in
 revision	af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 10 Jul 2015 09:55:18 -0700

[Message part 1 (text/plain, inline)]

On 07/10/2015 05:41 AM, Eli Zaretskii wrote:
>> Date: Fri, 10 Jul 2015 03:34:23 -0700
>> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
>>
>> I tried to pinpoint the commit that introduced the bug, and found revision af1a69f4d17a482c359d98c00ef86fac835b5fac by bisecting between 24.3 and master. Here's the commit in question:
>>
>>   af1a69f4d17a482c359d98c00ef86fac835b5fac is the first bad commit
>>   commit af1a69f4d17a482c359d98c00ef86fac835b5fac
>>   Author: Dmitry Antipov <dmantipov <at> yandex.ru>
>>   Date:   Wed Apr 2 17:24:19 2014 +0400
>>
>>       * font.c (font_list_entities): Do not add empty vector to font cache.
>>       (font_matching_entity): Likewise.  If matching entity is found, insert
>>       1-item vector with this entity instead of entity itself (Bug#17125).
> 
> Maybe I'm blind, but I don't see anything in that changeset that could
> explain a performance hit.  The modified code seems to do
> approximately the same amount of work, and create the same data
> structures, as the original one.  Are you sure that backing out those
> changes fixes your problem?

Thanks for looking into this. Yes, reverting these changes on master fixes the problem. I ran a number of experiments:

* I timed the sample code that I sent, before and after af1a69f4d17a482c359d98c00ef86fac835b5fac:

** On af1a69f4d17a482c359d98c00ef86fac835b5fac (the commit that I identified as bad):

   $ time src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (redisplay t) (kill-emacs))"
   real	0m2.249s
   user	0m1.259s
   sys	0m0.100s

   $ time src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 50) (insert \"日本国\n\")) (redisplay t) (kill-emacs))"
   real	0m5.749s
   user	0m2.717s
   sys	0m0.857s

** On 200c532bd04a67a89db602462d74706051c61178 (the previous commit)

   $ time src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (redisplay t) (kill-emacs))"
   real	0m2.214s
   user	0m1.247s
   sys	0m0.077s

   $ time src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 50) (insert \"日本国\n\")) (redisplay t) (kill-emacs))"
   real	0m2.226s
   user	0m1.222s
   sys	0m0.106s

Thus the operation of inserting these 50 lines takes about 3.5 seconds on the bad commit, while it's instantaneous in the previous commit.

* On master, timed the same code before and after reverting af1a69f4d17a482c359d98c00ef86fac835b5fac:

** Before reverting

   $ time emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (redisplay t) (kill-emacs))"
   real	0m2.441s
   user	0m1.363s
   sys	0m0.142s

   $ time emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 50) (insert \"日本国\n\")) (redisplay t) (kill-emacs))"
   real	0m5.149s
   user	0m2.569s
   sys	0m0.742s

** After reverting

   $ time src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (redisplay t) (kill-emacs))"
   real	0m2.413s
   user	0m1.351s
   sys	0m0.147s

   $ time src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 50) (insert \"日本国\n\")) (redisplay t) (kill-emacs))"
   real	0m2.570s
   user	0m1.608s
   sys	0m0.083s

The figures are very similar to the tests above: with that patch inserting 50 lines takes 3 seconds, and without it it's instantaneous. Thus I think it's safe to say that this particular patch is indeed responsible for the performance regression. But maybe I'm missing something?

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #23 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Antipov <dmantipov <at> yandex.ru>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 15 Jul 2015 15:32:36 +0300

On 07/10/2015 07:55 PM, Clément Pit--Claudel wrote:

> The figures are very similar to the tests above: with that patch inserting 50 lines takes 3 seconds,
> and without it it's instantaneous. Thus I think it's safe to say that this particular patch is indeed
> responsible for the performance regression. But maybe I'm missing something?

As of c40ea1328bb33abaec14f1fc92ac2349b5ee2715, I can't reproduce this issue, with your fontset setup
from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#17. Cursor motion and keyboard/mouse selection
are smooth, and CPU/memory usage looks normal.

My suggestions are:

1) Re-run your timed tests from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#20 under /usr/bin/time,
not time (the latter is a simple shell builtin). /usr/bin/time also shows memory usage; if "bad" (current)
instance consumes more memory than "good" (with reverted change) one, there may be nasty GC issue.

2) 3 seconds is large enough to leave the traces in profiled runs. On GNU/Linux, it may be worth trying
to run under perf, both "good" and "bad" cases, and comparing reports.

Dmitry

Message #26 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel
 <clement.pitclaudel <at> live.com>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sat, 18 Jul 2015 03:24:02 -0700

[Message part 1 (text/plain, inline)]

On 07/15/2015 05:32 AM, Dmitry Antipov wrote:
> On 07/10/2015 07:55 PM, Clément Pit--Claudel wrote:
> 
>> The figures are very similar to the tests above: with that patch inserting 50 lines takes 3 seconds,
>> and without it it's instantaneous. Thus I think it's safe to say that this particular patch is indeed
>> responsible for the performance regression. But maybe I'm missing something?
> 
> As of c40ea1328bb33abaec14f1fc92ac2349b5ee2715, I can't reproduce this issue, with your fontset setup
> from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#17. Cursor motion and keyboard/mouse selection
> are smooth, and CPU/memory usage looks normal.

Thanks for trying this out. Thanks for your suggestions, too.

> My suggestions are:
> 
> 1) Re-run your timed tests from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#20 under /usr/bin/time,
> not time (the latter is a simple shell builtin). /usr/bin/time also shows memory usage; if "bad" (current)
> instance consumes more memory than "good" (with reverted change) one, there may be nasty GC issue.

The timings are similar:

$ /usr/bin/time emacs-bad/src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 500) (insert \"日本国\n\")) (dotimes (_ 500) (previous-line) (redisplay t)) (kill-emacs))"
13.31user 8.57system 0:34.54elapsed 63%CPU (0avgtext+0avgdata 50592maxresident)k
0inputs+0outputs (0major+4958minor)pagefaults 0swaps

$ /usr/bin/time emacs-good/src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 500) (insert \"日本国\n\")) (dotimes (_ 500) (previous-line) (redisplay t)) (kill-emacs))"
0.58user 0.03system 0:01.05elapsed 58%CPU (0avgtext+0avgdata 50392maxresident)k
29840inputs+0outputs (105major+4911minor)pagefaults 0swaps

> 2) 3 seconds is large enough to leave the traces in profiled runs. On GNU/Linux, it may be worth trying
> to run under perf, both "good" and "bad" cases, and comparing reports.

I have attached the reports in the good and bad cases (200c532 and af1a69f respectively). I'm not sure what to conclude from them; I can provide the full trace data if needed.

Clément.

[bad-report (text/plain, attachment)]

[good-report (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #29 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel
 <clement.pitclaudel <at> live.com>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sat, 18 Jul 2015 04:16:16 -0700

[Message part 1 (text/plain, inline)]

Hi all,

I've spent a bit more time looking into this. The problem is due to the following three lines being conditioned on `!NILP(val)`: (that's around line 2780 of src/font.c)

    Lisp_Object copy = copy_font_spec (scratch_font_spec);
    ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
    XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));

Before af1a69f, these lines were always run. In af1a69f, they only run if `val` is non nil, causing the regression: on master, moving them back out of the if statement fixes the problem. For clarity I have attached the corresponding patch.

Clément.

On 07/18/2015 03:24 AM, Clément Pit--Claudel wrote:
> On 07/15/2015 05:32 AM, Dmitry Antipov wrote:
>> On 07/10/2015 07:55 PM, Clément Pit--Claudel wrote:
>>
>>> The figures are very similar to the tests above: with that patch inserting 50 lines takes 3 seconds,
>>> and without it it's instantaneous. Thus I think it's safe to say that this particular patch is indeed
>>> responsible for the performance regression. But maybe I'm missing something?
>>
>> As of c40ea1328bb33abaec14f1fc92ac2349b5ee2715, I can't reproduce this issue, with your fontset setup
>> from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#17. Cursor motion and keyboard/mouse selection
>> are smooth, and CPU/memory usage looks normal.
> 
> Thanks for trying this out. Thanks for your suggestions, too.
> 
>> My suggestions are:
>>
>> 1) Re-run your timed tests from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#20 under /usr/bin/time,
>> not time (the latter is a simple shell builtin). /usr/bin/time also shows memory usage; if "bad" (current)
>> instance consumes more memory than "good" (with reverted change) one, there may be nasty GC issue.
> 
> The timings are similar:
> 
> $ /usr/bin/time emacs-bad/src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 500) (insert \"日本国\n\")) (dotimes (_ 500) (previous-line) (redisplay t)) (kill-emacs))"
> 13.31user 8.57system 0:34.54elapsed 63%CPU (0avgtext+0avgdata 50592maxresident)k
> 0inputs+0outputs (0major+4958minor)pagefaults 0swaps
> 
> $ /usr/bin/time emacs-good/src/emacs -Q --eval "(progn (dotimes (_ 3) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (dotimes (_ 500) (insert \"日本国\n\")) (dotimes (_ 500) (previous-line) (redisplay t)) (kill-emacs))"
> 0.58user 0.03system 0:01.05elapsed 58%CPU (0avgtext+0avgdata 50392maxresident)k
> 29840inputs+0outputs (105major+4911minor)pagefaults 0swaps
> 
>> 2) 3 seconds is large enough to leave the traces in profiled runs. On GNU/Linux, it may be worth trying
>> to run under perf, both "good" and "bad" cases, and comparing reports.
> 
> I have attached the reports in the good and bad cases (200c532 and af1a69f respectively). I'm not sure what to conclude from them; I can provide the full trace data if needed.
> 
> Clément.
> 

[0001-Draft-of-a-fix-for-21028.patch (text/x-diff, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #32 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sat, 18 Jul 2015 14:26:21 +0300

> Date: Sat, 18 Jul 2015 04:16:16 -0700
> From: Clément Pit--Claudel
>  <clement.pitclaudel <at> live.com>
> CC: Eli Zaretskii <eliz <at> gnu.org>, 21028 <at> debbugs.gnu.org
> 
> I've spent a bit more time looking into this. The problem is due to the following three lines being conditioned on `!NILP(val)`: (that's around line 2780 of src/font.c)
> 
>     Lisp_Object copy = copy_font_spec (scratch_font_spec);
>     ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
>     XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));
> 
> Before af1a69f, these lines were always run. In af1a69f, they only run if `val` is non nil, causing the regression: on master, moving them back out of the if statement fixes the problem.

Thanks.  Do you understand how doing something only sometimes could be
slower than doing it always?

Given that your profile points to Fontconfig, the only explanation I
can come up with is that doing this conditionally means additional
work in Fontconfig.

Message #35 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel
 <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sat, 18 Jul 2015 13:08:09 -0700

[Message part 1 (text/plain, inline)]

On 07/18/2015 04:26 AM, Eli Zaretskii wrote:
>> Date: Sat, 18 Jul 2015 04:16:16 -0700
>> From: Clément Pit--Claudel
>>  <clement.pitclaudel <at> live.com>
>> CC: Eli Zaretskii <eliz <at> gnu.org>, 21028 <at> debbugs.gnu.org
>>
>> I've spent a bit more time looking into this. The problem is due to the following three lines being conditioned on `!NILP(val)`: (that's around line 2780 of src/font.c)
>>
>>     Lisp_Object copy = copy_font_spec (scratch_font_spec);
>>     ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
>>     XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));
>>
>> Before af1a69f, these lines were always run. In af1a69f, they only run if `val` is non nil, causing the regression: on master, moving them back out of the if statement fixes the problem.
> 
> Thanks.  Do you understand how doing something only sometimes could be
> slower than doing it always?

My guess would be that these lines themselves aren't slow, but omitting them makes things slow. In addition to the two perf traces that I sent, here are the two most common stack traces that I get when I interrupt the text insertion loop using [(gdb) run -Q --eval "(progn (dotimes (_ 10) (set-fontset-font t 'unicode (font-spec :name \"Arial\") nil 'append)) (redisplay t) (dotimes (_ 500) (insert \"日本国\n\")))"]

* 3/4 of the time I get this one:
(gdb) backtrace
#0  0x00007ffff164912d in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007fffee12db72 in ?? () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#2  0x00007fffee12f3ff in ?? () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#3  0x00007fffee12f512 in xcb_wait_for_reply () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#4  0x00007ffff4c9948f in _XReply () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#5  0x00007ffff4c950cd in XSync () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#6  0x00000000004bdfb5 in x_catch_errors (dpy=dpy <at> entry=0xbe58d0) at xterm.c:7471
#7  0x00000000005b06f9 in xfont_list_pattern (display=display <at> entry=0xbe58d0, 
    pattern=pattern <at> entry=0x7fffffff6f20 "Arial", registry=registry <at> entry=12339538, script=script <at> entry=12108082)
    at xfont.c:367
#8  0x00000000005b10ae in xfont_list (f=<optimized out>, spec=12261565) at xfont.c:560
#9  0x000000000056a879 in font_list_entities (f=f <at> entry=0x112a4b8, spec=spec <at> entry=18635693) at font.c:2756
#10 0x000000000056b242 in font_find_for_lface (f=f <at> entry=0x112a4b8, attrs=attrs <at> entry=0x1807320, 
    spec=<optimized out>, c=c <at> entry=22269) at font.c:3232
#11 0x00000000005b9481 in fontset_find_font (fontset=19770717, c=c <at> entry=22269, face=face <at> entry=0x1807320, 
    id=id <at> entry=-1, fallback=fallback <at> entry=false) at fontset.c:681
#12 0x00000000005b9940 in fontset_font (fontset=fontset <at> entry=18249853, c=c <at> entry=22269, face=face <at> entry=0x1807320, 
    id=-1) at fontset.c:754
#13 0x00000000005ba3ea in face_for_char (f=0x112a4b8, face=face <at> entry=0x1807320, c=22269, pos=<optimized out>, 
    object=<optimized out>) at fontset.c:956
#14 0x000000000043ff68 in get_next_display_element (it=it <at> entry=0x7fffffff8ff0) at xdisp.c:7064
#15 0x00000000004456e0 in display_line (it=it <at> entry=0x7fffffff8ff0) at xdisp.c:19867
#16 0x000000000044962a in try_window (window=window <at> entry=18003149, pos=..., flags=flags <at> entry=0) at xdisp.c:16685
#17 0x000000000045d312 in redisplay_window (window=18003149, just_this_one_p=just_this_one_p <at> entry=false)
    at xdisp.c:16367
#18 0x0000000000461193 in redisplay_window_0 (window=window <at> entry=18003149) at xdisp.c:14157
#19 0x00000000005545a6 in internal_condition_case_1 (bfun=bfun <at> entry=0x461160 <redisplay_window_0>, arg=18003149, 
    handlers=<optimized out>, hfun=hfun <at> entry=0x42b360 <redisplay_window_error>) at eval.c:1378
#20 0x000000000042fbfe in redisplay_windows (window=18003149) at xdisp.c:14137
#21 0x000000000044dfb1 in redisplay_internal () at xdisp.c:13736
#22 0x000000000044fdd5 in redisplay () at xdisp.c:13022
#23 0x00000000004ef9e1 in read_char (commandflag=1, map=map <at> entry=19387718, prev_event=12108082, 
    used_mouse_menu=used_mouse_menu <at> entry=0x7fffffffd71b, end_time=end_time <at> entry=0x0) at keyboard.c:2569
#24 0x00000000004f1353 in read_key_sequence (keybuf=keybuf <at> entry=0x7fffffffd7f0, prompt=12108082, 
    dont_downcase_last=dont_downcase_last <at> entry=false, can_return_switch_frame=can_return_switch_frame <at> entry=true, 
    fix_current_buffer=fix_current_buffer <at> entry=true, prevent_redisplay=prevent_redisplay <at> entry=false, bufsize=30)
    at keyboard.c:9081
#25 0x00000000004f2f70 in command_loop_1 () at keyboard.c:1449
#26 0x000000000055447e in internal_condition_case (bfun=bfun <at> entry=0x4f2d80 <command_loop_1>, 
    handlers=<optimized out>, hfun=hfun <at> entry=0x4e9d40 <cmd_error>) at eval.c:1354
#27 0x00000000004e546e in command_loop_2 (ignore=ignore <at> entry=12108082) at keyboard.c:1174
#28 0x000000000055438b in internal_catch (tag=12155586, func=func <at> entry=0x4e5450 <command_loop_2>, arg=12108082)
    at eval.c:1118
#29 0x00000000004e9967 in command_loop () at keyboard.c:1153
#30 recursive_edit_1 () at keyboard.c:777
#31 0x00000000004e9c52 in Frecursive_edit () at keyboard.c:845
#32 0x0000000000417f75 in main (argc=<optimized out>, argv=0x7fffffffdb48) at emacs.c:1654

* 1/4 of the time I get that one:
(gdb) backtrace
#0  0x00007ffff2453900 in ?? () from /usr/lib/x86_64-linux-gnu/libfontconfig.so.1
#1  0x00007ffff24543b6 in ?? () from /usr/lib/x86_64-linux-gnu/libfontconfig.so.1
#2  0x00007ffff246185d in ?? () from /usr/lib/x86_64-linux-gnu/libfontconfig.so.1
#3  0x00007ffff24619b0 in FcFontSetList () from /usr/lib/x86_64-linux-gnu/libfontconfig.so.1
#4  0x00007ffff24621ff in FcFontList () from /usr/lib/x86_64-linux-gnu/libfontconfig.so.1
#5  0x00000000005b3792 in ftfont_list (f=<optimized out>, spec=12261565) at ftfont.c:955
#6  0x00000000005b61fa in xftfont_list (f=<optimized out>, spec=<optimized out>) at xftfont.c:141
#7  0x000000000056a879 in font_list_entities (f=f <at> entry=0x112a4b8, spec=spec <at> entry=18192877) at font.c:2756
#8  0x000000000056b242 in font_find_for_lface (f=f <at> entry=0x112a4b8, attrs=attrs <at> entry=0x1807320, 
    spec=<optimized out>, c=c <at> entry=22269) at font.c:3232
#9  0x00000000005b9481 in fontset_find_font (fontset=19770717, c=c <at> entry=22269, face=face <at> entry=0x1807320, 
    id=id <at> entry=-1, fallback=fallback <at> entry=false) at fontset.c:681
#10 0x00000000005b9940 in fontset_font (fontset=fontset <at> entry=18249853, c=c <at> entry=22269, face=face <at> entry=0x1807320, 
    id=-1) at fontset.c:754
#11 0x00000000005ba3ea in face_for_char (f=0x112a4b8, face=face <at> entry=0x1807320, c=22269, pos=<optimized out>, 
    object=<optimized out>) at fontset.c:956
#12 0x000000000043ff68 in get_next_display_element (it=it <at> entry=0x7fffffff82f0) at xdisp.c:7064
#13 0x000000000043d8d1 in move_it_in_display_line_to (it=it <at> entry=0x7fffffff82f0, to_charpos=to_charpos <at> entry=2001, 
    to_x=to_x <at> entry=-1, op=op <at> entry=MOVE_TO_POS) at xdisp.c:8504
#14 0x00000000004450ec in move_it_to (it=it <at> entry=0x7fffffff82f0, to_charpos=to_charpos <at> entry=2001, 
    to_x=to_x <at> entry=-1, to_y=to_y <at> entry=-1, to_vpos=to_vpos <at> entry=-1, op=op <at> entry=8) at xdisp.c:9145
#15 0x000000000044999d in move_it_vertically_backward (it=it <at> entry=0x7fffffff9e60, dy=dy <at> entry=280) at xdisp.c:9318
#16 0x000000000045d26b in redisplay_window (window=18003149, just_this_one_p=just_this_one_p <at> entry=false)
    at xdisp.c:16330
#17 0x0000000000461193 in redisplay_window_0 (window=window <at> entry=18003149) at xdisp.c:14157
#18 0x00000000005545a6 in internal_condition_case_1 (bfun=bfun <at> entry=0x461160 <redisplay_window_0>, arg=18003149, 
    handlers=<optimized out>, hfun=hfun <at> entry=0x42b360 <redisplay_window_error>) at eval.c:1378
#19 0x000000000042fbfe in redisplay_windows (window=18003149) at xdisp.c:14137
#20 0x000000000044dfb1 in redisplay_internal () at xdisp.c:13736
#21 0x000000000044fdd5 in redisplay () at xdisp.c:13022
#22 0x00000000004ef9e1 in read_char (commandflag=1, map=map <at> entry=19387718, prev_event=12108082, 
    used_mouse_menu=used_mouse_menu <at> entry=0x7fffffffd71b, end_time=end_time <at> entry=0x0) at keyboard.c:2569
#23 0x00000000004f1353 in read_key_sequence (keybuf=keybuf <at> entry=0x7fffffffd7f0, prompt=12108082, 
    dont_downcase_last=dont_downcase_last <at> entry=false, can_return_switch_frame=can_return_switch_frame <at> entry=true, 
    fix_current_buffer=fix_current_buffer <at> entry=true, prevent_redisplay=prevent_redisplay <at> entry=false, bufsize=30)
    at keyboard.c:9081
#24 0x00000000004f2f70 in command_loop_1 () at keyboard.c:1449
#25 0x000000000055447e in internal_condition_case (bfun=bfun <at> entry=0x4f2d80 <command_loop_1>, 
    handlers=<optimized out>, hfun=hfun <at> entry=0x4e9d40 <cmd_error>) at eval.c:1354
#26 0x00000000004e546e in command_loop_2 (ignore=ignore <at> entry=12108082) at keyboard.c:1174
#27 0x000000000055438b in internal_catch (tag=12155586, func=func <at> entry=0x4e5450 <command_loop_2>, arg=12108082)
    at eval.c:1118
#28 0x00000000004e9967 in command_loop () at keyboard.c:1153
#29 recursive_edit_1 () at keyboard.c:777
#30 0x00000000004e9c52 in Frecursive_edit () at keyboard.c:845
#31 0x0000000000417f75 in main (argc=<optimized out>, argv=0x7fffffffdb48) at emacs.c:1654

Hopefully someone can make sense of this.

[signature.asc (application/pgp-signature, attachment)]

Message #38 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 2 Mar 2016 23:12:35 -0700

[Message part 1 (text/plain, inline)]

On 07/15/2015 06:32 AM, Dmitry Antipov wrote:
> On 07/10/2015 07:55 PM, Clément Pit--Claudel wrote:
> 
>> The figures are very similar to the tests above: with that patch inserting 50 lines takes 3 seconds,
>> and without it it's instantaneous. Thus I think it's safe to say that this particular patch is indeed
>> responsible for the performance regression. But maybe I'm missing something?
> 
> As of c40ea1328bb33abaec14f1fc92ac2349b5ee2715, I can't reproduce this issue, with your fontset setup
> from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#17. Cursor motion and keyboard/mouse selection
> are smooth, and CPU/memory usage looks normal.

I'm still receiving bug reports from users of my plugins about slowness; this doesn't seem to affect just me. I can reproduce it in the latest pretest. My bet is that this is due to specific fonts. That would explain why you couldn't reproduce this.

(I'm getting reports because my mode enables prettify-symbols-mode, and thus displays characters from fallback fonts)

I've removed the dependency on CJK fonts; on my machine, this does the trick, provided you have XITS Math and Ubuntu Mono installed:

emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\")))"

Scrolling around this buffer is horribly slow.

I would very much welcome help in tracking and fixing this bug. The culprit is af1a69f4d17a482c359d98c00ef86fac835b5fac, but if the commit itself can't be reversed, maybe we could find another way to achieve what it does?

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #41 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 2 Mar 2016 23:37:30 -0700

[Message part 1 (text/plain, inline)]

On 07/10/2015 09:42 AM, Glenn Morris wrote:
> Clément Pit--Claudel wrote:
> 
>> (by the way, the process was a bit slow; is there a way to make builds
>> faster, maybe by building only the emacs binary? `make src/emacs` does
>> not work, and `make src` spends some time loading el files and
>> generating autoloads)
> 
> I posted the recipe I use at
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19983#62

Belated thanks :)

[signature.asc (application/pgp-signature, attachment)]

Message #44 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 3 Mar 2016 00:33:42 -0700

[Message part 1 (text/plain, inline)]

On 03/03/2016 12:08 AM, Clément Pit--Claudel wrote:
> I've reattached both profiles.

Hi,

I've looked further into this. Comparing the profiles, there seems to be two significant differences:

1. ‘deliver_user_signal’ gets called constantly in the slow case, with many calls to ‘x_catch_errors_with_handler’, ‘x_catch_errors_with_handler’, ‘x_had_errors_p’, and ‘x_uncatch_errors’; there are few to no such calls in the fast case.
2. font_list_entities constantly calls xfont_list in the slow case, but not in the fast case:

In the slow case:

    -----------------------------------------------
                    0.04    1.01  554734/554734      font_find_for_lface [14]
    [16]    11.0    0.04    1.01  554734         font_list_entities [16]
                    0.04    0.91  554734/554734      xfont_list [17]
                    0.02    0.00  554734/2778892     Fnreverse [45]
                    0.00    0.02 1109468/1109469     assoc_no_quit [96]
                    0.01    0.00 1109468/1109470     xfont_get_cache [121]
                    0.01    0.00  555016/1289625     Fcons [80]
                    0.00    0.00     141/141         xftfont_list [212]
                    0.00    0.00     141/45490       concat [103]
                    0.00    0.00     141/138863      copy_font_spec [78]
                    0.00    0.00     141/6814        Fvconcat [832]
    -----------------------------------------------

In the fast case:

    -----------------------------------------------
                    0.06    0.41 1533134/1533134     font_find_for_lface [13]
    [25]    14.8    0.06    0.41 1533134         font_list_entities [25]
                    0.02    0.31 3066268/3066269     assoc_no_quit [29]
                    0.07    0.00 1533134/7672938     Fnreverse [28]
                    0.02    0.00 1534750/3266575     Fcons [62]
                    0.00    0.00     347/347         xftfont_list [209]
                    0.00    0.00     461/461         xfont_list [245]
                    0.00    0.00     808/385139      copy_font_spec [50]
                    0.00    0.00     347/56861       concat [111]
                    0.00    0.00 3066268/3066270     xfont_get_cache [605]
                    0.00    0.00     347/5788        Fvconcat [739]
    -----------------------------------------------

The difference there is this line in the slow profile:

    0.04    0.91  554734/554734      xfont_list [17]

(compare to this line:)

    0.00    0.00     461/461         xfont_list [245]

font_list_entities is in src/font.c; it call xfont_list here (old code):

    Lisp_Object copy;

    val = driver_list->driver->list (f, scratch_font_spec); // ← HERE
    if (NILP (val))
      val = zero_vector;
    else
      val = Fvconcat (1, &val);
    copy = copy_font_spec (scratch_font_spec);
    ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
    XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));

Here is the new code:

    val = driver_list->driver->list (f, scratch_font_spec);
    if (!NILP (val))
      {
        Lisp_Object copy = copy_font_spec (scratch_font_spec);

        val = Fvconcat (1, &val);
        ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
        XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));
      }

The commit message didn't explain why it was a good thing to not add an empty vector to the cache.

I'm a bit stuck at this point. I'm not sure how 1 and 2 are connected. Suggestions?

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #47 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 2 Mar 2016 23:42:14 -0700

[Message part 1 (text/plain, inline)]

On 03/02/2016 11:12 PM, Clément Pit--Claudel wrote:
> I've removed the dependency on CJK fonts; on my machine, this does
> the trick, provided you have XITS Math and Ubuntu Mono installed:
> 
> emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\")))"
> 
> Scrolling around this buffer is horribly slow.

Here's the few lines of a profile (full profile attached):

    Flat profile:

    Each sample counts as 0.01 seconds.
      %   cumulative   self              self     total
     time   seconds   seconds    calls   s/call   s/call  name
     69.14      6.61     6.61                             deliver_user_signal
      2.30      6.83     0.22      291     0.00     0.00  _init
      2.09      7.03     0.20  2446268     0.00     0.00  hash_lookup
      1.78      7.20     0.17  1109458     0.00     0.00  xfont_list_pattern
      1.36      7.33     0.13  7538722     0.00     0.00  char_table_ref
      1.36      7.46     0.13  2220018     0.00     0.00  ftfont_lookup_cache
      1.15      7.57     0.11  2774417     0.00     0.00  Fcompare_strings
      1.05      7.67     0.10  2778892     0.00     0.00  Fnreverse
      1.05      7.77     0.10 11525665     0.00     0.00  mark_object
      0.94      7.86     0.09 33105673     0.00     0.00  sub_char_table_ref

compare with this (with the aforementioned commit reverted):

    Flat profile:

    Each sample counts as 0.01 seconds.
      %   cumulative   self              self     total
     time   seconds   seconds    calls  ms/call  ms/call  name
      6.48      0.07     0.07  5576010     0.00     0.00  mark_object
      6.48      0.14     0.07  4987260     0.00     0.00  char_table_ref
      6.48      0.21     0.07                             deliver_user_signal
      5.56      0.27     0.06 12938424     0.00     0.00  sub_char_table_ref
      5.09      0.33     0.06  1407350     0.00     0.00  Fassq
      3.70      0.37     0.04    54802     0.00     0.00  face_before_or_after_it_pos
      1.85      0.39     0.02  1232573     0.00     0.00  lookup_char_property
      1.85      0.41     0.02   954743     0.00     0.00  Fcons
      1.85      0.43     0.02   723140     0.00     0.00  find_interval
      1.85      0.45     0.02   505158     0.00     0.00  hash_lookup
      1.85      0.47     0.02   497590     0.00     0.00  bidi_level_of_next_char
      1.85      0.49     0.02   497590     0.00     0.00  bidi_move_to_visually_next

I've attached the two profiles. Does this make any sense?

Clément.

[fast-trace-with-reverted-commit (text/plain, attachment)]

[slow-trace-with-plain-emacs (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #50 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Dmitry Antipov <dmantipov <at> yandex.ru>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 3 Mar 2016 00:08:38 -0700

[Message part 1 (text/plain, inline)]

On 03/02/2016 11:42 PM, Clément Pit--Claudel wrote:
> On 03/02/2016 11:12 PM, Clément Pit--Claudel wrote:
>> I've removed the dependency on CJK fonts; on my machine, this does
>> the trick, provided you have XITS Math and Ubuntu Mono installed:
>>
>> emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\")))"
>>
>> Scrolling around this buffer is horribly slow.
> 
> Here's the few lines of a profile (full profile attached):
> 
>     Flat profile:
> 
>     Each sample counts as 0.01 seconds.
>       %   cumulative   self              self     total
>      time   seconds   seconds    calls   s/call   s/call  name
>      69.14      6.61     6.61                             deliver_user_signal
>       2.30      6.83     0.22      291     0.00     0.00  _init
>       2.09      7.03     0.20  2446268     0.00     0.00  hash_lookup
>       1.78      7.20     0.17  1109458     0.00     0.00  xfont_list_pattern
>       1.36      7.33     0.13  7538722     0.00     0.00  char_table_ref
>       1.36      7.46     0.13  2220018     0.00     0.00  ftfont_lookup_cache
>       1.15      7.57     0.11  2774417     0.00     0.00  Fcompare_strings
>       1.05      7.67     0.10  2778892     0.00     0.00  Fnreverse
>       1.05      7.77     0.10 11525665     0.00     0.00  mark_object
>       0.94      7.86     0.09 33105673     0.00     0.00  sub_char_table_ref
> 
> compare with this (with the aforementioned commit reverted):

Sorry, this was the wrong profile. Here's the profile with a snappy emacs, after reverting the commit above

Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 16.93      0.54     0.54 14252708     0.00     0.00  internal_equal
 10.66      0.88     0.34  7672938     0.00     0.00  Fnreverse
  9.56      1.19     0.31 27776580     0.00     0.00  mark_object
  8.46      1.46     0.27  6640124     0.00     0.00  hash_lookup
  2.51      1.54     0.08  6134442     0.00     0.00  ftfont_lookup_cache
  2.19      1.61     0.07  3072176     0.00     0.00  casify_object
  2.19      1.68     0.07     8818     0.00     0.00  mark_char_table
  1.88      1.74     0.06 12567953     0.00     0.00  sub_char_table_ref
  1.88      1.80     0.06  1409718     0.00     0.00  Fassq
  1.88      1.86     0.06  1533134     0.00     0.00  font_list_entities
  1.57      1.91     0.05  6195131     0.00     0.00  assq_no_quit
  1.57      1.96     0.05  1533134     0.00     0.00  font_sort_entities
  1.57      2.01     0.05     3644     0.00     0.00  _init

I've reattached both profiles.

Thanks,
Clément.

[slow-trace-with-plain-emacs (text/plain, attachment)]

[fast-trace-with-reverted-commit (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #53 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Dmitry Antipov <dmantipov <at> yandex.ru>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 03 Mar 2016 12:10:22 -0500

Enormous attachments are likely to be rejected by the bug-gnu-emacs moderators.
Please remember to compress! :)

(Interested parties can see them on
https://debbugs.gnu.org/21028
so no need to resend.)

Message #56 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Dmitry Antipov <dmantipov <at> yandex.ru>
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 3 Mar 2016 10:28:17 -0700

[Message part 1 (text/plain, inline)]

On 03/03/2016 10:10 AM, Glenn Morris wrote:
> Enormous attachments are likely to be rejected by the bug-gnu-emacs moderators.
> Please remember to compress! :)

Thanks for the reminder; sorry!

[signature.asc (application/pgp-signature, attachment)]

Message #59 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 03 Mar 2016 22:27:42 +0200

> Cc: Eli Zaretskii <eliz <at> gnu.org>, 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Thu, 3 Mar 2016 00:33:42 -0700
> 
> I've looked further into this. Comparing the profiles, there seems to be two 
> significant differences:
> 
> 1. ‘deliver_user_signal’ gets called constantly in the slow case, with many 
> calls to ‘x_catch_errors_with_handler’, ‘x_catch_errors_with_handler’, 
> ‘x_had_errors_p’, and ‘x_uncatch_errors’; there are few to no such calls in the 
> fast case.

Not sure this is related.

> 2. font_list_entities constantly calls xfont_list in the slow case, but not in 
> the fast case:

Most probably because in the "fast" case it finds the empty vector in
the cache, and thus avoids the costly call to the font driver's 'list'
method.

> Here is the new code:
> 
>     val = driver_list->driver->list (f, scratch_font_spec);
>     if (!NILP (val))
>       {
>         Lisp_Object copy = copy_font_spec (scratch_font_spec);
> 
>         val = Fvconcat (1, &val);
>         ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
>         XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));
>       }
> 
> The commit message didn't explain why it was a good thing to not add an empty 
> vector to the cache.

Yes, it did, see http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#32:
the empty vector was causing an error when creating a new frame via
emacsclient.

> I'm a bit stuck at this point. I'm not sure how 1 and 2 are connected. 
> Suggestions?

Like I said, I'm not sure 1 is of any immediate concern to us.

The only way forward I see is to reproduce the problem described in
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#26 and in
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#29, in the "fast"
case (i.e. without the changes that fixed that bug), and then look for
a different solution for that problem.  If you can reproduce that,
please step through x_default_font_parameter and x_default_parameter,
and tell how does the empty vector in the cache come into play.  Then
we can try some other solution for that.

Thanks.

Message #62 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 20 Jul 2016 17:26:02 -0400

[Message part 1 (text/plain, inline)]

On 2016-03-03 15:27, Eli Zaretskii wrote:
> The only way forward I see is to reproduce the problem described in
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#26 and in
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#29, in the "fast"
> case (i.e. without the changes that fixed that bug), and then look for
> a different solution for that problem.  If you can reproduce that,
> please step through x_default_font_parameter and x_default_parameter,
> and tell how does the empty vector in the cache come into play.  Then
> we can try some other solution for that.

Belated thanks. I will try to look into this.

I've made more progress towards making this reproducible, too. My current repro requires XITS Math (https://github.com/khaledhosny/xits-math/blob/master/xits-mathbold.otf), and Segoe UI Emoji.  Do you have a machine with that font? If so, could you try the following?

    emacs-24.5 -Q -mm -l 21028.el 21028 -f prettify-symbols-mode

I've attached 21028.el and 21028. When I run this command, each full redisplay takes about 5 seconds; same for pointer motion.

Clément.

[21028.el (text/x-emacs-lisp, attachment)]

[21028 (text/plain, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #65 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 21 Jul 2016 17:21:03 +0300

> Cc: dmantipov <at> yandex.ru, 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Wed, 20 Jul 2016 17:26:02 -0400
> 
> I've made more progress towards making this reproducible, too. My current repro requires XITS Math (https://github.com/khaledhosny/xits-math/blob/master/xits-mathbold.otf), and Segoe UI Emoji.  Do you have a machine with that font? If so, could you try the following?
> 
>     emacs-24.5 -Q -mm -l 21028.el 21028 -f prettify-symbols-mode
> 
> I've attached 21028.el and 21028. When I run this command, each full redisplay takes about 5 seconds; same for pointer motion.

Thanks.  I think the evidence and the profiling data you presented
quite clearly indicate where the problem is.  So I think the efforts
should now be directed towards reproducing bug#17125 and then solving
it in some other way that doesn't suffer from performance problems.

Message #68 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 21 Jul 2016 10:33:46 -0400

[Message part 1 (text/plain, inline)]

On 2016-07-21 10:21, Eli Zaretskii wrote:
>> Cc: dmantipov <at> yandex.ru, 21028 <at> debbugs.gnu.org
>> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
>> Date: Wed, 20 Jul 2016 17:26:02 -0400
>>
>> I've made more progress towards making this reproducible, too. My current repro requires XITS Math (https://github.com/khaledhosny/xits-math/blob/master/xits-mathbold.otf), and Segoe UI Emoji.  Do you have a machine with that font? If so, could you try the following?
>>
>>     emacs-24.5 -Q -mm -l 21028.el 21028 -f prettify-symbols-mode
>>
>> I've attached 21028.el and 21028. When I run this command, each full redisplay takes about 5 seconds; same for pointer motion.
> 
> Thanks.  I think the evidence and the profiling data you presented
> quite clearly indicate where the problem is.  So I think the efforts
> should now be directed towards reproducing bug#17125 and then solving
> it in some other way that doesn't suffer from performance problems.

Ok; thanks very much for clarifying!

[signature.asc (application/pgp-signature, attachment)]

Message #71 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sat, 23 Jul 2016 16:50:04 -0400

[Message part 1 (text/plain, inline)]

On 2016-07-21 10:21, Eli Zaretskii wrote:
>> Cc: dmantipov <at> yandex.ru, 21028 <at> debbugs.gnu.org
>> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
>> Date: Wed, 20 Jul 2016 17:26:02 -0400
>>
>> I've made more progress towards making this reproducible, too. My current repro requires XITS Math (https://github.com/khaledhosny/xits-math/blob/master/xits-mathbold.otf), and Segoe UI Emoji.  Do you have a machine with that font? If so, could you try the following?
>>
>>     emacs-24.5 -Q -mm -l 21028.el 21028 -f prettify-symbols-mode
>>
>> I've attached 21028.el and 21028. When I run this command, each full redisplay takes about 5 seconds; same for pointer motion.
> 
> Thanks.  I think the evidence and the profiling data you presented
> quite clearly indicate where the problem is.  So I think the efforts
> should now be directed towards reproducing bug#17125 and then solving
> it in some other way that doesn't suffer from performance problems.

I have tried the recipe posted in 17125, but I cannot reproduce it (despite having reverted af1a69f4d17a482c359d98c00ef86fac835b5fac).
Dmitry, is this the right recipe?

  $ emacs -no-site-file -no-init-file --daemon=test

  Warning: due to a long standing Gtk+ bug
  http://bugzilla.gnome.org/show_bug.cgi?id=85715
  Emacs might crash when run in daemon mode and the X11 connection is unexpectedly lost.
  Using an Emacs configured with --with-x-toolkit=lucid does not have this problem.
  Starting Emacs daemon.

  $ emacsclient -nc -s test
  (exit with C-x 5 0)

  $ emacsclient -nc -s test
  (exit with C-x 5 0)

  $ emacsclient -nc -s test
  (exit with C-x 5 0)

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #74 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Jason Gross <jgross <at> mit.edu>
To: 21028 <at> debbugs.gnu.org
Subject: Re: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014)
Date: Tue, 04 Oct 2016 18:56:31 +0000

[Message part 1 (text/plain, inline)]

I ran into this bug (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028)
via prettify-symbols-mode, and applying Clément's patch fixed the issue.

I'm running GNU Emacs 24.5.1 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2016-04-17 on lgw01-04, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11803000
System Description: Ubuntu 16.04.1 LTS

-Jason

[Message part 2 (text/html, inline)]

Message #77 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org, dmantipov <at> yandex.ru
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Tue, 4 Oct 2016 18:17:33 -0400

[Message part 1 (text/plain, inline)]

On 2016-07-23 16:50, Clément Pit--Claudel wrote:
> On 2016-07-21 10:21, Eli Zaretskii wrote:
>>> […]
>> Thanks.  I think the evidence and the profiling data you presented
>> quite clearly indicate where the problem is.  So I think the efforts
>> should now be directed towards reproducing bug#17125 and then solving
>> it in some other way that doesn't suffer from performance problems.
> 
> I have tried the recipe posted in 17125, but I cannot reproduce it (despite having reverted af1a69f4d17a482c359d98c00ef86fac835b5fac).
> Dmitry, is this the right recipe?
> 
>   $ emacs -no-site-file -no-init-file --daemon=test
> 
>   Warning: due to a long standing Gtk+ bug
>   http://bugzilla.gnome.org/show_bug.cgi?id=85715
>   Emacs might crash when run in daemon mode and the X11 connection is unexpectedly lost.
>   Using an Emacs configured with --with-x-toolkit=lucid does not have this problem.
>   Starting Emacs daemon.
> 
>   $ emacsclient -nc -s test
>   (exit with C-x 5 0)
> 
>   $ emacsclient -nc -s test
>   (exit with C-x 5 0)
> 
>   $ emacsclient -nc -s test
>   (exit with C-x 5 0)

Ping on this?  I'd love to help fix this bug, but I can reproduce it.
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #80 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 26 Oct 2016 13:15:14 -0400

[Message part 1 (text/plain, inline)]

Note: inhibit-compact-font-caches doesn't solve this problem (was it supposed to?).  Reverting the commit in the title causes other crashes (24790).

On 2016-10-04 18:17, Clément Pit--Claudel wrote:
> On 2016-07-23 16:50, Clément Pit--Claudel wrote:
>> On 2016-07-21 10:21, Eli Zaretskii wrote:
>>>> […]
>>> Thanks.  I think the evidence and the profiling data you presented
>>> quite clearly indicate where the problem is.  So I think the efforts
>>> should now be directed towards reproducing bug#17125 and then solving
>>> it in some other way that doesn't suffer from performance problems.
>>
>> I have tried the recipe posted in 17125, but I cannot reproduce it (despite having reverted af1a69f4d17a482c359d98c00ef86fac835b5fac).
>> Dmitry, is this the right recipe?
>>
>>   $ emacs -no-site-file -no-init-file --daemon=test
>>
>>   Warning: due to a long standing Gtk+ bug
>>   http://bugzilla.gnome.org/show_bug.cgi?id=85715
>>   Emacs might crash when run in daemon mode and the X11 connection is unexpectedly lost.
>>   Using an Emacs configured with --with-x-toolkit=lucid does not have this problem.
>>   Starting Emacs daemon.
>>
>>   $ emacsclient -nc -s test
>>   (exit with C-x 5 0)
>>
>>   $ emacsclient -nc -s test
>>   (exit with C-x 5 0)
>>
>>   $ emacsclient -nc -s test
>>   (exit with C-x 5 0)
> 
> Ping on this?  I'd love to help fix this bug, but I can reproduce it.
> Clément.
> 

[signature.asc (application/pgp-signature, attachment)]

Message #83 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 27 Oct 2016 17:38:50 +0300

> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Wed, 26 Oct 2016 13:15:14 -0400
> 
> Note: inhibit-compact-font-caches doesn't solve this problem (was it supposed to?).  Reverting the commit in the title causes other crashes (24790).

The problem with reverting that commit is that you reverted too much.
The part of the commit shown below cannot be reverted, because the
code it fixed was inserting invalid data types into the font cache,
which then caused rare segfaults such as the one you had recently.

So put back that part of the changeset, and if your performance
problems are still solved by the rest, we might just revert the other
part on master, because no one seems to be able to reproduce the
problem with empty vectors in the font cache, which that part was
supposed to fix.


diff --git a/src/font.c b/src/font.c
index b49664b..e99141b 100644
--- a/src/font.c
+++ b/src/font.c
@@ -2804,7 +2803,6 @@ font_matching_entity (struct frame *f, Lisp_Object *attrs, Lisp_Object spec)
 	&& (NILP (ftype) || EQ (driver_list->driver->type, ftype)))
       {
 	Lisp_Object cache = font_get_cache (f, driver_list->driver);
-	Lisp_Object copy;
 
 	ASET (work, FONT_TYPE_INDEX, driver_list->driver->type);
 	entity = assoc_no_quit (work, XCDR (cache));
@@ -2813,9 +2811,14 @@ font_matching_entity (struct frame *f, Lisp_Object *attrs, Lisp_Object spec)
 	else
 	  {
 	    entity = driver_list->driver->match (f, work);
-	    copy = copy_font_spec (work);
-	    ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
-	    XSETCDR (cache, Fcons (Fcons (copy, entity), XCDR (cache)));
+	    if (!NILP (entity))
+	      {
+		Lisp_Object copy = copy_font_spec (work);
+		Lisp_Object match = Fvector (1, &entity);
+
+		ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
+		XSETCDR (cache, Fcons (Fcons (copy, match), XCDR (cache)));
+	      }
 	  }
 	if (! NILP (entity))
 	  break;

Message #86 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 27 Oct 2016 11:29:45 -0400

[Message part 1 (text/plain, inline)]

On 2016-10-27 10:38, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
>> Date: Wed, 26 Oct 2016 13:15:14 -0400
>>
>> Note: inhibit-compact-font-caches doesn't solve this problem (was it supposed to?).  Reverting the commit in the title causes other crashes (24790).
> 
> The problem with reverting that commit is that you reverted too much.
> The part of the commit shown below cannot be reverted, because the
> code it fixed was inserting invalid data types into the font cache,
> which then caused rare segfaults such as the one you had recently.

Indeed! Thanks for explaining Eli :) Based on your instructions I:

* Started from a clean checkout
* Reverted af1a69f4d17a482c359d98c00ef86fac835b5fac
* Applied the patch below

And indeed:

* Performance is great (bug #21028 is gone)
* I don't observe segfaults anymore (I can't reproduce bug #24790)

> So put back that part of the changeset, and if your performance
> problems are still solved by the rest, we might just revert the other
> part on master, because no one seems to be able to reproduce the
> problem with empty vectors in the font cache, which that part was
> supposed to fix.

That would be great.

Thanks!
Clément.

> diff --git a/src/font.c b/src/font.c
> index b49664b..e99141b 100644
> --- a/src/font.c
> +++ b/src/font.c
> @@ -2804,7 +2803,6 @@ font_matching_entity (struct frame *f, Lisp_Object *attrs, Lisp_Object spec)
>  	&& (NILP (ftype) || EQ (driver_list->driver->type, ftype)))
>        {
>  	Lisp_Object cache = font_get_cache (f, driver_list->driver);
> -	Lisp_Object copy;
>  
>  	ASET (work, FONT_TYPE_INDEX, driver_list->driver->type);
>  	entity = assoc_no_quit (work, XCDR (cache));
> @@ -2813,9 +2811,14 @@ font_matching_entity (struct frame *f, Lisp_Object *attrs, Lisp_Object spec)
>  	else
>  	  {
>  	    entity = driver_list->driver->match (f, work);
> -	    copy = copy_font_spec (work);
> -	    ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
> -	    XSETCDR (cache, Fcons (Fcons (copy, entity), XCDR (cache)));
> +	    if (!NILP (entity))
> +	      {
> +		Lisp_Object copy = copy_font_spec (work);
> +		Lisp_Object match = Fvector (1, &entity);
> +
> +		ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
> +		XSETCDR (cache, Fcons (Fcons (copy, match), XCDR (cache)));
> +	      }
>  	  }
>  	if (! NILP (entity))
>  	  break;
> 

[signature.asc (application/pgp-signature, attachment)]

Message #89 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 27 Oct 2016 18:55:11 +0300

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Thu, 27 Oct 2016 11:29:45 -0400
> 
> * Started from a clean checkout
> * Reverted af1a69f4d17a482c359d98c00ef86fac835b5fac
> * Applied the patch below
> 
> And indeed:
> 
> * Performance is great (bug #21028 is gone)
> * I don't observe segfaults anymore (I can't reproduce bug #24790)
> 
> > So put back that part of the changeset, and if your performance
> > problems are still solved by the rest, we might just revert the other
> > part on master, because no one seems to be able to reproduce the
> > problem with empty vectors in the font cache, which that part was
> > supposed to fix.
> 
> That would be great.

Come back in a month or two, and if there are no new issues, I will do
that.

Did you try the recipe described in

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#26

?  I tried, but couldn't see any errors, but I don't have a 7x13 font
on my system, so maybe the problem doesn't get trigger.

That's the only reason the other part of the patch was introduced, and
if it's no longer pertinent, we could go back.

Message #92 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 28 Oct 2016 10:23:36 -0400

[Message part 1 (text/plain, inline)]

On 2016-10-27 11:55, Eli Zaretskii wrote:
> Did you try the recipe described in
> 
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#26
> 
> ?  I tried, but couldn't see any errors, but I don't have a 7x13 font
> on my system, so maybe the problem doesn't get trigger.
> 
> That's the only reason the other part of the patch was introduced, and
> if it's no longer pertinent, we could go back.

I tried it, and I can't reproduce the error either.

[signature.asc (application/pgp-signature, attachment)]

Message #95 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 28 Oct 2016 17:33:43 +0300

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Fri, 28 Oct 2016 10:23:36 -0400
> 
> > Did you try the recipe described in
> > 
> >   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#26
> > 
> > ?  I tried, but couldn't see any errors, but I don't have a 7x13 font
> > on my system, so maybe the problem doesn't get trigger.
> > 
> > That's the only reason the other part of the patch was introduced, and
> > if it's no longer pertinent, we could go back.
> 
> I tried it, and I can't reproduce the error either.

Thanks.  So, like I said: let's revisit this in a month or two,
assuming no other problems surface.

Message #98 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: clement.pit <at> gmail.com
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 18 Nov 2016 10:55:33 +0200

> Date: Fri, 28 Oct 2016 17:33:43 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 21028 <at> debbugs.gnu.org
> 
> Thanks.  So, like I said: let's revisit this in a month or two,
> assuming no other problems surface.

Given my latest discoveries related to bug#24953, one other thing I'd
suggest to try, with an unpatched build from the current emacs-25
branch, is, instead of this:

  (set-fontset-font t 'unicode (font-spec :name "Arial") nil 'append)

to use this:

  (set-fontset-font t 'unicode
                    (font-spec :family "Arial" :registry "iso10646-1")
                    nil 'append)

IOW, explicitly tell that the font's registry is iso10646-1.  Does
this per chance resolve the performance problem?

Message #101 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 18 Dec 2016 12:33:10 -0500

[Message part 1 (text/plain, inline)]

On 2016-10-28 10:33, Eli Zaretskii wrote:
>> Cc: 21028 <at> debbugs.gnu.org
>> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
>> Date: Fri, 28 Oct 2016 10:23:36 -0400
>>
>>> Did you try the recipe described in
>>>
>>>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=17125#26
>>>
>>> ?  I tried, but couldn't see any errors, but I don't have a 7x13 font
>>> on my system, so maybe the problem doesn't get trigger.
>>>
>>> That's the only reason the other part of the patch was introduced, and
>>> if it's no longer pertinent, we could go back.
>>
>> I tried it, and I can't reproduce the error either.
> 
> Thanks.  So, like I said: let's revisit this in a month or two,
> assuming no other problems surface.

I've been using your patch since then, and everything works fine for me.  Is it a good time to revisit this?

Thanks!
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #104 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 18 Dec 2016 19:37:27 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Sun, 18 Dec 2016 12:33:10 -0500
> 
> > Thanks.  So, like I said: let's revisit this in a month or two,
> > assuming no other problems surface.
> 
> I've been using your patch since then, and everything works fine for me.  Is it a good time to revisit this?

What about my question here:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#98

Did you try that?  What were the results?

Message #107 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 18 Dec 2016 13:04:09 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-18 12:37, Eli Zaretskii wrote:
>> Cc: 21028 <at> debbugs.gnu.org
>> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
>> Date: Sun, 18 Dec 2016 12:33:10 -0500
>>
>>> Thanks.  So, like I said: let's revisit this in a month or two,
>>> assuming no other problems surface.
>>
>> I've been using your patch since then, and everything works fine for me.  Is it a good time to revisit this?
> 
> What about my question here:
> 
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#98
> 
> Did you try that?  What were the results?

Sorry, I missed that message.

It works!  But only for the original example (with Arial), AFAICT.  Do you know what the equivalent fix would be for the recipe given in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#62 ?

Thanks!
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #110 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 18 Dec 2016 21:52:37 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Sun, 18 Dec 2016 13:04:09 -0500
> 
> >   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#98
> > 
> > Did you try that?  What were the results?
> 
> Sorry, I missed that message.
> 
> It works!  But only for the original example (with Arial), AFAICT.  Do you know what the equivalent fix would be for the recipe given in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#62 ?

The same: use :family and :registry instead of :name.  What did you
try?

And why does that example use 'format', but without any format spec?
What am I missing here?

Message #113 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 18 Dec 2016 15:44:20 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-18 14:52, Eli Zaretskii wrote:
>> Cc: 21028 <at> debbugs.gnu.org
>> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
>> Date: Sun, 18 Dec 2016 13:04:09 -0500
>>
>>>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#98
>>>
>>> Did you try that?  What were the results?
>>
>> Sorry, I missed that message.
>>
>> It works!  But only for the original example (with Arial), AFAICT.  Do you know what the equivalent fix would be for the recipe given in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#62 ?
> 
> The same: use :family and :registry instead of :name.  What did you
> try?

I tried this, but it was didn't seem to make a difference:

(dolist (frame (frame-list))
  (dolist (fontset (fontset-list))
    (set-fontset-font fontset 'unicode (font-spec :family "Segoe UI Emoji" :registry "iso10646-1") frame 'append)
    (set-fontset-font fontset 'unicode (font-spec :family "XITS Math":registry "iso10646-1") frame 'append)))

(setq-default prettify-symbols-alist '(("=>" . 8658) (":=" . 8796)))

> And why does that example use 'format', but without any format spec?

It was a leftover from the code that I started from when trying to construct a minimal example.

Clément

[signature.asc (application/pgp-signature, attachment)]

Message #116 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 17:50:31 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Sun, 18 Dec 2016 15:44:20 -0500
> 
> >> It works!  But only for the original example (with Arial), AFAICT.  Do you know what the equivalent fix would be for the recipe given in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#62 ?
> > 
> > The same: use :family and :registry instead of :name.  What did you
> > try?
> 
> I tried this, but it was didn't seem to make a difference:
> 
> (dolist (frame (frame-list))
>   (dolist (fontset (fontset-list))
>     (set-fontset-font fontset 'unicode (font-spec :family "Segoe UI Emoji" :registry "iso10646-1") frame 'append)
>     (set-fontset-font fontset 'unicode (font-spec :family "XITS Math":registry "iso10646-1") frame 'append)))
                     ^^^^^^^
You want 'prepend, not 'append, I think.

But if using 'prepend' doesn't work, please explain what "didn't seem
to make a difference" means.  I tried the above with 'prepend', and
the Emoji characters are displayed using the Segoe font.

Also, using 'unicode' is gross, IMO; it's better to specify specific
ranges of characters you want to be displayed using the font family.

Message #119 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 11:25:28 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-19 10:50, Eli Zaretskii wrote:
> But if using 'prepend' doesn't work, please explain what "didn't seem
> to make a difference" means.  I tried the above with 'prepend', and
> the Emoji characters are displayed using the Segoe font.

Sorry for being unclear.  It didn't make a difference in terms of performance.  That is, after applying the changes, characters display just the same (which is good, since characters were already displayed in the right font), but performance is still bad (every redisplay takes about 5 seconds).

> Also, using 'unicode' is gross, IMO; it's better to specify specific
> ranges of characters you want to be displayed using the font family.

I'm not sure.  I want to use these two fonts as fallbacks — hence the append and the 'unicode range.

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #122 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 18:39:16 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Mon, 19 Dec 2016 11:25:28 -0500
> 
> Sorry for being unclear.  It didn't make a difference in terms of performance.  That is, after applying the changes, characters display just the same (which is good, since characters were already displayed in the right font), but performance is still bad (every redisplay takes about 5 seconds).

In "emacs -Q"?  If so, it didn't happen to me.  I tried both the Segoe
Symbol font (which nowadays includes all the Emoji, so the Emoji font
is not needed), and the Segoe Emoji one, and saw no slow-down.

So some other factor is at work here.

> > Also, using 'unicode' is gross, IMO; it's better to specify specific
> > ranges of characters you want to be displayed using the font family.
> 
> I'm not sure.  I want to use these two fonts as fallbacks — hence the append and the 'unicode range.

How can an Emoji font be a fallback for the entire range of Unicode
characters?  It's not what it's for.

Message #125 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 11:55:06 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-19 11:39, Eli Zaretskii wrote:
> In "emacs -Q"?  If so, it didn't happen to me.  I tried both the Segoe
> Symbol font (which nowadays includes all the Emoji, so the Emoji font
> is not needed), and the Segoe Emoji one, and saw no slow-down.

Let me try this again, then :)

> How can an Emoji font be a fallback for the entire range of Unicode
> characters?  It's not what it's for.

It's a fallback for everything that it covers; 'unicode was just a shorthand for "whatever the font includes".  It's a pain to have to determine exactly what the font covers, especially if the font is going to get updated to support a broader range in the future.

For Symbola, 'unicode was the intended meaning.

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #128 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 18:58:50 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Mon, 19 Dec 2016 11:55:06 -0500
> 
> On 2016-12-19 11:39, Eli Zaretskii wrote:
> > In "emacs -Q"?  If so, it didn't happen to me.  I tried both the Segoe
> > Symbol font (which nowadays includes all the Emoji, so the Emoji font
> > is not needed), and the Segoe Emoji one, and saw no slow-down.
> 
> Let me try this again, then :)

Thanks.

Also, do you see the slow-down when just displaying characters from
this font, or only in the prettify-symbols mode?

Message #131 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 12:00:50 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-19 11:39, Eli Zaretskii wrote:
> In "emacs -Q"?  If so, it didn't happen to me.  I tried both the Segoe
> Symbol font (which nowadays includes all the Emoji, so the Emoji font
> is not needed), and the Segoe Emoji one, and saw no slow-down.

The following command still results in a too-slow-to-be-usable Emacs for me.  I have attached the corresponding el files.

    /build/emacs/25/src/emacs -Q -l 21028.el 21028 -f prettify-symbols-mode

This is on the following system:

    In GNU Emacs 25.1.90.2 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
     of 2016-12-18 built on clem-w50-mint
    Repository revision: 3d94931cec5850fc4dc5ffc9f1bf88a291aa3a5b
    Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
    System Description:	Linux Mint 18 Sarah

Clément.

[21028 (text/plain, attachment)]

[21028.fixed.el (text/x-emacs-lisp, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #134 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 19 Dec 2016 12:13:31 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-19 11:58, Eli Zaretskii wrote:
> Also, do you see the slow-down when just displaying characters from
> this font, or only in the prettify-symbols mode?

Displaying symbols from that font is enough.  Given this, here's an updated recipe:

    /build/emacs/25/src/emacs -Q -l 21028.fixed.el 21028.fixed

Things are actually smoother using :family and :registry, but they are still unusable.  The problem is especially visible with vertical scrolling.

Here is a concrete experiment:

# With :name
$ time /build/emacs/25/src/emacs -Q -l 21028.el 21028.fixed --eval "(run-with-idle-timer 0 0 (lambda () (forward-line 25) (redisplay) (run-with-idle-timer 0 0 'kill-emacs)))"

real	0m3.500s
user	0m1.400s
sys	0m0.824s


# With :family and :registry
$ time /build/emacs/25/src/emacs -Q -l 21028.fixed.el 21028.fixed --eval "(run-with-idle-timer 0 0 (lambda () (forward-line 25) (redisplay) (run-with-idle-timer 0 0 'kill-emacs)))"

real	0m1.487s
user	0m0.752s
sys	0m0.188s


# With your patch from last month applied and the original one reverted:
$ time /build/emacs/master/src/emacs -Q -l 21028.fixed.el 21028.fixed --eval "(run-with-idle-timer 0 0 (lambda () (forward-line 25) (redisplay) (run-with-idle-timer 0 0 'kill-emacs)))"

real	0m0.486s
user	0m0.236s
sys	0m0.028s

Cheers,
Clément.

[21028.fixed (text/plain, attachment)]

[21028.fixed.el (text/x-emacs-lisp, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #137 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 22 Dec 2016 18:25:30 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Mon, 19 Dec 2016 12:13:31 -0500
> 
> Here is a concrete experiment:
> 
> # With :name
> $ time /build/emacs/25/src/emacs -Q -l 21028.el 21028.fixed --eval "(run-with-idle-timer 0 0 (lambda () (forward-line 25) (redisplay) (run-with-idle-timer 0 0 'kill-emacs)))"
> 
> real	0m3.500s
> user	0m1.400s
> sys	0m0.824s
> 
> 
> # With :family and :registry
> $ time /build/emacs/25/src/emacs -Q -l 21028.fixed.el 21028.fixed --eval "(run-with-idle-timer 0 0 (lambda () (forward-line 25) (redisplay) (run-with-idle-timer 0 0 'kill-emacs)))"
> 
> real	0m1.487s
> user	0m0.752s
> sys	0m0.188s
> 
> 
> # With your patch from last month applied and the original one reverted:
> $ time /build/emacs/master/src/emacs -Q -l 21028.fixed.el 21028.fixed --eval "(run-with-idle-timer 0 0 (lambda () (forward-line 25) (redisplay) (run-with-idle-timer 0 0 'kill-emacs)))"
> 
> real	0m0.486s
> user	0m0.236s
> sys	0m0.028s

I indeed see slow redisplay with your test files (but only after
changing 'append' to 'prepend', because otherwise Emacs selects a
different font for the arrows).  But when I looked at the display
closely, I found that the arrows are not displayed using Segoe UI
Emoji font, they are displayed by some other font.  In fact, the only
characters in that test which use the Segoe UI Emoji font are blanks!
Did you try using "C-u C-x =" to see which characters are displayed
using the Emoji font in your case?

I think that's because Segoe UI Emoji doesn't have glyphs for the
arrows, at least on my system.  So I modified your 21028.fixed.el
recipe as follows:

  (dolist (frame (frame-list))
    (dolist (fontset (fontset-list))
      (set-fontset-font fontset 'unicode (font-spec :family "Segoe UI Symbol" :registry "iso10646-1") frame 'prepend)
      (set-fontset-font fontset 'unicode (font-spec :family "XITS Math":registry "iso10646-1") frame 'prepend)))

  (setq-default prettify-symbols-alist '(("=>" . 8658) (":=" . 8796)))

and the problem went away: the arrows are displayed using Segoe UI
Symbol (which, at least on my system, includes both symbols and
Emoji), and there's no slowdown anymore.

Message #140 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 22 Dec 2016 11:49:09 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-22 11:25, Eli Zaretskii wrote:
> I indeed see slow redisplay with your test files (but only after
> changing 'append' to 'prepend', because otherwise Emacs selects a
> different font for the arrows).

I see, thanks; prepend is clearly not what I want, since I trying to set these fonts as a fallback.

> But when I looked at the display
> closely, I found that the arrows are not displayed using Segoe UI
> Emoji font, they are displayed by some other font.  In fact, the only
> characters in that test which use the Segoe UI Emoji font are blanks!
> Did you try using "C-u C-x =" to see which characters are displayed
> using the Emoji font in your case?

I did; append works fine here. Everything is displayed with XITS Math, as expected.  prepend uses Segoe or XITS for all text, which I don't want.  It's good for testing, though, since I guess that otherwise your Emacs picks other fonts that are available on your machine.

> I think that's because Segoe UI Emoji doesn't have glyphs for the
> arrows, at least on my system.  So I modified your 21028.fixed.el
> recipe as follows:

Thanks, but I'm not sure I see why you're suggesting this :/ If you see the slowdown, then it seems that managed to reproduce the problem, right?  Is there anything preventing us from applying the fix that you described last month?

The example only shows arrows because I tried to make it minimal; my bug report is about the fact that, when I set up Emacs to display characters the way I want them, it gets slow.  I readily admit that changing the font set-up may make the problem go away, but it's not a satisfactory solution (I still get support requests from users of my packages about degraded performance due to prettify-symbols-mode, and it's hard to tell them to use different fonts).

Recall that the original bug report didn't even use symbols: I first found out about this when trying to edit text written in Chinese.  I ran into the problem regularly in various settings since then, until I reverted the faulty commit (and later reapplied the safe part of it that you isolated).  Since then I've never run into this issue.  I've also confirmed on other users' laptops that reverting the faulty commit fixes the issue.

The fixes that you suggested to my minimal example do work around the issue in that case (another work around for me is to just remove the Segoe UI line), but these fixes are not applicable to my full font configuration.  I can post my full configuration if you want to suggest improvements that would work around the issue in my specific case (alternatively, it's available here: https://github.com/cpitclaudel/.emacs.d/blob/master/init/fonts.el), but this seems like a waste of your time: there's no doubt that others will run into the same issues (indeed, they already do), and fixing their configurations one by one isn't practical (assuming we even hear about them).

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #143 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pit <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 22 Dec 2016 20:04:35 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit <at> gmail.com>
> Date: Thu, 22 Dec 2016 11:49:09 -0500
> 
> I did; append works fine here. Everything is displayed with XITS Math, as expected.  prepend uses Segoe or XITS for all text, which I don't want.  It's good for testing, though, since I guess that otherwise your Emacs picks other fonts that are available on your machine.

Yes, that's why I used that.  I have Symbola installed, and it is used
for all those characters unless I use prepend, and the problem doesn't
happen.

> If you see the slowdown, then it seems that managed to reproduce the problem, right?

No, not really.  It doesn't seem useful to define a font that is only
used for blanks and can support no other characters in the buffer,
because that probably means Emacs is desperately trying to use it and
fails, which can explain the slowdown.

> Is there anything preventing us from applying the fix that you described last month?

You have to convince me it's needed ;-)

Is the other font a necessary part of this problem?  IOW, if you only
setup the fontset with the Segoe UI Emoji, does the problem still
happen?  I didn't try to install the other font, but if doing so is
necessary, I will try that.

> The example only shows arrows because I tried to make it minimal; my bug report is about the fact that, when I set up Emacs to display characters the way I want them, it gets slow.  I readily admit that changing the font set-up may make the problem go away, but it's not a satisfactory solution (I still get support requests from users of my packages about degraded performance due to prettify-symbols-mode, and it's hard to tell them to use different fonts).

I agree that changing the font is not a satisfactory solution (unless
it turns out the font you've been using is simply broken in some
way).  But neither does it make sense to make changes in the code we
don't understand well enough when it turns out the offending font is
not used for any of the characters in the buffer.

> The fixes that you suggested to my minimal example do work around the issue in that case (another work around for me is to just remove the Segoe UI line), but these fixes are not applicable to my full font configuration.  I can post my full configuration if you want to suggest improvements that would work around the issue in my specific case (alternatively, it's available here: https://github.com/cpitclaudel/.emacs.d/blob/master/init/fonts.el), but this seems like a waste of your time: there's no doubt that others will run into the same issues (indeed, they already do), and fixing their configurations one by one isn't practical (assuming we even hear about them).

I would like to try and understand the problem.  For that I need a
minimal setup that will show the problem while still making sense,
i.e. when the characters in the buffer are displayed by the offending
font.  A setup with a font that is not actually used for display
doesn't seem to be sensible; if showing the problem with real
characters requires a somewhat more elaborate setup, please show it.

One aspect that might make this issue easier to analyze is not to use
'unicode as the target of the fontset, but instead specify range(s) of
characters where you want it used.  Then the prepend/append issue will
go away, and the differences between our systems wrt other fonts will
not get in the way of reproducing the problem.

Thanks.

Message #146 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pit <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 22 Dec 2016 14:08:38 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-22 13:04, Eli Zaretskii wrote:
> I would like to try and understand the problem.  For that I need a
> minimal setup that will show the problem while still making sense,
> i.e. when the characters in the buffer are displayed by the offending
> font.  A setup with a font that is not actually used for display
> doesn't seem to be sensible; if showing the problem with real
> characters requires a somewhat more elaborate setup, please show it.

Thanks.  I'll try to make a more reproducible example.
This might suffer some delays, though; I need to send my laptop for repairs.

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #149 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 9 Feb 2017 23:45:24 -0500

[Message part 1 (text/plain, inline)]

On 2016-12-22 14:08, Clément Pit--Claudel wrote:
> On 2016-12-22 13:04, Eli Zaretskii wrote:
>> I would like to try and understand the problem.  For that I need a
>> minimal setup that will show the problem while still making sense,
>> i.e. when the characters in the buffer are displayed by the offending
>> font.  A setup with a font that is not actually used for display
>> doesn't seem to be sensible; if showing the problem with real
>> characters requires a somewhat more elaborate setup, please show it.

Hi Eli,

Here's a minimal recipe :) I followed these steps:

1. Downloaded a copy of Linux Mint 18.1 at https://linuxmint.com/edition.php?id=226
2. Installed it in VirtualBox, without proprietary add-ons (I turned on 3D acceleration in VBox's settings)
3. Installed VirtualBox's guest additions
4. Downloaded Emacs 25.1 from http://ftpmirror.gnu.org/emacs/
5. Extracted it to ~/emacs/emacs-25.1
6. Installed dependencies: sudo apt install libmagickwand-dev libgtk-3-dev libxpm-dev libjpeg-dev libgif-dev libtiff-dev libgconf2-dev librsvg2-dev libselinux1-dev libgnutls-dev libxml2-dev libfreetype6-dev libm17n-dev libotf-dev libsystemd-dev libncurses-dev texinfo
7. Compiled emacs: ./configure; make -j12
8. Ran emacs and confirmed that it was working normally
9. Downloaded XITS Math from https://raw.githubusercontent.com/khaledhosny/xits-math/master/xits-math.otf and saved it to ~/.fonts
10. Updated the font cache using fc-cache
11. Ran the following command: src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\")))"
12. Confirmed that XITS math was in use:
             position: 10472 of 10500 (100%), column: 13
            character: ⇒ (displayed as ⇒) (codepoint 8658, #o20722, #x21d2)
    preferred charset: unicode (Unicode (ISO10646))
code point in charset: 0x21D2
               script: symbol
               syntax: . 	which means: punctuation
             category: .:Base, h:Korean, j:Japanese
             to input: type "C-x 8 RET 21d2" or "C-x 8 RET RIGHTWARDS DOUBLE ARROW"
          buffer code: #xE2 #x87 #x92
            file code: #xE2 #x87 #x92 (encoded by coding system utf-8-unix)
              display: by this font (glyph code)
    xft:-STIX-XITS Math-normal-normal-normal-*-13-*-*-*-*-0-iso10646-1 (#x448)
13. Ran a simple benchmark: time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
    real	0m5.807s
    user	0m1.064s
    sys	0m1.260s
14. Compared this to the same benchmark without XITS Math: time src/emacs -Q --eval "(progn (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
    real	0m0.439s
    user	0m0.236s
    sys	0m0.032s

This sequence of steps yields an unusably slow copy of Emacs.  The font setup in that last step ensures that Emacs falls back to XITS Math for all characters that Ubuntu Mono does not handle.  This is my usual setup for programming with proof assistants (Agda and Coq), as I want a consistent fallback for symbols not covered by Ubuntu Mono.

Let me know if this helps!  I'll be happy to share a copy of the VM, too, if needed.

Cheers and thanks for your help,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #152 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 12 Mar 2017 07:38:13 -0400

[Message part 1 (text/plain, inline)]

On 2017-02-09 23:45, Clément Pit--Claudel wrote:
> On 2016-12-22 13:04, Eli Zaretskii wrote:
>> I would like to try and understand the problem.  For that I need a
>> minimal setup that will show the problem while still making sense,
> 
> Here's a minimal recipe :) I followed these steps: […]

Hi Eli and bug-gnu-emacs,

Any news on this? Is there anything more than I can do to help fix this issue?

Thanks!
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #155 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 12 Mar 2017 17:49:56 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Sun, 12 Mar 2017 07:38:13 -0400
> 
> Any news on this?

Not yet, sorry.  Having to install an OS is a bug turn-off for me.  I
will try reproducing on my system with the font you mentioned.

> Is there anything more than I can do to help fix this issue?

If you can come up with a recipe that doesn't require a particular OS
and font back-end, it will help a lot.

Message #158 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 12 Mar 2017 13:24:11 -0400

[Message part 1 (text/plain, inline)]

On 2017-03-12 11:49, Eli Zaretskii wrote:
>> Cc: 21028 <at> debbugs.gnu.org
>> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
>> Date: Sun, 12 Mar 2017 07:38:13 -0400
>>
>> Any news on this?
> 
> Not yet, sorry.  Having to install an OS is a bug turn-off for me.  I
> will try reproducing on my system with the font you mentioned.

I see. What about downloading a pre-built virtual machine, or running a Vagrant script?  Would any of these work better?

>> Is there anything more than I can do to help fix this issue?
> 
> If you can come up with a recipe that doesn't require a particular OS
> and font back-end, it will help a lot.

The last steps of the recipe reproduce the problem reliably on my machine, assuming Ubuntu Mono (http://font.ubuntu.com/) and XITS Math (https://github.com/khaledhosny/xits-math) are installed:

$ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m1.503s
user	0m0.540s
sys	0m0.244s

$ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m0.473s
user	0m0.216s
sys	0m0.040s

# With your latest patch
$ time emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m0.382s
user	0m0.252s
sys	0m0.020s

In fact, on my machine, I get consistent results (unusably slow Emacs) regardless of whether I pick XITS Math, Symbola, or Latin modern Math (on all of 24.4, 24.5, and 25.1, but not on 24.3 of course), and your patch solves the problem perfectly in all of these cases (amusingly, on 24.3, 24.4, and 24.5, I also get the very tall lines if I use Latin Modern, but that problem is solved in 25; thanks again!).

Let me know if I can help further!  I can also give you remote access to a machine displaying the problem, if that helps.

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #161 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 12 Mar 2017 19:48:46 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Sun, 12 Mar 2017 13:24:11 -0400
> 
> $ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> real	0m1.503s
> user	0m0.540s
> sys	0m0.244s
> 
> $ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> real	0m0.473s
> user	0m0.216s
> sys	0m0.040s
> 
> # With your latest patch
> $ time emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> real	0m0.382s
> user	0m0.252s
> sys	0m0.020s
> 
> In fact, on my machine, I get consistent results (unusably slow Emacs) regardless of whether I pick XITS Math, Symbola, or Latin modern Math (on all of 24.4, 24.5, and 25.1, but not on 24.3 of course), and your patch solves the problem perfectly in all of these cases

So you are saying that this:

  $ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"

takes about 1.5 sec on your system, is that right?  (Note that I
replaced XITS Math with Symbola here.)  If so, is the Ubuntu Mono part
necessary for reproducing the slow display, i.e. if you remove that
part, do you still get 1.5 sec?

Message #164 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 12 Mar 2017 15:19:48 -0400

[Message part 1 (text/plain, inline)]

On 2017-03-12 13:48, Eli Zaretskii wrote:
> So you are saying that this:
> 
>   $ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> 
> takes about 1.5 sec on your system, is that right?  (Note that I
> replaced XITS Math with Symbola here.)

Correct.  Here are extra timings:

# 24.3: before revision af1a69f4d17a482c359d98c00ef86fac835b5fac
$ time 24.3/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m0.451s
user	0m0.236s
sys	0m0.028s

# 24.5: slow
$ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m1.305s
user	0m0.444s
sys	0m0.192s

# 25.1: slow
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m1.495s
user	0m0.508s
sys	0m0.240s

# master with your patches: fast
$ time master/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m0.364s
user	0m0.216s
sys	0m0.020s

> If so, is the Ubuntu Mono part necessary for reproducing the slow display,
> i.e. if you remove that part, do you still get 1.5 sec?

The Ubuntu Mono part is necessary: here are timings:

# With Ubuntu Mono: slow
$ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m1.312s
user	0m0.440s
sys	0m0.184s

# With just Symbola added: fast (C-u C-x = shows that Symbola is used)
$ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
real	0m0.358s
user	0m0.212s
sys	0m0.032s

It doesn't need to be specifically Ubuntu Mono, though:

# With Noto Sans: slow
$ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"

real	0m1.593s
user	0m0.504s
sys	0m0.236s

# With Fira Sans: slow
$ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Fira Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"

real	0m1.511s
user	0m0.520s
sys	0m0.248s

Hope this helps,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #167 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 13 Mar 2017 17:46:25 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Sun, 12 Mar 2017 15:19:48 -0400
> 
> > So you are saying that this:
> > 
> >   $ time src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> > 
> > takes about 1.5 sec on your system, is that right?  (Note that I
> > replaced XITS Math with Symbola here.)
> 
> Correct.  Here are extra timings:
> [...]
> # With Noto Sans: slow
> $ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> 
> real    0m1.593s
> user    0m0.504s
> sys     0m0.236s
> 
> # With Fira Sans: slow
> $ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Fira Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 500) (insert (make-string 20 8658) \"\n\") (run-with-idle-timer 0 nil #'kill-emacs)))"
> 
> real    0m1.511s
> user    0m0.520s
> sys     0m0.248s

I installed Fira Sans and Noto Sans, and changed your program to insert
more of these characters and actually display all the characters it
inserts (see below).  I see timings like the one below:

  time emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs)))"

  real    00h00m06.069s
  user    00h00m05.272s
  sys     00h00m00.702s

If I don't specify the "normal" font in the fontset, only Symbola, or
if I don't specify fonts at all (Symbola is used by default for the
character in question), then I get timings like the one below:

  time emacs -Q --eval "(progn (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs)))"

  real    00h00m02.039s
  user    00h00m01.606s
  sys     00h00m00.358s

The above timings are with an optimized build of the stock Emacs 25.2
RC2 codebase.

So now I have the following questions:

  . Why is your 1.5 sec timing deemed "unusably slow" vs 0.35 sec
    which is the normal speed?  My timings above seem to produce a
    similar 4-fold slowdown, but if I use the same session (with
    hundreds of U+21D2 symbols) interactively, the display is quite
    responsive, and 6 sec for scrolling 5000 lines doesn't sound
    "unusable" to me.  Are you sure the above program exhibits your
    real-life problem?  Is it possible we are chasing the wrong geese
    here?

  . Why are you setting up your fontset with 2 fonts and claim that
    both support the entire Unicode range (as opposed to adding to the
    fontset just one font for specific ranges of codepoints not
    covered well by the default font)?  What happens if you do this
    instead:

      time emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" '(#x21D0 . #x21DF) \"Symbola\" nil 'prepend) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs)))"

    I get the "fast" timing for this test, and AFAIU it should produce
    the exact same display as for the "slow" timing.  Why can't
    something like this be used in your real-life use cases?

  . Finally, what are the values of the following variables in your
    Emacs used for timings?  (I don't see them anywhere in this bug's
    discussions, apologies if I missed something.)

      system-configuration-options
      system-configuration-features

Bottom line: it is strange that you alone see these problems, and no
one else AFAIR complained about unusably slow prettify-symbols-mode
display.  I'd like to understand what is special about your setup
before we make any code changes based on your single use case.

Thanks.

Message #170 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: 21028 <at> debbugs.gnu.org
Subject: Slow font rendering in emacs
Date: Mon, 13 Mar 2017 16:54:01 +0100

Hi all,

I think I ran into the issue described here as well:  If I am not
careful with the way I configure my fonts, emacs becomes pretty slow at
rendering them.  With my old laptop, actually merely navigating the
cursor large files (like
<https://gitlab.mpi-sws.org/robbertkrebbers/coq-stdpp/raw/master/theories/list.v>)
already was noticeably luggish.  My new laptop is fast enough to make
that not noticeable, but when I actually process the file (it's a file
to be run by the Coq proof assistant, I am using the ProofGeneral emacs
plugin for that), then it moves through the file much slower when I set
up my fonts in the "wrong" way.

Here's the setup which makes things behave badly:

(dolist (ft (fontset-list))
  ; Main font
  (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
  ; Fallback font
  (set-fontset-font ft 'unicode (font-spec :name "DejaVu Sans Mono"
:size 11.0) nil 'append)
)

And here's the one I use instead, which behaves much better:

(dolist (ft (fontset-list))
  ; Main font
  (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
  ; Fallback font
  (set-fontset-font ft nil (font-spec :name "DejaVu Sans Mono" :size 11.0))
)

(For the record, my "Monospace" font is "Fira Sans Mono".)

Configuring fonts in emacs to get proper unicode rendering is already
fairly difficult; it'd be great if this bug could be fixed so that one
doesn't have to care both about performance and about selecting the
right glyph for the right character.

Kind regards,
Ralf

Message #173 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 13 Mar 2017 12:36:25 -0400

On 2017-03-13 11:46, Eli Zaretskii wrote:
> I installed Fira Sans and Noto Sans, and changed your program to insert
> more of these characters and actually display all the characters it
> inserts (see below).

Thanks, that's a much better repro than mine :) Here are my timings:

# Stock 24.5
12:01:26 /build/emacs
$ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	1m1.862s
user	0m18.644s
sys	0m15.972s

# Stock 25.1
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	1m4.705s
user	0m18.296s
sys	0m18.444s

# Master with your patch
$ time master/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	0m1.207s
user	0m0.928s
sys	0m0.028s

> If I don't specify the "normal" font in the fontset, only Symbola, or
> if I don't specify fonts at all (Symbola is used by default for the
> character in question), then I get timings like the one below:

I get similar timings.

>   . Why is your 1.5 sec timing deemed "unusably slow" vs 0.35 sec
>     which is the normal speed?  

Because my repro wasn't very good, apparently :) As you see above, your tests takes 1 minute without your patch, and 1 second with it.

>     If I use the same session (with
>     hundreds of U+21D2 symbols) interactively, the display is quite
>     responsive, and 6 sec for scrolling 5000 lines doesn't sound
>     "unusable" to me.

Some redisplays cycles takes as much as a few seconds. That's enough to make it impossible to use Emacs.  In this concrete example, pressing the up arow introduces a half-second lag.  Pressing M-x doesn't show anything for close to .5 seconds.  Keeping the down arrow pressed for 1 second stops redisplay until I release the key, and then Emacs stays frozen for tens of seconds before redisplaying anything.

>     Are you sure the above program exhibits your
>     real-life problem?  Is it possible we are chasing the wrong geese
>     here?

I this it does exhibit my problem, as we saw above.

>   . Why are you setting up your fontset with 2 fonts and claim that
>     both support the entire Unicode range (as opposed to adding to the
>     fontset just one font for specific ranges of codepoints not
>     covered well by the default font)?  

Because you asked me for a minimal repro for my bug :)  In real life, I have more than 6 fonts, all covering different subsets (I linked to my own config in a previous post in this thread).  I don't know exactly what Ubuntu Mono covers, and in fact that changes from time to time, so I just want it to display everything it can (hence the first invocation with 'unicode). Then I want to use Segoe UI for emoji, so I use a range for that.  And then I then want Symbola to be used for any symbol that it covers that are not covered by the previous two fonts — hence the second call with 'unicode.  I then have a few fonts for various CJK ranges, but even there I don't know what exact range each font covers.

But I guess it boils down to: "⇒" is not the only symbol I use, and I don't want to encode all ranges supported by Symbola explicitly in my emacs config (nor, of course, all the CJK ranges supported by the barious fonts I use for east-asian scripts).  As I noted previously, remember that the original bug report didn't even use symbols (I ran into this when trying to edit text written in Chinese in Emacs.

>   . Finally, what are the values of the following variables in your
>     Emacs used for timings?  (I don't see them anywhere in this bug's
>     discussions, apologies if I missed something.)
> 
>       system-configuration-options
>       system-configuration-features

In both, system-configuration-options is ""
system-configuration-features is "XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11" in 25.1 (slow), and "XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 LIBSYSTEMD" in master with your patch (fast).

> Bottom line: it is strange that you alone see these problems, and no
> one else AFAIR complained about unusably slow prettify-symbols-mode
> display.

I'm not alone.  At least one other person posted to this bug list (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#74) and confirmed that the patch fixed their problem. The problem gets regularly mentioned elsewhere (when I opened this bug I linked to https://github.com/purcell/emacs.d/issues/273 "when I first try to input some chinese characters, it's very slow to moving [...] especially use C-n, C-p， emacs will hang for about 1~2 seconds."; a more recent example is at https://sympa.inria.fr/sympa/arc/coq-club/2017-03/msg00050.html "Appending to the 'unicode list makes emacs unbearably slow.").  Additionally, as the maintainer of company-coq, I've had multiple people tell me that it was just too slow to use, only to realize that turning off prettify-symbols-mode was enough to make everything snappy again for them.

Clément.

Message #176 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Mon, 13 Mar 2017 19:05:47 +0200

> From: Ralf Jung <post <at> ralfj.de>
> Date: Mon, 13 Mar 2017 16:54:01 +0100
> 
> I think I ran into the issue described here as well:  If I am not
> careful with the way I configure my fonts, emacs becomes pretty slow at
> rendering them.  With my old laptop, actually merely navigating the
> cursor large files (like
> <https://gitlab.mpi-sws.org/robbertkrebbers/coq-stdpp/raw/master/theories/list.v>)
> already was noticeably luggish.

I don't see any sluggish-ness in "emacs -Q" here, FWIW.

Do you have any measurements that would show the slowdown in numbers?
Also, what is your value of system-configuration-options and
system-configuration-features?  And which version of Emacs is that?

> Here's the setup which makes things behave badly:
> 
> (dolist (ft (fontset-list))
>   ; Main font
>   (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
>   ; Fallback font
>   (set-fontset-font ft 'unicode (font-spec :name "DejaVu Sans Mono"
> :size 11.0) nil 'append)
> )
> 
> And here's the one I use instead, which behaves much better:
> 
> (dolist (ft (fontset-list))
>   ; Main font
>   (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
>   ; Fallback font
>   (set-fontset-font ft nil (font-spec :name "DejaVu Sans Mono" :size 11.0))
> )
> 
> (For the record, my "Monospace" font is "Fira Sans Mono".)

What happens if you don't make any of the above customizations?

Also, do you have the Symbola font installed?  If not, can you install
it and see if that makes things better?

> Configuring fonts in emacs to get proper unicode rendering is already
> fairly difficult

Actually, the default fontset is supposed to "just work", so I'd be
interested to hear why you needed to customize that.

> it'd be great if this bug could be fixed

I'm trying.  But it's hard, since for now I cannot even reproduce it.

Thanks.

Message #179 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 13 Mar 2017 19:22:45 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Mon, 13 Mar 2017 12:36:25 -0400
> 
> On 2017-03-13 11:46, Eli Zaretskii wrote:
> > I installed Fira Sans and Noto Sans, and changed your program to insert
> > more of these characters and actually display all the characters it
> > inserts (see below).
> 
> Thanks, that's a much better repro than mine :) Here are my timings:
> 
> # Stock 24.5
> 12:01:26 /build/emacs
> $ time 24.5/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
> real	1m1.862s
> user	0m18.644s
> sys	0m15.972s
> 
> # Stock 25.1
> $ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
> real	1m4.705s
> user	0m18.296s
> sys	0m18.444s
> 
> # Master with your patch
> $ time master/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
> real	0m1.207s
> user	0m0.928s
> sys	0m0.028s

I see nothing of the kind, as you saw.  With the same fonts as you
use.  So it's definitely not a problem with the fonts.  Which is
strange, since the profiles you provided seem to indicate a
font-related problem.

> > If I don't specify the "normal" font in the fontset, only Symbola, or
> > if I don't specify fonts at all (Symbola is used by default for the
> > character in question), then I get timings like the one below:
> 
> I get similar timings.

So why not use Symbola (i.e. the default fontset setup)?

>In real life, I have more than 6 fonts, all covering different subsets (I linked to my own config in a previous post in this thread).  I don't know exactly what Ubuntu Mono covers, and in fact that changes from time to time, so I just want it to display everything it can (hence the first invocation with 'unicode). Then I want to use Segoe UI for emoji, so I use a range for that.  And then I then want Symbola to be used for any symbol that it covers that are not covered by the previous two fonts — hence the second call with 'unicode.  I then have a few fonts for various CJK ranges, but even there I don't know what exact range each font covers.

Symbola supports emoji in the default fontset, so neither Segoe UI
nort Symbola should be needed in your setup.  As for Ubuntu Mono,
isn't that the default monospaced font?

In short, are you sure you really need those customizations?  You may
be creating your own problems.

> But I guess it boils down to: "⇒" is not the only symbol I use, and I don't want to encode all ranges supported by Symbola explicitly in my emacs config (nor, of course, all the CJK ranges supported by the barious fonts I use for east-asian scripts).  As I noted previously, remember that the original bug report didn't even use symbols (I ran into this when trying to edit text written in Chinese in Emacs.

The default fontset should already provide you with the fonts that
display all of these, including CJK.  If you use the default, what
characters aren't displayed well enough, or at all?

Is this the complete fontset setup you are using:

  (let ((cjk-font "WenQuanYi Micro Hei Mono")
	(fontset t)) ; "fontset-default"
    (set-fontset-font t (cons ?≔ ?≔) "FreeSerif" nil 'prepend)
    (set-fontset-font fontset 'unicode (font-spec :name "Consolas") nil 'append)
    (set-fontset-font fontset 'unicode (font-spec :name "Symbola") nil 'append)
    (set-fontset-font fontset '(#x4E00 . #x9FFF) (font-spec :name cjk-font))
    (set-fontset-font fontset '(#x3400 . #x4DFF) (font-spec :name cjk-font))
    (set-fontset-font fontset '(#x20000 . #x2A6DF) (font-spec :name cjk-font))
    (set-fontset-font fontset '(#xF900 . #xFAFF) (font-spec :name cjk-font))
    (set-fontset-font fontset '(#x2F800 . #x2FA1F) (font-spec :name cjk-font)))

If it is, I will try to reproduce using this fontset.

Message #182 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Mon, 13 Mar 2017 19:12:10 +0100

Hi Eli,

Wow, that was a quick reply!

>> From: Ralf Jung <post <at> ralfj.de>
>> Date: Mon, 13 Mar 2017 16:54:01 +0100
>>
>> I think I ran into the issue described here as well:  If I am not
>> careful with the way I configure my fonts, emacs becomes pretty slow at
>> rendering them.  With my old laptop, actually merely navigating the
>> cursor large files (like
>> <https://gitlab.mpi-sws.org/robbertkrebbers/coq-stdpp/raw/master/theories/list.v>)
>> already was noticeably luggish.
> 
> I don't see any sluggish-ness in "emacs -Q" here, FWIW.
> 
> Do you have any measurements that would show the slowdown in numbers?

I know very little about emacs, so I wouldn't know how to make any
measurements.  Also, I don't have access to my old machine any more.
I may try adapting some of the timing scripts that you and Clément were
using, and see if I can quantify anything.

> Also, what is your value of system-configuration-options and
> system-configuration-features?  And which version of Emacs is that?

Sorry, I have no idea how to read those variables and a quick google
search was not very fruitful.
This is emacs 24.5+1-8 as packaged by Debian.

> What happens if you don't make any of the above customizations?

Emacs is fine in terms of speed, but it uses the wrong font for
characters not supported by Fira Sans Mono.
For example, for ∃ it picks "STIX", which seems to have broken metrics
because the character is higher than all the others (it extends ~20%
below the bottom bar, breaking the height of the entire line).  For ∗ it
picks DejaVu Math TeX Gyre, which is too wide.  For ⋅ it picks EB
Garamond SC which is way too small.

Opening the same file on the same system in Kate or gedit results in the
expected rendering -- all characters fit the grid; unfortunately I don't
know how to figure out which fonts they picked for what.  I did no
configuration whatsoever to achieve this, besides installing Fira Sans.
This was the case even before I installed Symbola.

> Also, do you have the Symbola font installed?  If not, can you install
> it and see if that makes things better?

I do have it installed.  emacs doesn't seem to pick it per default.
The reason I have it installed is that some characters (like ∖) are
supported neither by Fira Sans Mono nor by DejaVu Mono, so then I made
it fall back to Symbola.  These are very few rarely used characters, so
the fact that the widths don't match up is much less of a problem.  My
full configuration is:


;; Fonts
(dolist (ft (fontset-list))
  ; Main font
  (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
  ; Fallback font
  ; Appending to the 'unicode list makes emacs unbearably slow.
  ;(set-fontset-font ft 'unicode (font-spec :name "DejaVu Sans Mono"
:size 11.0) nil 'append)
  (set-fontset-font ft nil (font-spec :name "DejaVu Sans Mono" :size 11.0))
)
; Fallback-fallback font
; If we 'append this to all fontsets, it picks Symbola even for some
cases where DejaVu could
; be used. Adding it only to the "t" table makes it Do The Right Thing (TM).
(set-fontset-font t nil (font-spec :name "Symbola" :size 11.0))

And just in case you wonder -- no, I have no idea what I am doing here.
I arrived at that configuration after several hours of random, fairly
unguided modifications to it until all my test characters picked the
font I wanted:
→  Fira Sans Mono (even though DejaVu can also do it, but the arrow in
   Fira looks much better IMHO)
∃  DejaVu Mono (not supported by Fira, it seems)
∀  DejaVu Mono (not supported by Fira, it seems)
∗  DejaVu Mono (not supported by Fira, it seems)
⋅  DejaVu Mono (not supported by Fira, it seems; the one in Symbola
   is *tiny*)
∖  Symbola (for lack of an alternative)

I shortened the configuration in my first mail because I was focusing on
the performance issue.  While I do consider it a bug that emacs picks
the wrong fonts per default, shouldn't emacs be reasonably fast to use
even if I do some manual configuration?  The bug at hand IMHO wouldn't
be fixed if the default emacs configuration was making better choices
for the fonts (though of course I wouldn't be affected by it anymore;
but then, neither am I now since I found a way to work around it).

>> Configuring fonts in emacs to get proper unicode rendering is already
>> fairly difficult
> 
> Actually, the default fontset is supposed to "just work", so I'd be
> interested to hear why you needed to customize that.

> I'm trying.  But it's hard, since for now I cannot even reproduce it.

I acknowledge that it'd be much easier if you could reproduce the
problem.  That said, there's a patch that Clément said works.  Honestly
I am a little puzzled why you are so hesitant to apply a patch that you
wrote, and that reportedly fixes the problem.  Do you expect that patch
to have negative side-effects?  Of course this is ultimately up to you,
I am just expressing my surprise here.

Kind regards,
Ralf

Message #185 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 13 Mar 2017 15:04:24 -0400

On 2017-03-13 13:22, Eli Zaretskii wrote:
>> Cc: 21028 <at> debbugs.gnu.org From: Clément Pit--Claudel
>> <clement.pitclaudel <at> live.com> Date: Mon, 13 Mar 2017 12:36:25
>> -0400
>> 
>> On 2017-03-13 11:46, Eli Zaretskii wrote:
>>> I installed Fira Sans and Noto Sans, and changed your program to
>>> insert more of these characters and actually display all the
>>> characters it inserts (see below).
>> 
>> Thanks, that's a much better repro than mine :) Here are my
>> timings:
>> 
>> # Stock 24.5 12:01:26 /build/emacs $ time 24.5/src/emacs -Q --eval
>> "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto
>> Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode
>> \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20
>> 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case
>> nil (while t (scroll-up) (sit-for 0)) (error nil))
>> (run-with-idle-timer 0 nil #'kill-emacs))" real	1m1.862s user
>> 0m18.644s sys	0m15.972s
>> 
>> # Stock 25.1 $ time 25.1/src/emacs -Q --eval "(progn
>> (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil)
>> (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil
>> 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\"))
>> (goto-char (point-min)) (sit-for 0) (condition-case nil (while t
>> (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil
>> #'kill-emacs))" real	1m4.705s user	0m18.296s sys	0m18.444s
>> 
>> # Master with your patch $ time master/src/emacs -Q --eval "(progn
>> (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil)
>> (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil
>> 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\"))
>> (goto-char (point-min)) (sit-for 0) (condition-case nil (while t
>> (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil
>> #'kill-emacs))" real	0m1.207s user	0m0.928s sys	0m0.028s
> 
> I see nothing of the kind, as you saw.  With the same fonts as you 
> use.  So it's definitely not a problem with the fonts.  Which is 
> strange, since the profiles you provided seem to indicate a 
> font-related problem.

We know the exact commit that caused this regression, right?  So how could the problem not be font-related, since reverting it fixes the problem?

> So why not use Symbola (i.e. the default fontset setup)?

Because I don't use Symbola proper, but a monospacified variant.  In fact, I use XITS Math (monospacified) now, followed by Symbola as the fallback.
And in any case, we're not just trying to fix my problem: I regularly get support requests about company-coq because people run into performance issues with Emacs and prettify-symbols-mode. 

> Symbola supports emoji in the default fontset, so neither Segoe UI 
> nort Symbola should be needed in your setup.

Except I don't want the Emoji from Symbola — I want the ones from Segoe.  And again, this isn't about fixing my setup (I've perfectly happy running my self-compiled fork for the last year-and-a-half).

> As for Ubuntu Mono, isn't that the default monospaced font?

Not on all my machines, no.

> In short, are you sure you really need those customizations?  You
> may be creating your own problems.

I'm sure: commit af1a69f4d17a482c359d98c00ef86fac835b5fac is creating my problems :)

> The default fontset should already provide you with the fonts that 
> display all of these, including CJK.  If you use the default, what 
> characters aren't displayed well enough, or at all?

Depends on the machine: not well enough on some, not at all on other, depending on the installed set of fonts.

> Is this the complete fontset setup you are using […] If it is, I will try to reproduce using this fontset.

No, not exactly.  The full thing is at https://github.com/cpitclaudel/.emacs.d/blob/master/init/fonts.el#L7, but the problem is reproducible in a clean install an a much simpler set up.

What about the VM approach, or simply remote access on a VM that I could provide? I'm afraid that you'll waste your time reproducing my complex set-up, with no actual benefit.  Do you want to reproduce the problem to debug it locally, or something else? If the former, it there nothing I could do to help?

Would it help if I tried to find other people who can reproduce the problem on their machines by following my VM-based reproduction steps?

Cheers,
Clément.

Message #188 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Mon, 13 Mar 2017 22:39:48 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Ralf Jung <post <at> ralfj.de>
> Date: Mon, 13 Mar 2017 19:12:10 +0100
> 
> > Do you have any measurements that would show the slowdown in numbers?
> 
> I know very little about emacs, so I wouldn't know how to make any
> measurements.

If some reaction time is slow, just estimating how many seconds it
takes might be good enough, if you describe the slow command and what
is on the screen when it is slow.

> > Also, what is your value of system-configuration-options and
> > system-configuration-features?  And which version of Emacs is that?
> 
> Sorry, I have no idea how to read those variables

Type

  M-: system-configuration-options RET

and tell what Emacs displays.  the same with the other one.

> This is emacs 24.5+1-8 as packaged by Debian.
> 
> > What happens if you don't make any of the above customizations?
> 
> Emacs is fine in terms of speed, but it uses the wrong font for
> characters not supported by Fira Sans Mono.
> For example, for ∃ it picks "STIX"

The default fontset was improved in Emacs 25.1, so there you should
have Symbola for this character automatically.

> I acknowledge that it'd be much easier if you could reproduce the
> problem.  That said, there's a patch that Clément said works.  Honestly
> I am a little puzzled why you are so hesitant to apply a patch that you
> wrote, and that reportedly fixes the problem.  Do you expect that patch
> to have negative side-effects?  Of course this is ultimately up to you,
> I am just expressing my surprise here.

I'm hesitant because no one understands why that patch should have any
profound effect on performance.  We arrived at the patch by
selectively reverting a particular commit that was fond by bisection.
But that commit fixed an unrelated bug, so it wasn't done by mistake.

Message #191 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Mon, 13 Mar 2017 22:53:35 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Mon, 13 Mar 2017 15:04:24 -0400
> 
> > I see nothing of the kind, as you saw.  With the same fonts as you 
> > use.  So it's definitely not a problem with the fonts.  Which is 
> > strange, since the profiles you provided seem to indicate a 
> > font-related problem.
> 
> We know the exact commit that caused this regression, right?  So how could the problem not be font-related, since reverting it fixes the problem?

Because using the same fonts here doesn't reproduce the problem.
Which means that the font data is not the culprit, there's some other
factor at work here.

> > Is this the complete fontset setup you are using […] If it is, I will try to reproduce using this fontset.
> 
> No, not exactly.  The full thing is at https://github.com/cpitclaudel/.emacs.d/blob/master/init/fonts.el#L7, but the problem is reproducible in a clean install an a much simpler set up.

Thanks, I will try using one of these, maybe it will help me reproduce
the problem.

> What about the VM approach, or simply remote access on a VM that I could provide?

Debugging display problems without seeing the display is not
practical.

> Do you want to reproduce the problem to debug it locally, or something else? If the former, it there nothing I could do to help?

The former, of course.  This happens in complicated code I know very
little about, and no one else on board knows more.  Stepping through
the code with a debugger, comparing the "fast" path with the "slow"
path is the only way to understand what is going on there.

> Would it help if I tried to find other people who can reproduce the problem on their machines by following my VM-based reproduction steps?

If they can debug the problem and explain why the slowdown is so
massive, yes.  Otherwise, we are back to square one.

Message #194 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Tue, 14 Mar 2017 17:45:30 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Mon, 13 Mar 2017 15:04:24 -0400
> 
> # Stock 25.1
> $ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
> real	1m4.705s
> user	0m18.296s
> sys	0m18.444s

I checked: all the fonts you use as the default -- Noto Sans, Fira
Sans, Ubuntu Mono -- all of them basically support only Latin, Greek,
and Cyrillic blocks, and very little else.  By contrast, the above
fontset specification claims that they support the entire Unicode
range of characters, which causes Emacs waste cycles trying to use
these fonts for display of characters they don't support.

In addition, the font-spec doesn't specify the registry of the fonts,
leaving that to the default, which IME is inadequate.  So please try
the following, and see if you get any significant speedup:

  time emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 'latin '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs)))"

I get a 4-fold speedup with this, it would be interesting to know
whether you see any improvements.

Other than that, I'm sorry to say that I couldn't reproduce any of the
slowdowns you mention in this bug report and in the referenced issues.
In each of the recipes I tried, the slowdown disappears either if I
set inhibit-compacting-font-caches (which you said doesn't affect your
use cases) or if I define the fontset correctly, like specify a
registry for a font or the scripts it really supports.  It's possible
that these factors somehow have a much more significant effect on your
system, I don't know why.  Perhaps you have an awful lot of fonts
installed?  What does the following yield when evaluated:

  (length (x-list-fonts "-*-*-*-r-*-*-*-*-*-*-*-*-iso10646-1"))

(I get 1398.)

Allow me a few comments on your real-life setup, they could be
relevant even if they don't resolve the slow display with stock Emacs:

  (defun my-configure-one-fontset (fontset &optional size)
    "Add fallbacks to FONTSET with SIZE."
    (let* ((size (when size `(:size ,(/ size 10.0))))
	   (base-spec (apply #'font-spec :name ~/main-font size))
	   (emoji-spec (apply #'font-spec :name (format "Segoe UI Emoji monospacified for %s" ~/main-font) size))
	   (symbol-spec (apply #'font-spec :name (format "XITS Math monospacified for %s" ~/main-font) size))
	   (fallback-spec (apply #'font-spec :name (format "Symbola monospacified for %s" ~/main-font) size))
	   (cjk-spec (apply #'font-spec :name "WenQuanYi Micro Hei Mono" size)))

All of the above specs lack a proper :registry value, which should be
'iso10646-1.

      (set-fontset-font fontset 'unicode base-spec nil)

This should only specify 'latin, 'greek, and 'cyrillic (one such line
for each of them), as 'unicode is a blatant lie.

      (set-fontset-font fontset 'unicode emoji-spec nil 'append)

It is better o have a definitive list of codepoint ranges here, since
again 'unicode is not what you want.  Once you have the ranges, you
can use 'prepend, which will speed up things a bit more, because Emacs
won't need to go through all the fonts you don't want to see.

      (set-fontset-font fontset 'unicode symbol-spec nil 'append)

Likewise here.

      (set-fontset-font fontset 'unicode fallback-spec nil 'append)

This you shouldn't need doing, as Symbola is already in the default
fontset, and set up according to the characters where it shines.

      (set-fontset-font fontset (cons ?😱 ?😱) "Symbola" nil 'prepend)
      (set-fontset-font fontset (cons #x2009 #x2009) "Symbola" nil 'prepend) ;; Thin space

Likewise.

      (dolist (cjk-block '((#x3000 . #x303F)
			   (#x3040 . #x309F)
			   (#x30A0 . #x30FF)
			   (#x3400 . #x4DFF)
			   (#x4E00 . #x9FFF)
			   (#xF900 . #xFAFF)
			   (#x20000 . #x2A6DF)
			   (#x2A700 . #x2B73F)
			   (#x2B740 . #x2B81F)
			   (#x2B820 . #x2CEAF)
			   (#x2F800 . #x2FA1F)))
  (set-fontset-font fontset cjk-block cjk-spec nil 'append))))

This should use 'prepend, not 'append, IMO.

One more comment: any reasons why you set this up for all the
fontsets, not just for fontset-default?  AFAIU, doing these changes in
all of the fonts might slow down things even more, for no good reason,
because fontset-default is the fallback for all the other fontsets
anyway, so anything you set up in it will be in effect for any other
fontset.

> I'm not alone.  At least one other person posted to this bug list (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21028#74) and confirmed that the patch fixed their problem.

That user's fontset setup makes even less sense to me than yours.  I
will respond to that in a separate message.

> The problem gets regularly mentioned elsewhere (when I opened this bug I linked to https://github.com/purcell/emacs.d/issues/273 "when I first try to input some chinese characters, it's very slow to moving [...] especially use C-n, C-p， emacs will hang for about 1~2 seconds.";

The slowdown in that example disappears for me once I set
inhibit-compacting-font-caches non-nil.  So the factors at work in
that example on my system are somehow different from the factors on
your system (unless you will tell that inhibit-compacting-font-caches
solves the problem for you as well in that case).

> a more recent example is at https://sympa.inria.fr/sympa/arc/coq-club/2017-03/msg00050.html "Appending to the 'unicode list makes emacs unbearably slow.").

This is from the same user who wrote here yesterday.  I comment on his
setup in a separate message.

> Additionally, as the maintainer of company-coq, I've had multiple people tell me that it was just too slow to use, only to realize that turning off prettify-symbols-mode was enough to make everything snappy again for them.

Can you ask some of those people to show their fontset setup?  I'd
like to know how different they are from your setup.

Btw, I'm still trying to understand the significance of the proposed
patch, although it's hard without a clear-cut test case.

Thanks.

Message #197 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 17:47:49 +0200

> From: Ralf Jung <post <at> ralfj.de>
> Date: Mon, 13 Mar 2017 16:54:01 +0100
> 
> Here's the setup which makes things behave badly:
> 
> (dolist (ft (fontset-list))
>   ; Main font
>   (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
>   ; Fallback font
>   (set-fontset-font ft 'unicode (font-spec :name "DejaVu Sans Mono"
> :size 11.0) nil 'append)
> )
> 
> And here's the one I use instead, which behaves much better:
> 
> (dolist (ft (fontset-list))
>   ; Main font
>   (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
>   ; Fallback font
>   (set-fontset-font ft nil (font-spec :name "DejaVu Sans Mono" :size 11.0))
> )
> 
> (For the record, my "Monospace" font is "Fira Sans Mono".)

See, this setup makes very little sense.  It sets up no less than 2
monospaced fonts for the entire range of Unicode codepoints, when both
of these fonts support only Latin, Greek, and Cyrillic blocks.  What's
more, this is done in all the fontsets, so Emacs will waste many
cycles on futile search through these fonts, multiple times!

Try this instead:

  (set-fontset-font "fontset-default" 'greek
                    (font-spec :name "DejaVu Sans Mono"
		    :size 11.0 :registry "iso10646-1") nil 'prepend)
  (set-fontset-font "fontset-default" 'cyrillic
                    (font-spec :name "DejaVu Sans Mono"
		    :size 11.0 :registry "iso10646-1") nil 'prepend)
  (set-fontset-font "fontset-default" 'latin
                    (font-spec :name "DejaVu Sans Mono"
		    :size 11.0 :registry "iso10646-1") nil 'prepend)
  (set-fontset-font "fontset-default" 'greek
                    (font-spec :name "Monospace"
		    :size 11.0 :registry "iso10646-1") nil 'prepend)
  (set-fontset-font "fontset-default" 'cyrillic
                    (font-spec :name "Monospace"
		    :size 11.0 :registry "iso10646-1") nil 'prepend)
  (set-fontset-font "fontset-default" 'latin
                    (font-spec :name "Monospace"
		    :size 11.0 :registry "iso10646-1") nil 'prepend)
  (set-fontset-font t nil (font-spec :name "Symbola"
                                     :size 11.0 :registry "iso10646-1")
                          nil 'append)

(I'd also drop the :size part, but maybe you have a good reason for
that.)

In Emacs 25.2, the last form is not necessary, as Symbola is already
set up by default.

> And just in case you wonder -- no, I have no idea what I am doing here.
> I arrived at that configuration after several hours of random, fairly
> unguided modifications to it until all my test characters picked the
> font I wanted:

I suggest to look at fontset.el in the Emacs sources for inspiration
on how to set up the fontset correctly.  Or ask on emacs-devel.  I
very much doubt that random modifications are likely to come up with
correct and efficient setup of the fonts, since this is indeed a
tricky issue.  (I think Emacs 25.2 improves the situation quite a bit,
so you may wish to try the latest version.)

> While I do consider it a bug that emacs picks the wrong fonts per
> default, shouldn't emacs be reasonably fast to use even if I do some
> manual configuration?

If the manual configuration forces Emacs to unnecessarily search fonts
for characters that aren't there, then it could get unreasonably slow.
We are giving the users here enough rope to hang themselves, in the
hope that they won't try doing that too hard...

Message #200 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 16:57:33 +0100

Hi,

>> I know very little about emacs, so I wouldn't know how to make any
>> measurements.
> 
> If some reaction time is slow, just estimating how many seconds it
> takes might be good enough, if you describe the slow command and what
> is on the screen when it is slow.

Okay, so what I do as actually quite simple: I open the aforementioned
file in emacs, and then I use the scroll wheel on my mouse to scroll
around quickly (move wheel up fast - move wheel down fast - repeat).
When I do this with the "good" setup, text keeps flying over my screen
and everything reacts instantaneously.  When I do this with the "bad"
setup, nothing moves until I stop turning the scroll wheel and wait a
little more, where the waiting time is roughly half the time I spent
scrolling quickly.

>>> Also, what is your value of system-configuration-options and
>>> system-configuration-features?  And which version of Emacs is that?
>>
>> Sorry, I have no idea how to read those variables
> 
> Type
> 
>   M-: system-configuration-options RET
> 
> and tell what Emacs displays.  the same with the other one.

The -options variable says

"--build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
--libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
--build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
--libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
--with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g
-O2 -fdebug-prefix-map=/build/emacs24-OFSh0x/emacs24-24.5+1=.
-fstack-protector-strong -Wformat -Werror=format-security -Wall
-fno-PIE' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
'LDFLAGS=-Wl,-z,relro -no-pie'"

And the -features one says

Symbol's value as variable is void: system-configuration-features

>> Emacs is fine in terms of speed, but it uses the wrong font for
>> characters not supported by Fira Sans Mono.
>> For example, for ∃ it picks "STIX"
> 
> The default fontset was improved in Emacs 25.1, so there you should
> have Symbola for this character automatically.

But I don't want Symbola for this character.  Symbola is not monospace,
so it's not going to look right in a document that otherwise uses
monospace characters.

> I'm hesitant because no one understands why that patch should have any
> profound effect on performance.  We arrived at the patch by
> selectively reverting a particular commit that was fond by bisection.
> But that commit fixed an unrelated bug, so it wasn't done by mistake.

I see; thanks.

Kind regards,
Ralf

Message #203 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 19:11:06 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Ralf Jung <post <at> ralfj.de>
> Date: Tue, 14 Mar 2017 16:57:33 +0100
> 
> Okay, so what I do as actually quite simple: I open the aforementioned
> file in emacs, and then I use the scroll wheel on my mouse to scroll
> around quickly (move wheel up fast - move wheel down fast - repeat).
> When I do this with the "good" setup, text keeps flying over my screen
> and everything reacts instantaneously.  When I do this with the "bad"
> setup, nothing moves until I stop turning the scroll wheel and wait a
> little more, where the waiting time is roughly half the time I spent
> scrolling quickly.

In the "bad" setup, if you press C-n just once, how much time does it
take for Emacs to react?  One second, 10 sec, more?  Same question
regarding C-v.

> >> Emacs is fine in terms of speed, but it uses the wrong font for
> >> characters not supported by Fira Sans Mono.
> >> For example, for ∃ it picks "STIX"
> > 
> > The default fontset was improved in Emacs 25.1, so there you should
> > have Symbola for this character automatically.
> 
> But I don't want Symbola for this character.  Symbola is not monospace,
> so it's not going to look right in a document that otherwise uses
> monospace characters.

I see.  Well, I don't think there are many fonts out there with good
coverage of symbols, punctuation, and other special characters, that
are monospaced.  Certainly not free fonts.  So if you want these
symbols to be displayed monospaced as much as possible, you have 2
alternatives:

  1) find a monospaced font with good coverage of both Latin and
     symbol/punctuation blocks and use it as the default font; Emacs
     will try to use the default font for any symbol/punctuation
     characters that font supports, before trying other fonts

  2) set up a very detailed fontset with explicit ranges of codepoints
     allotted to each font that supports the respective characters
     well and whose looks on display you like

Note that the above is probably relevant to Emacs 25.1 and later (the
first item definitely), so I suggest to upgrade, because this stuff
should work much better in Emacs 25, and some of your problems might
just go away without any need for customization.

Message #206 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 19:41:15 +0100

Hi,

>> Here's the setup which makes things behave badly:
>>
>> (dolist (ft (fontset-list))
>>   ; Main font
>>   (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
>>   ; Fallback font
>>   (set-fontset-font ft 'unicode (font-spec :name "DejaVu Sans Mono"
>> :size 11.0) nil 'append)
>> )
>>
>> And here's the one I use instead, which behaves much better:
>>
>> (dolist (ft (fontset-list))
>>   ; Main font
>>   (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
>>   ; Fallback font
>>   (set-fontset-font ft nil (font-spec :name "DejaVu Sans Mono" :size 11.0))
>> )
>>
>> (For the record, my "Monospace" font is "Fira Sans Mono".)
> 
> See, this setup makes very little sense.  It sets up no less than 2
> monospaced fonts for the entire range of Unicode codepoints, when both
> of these fonts support only Latin, Greek, and Cyrillic blocks.

Well, maybe it makes little sense to you -- however, this requires
expert knowledge about unicode blocks.  I don't have that knowledge, and
I think it is a bug if configuring a straight-forward fallback chain of
fonts requires such knowledge.  *Not* knowing these details about which
font supports what, I'd say my setup makes a lot of sense.
Well, actually, the following would make even more sense:

(set-fontset-font "fontset-default" 'unicode (font-spec :name
"Monospace" :size 11.0))
(set-fontset-font "fontset-default" 'unicode (font-spec :name "DejaVu
Sans Mono" :size 11.0) nil 'append)

I think back then I didn't yet know about 'append, so I used the nil
range for the fallback font (the documentation even said something
related to fallback for the nil range) and then I had to do this for all
fontsets for it to have any effect...
Anyway, I just tried the above, and the result is that things are just
as slow as with my previous "bad" setup.  And this time, I honestly
can't think of a clearer / more sensible way to tell emacs that these
are the fonts I (mainly) want to use for unicode characters.

>  What's
> more, this is done in all the fontsets, so Emacs will waste many
> cycles on futile search through these fonts, multiple times!

So the slowdown happens even if I do this in just one fontset.
How can searching two fonts take so much time?  95% of the characters in
my document are present in Fira Sans, and only about 0.1% of the
characters (I am guessing here, I suspect actually it's even less) are
present in neither DejaVu nor Fira Sans.  So having these two fonts
first should result in very quick, successful lookups.

> Try this instead:
> 
>   (set-fontset-font "fontset-default" 'greek
>                     (font-spec :name "DejaVu Sans Mono"
> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>   (set-fontset-font "fontset-default" 'cyrillic
>                     (font-spec :name "DejaVu Sans Mono"
> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>   (set-fontset-font "fontset-default" 'latin
>                     (font-spec :name "DejaVu Sans Mono"
> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>   (set-fontset-font "fontset-default" 'greek
>                     (font-spec :name "Monospace"
> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>   (set-fontset-font "fontset-default" 'cyrillic
>                     (font-spec :name "Monospace"
> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>   (set-fontset-font "fontset-default" 'latin
>                     (font-spec :name "Monospace"
> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>   (set-fontset-font t nil (font-spec :name "Symbola"
>                                      :size 11.0 :registry "iso10646-1")
>                           nil 'append)

With this configuration, emacs picks STIX to render ∃.  It also picks
"DejaVu Math TeX Gyre" for ↦.  This breaks the monospace grid.  Both of
these characters are supported by DejaVu Sans Mono.

You say these fonts support only Latin, Cyrillic and Greek -- but for
example Fira Sans Mono supports → and … and ↑, and DejaVu Mono supports
∃ and ↦ and ▷.  Are these all in one of these ranges?

> (I'd also drop the :size part, but maybe you have a good reason for
> that.)

Well, I want to use a smaller font size than the default.  If I use the
"normal" way to do that through the "Set Default Font" UI, the fonts of
most characters changed, but when some fallback font was picked, it used
the default size again so the characters were way too big.  I fixed this
by adding the size to every single fontspec.
So, no, no good reason. I was just desperate.

> I suggest to look at fontset.el in the Emacs sources for inspiration
> on how to set up the fontset correctly.

Frankly, I do not plan to figure out which of my fonts support which
unicode block, nor do I want to spend hours manually selecting the right
font for each individual character.  I believe that this is something
the machine can figure out for me.  With more than two billion
instructions per second, it should be capable of querying two (!) fonts
for every one of the maybe 1000 characters it has to display at any
given time.

> Or ask on emacs-devel.

Well, yes, I admit I should/could have done that.  After spending
something like 2 or 3 hours on this issue, I was frustrated and didn't
find the motivation any more to describe in detail what I tried and what
went wrong.

> If the manual configuration forces Emacs to unnecessarily search fonts
> for characters that aren't there, then it could get unreasonably slow.
> We are giving the users here enough rope to hang themselves, in the
> hope that they won't try doing that too hard...

I had no intention to hang myself, but may have accidentally strangled
myself in the 25 ropes that emacs gave me to configure fonts -- where
all I wanted was for things to look just like they look in any other
editor...

Kind regards,
Ralf

Message #209 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 19:50:22 +0100

Hi,

>> Okay, so what I do as actually quite simple: I open the aforementioned
>> file in emacs, and then I use the scroll wheel on my mouse to scroll
>> around quickly (move wheel up fast - move wheel down fast - repeat).
>> When I do this with the "good" setup, text keeps flying over my screen
>> and everything reacts instantaneously.  When I do this with the "bad"
>> setup, nothing moves until I stop turning the scroll wheel and wait a
>> little more, where the waiting time is roughly half the time I spent
>> scrolling quickly.
> 
> In the "bad" setup, if you press C-n just once, how much time does it
> take for Emacs to react?  One second, 10 sec, more?  Same question
> regarding C-v.

This takes no noticeable time (<0.1s).

> I see.  Well, I don't think there are many fonts out there with good
> coverage of symbols, punctuation, and other special characters, that
> are monospaced.  Certainly not free fonts.

I don't think that's true.  The combination of Fira Sans Mono and DejaVu
Mono covers all characters I mentioned, and all characters I ever needed
so far (including even ✓), except for ∖.

> Note that the above is probably relevant to Emacs 25.1 and later (the
> first item definitely), so I suggest to upgrade, because this stuff
> should work much better in Emacs 25, and some of your problems might
> just go away without any need for customization.

I noticed that emacs 25.1 is also available in Debian testing, so I'll
upgrade and report on my observations.

Kind regards,
Ralf

Message #212 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 21:16:05 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Ralf Jung <post <at> ralfj.de>
> Date: Tue, 14 Mar 2017 19:41:15 +0100
> 
> Well, maybe it makes little sense to you -- however, this requires
> expert knowledge about unicode blocks.  I don't have that knowledge, and
> I think it is a bug if configuring a straight-forward fallback chain of
> fonts requires such knowledge.

I don't think there's a bug here.  Fontsets are complex and delicate
means of customizing how Emacs looks for fonts, so using them does
require expert knowledge.  Emacs believes the specifications in the
fontset blindly, so a misconfigured fontset could cause a real
slowdown.

The way we try to solve these issues is by having Emacs DTRT by
default, at least on most modern platforms.  Emacs 25.1 made a
significant step forward in this regard.  If that's still not enough,
then the way to improve that is to describe your needs and installed
fonts, and ask for a fontset that would exist in Emacs by default,
which will do what you (and presumably others) would want.

IOW, the way to fix this is to augment the existing default fonts or
maybe create an additional ready-to-be-used fontset that users could
simply tell Emacs to use, without any tinkering.

> *Not* knowing these details about which
> font supports what, I'd say my setup makes a lot of sense.
> Well, actually, the following would make even more sense:
> 
> (set-fontset-font "fontset-default" 'unicode (font-spec :name
> "Monospace" :size 11.0))
> (set-fontset-font "fontset-default" 'unicode (font-spec :name "DejaVu
> Sans Mono" :size 11.0) nil 'append)

Not in general.  Maybe if you only need a small subset of Unicode the
above is good enough.  But with today's increasing use of special
characters and symbols, and also emoji, more and more Unicode blocks
need to be supported by the fonts you set up, and the above simply
won't cut it.

> Anyway, I just tried the above, and the result is that things are just
> as slow as with my previous "bad" setup.  And this time, I honestly
> can't think of a clearer / more sensible way to tell emacs that these
> are the fonts I (mainly) want to use for unicode characters.

The more sensible way is to specify explicit ranges of codepoints
where you want certain fonts, and leave the rest to the defaults.

> How can searching two fonts take so much time?

They are searched many times, including their variants.

> 95% of the characters in
> my document are present in Fira Sans, and only about 0.1% of the
> characters (I am guessing here, I suspect actually it's even less) are
> present in neither DejaVu nor Fira Sans.  So having these two fonts
> first should result in very quick, successful lookups.

These lookups are done many times, and Emacs also looks in other fonts
on your system, because the above fontset specification doesn't remove
all the other fonts that are in the fontset.  IOW, your spec is too
general, and thus doesn't allow Emacs to efficiently zero in on the
font you want in each case.

> > Try this instead:
> > 
> >   (set-fontset-font "fontset-default" 'greek
> >                     (font-spec :name "DejaVu Sans Mono"
> > 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
> >   (set-fontset-font "fontset-default" 'cyrillic
> >                     (font-spec :name "DejaVu Sans Mono"
> > 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
> >   (set-fontset-font "fontset-default" 'latin
> >                     (font-spec :name "DejaVu Sans Mono"
> > 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
> >   (set-fontset-font "fontset-default" 'greek
> >                     (font-spec :name "Monospace"
> > 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
> >   (set-fontset-font "fontset-default" 'cyrillic
> >                     (font-spec :name "Monospace"
> > 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
> >   (set-fontset-font "fontset-default" 'latin
> >                     (font-spec :name "Monospace"
> > 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
> >   (set-fontset-font t nil (font-spec :name "Symbola"
> >                                      :size 11.0 :registry "iso10646-1")
> >                           nil 'append)
> 
> With this configuration, emacs picks STIX to render ∃.  It also picks
> "DejaVu Math TeX Gyre" for ↦.  This breaks the monospace grid.  Both of
> these characters are supported by DejaVu Sans Mono.

Well, I don't really know the details of what you want to achieve, so
the above might need some tweaking, like using explicit ranges of
codepoints.

> You say these fonts support only Latin, Cyrillic and Greek -- but for
> example Fira Sans Mono supports → and … and ↑, and DejaVu Mono supports
> ∃ and ↦ and ▷.  Are these all in one of these ranges?

No.  But the rest of the blocks are covered only very partially, so my
suggestion would be to find a single monospaced font that covers the
other characters (symbols and punctuation), and use only it, instead
of having some of them displayed by one font and the rest by another
(and probably quite a few not covered by any of them).

The default fontset tries to achieve this by using Symbola, but if you
want a monospaced font in those areas as well, you will have to find a
suitable font.  You can then set up your fontset like we do with
Symbola in fontset.el (in the Emacs 25 sources).

> > I suggest to look at fontset.el in the Emacs sources for inspiration
> > on how to set up the fontset correctly.
> 
> Frankly, I do not plan to figure out which of my fonts support which
> unicode block, nor do I want to spend hours manually selecting the right
> font for each individual character.  I believe that this is something
> the machine can figure out for me.

The machine cannot possibly know your desires, or which fonts you do
and don't like.  It has a fixed set of criteria, and it applies them
in some predefined order.  When the results are not to your liking,
you need to make amends, and the recommended way of doing so is by
small changes that correct only the specific cases which don't look
good by default.  By contrast, your setup affects the entire Unicode
range, and this runs a greater risk of causing trouble.

Message #215 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 20:17:00 +0100

Hi again,

> Note that the above is probably relevant to Emacs 25.1 and later (the
> first item definitely), so I suggest to upgrade, because this stuff
> should work much better in Emacs 25, and some of your problems might
> just go away without any need for customization.

All right, I am on emacs 25.1.1 in Debian testing now.  If I just remove
all font configuration, I still see emacs picking the wrong characters:
While the Latin and Greek letters and the arrows are all Fira Sans Mono
(my system-wide default monospace font), emacs still picks STIX to
render ∃ and "DejaVu Math TeX Gyre" for ↦ and ∖.  (Actually the
situation with ∖ is even more weird -- once I *select* that character
(i.e., I use shift+arrow key to mark it -- sorry, I don't know the emacs
terminology here), it actually changes to some other font, which I can
tell because it suddenly is only half the width.  Unfortunately I don't
know how to query that font because that would remove the selection, but
it looks a lot like the one from Symbola.)

So I want to tell emacs to use DejaVu for everything the default font
doesn't support.  I tried

(set-fontset-font "fontset-default" 'unicode (font-spec :name "DejaVu
Sans Mono" :size 11.0))

but that just results in *fewer* characters being rendered in Fira Sans.
 For example, Σ is now rendered in TeX Gyre Schola Math. (How can
setting DejaVu as the 'unicode font result in emacs switching from Fira
Sans to TeX Gyre?!?)  For additional confusion, if I select the Σ, it
switches to some other font; I can't tell which.

This brings me back to

(set-fontset-font "fontset-default" 'unicode (font-spec :name
"Monospace" :size 11.0))
(set-fontset-font "fontset-default" 'unicode (font-spec :name "DejaVu
Sans Mono" :size 11.0) nil 'append)

which fixes Σ, but still leaves all the other characters with their
wrong fonts.
So I do

(dolist (ft (fontset-list))
  ; Main font
  (set-fontset-font ft 'unicode (font-spec :name "Monospace" :size 11.0))
  ; Fallback font
  (set-fontset-font ft 'unicode (font-spec :name "DejaVu Sans Mono"
:size 11.0) nil 'append)
)

And now it uses the right font for things supported by either Fira or
DejaVu. For ∖, it picks STIXVariants, which is slightly too high so the
line grid is not perfect; that's why I usually try to make Symbola the
fallback-fallback font.
In any case, the above configuration is *slow*.  As in, way slower than
Emacs 24 was.  Just clicking somewhere results in a noticeably delay of
~0.5s until the cursor actually moves.  Same for C-v.  C-n is still
instantaneous.  This is all at the top of
<https://gitlab.mpi-sws.org/FP/LambdaRust-coq/raw/master/theories/typing/lib/rc.v>.
 The way larger list.v file I mentioned earlier is less affected, i.e.,
it is still slow but not slower than Emacs 24.  This may be related to
the fact that rc.v uses much more unicode characters than list.v.

Lucky enough, my work-around still works; I can keep my old "weird"
configuration.

Kind regards,
Ralf

Message #218 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 21:16:39 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Ralf Jung <post <at> ralfj.de>
> Date: Tue, 14 Mar 2017 19:50:22 +0100
> 
> I noticed that emacs 25.1 is also available in Debian testing, so I'll
> upgrade and report on my observations.

Thanks.

Message #221 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Tue, 14 Mar 2017 15:35:01 -0400

[Message part 1 (text/plain, inline)]

On 2017-03-14 11:45, Eli Zaretskii wrote:
> I checked: all the fonts you use as the default -- Noto Sans, Fira 
> Sans, Ubuntu Mono -- all of them basically support only Latin,
> Greek, and Cyrillic blocks, and very little else.  By contrast, the
> above fontset specification claims that they support the entire
> Unicode range of characters, which causes Emacs waste cycles trying
> to use these fonts for display of characters they don't support.

Is there a way to tell Emacs to use them for everything they support? For example, Ubuntu Mono supports box-drawing characters (so I could add that range), but it also supports U+2202 "partial diff", U+2206 "Delta", U+2211 "summation", and U+222B "integral", and a few other symbols (≠, ≤, …).

> In addition, the font-spec doesn't specify the registry of the
> fonts, leaving that to the default, which IME is inadequate.  So
> please try the following, and see if you get any significant
> speedup:

I do :)  See timings below:

# With your patch
$ time master/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 'latin '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	0m0.532s
user	0m0.404s
sys	0m0.024s

# Without your patch
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 'latin '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	0m0.577s
user	0m0.392s
sys	0m0.020s

And more experiments:

# With just 'latin (no registry) on stock 25.1:
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 'latin \"Noto Sans\" nil) (set-fontset-font \"fontset-default\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	0m0.587s
user	0m0.416s
sys	0m0.012s

# With just the registries (and 'unicode instead of 'latin) on stock 25.1
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 'unicode '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	0m42.250s
user	0m22.148s
sys	0m6.220s

# With 'unicode and inhibit-compacting-font-caches on stock 25.1:
$ time 25.1/src/emacs -Q --eval "(progn (setq inhibit-compacting-font-caches t) (set-fontset-font \"fontset-default\" 'unicode '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	0m42.105s
user	0m22.340s
sys	0m6.124s

> Other than that, I'm sorry to say that I couldn't reproduce any of
> the slowdowns you mention in this bug report and in the referenced
> issues. In each of the recipes I tried, the slowdown disappears
> either if I set inhibit-compacting-font-caches (which you said
> doesn't affect your use cases) or if I define the fontset correctly,
> like specify a registry for a font or the scripts it really supports.
> 
For me specifying the registry doesn't seem to do anything, but restricting the font to 'latin does.  Is that example right though?  On my machine, running the code above gives me an Emacs that uses Ubuntu Mono and Symbola, not Noto Sans.  I need to use "fontset-startup" for the font to be taken into account.

To clarify, the following uses Noto Sans and XITS Math, and is very slow:

$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"

The following uses *Ubuntu Mono* (?!) and XITS Math, and is very slow
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 'unicode \"Noto Sans\" nil) (set-fontset-font \"fontset-default\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"

The following uses Noto Sans and XITS Math, and is very fast:
$ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'latin \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS Math\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"

I'm a bit confused, though: the restriction to latin still allows Emacs to use Noto Sans for characters like "⅔", "≠", and "√"?

> It's possible that these factors somehow have a much more significant
> effect on your system, I don't know why.  Perhaps you have an awful
> lot of fonts installed?  What does the following yield when
> evaluated:
> 
> (length (x-list-fonts "-*-*-*-r-*-*-*-*-*-*-*-*-iso10646-1"))

I get 874.

> Allow me a few comments on your real-life setup, they could be 
> relevant even if they don't resolve the slow display with stock
> Emacs:
>
> All of the above specs lack a proper :registry value, which should
> be 'iso10646-1.

Thanks.

> (set-fontset-font fontset 'unicode base-spec nil)
> 
> This should only specify 'latin, 'greek, and 'cyrillic (one such
> line for each of them), as 'unicode is a blatant lie.

But I want more tha 'latin, 'greek, and 'cyrillic: I want "any character that this font supports".

> (set-fontset-font fontset 'unicode emoji-spec nil 'append)
> 
> It is better o have a definitive list of codepoint ranges here,
> since again 'unicode is not what you want.  Once you have the ranges,
> you can use 'prepend, which will speed up things a bit more, because
> Emacs won't need to go through all the fonts you don't want to see.

Possibly — but this will break next time the font is updated with more Emoji, right?

> (set-fontset-font fontset 'unicode fallback-spec nil 'append)
> 
> This you shouldn't need doing, as Symbola is already in the default 
> fontset, and set up according to the characters where it shines.

I used a variant of Symbola, not Symbola itself.

> (dolist (cjk-block '((#x3000 . #x303F) (#x3040 . #x309F) (#x30A0 .
> #x30FF) (#x3400 . #x4DFF) (#x4E00 . #x9FFF) (#xF900 . #xFAFF) 
> (#x20000 . #x2A6DF) (#x2A700 . #x2B73F) (#x2B740 . #x2B81F) (#x2B820
> . #x2CEAF) (#x2F800 . #x2FA1F))) (set-fontset-font fontset cjk-block
> cjk-spec nil 'append))))
> 
> This should use 'prepend, not 'append, IMO.

Thanks, good point!

> One more comment: any reasons why you set this up for all the 
> fontsets, not just for fontset-default?  AFAIU, doing these changes
> in all of the fonts might slow down things even more, for no good
> reason, because fontset-default is the fallback for all the other
> fontsets anyway, so anything you set up in it will be in effect for
> any other fontset.

I'm not sure I understand.  Are you saying that e.g. running the following should show display a *scratch* buffer in Noto Sans?

$ 25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'latin \"Noto Sans\" nil)"

It doesn't for me.  I need to set fontset-startup instead.

$ 25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-startup\" 'latin \"Noto Sans\" nil)"

But then I run into trouble when I change the font size (with set-face-attribute), because that creates new fontsets.

>> The problem gets regularly mentioned elsewhere (when I opened this
>> bug I linked to https://github.com/purcell/emacs.d/issues/273 "when
>> I first try to input some chinese characters, it's very slow to
>> moving [...] especially use C-n, C-p， emacs will hang for about 1~2
>> seconds.";
> 
> The slowdown in that example disappears for me once I set 
> inhibit-compacting-font-caches non-nil.  So the factors at work in 
> that example on my system are somehow different from the factors on 
> your system (unless you will tell that
> inhibit-compacting-font-caches solves the problem for you as well in
> that case).

For me that example is fast in all configurations: the problem I ran into with CJK scripts was 2 years ago, and I haven't managed to reproduce it.

> Can you ask some of those people to show their fontset setup?  I'd 
> like to know how different they are from your setup.

They essentially use this:

(dolist (ft (fontset-list))
  (set-fontset-font ft 'unicode (font-spec :name "YOUR-USUAL-FONT"))
  (set-fontset-font ft 'unicode (font-spec :name "Symbola") nil 'append))

because that's what I recommend (that's the only one that I've found to work reliably: not specifying 'unicode uses Symbola too often, not looping on the fontsets causes issues when changing font sizes, and not specifying Symbola sometimes falls back to other fonts).  It used to work great in Emacs 24.3, which was the default on Ubuntu for a while (and still is for the long-term support release)

> Btw, I'm still trying to understand the significance of the proposed 
> patch, although it's hard without a clear-cut test case.

Thanks for your help!
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #224 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 20:39:56 +0100

Hi,

>> Well, maybe it makes little sense to you -- however, this requires
>> expert knowledge about unicode blocks.  I don't have that knowledge, and
>> I think it is a bug if configuring a straight-forward fallback chain of
>> fonts requires such knowledge.
> 
> I don't think there's a bug here.  Fontsets are complex and delicate
> means of customizing how Emacs looks for fonts, so using them does
> require expert knowledge.  Emacs believes the specifications in the
> fontset blindly, so a misconfigured fontset could cause a real
> slowdown.
> 
> The way we try to solve these issues is by having Emacs DTRT by
> default, at least on most modern platforms.  Emacs 25.1 made a
> significant step forward in this regard.  If that's still not enough,
> then the way to improve that is to describe your needs and installed
> fonts, and ask for a fontset that would exist in Emacs by default,
> which will do what you (and presumably others) would want.
> 
> IOW, the way to fix this is to augment the existing default fonts or
> maybe create an additional ready-to-be-used fontset that users could
> simply tell Emacs to use, without any tinkering.

I think we can agree that ideally, the default should DTRT.
Unfortunately, I noticed no change when upgrading to Emacs 25; that may
be because of which exact fonts I have installed.

>> Anyway, I just tried the above, and the result is that things are just
>> as slow as with my previous "bad" setup.  And this time, I honestly
>> can't think of a clearer / more sensible way to tell emacs that these
>> are the fonts I (mainly) want to use for unicode characters.
> 
> The more sensible way is to specify explicit ranges of codepoints
> where you want certain fonts, and leave the rest to the defaults.

Well, IMHO it is not very sensible to do this manually.  For example,
the supported codepoint ranges of the fonts won't change during the
runtime of Emacs, so Emacs could perfectly well build a more refined
per-block fallback cascade from the coarse-grained information I provide
(essentially caching the information whether a font supports no, some,
or all characters of any given block).

>> How can searching two fonts take so much time?
> 
> They are searched many times, including their variants.
> 
>> 95% of the characters in
>> my document are present in Fira Sans, and only about 0.1% of the
>> characters (I am guessing here, I suspect actually it's even less) are
>> present in neither DejaVu nor Fira Sans.  So having these two fonts
>> first should result in very quick, successful lookups.
> 
> These lookups are done many times, and Emacs also looks in other fonts
> on your system, because the above fontset specification doesn't remove
> all the other fonts that are in the fontset.  IOW, your spec is too
> general, and thus doesn't allow Emacs to efficiently zero in on the
> font you want in each case.

Oh, I would be happy if there was a way to remove all the other fonts
from the fontsets.  Then at least I could be sure that there's not one
or two random characters left for which it still picks TeX Gyre or
whatever other random font I happen to have installed -- or maybe that
font is picked only when I select the character, or so.  I suppose I
wouldn't even need to add my fonts to multiple fontsets then for them to
actually have any effect.

Of course I would be even more happy if I wouldn't have to do anything.

>>> Try this instead:
>>>
>>>   (set-fontset-font "fontset-default" 'greek
>>>                     (font-spec :name "DejaVu Sans Mono"
>>> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>>>   (set-fontset-font "fontset-default" 'cyrillic
>>>                     (font-spec :name "DejaVu Sans Mono"
>>> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>>>   (set-fontset-font "fontset-default" 'latin
>>>                     (font-spec :name "DejaVu Sans Mono"
>>> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>>>   (set-fontset-font "fontset-default" 'greek
>>>                     (font-spec :name "Monospace"
>>> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>>>   (set-fontset-font "fontset-default" 'cyrillic
>>>                     (font-spec :name "Monospace"
>>> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>>>   (set-fontset-font "fontset-default" 'latin
>>>                     (font-spec :name "Monospace"
>>> 		    :size 11.0 :registry "iso10646-1") nil 'prepend)
>>>   (set-fontset-font t nil (font-spec :name "Symbola"
>>>                                      :size 11.0 :registry "iso10646-1")
>>>                           nil 'append)
>>
>> With this configuration, emacs picks STIX to render ∃.  It also picks
>> "DejaVu Math TeX Gyre" for ↦.  This breaks the monospace grid.  Both of
>> these characters are supported by DejaVu Sans Mono.
> 
> Well, I don't really know the details of what you want to achieve, so
> the above might need some tweaking, like using explicit ranges of
> codepoints.

On a high level, what I want to achieve is for things to look like they
do in Kate/gedit. ;)  Now that's probably not very instructive, so here
is what I think these editors pick for various characters, just to give
you some examples (deduced by checking that the characters look the same
in emacs and Kate, and then asking emacs which font it used):

all basic ASCII, Latin, Greek characters: Fira Sans Mono
→  Fira Sans Mono
∃  DejaVu Mono (not supported by Fira, it seems)
∀  DejaVu Mono (not supported by Fira, it seems)
∗  DejaVu Mono (not supported by Fira, it seems)
⋅  DejaVu Mono (not supported by Fira, it seems)

Of course this is just an excerpt of the characters we use.  From all I
can tell, the general rule is "if the character is supported by Fira,
use that font; else of it is supported by DejaVu, use that font; else do
<no idea what it does>"

>> You say these fonts support only Latin, Cyrillic and Greek -- but for
>> example Fira Sans Mono supports → and … and ↑, and DejaVu Mono supports
>> ∃ and ↦ and ▷.  Are these all in one of these ranges?
> 
> No.  But the rest of the blocks are covered only very partially,

Wait -- if Fira already covers the entire Latic, Greek and Cyrillic
range, how can it make any sense to even have a fallback to DejaVu here?
 I thought the entire purpose of these fallback chains was to deal with
fonts supporting random characters in some blocks, so that one cannot
tell in advance exactly which font to use for which block.
So, maybe I could figure out which 2 or 3 unicode blocks all the special
characters are in, and then I could set up Fira followed by DejaVu just
for these blocks.  Then I could have emacs pick the fonts I want without
being slow.
Of course, I already have a configuration that picks the fonts I want
and is not slow -- and the one I have doesn't even require me to
maintain a list of unicode blocks we use, which is a clear advantage.

> so my
> suggestion would be to find a single monospaced font that covers the
> other characters (symbols and punctuation), and use only it, instead
> of having some of them displayed by one font and the rest by another
> (and probably quite a few not covered by any of them).

I don't think the font is to blame here.  After all, other applications
manage to deal with exactly the same fonts just fine.
Unfortunately, I don't know *how* everybody else is selecting fonts, I
only know they do a better job at it than emacs, and they have no
problem dealing with fonts that only partially support some blocks.
Probably fontconfig is doing most of the work here, but I am really just
guessing.

Kind regards,
Ralf

Message #227 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Tue, 14 Mar 2017 15:45:16 -0400

[Message part 1 (text/plain, inline)]

On 2017-03-13 16:53, Eli Zaretskii wrote:
>> We know the exact commit that caused this regression, right?  So
>> how could the problem not be font-related, since reverting it fixes
>> the problem?
> 
> Because using the same fonts here doesn't reproduce the problem. 
> Which means that the font data is not the culprit, there's some
> other factor at work here.

Ok, thanks.

>> What about the VM approach, or simply remote access on a VM that I
>> could provide?
> 
> Debugging display problems without seeing the display is not 
> practical.

I was offering an ssh -X session, or VNC, or whichever remote desktop application could help, so as to give you access to the graphic display.  Would that help?

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #230 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 20:45:51 +0100

Hi again,

> On a high level, what I want to achieve is for things to look like they
> do in Kate/gedit. ;)

I should add that actually, on an even more higher level, I want things
to look "properly monospaced", with all characters (or as many as
possible) being aligned in the grid.  I bring up Kate/gedit because they
demonstrate that this is actually possible without any configuration
being required by the user.  It would be of course fine if Emacs looked
different, as long as it looked "better" (whatever that means ;) and,
most importantly, managed to maintain the grid wherever possible.

; Ralf

Message #233 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: John Mastro <john.b.mastro <at> gmail.com>
To: 21028 <at> debbugs.gnu.org
Cc: Eli Zaretskii <eliz <at> gnu.org>, Ralf Jung <post <at> ralfj.de>
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Tue, 14 Mar 2017 13:49:59 -0700

Eli Zaretskii <eliz <at> gnu.org> wrote:
>   2) set up a very detailed fontset with explicit ranges of codepoints
>      allotted to each font that supports the respective characters
>      well and whose looks on display you like

Is it possible to get the codepoint ranges supported by a given font
programmatically within Emacs? It sounds like that might help a user
tell Emacs "use this font for everything it supports" without specifying
an overly-broad target (like `unicode') or finding and spelling out the
ranges of codepoints manually.

In other words, I'm imagining something like this:

(dolist (target (targets-supported-by-font "DejaVu Sans Mono"))
  (set-fontset-font "fontset-default" target
                    (font-spec :name "DejaVu Sans Mono" :registry "iso10646-1")
                    nil 'prepend))

Where `targets-supported-by-font' is the hypothetical function I'm
imagining.

Apologies if this doesn't actually make sense :)

        John

Message #236 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 15 Mar 2017 17:36:12 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Tue, 14 Mar 2017 15:35:01 -0400
> 
> > I checked: all the fonts you use as the default -- Noto Sans, Fira 
> > Sans, Ubuntu Mono -- all of them basically support only Latin,
> > Greek, and Cyrillic blocks, and very little else.  By contrast, the
> > above fontset specification claims that they support the entire
> > Unicode range of characters, which causes Emacs waste cycles trying
> > to use these fonts for display of characters they don't support.
> 
> Is there a way to tell Emacs to use them for everything they support?

The problem here is that opening a font and looking up a character is
very expensive, certainly when there are hundreds of fonts installed.
So Emacs filters the fonts according to the scripts they claim to
support, and only opens those which appear to be valid candidates for
the script of the character it needs to display.  By using 'unicode as
the script when you set up your fontset, you actually trip Emacs by
telling it to try this font for every character it needs to display.
You should instead specify the scripts which the fonts supports well.
And for the default font (i.e. the font of the default face) I think
you don't have to put it into the fontset at all, just specify it as
the default font (via default-frame-alist, for example).

> For example, Ubuntu Mono supports box-drawing characters (so I could
> add that range), but it also supports U+2202 "partial diff", U+2206
> "Delta", U+2211 "summation", and U+222B "integral", and a few other
> symbols (≠, ≤, …).

That shouldn't be a problem in Emacs 25: it by default uses the
default face's font for any punctuation and symbol characters for
which the font has glyphs, even if the fontset specifies a different
font for punctuation and symbol blocks.

> > In addition, the font-spec doesn't specify the registry of the
> > fonts, leaving that to the default, which IME is inadequate.  So
> > please try the following, and see if you get any significant
> > speedup:
> 
> I do :)  See timings below:
> 
> # With your patch
> $ time master/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 
> 'latin '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font 
> \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) 
> (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char 
> (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) 
> (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
> real    0m0.532s
> user    0m0.404s
> sys     0m0.024s
> 
> # Without your patch
> $ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-default\" 
> 'latin '(\"Noto Sans\" . \"iso10646-1\") nil) (set-fontset-font 
> \"fontset-default\" 'unicode '(\"Symbola\" . \"iso10646-1\") nil 'append) 
> (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char 
> (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) 
> (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
> real    0m0.577s
> user    0m0.392s
> sys     0m0.020s

So using more accurate scripts gives a major improvement.  Good.

> For me specifying the registry doesn't seem to do anything

That could sometimes be the case, but I have found that in some cases
omitting the registry for characters beyond Latin-1 can be a major
setback.  So I recommend to always use it.

> The following uses *Ubuntu Mono* (?!) and XITS Math, and is very slow

I'm guessing that Ubuntu Mono is your system's default font which
Emacs picks up.  The fontset-default specifies fallbacks, it doesn't
affect the default font.

I'm not sure what exactly do you want to happen, because I don't
understand why you need 2 monospaced fonts in your setup.  So I cannot
yet suggest how to set your fonts for them to do what you want.  But
see below for some proposals.

> The following uses Noto Sans and XITS Math, and is very fast:
> $ time 25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 
> 'latin \"Noto Sans\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"XITS 
> Math\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) 
> (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) 
> (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"

> I'm a bit confused, though: the restriction to latin still allows Emacs to use 
> Noto Sans for characters like "⅔", "≠", and "√"?

See above: that's a feature of Emacs 25 -- it by default uses the
default face's font for punctuation and symbol characters.  You can
use use-default-font-for-symbols, new in Emacs 25.2, to disable this
feature and go by the fontsets instead, but I believe the default
produces better results in most cases.

> > (set-fontset-font fontset 'unicode base-spec nil)
> > 
> > This should only specify 'latin, 'greek, and 'cyrillic (one such
> > line for each of them), as 'unicode is a blatant lie.
> 
> But I want more than 'latin, 'greek, and 'cyrillic: I want "any character that 
> this font supports".

Given that symbols and punctuation characters already use that font,
why do you need more than that, and in what Unicode blocks?

You can use a font utility, such as Fontforge, to see which blocks the
font supports, and how many characters from each block it can display.
My conclusion from looking at Ubuntu Mono is that the above 3 scripts
are the only ones it supports well; the rest are not covered well at
all.  You can, of course, add more scripts if you need them, but the
downside will be that some of those scripts will be displayed by a mix
of more than one font, which I think will make the display ugly.
Moreover, Emacs cannot compose glyphs that come from different fonts,
so you will sometimes see decomposed display if you request a font for
scripts where its support is incomplete.  I think this is an important
factor for users of prettify-symbols-mode in particular.

> > (set-fontset-font fontset 'unicode emoji-spec nil 'append)
> > 
> > It is better o have a definitive list of codepoint ranges here,
> > since again 'unicode is not what you want.  Once you have the ranges,
> > you can use 'prepend, which will speed up things a bit more, because
> > Emacs won't need to go through all the fonts you don't want to see.
> 
> Possibly — but this will break next time the font is updated with more Emoji, 
> right?

Not necessarily: you could specify the full range of codepoints from
the Emoji block, even if some of them are not yet available.

> > (set-fontset-font fontset 'unicode fallback-spec nil 'append)
> > 
> > This you shouldn't need doing, as Symbola is already in the default 
> > fontset, and set up according to the characters where it shines.
> 
> I used a variant of Symbola, not Symbola itself.

Then set it up by copying the default setup from fontset.el, as that
setup is based on a lot of thought and careful testing on several
systems.

> > One more comment: any reasons why you set this up for all the 
> > fontsets, not just for fontset-default?  AFAIU, doing these changes
> > in all of the fonts might slow down things even more, for no good
> > reason, because fontset-default is the fallback for all the other
> > fontsets anyway, so anything you set up in it will be in effect for
> > any other fontset.
> 
> I'm not sure I understand.  Are you saying that e.g. running the following 
> should show display a *scratch* buffer in Noto Sans?
> 
> $ 25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'latin \"Noto 
> Sans\" nil)"

No, I'm saying that you have no reason to repeat the fallback fonts in
any fontset but fontset-default.  The default font should be
customized differently, not through fontsets.

> It doesn't for me.  I need to set fontset-startup instead.

That's again because you use fontsets for the default face's font.
You should instead use default-frame-alist for that, and leave
fontsets only for when a character might be unavailable in the default
font.

> > Can you ask some of those people to show their fontset setup?  I'd 
> > like to know how different they are from your setup.
> 
> They essentially use this:

> (dolist (ft (fontset-list))
>   (set-fontset-font ft 'unicode (font-spec :name "YOUR-USUAL-FONT"))
>   (set-fontset-font ft 'unicode (font-spec :name "Symbola") nil 'append))

Well, maybe with the new insights you have now, you can recommend them
better setups which don't use 'unicode'.

Message #239 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Wed, 15 Mar 2017 17:36:57 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Ralf Jung <post <at> ralfj.de>
> Date: Tue, 14 Mar 2017 20:39:56 +0100
> 
> > IOW, the way to fix this is to augment the existing default fonts or
> > maybe create an additional ready-to-be-used fontset that users could
> > simply tell Emacs to use, without any tinkering.
> 
> I think we can agree that ideally, the default should DTRT.

I was indeed talking what the default should do.  If it doesn't do TRT
in your case (without any customizations of the fontsets), I suggest
to describe the problems in a bug report, so that the defaults could
be augmented to cater to your use cases.

> Unfortunately, I noticed no change when upgrading to Emacs 25; that may
> be because of which exact fonts I have installed.

Could be, I don't know.  Once again, if the defaults, without any font
customizations, are somehow not good for you, please report the
details in a bug report.

> > The more sensible way is to specify explicit ranges of codepoints
> > where you want certain fonts, and leave the rest to the defaults.
> 
> Well, IMHO it is not very sensible to do this manually.  For example,
> the supported codepoint ranges of the fonts won't change during the
> runtime of Emacs, so Emacs could perfectly well build a more refined
> per-block fallback cascade from the coarse-grained information I provide
> (essentially caching the information whether a font supports no, some,
> or all characters of any given block).

If you are asking for an Emacs feature that could map the installed
fonts and find out which characters they support, this should be
doable, if someone writes the code.  But the assumption that the
installed fonts rarely change is not a universally valid one: I know
people who install new fonts every day.

> On a high level, what I want to achieve is for things to look like they
> do in Kate/gedit. ;)  Now that's probably not very instructive, so here
> is what I think these editors pick for various characters, just to give
> you some examples (deduced by checking that the characters look the same
> in emacs and Kate, and then asking emacs which font it used):

> all basic ASCII, Latin, Greek characters: Fira Sans Mono
> →  Fira Sans Mono
> ∃  DejaVu Mono (not supported by Fira, it seems)
> ∀  DejaVu Mono (not supported by Fira, it seems)
> ∗  DejaVu Mono (not supported by Fira, it seems)
> ⋅  DejaVu Mono (not supported by Fira, it seems)

I would begin by specifying Fira Sans Mono as your default font, via
default-frame-alist.  Then see if some of the above are not displayed
as you like, and fix that via fontset-default.  It sounds like all the
characters for which you need DejaVu Mono are punctuation and symbols,
so setting up DejaVu Mono for the range '(#x2200 . #x22FF) should do.
If you want more symbols, try '(#x2200 . #x232F), it looks like DejaVu
Mono has good coverage in this larger range.

> Of course this is just an excerpt of the characters we use.  From all I
> can tell, the general rule is "if the character is supported by Fira,
> use that font; else of it is supported by DejaVu, use that font; else do
> <no idea what it does>"

As you have seen, this does work as you want, but it's slow.  We are
talking about getting you the same functionality, but faster.  That
comes for a price of more accurate fontset setup.

> >> You say these fonts support only Latin, Cyrillic and Greek -- but for
> >> example Fira Sans Mono supports → and … and ↑, and DejaVu Mono supports
> >> ∃ and ↦ and ▷.  Are these all in one of these ranges?
> > 
> > No.  But the rest of the blocks are covered only very partially,
> 
> Wait -- if Fira already covers the entire Latic, Greek and Cyrillic
> range, how can it make any sense to even have a fallback to DejaVu here?

You tell me, I don't know your exact needs.  Maybe I guessed them
above, and all you need is more punctuation and symbols, because Fira
has very incomplete coverage of those.  Emacs 25 will use the default
face's font for any symbol and punctuation characters supported by
that font, so you only need fallbacks where those characters are not
available in Fira.  That's what the above blocks should achieve by
configuring DejaVu Sans for them.

>  I thought the entire purpose of these fallback chains was to deal with
> fonts supporting random characters in some blocks, so that one cannot
> tell in advance exactly which font to use for which block.

It can be used like that, but then it could be slow.

> So, maybe I could figure out which 2 or 3 unicode blocks all the special
> characters are in, and then I could set up Fira followed by DejaVu just
> for these blocks.  Then I could have emacs pick the fonts I want without
> being slow.

My suggestions for that are above.  Maybe that's what you need.

> I don't think the font is to blame here.  After all, other applications
> manage to deal with exactly the same fonts just fine.
> Unfortunately, I don't know *how* everybody else is selecting fonts, I
> only know they do a better job at it than emacs, and they have no
> problem dealing with fonts that only partially support some blocks.
> Probably fontconfig is doing most of the work here, but I am really just
> guessing.

Most other applications don't deal with multi-lingual text, so their
job is easier.  Emacs attempts to solve a harder problem here.

Message #242 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 15 Mar 2017 17:37:39 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Tue, 14 Mar 2017 15:45:16 -0400
> 
> >> What about the VM approach, or simply remote access on a VM that I
> >> could provide?
> > 
> > Debugging display problems without seeing the display is not 
> > practical.
> 
> I was offering an ssh -X session, or VNC, or whichever remote desktop 
> application could help, so as to give you access to the graphic display.  Would 
> that help?

AFAIU, "ssh -X" uses the fonts from my machine, because that's where
the X server runs.  Right?

And I don't believe VNC will be practical for such a long distance.

But thanks anyway.

Message #245 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: John Mastro <john.b.mastro <at> gmail.com>
Cc: 21028 <at> debbugs.gnu.org, post <at> ralfj.de
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Wed, 15 Mar 2017 17:40:17 +0200

> From: John Mastro <john.b.mastro <at> gmail.com>
> Date: Tue, 14 Mar 2017 13:49:59 -0700
> Cc: Eli Zaretskii <eliz <at> gnu.org>, Ralf Jung <post <at> ralfj.de>
> 
> >   2) set up a very detailed fontset with explicit ranges of codepoints
> >      allotted to each font that supports the respective characters
> >      well and whose looks on display you like
> 
> Is it possible to get the codepoint ranges supported by a given font
> programmatically within Emacs?

Yes (although this capability doesn't seem to be exposed to Lisp as of
now).  But it's expensive when done for many fonts, because it
requires opening each such font.

> It sounds like that might help a user tell Emacs "use this font for
> everything it supports" without specifying an overly-broad target
> (like `unicode') or finding and spelling out the ranges of
> codepoints manually.

Yes, this could be a base of a useful feature.

> In other words, I'm imagining something like this:
> 
> (dolist (target (targets-supported-by-font "DejaVu Sans Mono"))
>   (set-fontset-font "fontset-default" target
>                     (font-spec :name "DejaVu Sans Mono" :registry "iso10646-1")
>                     nil 'prepend))
> 
> Where `targets-supported-by-font' is the hypothetical function I'm
> imagining.

This is not suitable for running at Emacs startup time, IMO, as it
will probably slow down the startup too much.

> Apologies if this doesn't actually make sense :)

No apologies needed.  This is a complex issue, and I'm not an expert
either.

Message #248 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Wed, 15 Mar 2017 16:46:33 -0400

[Message part 1 (text/plain, inline)]

On 2017-03-15 11:36, Eli Zaretskii wrote:
> The problem here is that opening a font and looking up a character is
> very expensive, certainly when there are hundreds of fonts installed.
> So Emacs filters the fonts according to the scripts they claim to
> support, and only opens those which appear to be valid candidates for
> the script of the character it needs to display.  By using 'unicode as
> the script when you set up your fontset, you actually trip Emacs by
> telling it to try this font for every character it needs to display.
> You should instead specify the scripts which the fonts supports well.

Ok — but why wasn't it slow in 24.3?  And why does the number of fonts matter, if I'm only adding one (Symbola) with 'unicode?

> That shouldn't be a problem in Emacs 25: it by default uses the
> default face's font for any punctuation and symbol characters for
> which the font has glyphs, even if the fontset specifies a different
> font for punctuation and symbol blocks.

Neat.  Though of course that doesn't help for Emacs < 24.

> So using more accurate scripts gives a major improvement.  Good.

It also seems to negate the previous conclusion, right? In these fast example I'm still adding Symbola with 'unicode — but adding Ubuntu Mono with 'latin.  Shouldn't that be slow, according to your explanation at the top of this post?

>> For me specifying the registry doesn't seem to do anything
> 
> That could sometimes be the case, but I have found that in some cases
> omitting the registry for characters beyond Latin-1 can be a major
> setback.  So I recommend to always use it.

Ok.  Could Emacs not infer it?  I have no clue what it means.

>>> (set-fontset-font fontset 'unicode base-spec nil)
>>>
>>> This should only specify 'latin, 'greek, and 'cyrillic (one such
>>> line for each of them), as 'unicode is a blatant lie.
>>
>> But I want more than 'latin, 'greek, and 'cyrillic: I want "any character that 
>> this font supports".

Ok, noted for Emacs >= 25.

> You can use a font utility, such as Fontforge, to see which blocks the
> font supports, and how many characters from each block it can display.
> My conclusion from looking at Ubuntu Mono is that the above 3 scripts
> are the only ones it supports well; the rest are not covered well at
> all.  You can, of course, add more scripts if you need them, but the
> downside will be that some of those scripts will be displayed by a mix
> of more than one font, which I think will make the display ugly.

I don't care about the display being pretty as much as it being properly align (vertically), which means that using Ubuntu Mono helps a lot.

> Moreover, Emacs cannot compose glyphs that come from different fonts,
> so you will sometimes see decomposed display if you request a font for
> scripts where its support is incomplete.  I think this is an important
> factor for users of prettify-symbols-mode in particular.

Why? prettify-symbols-mode composes strings (typically) into single characters, so why would this matter?

>>> Can you ask some of those people to show their fontset setup?  I'd 
>>> like to know how different they are from your setup.
>>
>> They essentially use this:
> 
>> (dolist (ft (fontset-list))
>>   (set-fontset-font ft 'unicode (font-spec :name "YOUR-USUAL-FONT"))
>>   (set-fontset-font ft 'unicode (font-spec :name "Symbola") nil 'append))
> 
> Well, maybe with the new insights you have now, you can recommend them
> better setups which don't use 'unicode'.

I think I still have a blurry picture of the whole process, or at least it sounds very complicated.  Is it really the following?

* Pick a good monospace font.  Set that as the default Emacs font (this is easy)
* Pick a good math font.  Symbola is easier than others, because Emacs knows about it.
* Download and install fontforge.  Figure out which ranges the math font support, and all the characters that it supports that are not punctuation or symbols, too
* Create a set-fontset-font rule for each range and character.  Figure out the font's registry too.  Add all these rules with 'append, though for better performance you can use 'prepend, but for that you'll need to know which ranges the first font supports, too.

This sounds too complicated, given that the simpler approach used to work in Emacs < 24.4.  In CSS, the rule would be 'font-family: "Ubuntu Mono", "XITS Math"'.  In Emacs < 24.4, the following worked — and it works in Emacs 25 with your patch:

(dolist (ft (fontset-list))
   (set-fontset-font ft 'unicode (font-spec :name "Ubuntu Mono"))
   (set-fontset-font ft 'unicode (font-spec :name "Symbola") nil 'append))

In Emacs 25, given that symbols use the default font,  it looks like the following should work, at least with your patch:

  25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'unicode (font-spec :name \"XITS Math\") nil 'prepend)"

It seems to work ok — but without your patch, it's unbearably slow too.  This is better, performance-wise:

  25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'unicode (font-spec :name \"XITS Math\") nil nil)"

but it causes Emacs to use different fonts in different buffers (the concrete example is that when I press C-x 8 RET, type alpha, press TAB, and copy the contents of the completion buffer to *scratch*, the symbols don't appear in the same font in both buffers.

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #251 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 16 Mar 2017 17:27:09 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Wed, 15 Mar 2017 16:46:33 -0400
> 
> > The problem here is that opening a font and looking up a character is
> > very expensive, certainly when there are hundreds of fonts installed.
> > So Emacs filters the fonts according to the scripts they claim to
> > support, and only opens those which appear to be valid candidates for
> > the script of the character it needs to display.  By using 'unicode as
> > the script when you set up your fontset, you actually trip Emacs by
> > telling it to try this font for every character it needs to display.
> > You should instead specify the scripts which the fonts supports well.
> 
> Ok — but why wasn't it slow in 24.3?

That's the question about the significance of the zero_vector in the
font-cache, the patch you want to apply.  I'm still struggling with
understanding why it speeds up the font search.

> And why does the number of fonts matter, 
> if I'm only adding one (Symbola) with 'unicode?

Any font has several variants, and they are all checked.  Moreover,
checking too many fonts conses a lot of Lisp data, which is likely to
trigger GC, which is likely to throw away font caches, which will then
require us to check those fonts again.

> > That shouldn't be a problem in Emacs 25: it by default uses the
> > default face's font for any punctuation and symbol characters for
> > which the font has glyphs, even if the fontset specifies a different
> > font for punctuation and symbol blocks.
> 
> Neat.  Though of course that doesn't help for Emacs < 24.

Users who care about fonts and exotic characters shouldn't stay with
Emacs 24.

> > So using more accurate scripts gives a major improvement.  Good.
> 
> It also seems to negate the previous conclusion, right?

Not really.

> In these fast example I'm still adding Symbola with 'unicode — but
> adding Ubuntu Mono with 'latin.

Ubuntu Mono _is_ the problem.  Symbola has very good coverage of
almost all Unicode blocks, so it is not the problem.

> > That could sometimes be the case, but I have found that in some cases
> > omitting the registry for characters beyond Latin-1 can be a major
> > setback.  So I recommend to always use it.
> 
> Ok.  Could Emacs not infer it?

I think Emacs still goes by the XLFD rules, which require iso8859-1 be
the default.  I don't know how important it is to keep that
compatibility.

> > You can use a font utility, such as Fontforge, to see which blocks the
> > font supports, and how many characters from each block it can display.
> > My conclusion from looking at Ubuntu Mono is that the above 3 scripts
> > are the only ones it supports well; the rest are not covered well at
> > all.  You can, of course, add more scripts if you need them, but the
> > downside will be that some of those scripts will be displayed by a mix
> > of more than one font, which I think will make the display ugly.
> 
> I don't care about the display being pretty as much as it being properly align 
> (vertically), which means that using Ubuntu Mono helps a lot.

That's fine, just don't tell Emacs Ubuntu Mono supports the entire
Unicode.

> > Moreover, Emacs cannot compose glyphs that come from different fonts,
> > so you will sometimes see decomposed display if you request a font for
> > scripts where its support is incomplete.  I think this is an important
> > factor for users of prettify-symbols-mode in particular.
> 
> Why? prettify-symbols-mode composes strings (typically) into single characters, 
> so why would this matter?

I imagine that prettify-symbols-mode is not limited to single existing
characters, but if it is, this issue is indeed not relevant to that
mode.

> >> (dolist (ft (fontset-list))
> >>   (set-fontset-font ft 'unicode (font-spec :name "YOUR-USUAL-FONT"))
> >>   (set-fontset-font ft 'unicode (font-spec :name "Symbola") nil 'append))
> > 
> > Well, maybe with the new insights you have now, you can recommend them
> > better setups which don't use 'unicode'.

> I think I still have a blurry picture of the whole process, or at least it 
> sounds very complicated.  Is it really the following?
> 
> * Pick a good monospace font.  Set that as the default Emacs font (this is easy)

Yes.

> * Pick a good math font.  Symbola is easier than others, because Emacs knows 
> about it.

Yes.

> * Download and install fontforge.  Figure out which ranges the math font 
> support, and all the characters that it supports that are not punctuation or 
> symbols, too

Not needed, as long as the default font is set up via
default-frame-alist.

> * Create a set-fontset-font rule for each range and character.  Figure out the 
> font's registry too.  Add all these rules with 'append, though for better 
> performance you can use 'prepend, but for that you'll need to know which ranges 
> the first font supports, too.

Should not be needed at all, as math symbols are already set up in the
default fontset (if I understand correctly what you need the math font
for, see below).  Definitely not needed if Symbola is to be used.  For
other Math fonts, I expect the default to just work; if it doesn't,
I'd like to see bug reports.

> In Emacs 25, given that symbols use the default font,  it looks like the 
> following should work, at least with your patch:
> 
>   25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'unicode 
> (font-spec :name \"XITS Math\") nil 'prepend)"

What happens if you don't do this?  Which font is used for math
symbols?  Or are the problems with symbols other than math symbols?
IOW, I don't understand fully why you need to set up XITS Math in the
fontset, what is missing/incorrect/ugly if you don't?

Message #254 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Thu, 16 Mar 2017 17:23:13 -0400

On 2017-03-16 11:27, Eli Zaretskii wrote:
>>> You should instead specify the scripts which the fonts supports well.
>>
>> Ok — but why wasn't it slow in 24.3?
> 
> That's the question about the significance of the zero_vector in the
> font-cache, the patch you want to apply.  I'm still struggling with
> understanding why it speeds up the font search.

Got it, thanks.  I don't know whether that'll be useful, but I uploaded a pre-built virtual machine to https://people.csail.mit.edu/cpitcla/emacs-bug-21028.ova ; maybe this is a viable option (it should be much easier to download a VM than to rebuild one, I hope).

It was produced using the instructions that I posted earlier.  On that machine, here are the times I get for the benchmark you posted earlier:

emacs <at> emacs-VirtualBox ~ $ time emacs/emacs-25.1/src/emacs -Q --eval "(progn (set-fontset-font \"fontset-startup\" 'unicode \"Ubuntu Mono\" nil) (set-fontset-font \"fontset-startup\" 'unicode \"Symbola\" nil 'append) (dotimes (_ 5000) (insert (make-string 20 8658) \"\n\")) (goto-char (point-min)) (sit-for 0) (condition-case nil (while t (scroll-up) (sit-for 0)) (error nil)) (run-with-idle-timer 0 nil #'kill-emacs))"
real	5m54.574s
user	0m54.612s
sys	1m17.496s

I loaded this VM on a different machine and confirmed that I could reproduce the problem in the guest (2s without fontset declarations, and >4m with fontset declarations).  I wasn't able to reproduce the bug on the host, though, mirroring your unsuccessful reproduction attempts.  I wonder if something's wrong in Linux Mint… that host wasn't running Mint.

Maybe this pre-built VM can help reproduce, and thus understand, the underlying problem?  Otherwise, maybe you could recommend a resource that I should look into to debug the problem myself locally?

>> And why does the number of fonts matter,
>> if I'm only adding one (Symbola) with 'unicode?
> 
> Any font has several variants, and they are all checked.  Moreover,
> checking too many fonts conses a lot of Lisp data, which is likely to
> trigger GC, which is likely to throw away font caches, which will then
> require us to check those fonts again.

Thanks, understood.

>>> That shouldn't be a problem in Emacs 25: it by default uses the
>>> default face's font for any punctuation and symbol characters for
>>> which the font has glyphs, even if the fontset specifies a different
>>> font for punctuation and symbol blocks.
>>
>> Neat.  Though of course that doesn't help for Emacs < 24.
> 
> Users who care about fonts and exotic characters shouldn't stay with
> Emacs 24.

Understood as well.

>> In these fast example I'm still adding Symbola with 'unicode — but
>> adding Ubuntu Mono with 'latin.
> 
> Ubuntu Mono _is_ the problem.  Symbola has very good coverage of
> almost all Unicode blocks, so it is not the problem.

Ok, I think I understand better now.  But I'm still confused: why would the following be slow, then?

    25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'unicode (font-spec :name \"Symbola\") nil 'prepend)"

To clarify: it isn't slow with the example of plenty of ⇒ characters, but it's still slow if I use a more diverse set of symbols or letters, like e.g.

    ⍺⍶ᷧΑΆἉἍᾍἏᾏᾉἋᾋᾹΆᾼἈἌᾌἎᾎᾈἊᾊΆᾺᾸαάἁἅᾅἇᾇἃᾃᾁᾱάᾴᾶᾷἀἄᾄἆᾆἂᾂᾀάὰᾲᾰᾳⱭⱰɑᶐꬰꭤɒ𝚨𝜜𝜶𝛂𝛢𝛼𝝖𝞐𝞪𝝰ᵅᶛ

So although I think we're converging to a fix for my font config and the use case where Symbola covers everything, there's still an underlying problem that would be great to fix (and your patch does fix it).

>>> That could sometimes be the case, but I have found that in some cases
>>> omitting the registry for characters beyond Latin-1 can be a major
>>> setback.  So I recommend to always use it.
>>
>> Ok.  Could Emacs not infer it?
> 
> I think Emacs still goes by the XLFD rules, which require iso8859-1 be
> the default.  I don't know how important it is to keep that
> compatibility.

Thanks.

>>> Moreover, Emacs cannot compose glyphs that come from different fonts,
>>> so you will sometimes see decomposed display if you request a font for
>>> scripts where its support is incomplete.  I think this is an important
>>> factor for users of prettify-symbols-mode in particular.
>>
>> Why? prettify-symbols-mode composes strings (typically) into single characters, 
>> so why would this matter?
> 
> I imagine that prettify-symbols-mode is not limited to single existing
> characters, but if it is, this issue is indeed not relevant to that
> mode.

The basic interface that it provides is indeed limited to that, so you can trick it by adding arbitrary composition strings to prettify-symbols-alist.

>> * Download and install fontforge.  Figure out which ranges the math font
>> support, and all the characters that it supports that are not punctuation or 
>> symbols, too
> 
> Not needed, as long as the default font is set up via
> default-frame-alist.

Ah, I see.

> What happens if you don't do this?  Which font is used for math
> symbols?  Or are the problems with symbols other than math symbols?
> IOW, I don't understand fully why you need to set up XITS Math in the
> fontset, what is missing/incorrect/ugly if you don't?

I think this is due to my not explaining it properly, or trying to hard to make minimal examples.  In can answer separately for myself and for the other user configurations I've looked at.

For myself, it's a matter of monospace fonts: that is, I want all characters to occupy the same width, so I don't use Symbola: instead, I use a version of XITS Math edited with FontForge to have equal widths for all characters.  I also prefer as many characters to be displayed by the same font.

For other users, we had cases were Emacs displayed an empty rectangle until we added an explicit set-fontset-font with Symbola.

Cheers,
Clément.

Message #257 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Fri, 17 Mar 2017 10:15:33 +0200

> Cc: 21028 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Thu, 16 Mar 2017 17:23:13 -0400
> 
> >> In these fast example I'm still adding Symbola with 'unicode — but
> >> adding Ubuntu Mono with 'latin.
> > 
> > Ubuntu Mono _is_ the problem.  Symbola has very good coverage of
> > almost all Unicode blocks, so it is not the problem.
> 
> Ok, I think I understand better now.  But I'm still confused: why would the following be slow, then?
> 
>     25.1/src/emacs -Q --eval "(set-fontset-font \"fontset-default\" 'unicode (font-spec :name \"Symbola\") nil 'prepend)"
> 
> To clarify: it isn't slow with the example of plenty of ⇒ characters, but it's still slow if I use a more diverse set of symbols or letters, like e.g.
> 
>     ⍺⍶ᷧΑΆἉἍᾍἏᾏᾉἋᾋᾹΆᾼἈἌᾌἎᾎᾈἊᾊΆᾺᾸαάἁἅᾅἇᾇἃᾃᾁᾱάᾴᾶᾷἀἄᾄἆᾆἂᾂᾀάὰᾲᾰᾳⱭⱰɑᶐꬰꭤɒ𝚨𝜜𝜶𝛂𝛢𝛼𝝖𝞐𝞪𝝰ᵅᶛ

If you show a concrete test case, I could look into it when I have
time.

> > What happens if you don't do this?  Which font is used for math
> > symbols?  Or are the problems with symbols other than math symbols?
> > IOW, I don't understand fully why you need to set up XITS Math in the
> > fontset, what is missing/incorrect/ugly if you don't?
> 
> I think this is due to my not explaining it properly, or trying to hard to make minimal examples.  In can answer separately for myself and for the other user configurations I've looked at.
> 
> For myself, it's a matter of monospace fonts: that is, I want all characters to occupy the same width, so I don't use Symbola: instead, I use a version of XITS Math edited with FontForge to have equal widths for all characters.

So you could instead edit Symbola to have a fixed width, and then you
wouldn't need to touch the default fontset, is that right?

> I also prefer as many characters to be displayed by the same font.

That already works in Emacs 25, unless I misunderstand what you mean.
The default font is used for all the characters it supports.  For
fallback fonts, those from fontset-default, you need to set up their
supported scripts for your liking, based on what they support and on
whether or not you like the glyphs they produce for each script they
support.  Using 'unicode' is only recommended for a few fonts whose
coverage is really almost universal.

> For other users, we had cases were Emacs displayed an empty rectangle until we added an explicit set-fontset-font with Symbola.

This should be fixed in Emacs 25 out of the box, AFAIU.

Message #262 received at 21028-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: clement.pitclaudel <at> live.com, Kenichi Handa <handa <at> gnu.org>
Cc: 21028-done <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 16 Apr 2017 10:48:04 +0300

> Date: Thu, 16 Mar 2017 17:27:09 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 21028 <at> debbugs.gnu.org
> 
> > Cc: 21028 <at> debbugs.gnu.org
> > From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> > Date: Wed, 15 Mar 2017 16:46:33 -0400
> > 
> > > The problem here is that opening a font and looking up a character is
> > > very expensive, certainly when there are hundreds of fonts installed.
> > > So Emacs filters the fonts according to the scripts they claim to
> > > support, and only opens those which appear to be valid candidates for
> > > the script of the character it needs to display.  By using 'unicode as
> > > the script when you set up your fontset, you actually trip Emacs by
> > > telling it to try this font for every character it needs to display.
> > > You should instead specify the scripts which the fonts supports well.
> > 
> > Ok — but why wasn't it slow in 24.3?
> 
> That's the question about the significance of the zero_vector in the
> font-cache, the patch you want to apply.  I'm still struggling with
> understanding why it speeds up the font search.

After looking at the code and discussing with Handa-san, I concluded
that we should restore putting the zero_vector in the font-cache when
no fonts matching a spec were found.

So I've reverted part of the changes introduced during fixing
bug#17125, and I'm marking this bug done.  If these changes cause some
trouble, we will have to debug that when the problems are reported.

Thanks.

Message #265 received at 21028-done <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Kenichi Handa <handa <at> gnu.org>
Cc: 21028-done <at> debbugs.gnu.org
Subject: Re: bug#21028: Performance regression in revision
 af1a69f4d17a482c359d98c00ef86fac835b5fac (Apr 2014).
Date: Sun, 16 Apr 2017 11:28:10 -0400

[Message part 1 (text/plain, inline)]

On 2017-04-16 03:48, Eli Zaretskii wrote:
> So I've reverted part of the changes introduced during fixing
> bug#17125, and I'm marking this bug done.  If these changes cause some
> trouble, we will have to debug that when the problems are reported.

Wonderful; thanks.  I've confirmed the fix.
I'll be happy to help debug any issues that pop up.  Thanks again for your patience and dedication.

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #268 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Jung <post <at> ralfj.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Sat, 22 Apr 2017 10:54:56 +0200

Hi,

> Could be, I don't know.  Once again, if the defaults, without any font
> customizations, are somehow not good for you, please report the
> details in a bug report.

All right, I put that on my ToDo list.

>> Of course this is just an excerpt of the characters we use.  From all I
>> can tell, the general rule is "if the character is supported by Fira,
>> use that font; else of it is supported by DejaVu, use that font; else do
>> <no idea what it does>"
> 
> As you have seen, this does work as you want, but it's slow.  We are
> talking about getting you the same functionality, but faster.  That
> comes for a price of more accurate fontset setup.

Well, we also have a patch fixing that slowness, so it doesn't seem to
be an inherent problem, just some implementation artifact.

>> I don't think the font is to blame here.  After all, other applications
>> manage to deal with exactly the same fonts just fine.
>> Unfortunately, I don't know *how* everybody else is selecting fonts, I
>> only know they do a better job at it than emacs, and they have no
>> problem dealing with fonts that only partially support some blocks.
>> Probably fontconfig is doing most of the work here, but I am really just
>> guessing.
> 
> Most other applications don't deal with multi-lingual text, so their
> job is easier.  Emacs attempts to solve a harder problem here.

What exactly does this mean?  I sure would expect all these editors to
correctly display text that mixes Latin, Greek, Cyrillic and Japanese
characters.  I believe they can handle this, but have to admit I did not
try that (mostly for lack of a personal use-case).  Is there an example
that can be used to test this?

Kind regards,
Ralf

Message #271 received at 21028 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ralf Jung <post <at> ralfj.de>
Cc: 21028 <at> debbugs.gnu.org
Subject: Re: bug#21028: Slow font rendering in emacs
Date: Sat, 22 Apr 2017 16:40:50 +0300

> Cc: 21028 <at> debbugs.gnu.org
> From: Ralf Jung <post <at> ralfj.de>
> Date: Sat, 22 Apr 2017 10:54:56 +0200
> 
> >> Of course this is just an excerpt of the characters we use.  From all I
> >> can tell, the general rule is "if the character is supported by Fira,
> >> use that font; else of it is supported by DejaVu, use that font; else do
> >> <no idea what it does>"
> > 
> > As you have seen, this does work as you want, but it's slow.  We are
> > talking about getting you the same functionality, but faster.  That
> > comes for a price of more accurate fontset setup.
> 
> Well, we also have a patch fixing that slowness, so it doesn't seem to
> be an inherent problem, just some implementation artifact.

That patch has been applied, in case you weren't tracking this bug.

> >> I don't think the font is to blame here.  After all, other applications
> >> manage to deal with exactly the same fonts just fine.
> >> Unfortunately, I don't know *how* everybody else is selecting fonts, I
> >> only know they do a better job at it than emacs, and they have no
> >> problem dealing with fonts that only partially support some blocks.
> >> Probably fontconfig is doing most of the work here, but I am really just
> >> guessing.
> > 
> > Most other applications don't deal with multi-lingual text, so their
> > job is easier.  Emacs attempts to solve a harder problem here.
> 
> What exactly does this mean?  I sure would expect all these editors to
> correctly display text that mixes Latin, Greek, Cyrillic and Japanese
> characters.

Most editors assume each file will only ever include text in a single
language.  Emacs explicitly tries to do better when several languages
and scripts are mixed in the same file/buffer.

Emacs also has some rules for selecting fonts based on cultural
preferences, so it could use different fonts for the same Unicode
codepoints in different locales.

> I believe they can handle this, but have to admit I did not try that
> (mostly for lack of a personal use-case).  Is there an example that
> can be used to test this?

I'd begin with the HELLO file.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.