GNU logs - #61285, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#61285: (Sometimes very) slow font-lock after %w in ruby-ts-mode
Resent-From: Dmitry Gutov <dgutov@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sun, 05 Feb 2023 00:40:02 +0000
Resent-Message-ID: <handler.61285.B.167555759411705 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 61285 <at> debbugs.gnu.org
Message-ID: <17a29b2e-beb2-f246-fe1c-2c3503c1bfe6@HIDDEN>
Date: Sun, 5 Feb 2023 02:39:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Content-Language: en-US
From: Dmitry Gutov <dgutov@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=2a00:1450:4864:20::532;
 envelope-from=raaahh@HIDDEN; helo=mail-ed1-x532.google.com
X-Spam_score_int: -14
X-Spam_score: -1.5
X-Spam_bar: -
X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.25,
 FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.1 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

This probably involves a parser bug and/or maybe a tree-sitter one. But 
I'm posting this here anyway because this might not be the only way to 
trigger this problem. Or it could give us some optimization insights.

Also, while it involves a node which is parsed to have a large number of 
descendants, the performance depends heavily on whether the node is at 
the top level of the program (then it's slow), or not.

To repro:

1. Visit test/lisp/progmodes/ruby-mode-resources/ruby.rb
2. add 'a = %w' (without quotes) as a separate new line before all of 
the existing code.
3. Notice the delay in redisplay after you type 'w'.

In you do that with a larger file, BTW, this delay may be on the order 
of a minute. Here's an example of such file: 
https://github.com/rails/rails/blob/main/activerecord/lib/active_record/associations.rb

The superficial reason for this delay is that %w opens a new "array of 
strings" literal which parses every separate word in the rest of the 
buffer as a separate string. So we get a node with thousands of 
children, in the case of associations.rb. Or just ~1000 in the case of 
ruby.rb.

I also tried setting treesit--font-lock-fast-mode to t: no effect.

But! If we do the same not on top-level -- say, put the 'a = %w' line 
after the 'foo' line inside the first 'if' statement (i.e. on line 7), 
the delay is much smaller -- not noticeable in ruby.rb, and still 
apparent but much more bearable in associations.rb (you can put that 
statement right after 'module ActiveRecord') -- even though the size of 
the tree is changed minimally, and the number of children nodes for that 
"array of strings" still counts in the thousands (e.g. 13319).

Perf report for the "bad" highlighting delay looks like this:

   61.19%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_current_status
   30.88%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_parent_node
    7.44%  emacs libtree-sitter.so.0.0  [.] ts_language_symbol_metadata
    0.06%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_goto_first_child
    0.05%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_goto_next_sibling
    0.03%  emacs libtree-sitter.so.0.0  [.] ts_node_end_byte

And like this in the "good" case (with many type-backspace repetitions):

   32.10%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_current_status
    9.50%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_goto_first_child
    7.89%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_goto_next_sibling
    7.51%  emacs libtree-sitter.so.0.0  [.] ts_language_symbol_metadata
    6.45%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_parent_node
    1.87%  emacs libtree-sitter.so.0.0  [.] ts_node_start_point
    1.85%  emacs emacs  [.] process_mark_stack
    0.93%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_current_node

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Dmitry Gutov <dgutov@HIDDEN>
Subject: bug#61285: Acknowledgement ((Sometimes very) slow font-lock after
 %w in ruby-ts-mode)
Message-ID: <handler.61285.B.167555759411705.ack <at> debbugs.gnu.org>
References: <17a29b2e-beb2-f246-fe1c-2c3503c1bfe6@HIDDEN>
X-Gnu-PR-Message: ack 61285
X-Gnu-PR-Package: emacs
Reply-To: 61285 <at> debbugs.gnu.org
Date: Sun, 05 Feb 2023 00:40:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 61285 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
61285: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D61285
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#61285: (Sometimes very) slow font-lock after %w in  ruby-ts-mode
References: <17a29b2e-beb2-f246-fe1c-2c3503c1bfe6@HIDDEN>
In-Reply-To: <17a29b2e-beb2-f246-fe1c-2c3503c1bfe6@HIDDEN>
Resent-From: Yuan Fu <casouri@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Mon, 06 Feb 2023 00:09:02 +0000
Resent-Message-ID: <handler.61285.B61285.167564213726785 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Dmitry Gutov <dgutov@HIDDEN>
Cc: 61285 <at> debbugs.gnu.org
From: Yuan Fu <casouri@HIDDEN>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\))
Message-Id: <2C6F83F0-DD84-4549-9D91-3D820188674D@HIDDEN>
Date: Sun, 5 Feb 2023 16:08:36 -0800
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>


Dmitry Gutov <dgutov@HIDDEN> writes:

> This probably involves a parser bug and/or maybe a tree-sitter one.
> But I'm posting this here anyway because this might not be the only
> way to trigger this problem. Or it could give us some optimization
> insights.
>
> Also, while it involves a node which is parsed to have a large number
> of descendants, the performance depends heavily on whether the node is
> at the top level of the program (then it's slow), or not.
>
> To repro:
>
> 1. Visit test/lisp/progmodes/ruby-mode-resources/ruby.rb
> 2. add 'a =3D %w' (without quotes) as a separate new line before all =
of
> the existing code.
> 3. Notice the delay in redisplay after you type 'w'.
>
> In you do that with a larger file, BTW, this delay may be on the order
> of a minute. Here's an example of such file:
> =
https://github.com/rails/rails/blob/main/activerecord/lib/active_record/as=
sociations.rb
>
> The superficial reason for this delay is that %w opens a new "array of
> strings" literal which parses every separate word in the rest of the
> buffer as a separate string. So we get a node with thousands of
> children, in the case of associations.rb. Or just ~1000 in the case of
> ruby.rb.
>
> I also tried setting treesit--font-lock-fast-mode to t: no effect.
>
> But! If we do the same not on top-level -- say, put the 'a =3D %w' =
line
> after the 'foo' line inside the first 'if' statement (i.e. on line 7),
> the delay is much smaller -- not noticeable in ruby.rb, and still
> apparent but much more bearable in associations.rb (you can put that
> statement right after 'module ActiveRecord') -- even though the size
> of the tree is changed minimally, and the number of children nodes for
> that "array of strings" still counts in the thousands (e.g. 13319).
>
> Perf report for the "bad" highlighting delay looks like this:
>
>   61.19%  emacs libtree-sitter.so.0.0  [.] =
ts_tree_cursor_current_status
>   30.88%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_parent_node
>    7.44%  emacs libtree-sitter.so.0.0  [.] ts_language_symbol_metadata
>    0.06%  emacs libtree-sitter.so.0.0  [.] =
ts_tree_cursor_goto_first_child
>    0.05%  emacs libtree-sitter.so.0.0  [.] =
ts_tree_cursor_goto_next_sibling
>    0.03%  emacs libtree-sitter.so.0.0  [.] ts_node_end_byte
>
> And like this in the "good" case (with many type-backspace =
repetitions):
>
>   32.10%  emacs libtree-sitter.so.0.0  [.] =
ts_tree_cursor_current_status
>    9.50%  emacs libtree-sitter.so.0.0  [.] =
ts_tree_cursor_goto_first_child
>    7.89%  emacs libtree-sitter.so.0.0  [.] =
ts_tree_cursor_goto_next_sibling
>    7.51%  emacs libtree-sitter.so.0.0  [.] ts_language_symbol_metadata
>    6.45%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_parent_node
>    1.87%  emacs libtree-sitter.so.0.0  [.] ts_node_start_point
>    1.85%  emacs emacs  [.] process_mark_stack
>    0.93%  emacs libtree-sitter.so.0.0  [.] ts_tree_cursor_current_node

Interesting. Perhaps it has to do with how tree-sitter implements the
"incremental" part of the parser? But the profile doesn=E2=80=99t look =
like it=E2=80=99s
spending time parsing, I need to look at what does
ts_tree_cursor_current_status actually do (maybe it=E2=80=99s used in =
parsing?)

Yuan

Last modified: Mon, 6 Feb 2023 00:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.