GNU bug report logs - #78402
treesit after-change-functions

Previous Next

To reply to this bug, email your comments to 78402 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: bug-gnu-emacs <at> gnu.org
Subject: treesit after-change-functions
Date: Tue, 13 May 2025 09:27:45 +0300

This new request is the continuation from bug#77256 and
https://lists.gnu.org/archive/html/emacs-devel/2025-05/msg00300.html
about updating treesit ranges for after-change-functions,
particularly for outline--fix-buttons-after-change that uses ranges.

Hopefully Yuan could verify how implementable is this:

> BTW the fix should be fairly simple, AFAICT: the `treesit.c` code is
> told about the buffer change before `after-change-functions` runs, so it
> can record "ranges need to be updated" somewhere, and then lazily update
> the ranges next time some code uses treesit functions that depend on the
> ranges (e.g. outline's after-change function).  If the fully automatic
> lazy update is too eager, we can delay it to an explicit call to
> `treesit-ensure-ranges-are-uptodate` which outline's after-change
> function would have to call manually (same as all other current calls
> to `treesit-update-ranges`)

Message #8 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Thu, 15 May 2025 00:10:57 -0700

> On May 12, 2025, at 11:27 PM, Juri Linkov <juri <at> linkov.net> wrote:
> 
> This new request is the continuation from bug#77256 and
> https://lists.gnu.org/archive/html/emacs-devel/2025-05/msg00300.html
> about updating treesit ranges for after-change-functions,
> particularly for outline--fix-buttons-after-change that uses ranges.
> 
> Hopefully Yuan could verify how implementable is this:
> 
>> BTW the fix should be fairly simple, AFAICT: the `treesit.c` code is
>> told about the buffer change before `after-change-functions` runs, so it
>> can record "ranges need to be updated" somewhere, and then lazily update
>> the ranges next time some code uses treesit functions that depend on the
>> ranges (e.g. outline's after-change function).  If the fully automatic
>> lazy update is too eager, we can delay it to an explicit call to
>> `treesit-ensure-ranges-are-uptodate` which outline's after-change
>> function would have to call manually (same as all other current calls
>> to `treesit-update-ranges`)

First of all, the affected range could be larger than the changed range (think inserting “/*” into a C buffer, everything behind the /* is affected). And you don’t really need to remember the changed range, we already does that in the form of updating the existing tree-sitter parse-tree with buffer edit information. When we finally re-parse the buffer, the parser knows the affected range. And we already rely on that information to selectively update ranges. Specifically, we get the affected regions of the primary parser (which should be the superset of all the affected ranges of all the embedded parsers), and update embedded parser ranges in those regions.

It seems that we only need to invoke treesit--pre-redisplay whenever the root node is accessed (you have to get the root node to accessing any node in the parse tree). treesit--pre-redisplay, if haven’t called since last buffer-chars-modified-tick, will force a re-parse on the primary parser, get the affected region, and update ranges for embedded parsers on the region.

Let me ponder on this a bit more to make sure it’s actually a good idea.

Btw, what’s the issue we’re trying to fix here? If some lisp want to have up-to-date ranges, it can just call treesit--pre-redisplay or treesit-update-ranges. Why do we get the node-outdated error?

Yuan

Message #11 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Thu, 15 May 2025 20:06:14 +0300

> Btw, what’s the issue we’re trying to fix here? If some lisp want to
> have up-to-date ranges, it can just call treesit--pre-redisplay or
> treesit-update-ranges. Why do we get the node-outdated error?

One example where this is needed is updating the outline buttons
based on the changed text in a ts-mode buffer.

Either after-change-functions or jit-lock-function calls
outline--fix-up-all-buttons that uses treesit-outline-search
that requires the updated ranges.

I see that treesit-indent and treesit-indent-region
explicitly calls treesit-update-ranges at the beginning.
Is it how treesit command are intended to update ranges
for themselves?  It would be inefficient to call
treesit-update-ranges in treesit-outline-search.

Message #14 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 78402 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#78402: treesit after-change-functions
Date: Sat, 17 May 2025 17:12:25 -0400

> Btw, what’s the issue we’re trying to fix here? If some lisp want to have
> up-to-date ranges, it can just call treesit--pre-redisplay or
> treesit-update-ranges. Why do we get the node-outdated error?

Hmm... so what you're saying is that we should do something like the
patch below?


        Stefan


diff --git a/lisp/treesit.el b/lisp/treesit.el
index a353bc942d3..0c368de6a03 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -4050,6 +4050,9 @@ treesit-outline-search
   "Search for the next outline heading in the syntax tree.
 For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
 `outline-search-function'."
+  (if backward
+      (treesit-update-ranges bound (point))
+    (treesit-update-ranges (point) bound))
   (if looking-at
       (when (treesit-outline--at-point) (pos-bol))
 
@@ -4136,11 +4139,6 @@ treesit-outline-level
 
     level))
 
-(defun treesit--after-change (beg end _len)
-  "Force updating the ranges in BEG...END.
-Expected to be called after each text change."
-  (treesit-update-ranges beg end))
-
 ;;; Hideshow mode
 
 (defun treesit-hs-block-end ()
@@ -4438,8 +4436,7 @@ treesit-major-mode-setup
       (setq treesit-outline-predicate
             #'treesit-outline-predicate--from-imenu))
     (setq-local outline-search-function #'treesit-outline-search
-                outline-level #'treesit-outline-level)
-    (add-hook 'outline-after-change-functions #'treesit--after-change nil t))
+                outline-level #'treesit-outline-level))
 
   ;; Remove existing local parsers.
   (dolist (ov (overlays-in (point-min) (point-max)))

Message #17 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Yuan Fu <casouri <at> gmail.com>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Mon, 19 May 2025 09:38:40 +0300

>> Btw, what’s the issue we’re trying to fix here? If some lisp want to have
>> up-to-date ranges, it can just call treesit--pre-redisplay or
>> treesit-update-ranges. Why do we get the node-outdated error?
>
> Hmm... so what you're saying is that we should do something like the
> patch below?

Maybe Yuan could confirm if `treesit-update-ranges`
is already optimized for frequent calls on overlapping ranges
that don't need updating and where 'beg' and 'end' fall back
to (point-min) and (point-max).

> +  (if backward
> +      (treesit-update-ranges bound (point))
> +    (treesit-update-ranges (point) bound))

This also requires turning markers into integers before:

  (when (markerp bound) (setq bound (marker-position bound)))

Message #20 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Yuan Fu <casouri <at> gmail.com>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Wed, 21 May 2025 09:35:18 +0300

>>> Btw, what’s the issue we’re trying to fix here? If some lisp want to have
>>> up-to-date ranges, it can just call treesit--pre-redisplay or
>>> treesit-update-ranges. Why do we get the node-outdated error?
>>
>> Hmm... so what you're saying is that we should do something like the
>> patch below?
>
> Maybe Yuan could confirm if `treesit-update-ranges`
> is already optimized for frequent calls on overlapping ranges
> that don't need updating and where 'beg' and 'end' fall back
> to (point-min) and (point-max).

It seems not optimized since in my tests outline-minor-mode
is much slower with this change.

Message #23 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Wed, 21 May 2025 00:21:39 -0700

> On May 20, 2025, at 11:35 PM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>>>> Btw, what’s the issue we’re trying to fix here? If some lisp want to have
>>>> up-to-date ranges, it can just call treesit--pre-redisplay or
>>>> treesit-update-ranges. Why do we get the node-outdated error?
>>> 
>>> Hmm... so what you're saying is that we should do something like the
>>> patch below?
>> 
>> Maybe Yuan could confirm if `treesit-update-ranges`
>> is already optimized for frequent calls on overlapping ranges
>> that don't need updating and where 'beg' and 'end' fall back
>> to (point-min) and (point-max).
> 
> It seems not optimized since in my tests outline-minor-mode
> is much slower with this change.

Hey, sorry for the delay. I've been busy with real life. As you said, `treesit-update-ranges` doesn’t have any optimization for repeated calls. OTOH, `treesit--pre-redisplay` has optimization for repeated calls while buffer content doesn’t change. Maybe the patch can use that instead?

Yuan

Message #26 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 78402 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#78402: treesit after-change-functions
Date: Wed, 21 May 2025 13:47:09 -0400

Yuan Fu [2025-05-21 00:21:39] wrote:
>> On May 20, 2025, at 11:35 PM, Juri Linkov <juri <at> linkov.net> wrote:
>>>>> Btw, what’s the issue we’re trying to fix here? If some lisp want to have
>>>>> up-to-date ranges, it can just call treesit--pre-redisplay or
>>>>> treesit-update-ranges. Why do we get the node-outdated error?
>>>> 
>>>> Hmm... so what you're saying is that we should do something like the
>>>> patch below?
>>> 
>>> Maybe Yuan could confirm if `treesit-update-ranges`
>>> is already optimized for frequent calls on overlapping ranges
>>> that don't need updating and where 'beg' and 'end' fall back
>>> to (point-min) and (point-max).
>> 
>> It seems not optimized since in my tests outline-minor-mode
>> is much slower with this change.
>
> Hey, sorry for the delay. I've been busy with real life. As you said,
> `treesit-update-ranges` doesn’t have any optimization for repeated
> calls.

That seems like a potential problem for existing uses of that function
(we're just "lucky" that they're expected to be used once-per-command).

> OTOH, `treesit--pre-redisplay` has optimization for repeated calls
> while buffer content doesn’t change. Maybe the patch can use that instead?

Like in the patch below?

That sounds OK, but then `treesit--pre-redisplay` needs to be renamed to
something like `treesit--update` or `treesit--sync`, and maybe
`treesit-font-lock-fontify-region`, `treesit-indent`, and
`treesit-indent-region` should use that function instead of
`treesit-update-ranges`?


        Stefan


diff --git a/lisp/treesit.el b/lisp/treesit.el
index 5df8eb70cbf..89edc700f75 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -2615,6 +2615,7 @@ treesit--indent-1
 
 (defun treesit-indent ()
   "Indent according to the result of `treesit-indent-function'."
+  ;; FIXME: Use `prog-indent-line'?
   (treesit-update-ranges (line-beginning-position)
                          (line-end-position))
   ;; We don't return 'noindent even if no rules match, because
@@ -4064,6 +4065,7 @@ treesit-outline-search
   "Search for the next outline heading in the syntax tree.
 For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
 `outline-search-function'."
+  (treesit--pre-redisplay)
   (if looking-at
       (when (treesit-outline--at-point) (pos-bol))
 
@@ -4150,11 +4152,6 @@ treesit-outline-level
 
     level))
 
-(defun treesit--after-change (beg end _len)
-  "Force updating the ranges in BEG...END.
-Expected to be called after each text change."
-  (treesit-update-ranges beg end))
-
 ;;; Hideshow mode
 
 (defun treesit-hs-block-end ()
@@ -4452,8 +4449,7 @@ treesit-major-mode-setup
       (setq treesit-outline-predicate
             #'treesit-outline-predicate--from-imenu))
     (setq-local outline-search-function #'treesit-outline-search
-                outline-level #'treesit-outline-level)
-    (add-hook 'outline-after-change-functions #'treesit--after-change nil t))
+                outline-level #'treesit-outline-level))
 
   ;; Remove existing local parsers.
   (dolist (ov (overlays-in (point-min) (point-max)))

Message #29 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 78402 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#78402: treesit after-change-functions
Date: Wed, 21 May 2025 22:36:06 -0700

> On May 21, 2025, at 10:47 AM, Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:
> 
> Yuan Fu [2025-05-21 00:21:39] wrote:
>>> On May 20, 2025, at 11:35 PM, Juri Linkov <juri <at> linkov.net> wrote:
>>>>>> Btw, what’s the issue we’re trying to fix here? If some lisp want to have
>>>>>> up-to-date ranges, it can just call treesit--pre-redisplay or
>>>>>> treesit-update-ranges. Why do we get the node-outdated error?
>>>>> 
>>>>> Hmm... so what you're saying is that we should do something like the
>>>>> patch below?
>>>> 
>>>> Maybe Yuan could confirm if `treesit-update-ranges`
>>>> is already optimized for frequent calls on overlapping ranges
>>>> that don't need updating and where 'beg' and 'end' fall back
>>>> to (point-min) and (point-max).
>>> 
>>> It seems not optimized since in my tests outline-minor-mode
>>> is much slower with this change.
>> 
>> Hey, sorry for the delay. I've been busy with real life. As you said,
>> `treesit-update-ranges` doesn’t have any optimization for repeated
>> calls.
> 
> That seems like a potential problem for existing uses of that function
> (we're just "lucky" that they're expected to be used once-per-command).

Kind of, yeah. Most of the time fontification updates the ranges early enough that other lisp commands/functions never see the state when the ranges are not up to date.

> 
>> OTOH, `treesit--pre-redisplay` has optimization for repeated calls
>> while buffer content doesn’t change. Maybe the patch can use that instead?
> 
> Like in the patch below?
> 
> That sounds OK, but then `treesit--pre-redisplay` needs to be renamed to
> something like `treesit--update` or `treesit--sync`, and maybe
> `treesit-font-lock-fontify-region`, `treesit-indent`, and
> `treesit-indent-region` should use that function instead of
> `treesit-update-ranges`?
> 

Yeah. On top of that, we can try calling `treesit--pre-redisplay` in `treesit-node-at`, which should ensure range is always up-to-date. The downside is the potential cost of updating range for the whole buffer, especially when an edit changes actually does affect the whole buffer (like inserting /* in C) and we have to add/remove hundred of local parsers (eg, for doxygen).

Yuan

Message #32 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Yuan Fu <casouri <at> gmail.com>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Thu, 22 May 2025 20:06:07 +0300

> @@ -4064,6 +4065,7 @@ treesit-outline-search
>    "Search for the next outline heading in the syntax tree.
>  For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
>  `outline-search-function'."
> +  (treesit--pre-redisplay)
>    (if looking-at
>        (when (treesit-outline--at-point) (pos-bol))

Alas, the same error:

Error: treesit-node-outdated (#<treesit-node-outdated>)
  treesit-node-start(#<treesit-node-outdated>)
  treesit-node-enclosed-p(#<treesit-node-outdated> #<treesit-node element in 26-373>)
  treesit-navigate-thing(252 1 beg html-ts-mode--outline-predicate)
  treesit-outline-search(#<marker at 274 in html-multi.html>)
  outline-map-region(#<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_48> 253 274)
  outline--fix-up-all-buttons(253 274)
  outline--fix-buttons(253 257)
  #f(compiled-function (fun) #<bytecode 0x1bafbf8e90b212bf>)(outline--fix-buttons)
  run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x1bafbf8e90b212bf>) outline--fix-buttons)
  jit-lock--run-functions(253 257)
  jit-lock-fontify-now(253 401)
  jit-lock-function(253)
  redisplay_internal\ \(C\ function\)()

Message #35 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Sun, 25 May 2025 20:55:05 -0700

> On May 22, 2025, at 10:06 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>> @@ -4064,6 +4065,7 @@ treesit-outline-search
>>   "Search for the next outline heading in the syntax tree.
>> For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
>> `outline-search-function'."
>> +  (treesit--pre-redisplay)
>>   (if looking-at
>>       (when (treesit-outline--at-point) (pos-bol))
> 
> Alas, the same error:

Gah! Sorry, I don’t have time to properly look into this right now. But I’ll try to find some time as soon as possible (pinky promise).

Yuan

Message #38 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Sun, 1 Jun 2025 15:31:01 -0700

> On May 25, 2025, at 8:55 PM, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>> On May 22, 2025, at 10:06 AM, Juri Linkov <juri <at> linkov.net> wrote:
>> 
>>> @@ -4064,6 +4065,7 @@ treesit-outline-search
>>>  "Search for the next outline heading in the syntax tree.
>>> For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
>>> `outline-search-function'."
>>> +  (treesit--pre-redisplay)
>>>  (if looking-at
>>>      (when (treesit-outline--at-point) (pos-bol))
>> 
>> Alas, the same error:
> 
> Gah! Sorry, I don’t have time to properly look into this right now. But I’ll try to find some time as soon as possible (pinky promise).

Hi Juri, how did you produce this error? I tried to use the liquid-generic-ts-mode example in bug#77256 but that didn’t trigger the error.

BTW, I noticed that when tree-sitter-provided outline is in effect, pressing tab on the lines determined as outline titles—like first line of a defun—toggles folding instead of indenting the code. That feels intrusive IMHO. It’s not a problem for “traditional” kind of outline headers in comments, but code is another story.

Take this code as example:

function MyBigFunc(param1
                  param2) {
  const abc = 'def';
  function embed() {
    return true;
  }
}

Right now I can’t indent the embed function, because pressing TAB folds it. Another common practice is to select the whole MyBigFunc function and press TAB to indent everything in the region, that wouldn’t work if point happens to be on the first line, which is common.

Maybe I should create a separate bug report to discuss/address this?

Yuan

Message #41 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Tue, 03 Jun 2025 18:29:45 +0300

>>>> @@ -4064,6 +4065,7 @@ treesit-outline-search
>>>>  "Search for the next outline heading in the syntax tree.
>>>> For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
>>>> `outline-search-function'."
>>>> +  (treesit--pre-redisplay)
>>>>  (if looking-at
>>>>      (when (treesit-outline--at-point) (pos-bol))
>>> 
>>> Alas, the same error:
>> 
>> Gah! Sorry, I don’t have time to properly look into this right now. But I’ll try to find some time as soon as possible (pinky promise).
>
> Hi Juri, how did you produce this error? I tried to use the
> liquid-generic-ts-mode example in bug#77256 but that didn’t trigger
> the error.

You can produce this error by visiting test/manual/indent/html-multi.html
with enabled liquid-generic-ts-mode, then delete the word "script"
(outside of comments).

> BTW, I noticed that when tree-sitter-provided outline is in effect,
> pressing tab on the lines determined as outline titles—like first line
> of a defun—toggles folding instead of indenting the code. That feels
> intrusive IMHO. It’s not a problem for “traditional” kind of outline
> headers in comments, but code is another story.
>
> Take this code as example:
>
> function MyBigFunc(param1
>                   param2) {
>   const abc = 'def';
>   function embed() {
>     return true;
>   }
> }
>
> Right now I can’t indent the embed function, because pressing TAB
> folds it. Another common practice is to select the whole MyBigFunc
> function and press TAB to indent everything in the region, that
> wouldn’t work if point happens to be on the first line, which is
> common.

Indeed, there is a clash between TAB keybindings.
So to address this ambiguity we created a special option
'outline-minor-mode-cycle-filter' that defines in what context
TAB should fold outlines.  For example, you can customize:

  (setopt outline-minor-mode-cycle-filter (lambda () (bolp)))

Then TAB will fold only when pressed at the beginning of the line.
Anywhere else TAB will indent the line.

Message #44 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Wed, 4 Jun 2025 00:30:59 -0700

> On Jun 3, 2025, at 8:29 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>>>>> @@ -4064,6 +4065,7 @@ treesit-outline-search
>>>>> "Search for the next outline heading in the syntax tree.
>>>>> For BOUND, MOVE, BACKWARD, LOOKING-AT, see the descriptions in
>>>>> `outline-search-function'."
>>>>> +  (treesit--pre-redisplay)
>>>>> (if looking-at
>>>>>     (when (treesit-outline--at-point) (pos-bol))
>>>> 
>>>> Alas, the same error:
>>> 
>>> Gah! Sorry, I don’t have time to properly look into this right now. But I’ll try to find some time as soon as possible (pinky promise).
>> 
>> Hi Juri, how did you produce this error? I tried to use the
>> liquid-generic-ts-mode example in bug#77256 but that didn’t trigger
>> the error.
> 
> You can produce this error by visiting test/manual/indent/html-multi.html
> with enabled liquid-generic-ts-mode, then delete the word "script"
> (outside of comments).

Hmm, not sure what I’m doing wrong but I still can’t trigger the error. I’m using emacs -q, with Stefan’s patch, enabled liquid mode and outline-minor-mode, and deleted the <script> tag on line 19. Any ideas?

> 
>> BTW, I noticed that when tree-sitter-provided outline is in effect,
>> pressing tab on the lines determined as outline titles—like first line
>> of a defun—toggles folding instead of indenting the code. That feels
>> intrusive IMHO. It’s not a problem for “traditional” kind of outline
>> headers in comments, but code is another story.
>> 
>> Take this code as example:
>> 
>> function MyBigFunc(param1
>>                  param2) {
>>  const abc = 'def';
>>  function embed() {
>>    return true;
>>  }
>> }
>> 
>> Right now I can’t indent the embed function, because pressing TAB
>> folds it. Another common practice is to select the whole MyBigFunc
>> function and press TAB to indent everything in the region, that
>> wouldn’t work if point happens to be on the first line, which is
>> common.
> 
> Indeed, there is a clash between TAB keybindings.
> So to address this ambiguity we created a special option
> 'outline-minor-mode-cycle-filter' that defines in what context
> TAB should fold outlines.  For example, you can customize:
> 
>  (setopt outline-minor-mode-cycle-filter (lambda () (bolp)))
> 
> Then TAB will fold only when pressed at the beginning of the line.
> Anywhere else TAB will indent the line.

Thanks. Digging further, I’m only hitting this problem because I have outline-minor-mode-cycle set to t. So technically with default settings, one wouldn’t have this problem. 

But I can’t be the only one that sets outline-minor-mode-cycle to t, expecting to only cycle outline sections in comments. Can we do something to prevent people tripping on this? Maybe change the default value of outline-minor-mode-cycle-filter to exclude code?

Yuan

Message #47 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Wed, 04 Jun 2025 20:16:02 +0300

> Hmm, not sure what I’m doing wrong but I still can’t trigger the
> error. I’m using emacs -q, with Stefan’s patch, enabled liquid mode
> and outline-minor-mode, and deleted the <script> tag on line 19.
> Any ideas?

Sorry, I missed essential non-default settings:

  (setq outline-minor-mode-cycle t)
  (setq outline-minor-mode-use-buttons t)

Also (setq debug-on-error t) is nice to have.
So here is a complete test case:

0. emacs -Q
1. M-: (setq outline-minor-mode-cycle t)
2. M-: (setq outline-minor-mode-use-buttons t)
3. M-: (setq debug-on-error t)
4. C-x p f test/manual/indent/html-multi.html
5. M-x load-library RET treesit-x
6. M-x liquid-generic-ts-mode
7. M-x outline-minor-mode
8. move point to the first 's' in 'script' on line 19
9. type 'M-d' ('kill-word')

>>> BTW, I noticed that when tree-sitter-provided outline is in effect,
>>> pressing tab on the lines determined as outline titles—like first line
>>> of a defun—toggles folding instead of indenting the code. That feels
>>> intrusive IMHO. It’s not a problem for “traditional” kind of outline
>>> headers in comments, but code is another story.
>>> 
>>> Take this code as example:
>>> 
>>> function MyBigFunc(param1
>>>                  param2) {
>>>  const abc = 'def';
>>>  function embed() {
>>>    return true;
>>>  }
>>> }
>>> 
>>> Right now I can’t indent the embed function, because pressing TAB
>>> folds it. Another common practice is to select the whole MyBigFunc
>>> function and press TAB to indent everything in the region, that
>>> wouldn’t work if point happens to be on the first line, which is
>>> common.
>> 
>> Indeed, there is a clash between TAB keybindings.
>> So to address this ambiguity we created a special option
>> 'outline-minor-mode-cycle-filter' that defines in what context
>> TAB should fold outlines.  For example, you can customize:
>> 
>>  (setopt outline-minor-mode-cycle-filter (lambda () (bolp)))
>> 
>> Then TAB will fold only when pressed at the beginning of the line.
>> Anywhere else TAB will indent the line.
>
> Thanks. Digging further, I’m only hitting this problem because I have
> outline-minor-mode-cycle set to t. So technically with default
> settings, one wouldn’t have this problem.
>
> But I can’t be the only one that sets outline-minor-mode-cycle to t,
> expecting to only cycle outline sections in comments. Can we do something
> to prevent people tripping on this? Maybe change the default value of
> outline-minor-mode-cycle-filter to exclude code?

Do you use the default 'outline-regexp'?  For example in emacs-lisp-mode
it matches not only comments, but also opening parens at the beginning
of function definitions in code.  So I don't understand how it would be
useful to fold only comments, but not code.

Message #50 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Fri, 6 Jun 2025 00:55:04 -0700

> On Jun 4, 2025, at 10:16 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>> Hmm, not sure what I’m doing wrong but I still can’t trigger the
>> error. I’m using emacs -q, with Stefan’s patch, enabled liquid mode
>> and outline-minor-mode, and deleted the <script> tag on line 19.
>> Any ideas?
> 
> Sorry, I missed essential non-default settings:
> 
>  (setq outline-minor-mode-cycle t)
>  (setq outline-minor-mode-use-buttons t)
> 
> Also (setq debug-on-error t) is nice to have.
> So here is a complete test case:
> 
> 0. emacs -Q
> 1. M-: (setq outline-minor-mode-cycle t)
> 2. M-: (setq outline-minor-mode-use-buttons t)
> 3. M-: (setq debug-on-error t)
> 4. C-x p f test/manual/indent/html-multi.html
> 5. M-x load-library RET treesit-x
> 6. M-x liquid-generic-ts-mode
> 7. M-x outline-minor-mode
> 8. move point to the first 's' in 'script' on line 19
> 9. type 'M-d' ('kill-word')

Aha, I reproduced it. It didn’t happen when I kill-word, but happened when I undo. I’m still trying to understand why the node-outdated error can happen. Each parser has a tick, which is incremented whenever it re-parses. When we create a node from the parser, the node inherits the current tick number of the parser. We only consider a node outdated when its tick is less than its parser’s tick, meaning the parser re-parsed after the node is created. For a parser to re-parse, it has to a) receive a buffer edit or b) change it’s ranges; and then someone needs to request a node from it.

That means in the following backtrace, the parser received a buffer edit or changed its ranges after the `prev` node is created. And that’s baffling to me.

  treesit-node-start(#<treesit-node-outdated>)
  (if (consp smaller) (car smaller) (treesit-node-start smaller))
  (let ((larger-start (if (consp larger) (car larger) (treesit-node-start larger))) (larger-end (if (consp larger) (cdr larger) (treesit-node-end larger))) (smaller-start (if (consp smaller) (car smaller) (treesit-node-start smaller))) (smaller-end (if (consp smaller) (cdr smaller) (treesit-node-end smaller)))) (cond ((eq strict 't) (let nil (and (< larger-start smaller-start) (< smaller-end larger-end)))) ((eq strict 'partial) (let nil (and (or (not (eq larger-start smaller-start)) (not (eq larger-end smaller-end))) (<= larger-start smaller-start smaller-end larger-end)))) (t (let nil (<= larger-start smaller-start smaller-end larger-end)))))
  treesit-node-enclosed-p(#<treesit-node-outdated> #<treesit-node element in 26-379>)
  (not (treesit-node-enclosed-p prev parent))
  (and parent prev (not (treesit-node-enclosed-p prev parent)))
  (if (and parent prev (not (treesit-node-enclosed-p prev parent))) (progn (setq prev nil)))
  (let ((prev (treesit-thing-prev pos thing)) (next (treesit-thing-next pos thing)) (parent (treesit-thing-at pos thing t))) (if (and parent prev (not (treesit-node-enclosed-p prev parent))) (progn (setq prev nil))) (if (and parent next (not (treesit-node-enclosed-p next parent))) (progn (setq next nil))) (if (and (eq tactic 'top-level) parent) (progn (progn (setq parent (treesit-node-top-level parent thing t)) (setq prev nil) (setq next nil)))) (if (eq tactic 'restricted) (setq pos (funcall advance (cond ((and (null next) (null prev)) parent) ((> arg 0) next) (t prev)))) (if (> arg 0) (if (and (eq side 'beg) (cond (next (and ... ...)) (parent t))) (setq pos (or (treesit-navigate-thing (treesit-node-end ...) 1 'beg thing tactic t) (throw 'term nil))) (setq pos (funcall advance (or next parent)))) (if (and (eq side 'end) (cond (prev (and ... ...)) (parent t))) (setq pos (or (treesit-navigate-thing (treesit-node-start ...) -1 'end thing tactic t) (throw 'term nil))) (setq pos (funcall advance (or prev parent)))))) (setq counter (- counter 1)))



> 
>>>> BTW, I noticed that when tree-sitter-provided outline is in effect,
>>>> pressing tab on the lines determined as outline titles—like first line
>>>> of a defun—toggles folding instead of indenting the code. That feels
>>>> intrusive IMHO. It’s not a problem for “traditional” kind of outline
>>>> headers in comments, but code is another story.
>>>> 
>>>> Take this code as example:
>>>> 
>>>> function MyBigFunc(param1
>>>>                 param2) {
>>>> const abc = 'def';
>>>> function embed() {
>>>>   return true;
>>>> }
>>>> }
>>>> 
>>>> Right now I can’t indent the embed function, because pressing TAB
>>>> folds it. Another common practice is to select the whole MyBigFunc
>>>> function and press TAB to indent everything in the region, that
>>>> wouldn’t work if point happens to be on the first line, which is
>>>> common.
>>> 
>>> Indeed, there is a clash between TAB keybindings.
>>> So to address this ambiguity we created a special option
>>> 'outline-minor-mode-cycle-filter' that defines in what context
>>> TAB should fold outlines.  For example, you can customize:
>>> 
>>> (setopt outline-minor-mode-cycle-filter (lambda () (bolp)))
>>> 
>>> Then TAB will fold only when pressed at the beginning of the line.
>>> Anywhere else TAB will indent the line.
>> 
>> Thanks. Digging further, I’m only hitting this problem because I have
>> outline-minor-mode-cycle set to t. So technically with default
>> settings, one wouldn’t have this problem.
>> 
>> But I can’t be the only one that sets outline-minor-mode-cycle to t,
>> expecting to only cycle outline sections in comments. Can we do something
>> to prevent people tripping on this? Maybe change the default value of
>> outline-minor-mode-cycle-filter to exclude code?
> 
> Do you use the default 'outline-regexp'?  For example in emacs-lisp-mode
> it matches not only comments, but also opening parens at the beginning
> of function definitions in code.  So I don't understand how it would be
> useful to fold only comments, but not code.

Ah, I guess never pressed TAB on the first line of a defun in elisp mode. I still think it’ll be problematic on languages that has nested defun (eg, javascript). But maybe I’m being old man yell at cloud. I guess we can see if there’s someone that actually complains about it.

Yuan

Message #53 received at 78402 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 78402 <at> debbugs.gnu.org
Subject: Re: bug#78402: treesit after-change-functions
Date: Fri, 06 Jun 2025 18:42:57 +0300

>> 1. M-: (setq outline-minor-mode-cycle t)
>> 2. M-: (setq outline-minor-mode-use-buttons t)
>> 3. M-: (setq debug-on-error t)
>> 4. C-x p f test/manual/indent/html-multi.html
>> 5. M-x load-library RET treesit-x
>> 6. M-x liquid-generic-ts-mode
>> 7. M-x outline-minor-mode
>> 8. move point to the first 's' in 'script' on line 19
>> 9. type 'M-d' ('kill-word')
>
> Aha, I reproduced it. It didn’t happen when I kill-word, but happened when
> I undo.

This is what I observe too: when it doesn't fail on edit,
then in any case it fails on undo.

> I’m still trying to understand why the node-outdated error can
> happen. Each parser has a tick, which is incremented whenever it
> re-parses. When we create a node from the parser, the node inherits the
> current tick number of the parser. We only consider a node outdated when
> its tick is less than its parser’s tick, meaning the parser re-parsed after
> the node is created. For a parser to re-parse, it has to a) receive
> a buffer edit or b) change it’s ranges; and then someone needs to request
> a node from it.
>
> That means in the following backtrace, the parser received a buffer
> edit or changed its ranges after the `prev` node is created. And
> that’s baffling to me.

Maybe calling this from after-change-functions affects the order of
calling parser functions.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.