GNU bug report logs - #60894
30.0.50; [PATCH] Add treesit-forward-sexp

Previous Next

Package: emacs;

Reported by: Theodor Thornhill <theo <at> thornhill.no>

Date: Tue, 17 Jan 2023 20:45:01 UTC

Severity: normal

Tags: patch

Found in version 30.0.50

Fixed in version 30.1

Done: Theodor Thornhill <theo <at> thornhill.no>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 60894 in the body.
You can then email your comments to 60894 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to juri <at> linkov.net, mickey <at> masteringemacs.org, monnier <at> iro.umontreal.ca, bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Tue, 17 Jan 2023 20:45:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Theodor Thornhill <theo <at> thornhill.no>:
New bug report received and forwarded. Copy sent to juri <at> linkov.net, mickey <at> masteringemacs.org, monnier <at> iro.umontreal.ca, bug-gnu-emacs <at> gnu.org. (Tue, 17 Jan 2023 20:45:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Tue, 17 Jan 2023 21:44:35 +0100
[Message part 1 (text/plain, inline)]
Hi Emacs (also Juri and Mickey as you've expressed some interest for this)

This is an example patch for sexp movement with tree sitter.  I want to
put it out here to hopefully produce some discussion, sooner rather than
later.

Three initial questions:

1. What should a sexp be?

  Is it basically "everything", or is there a distincition between
  "word", "sexp" and "sentence"?  For lisp forward-sexp looks like a
  "jump over words, or a balanced pair of parens".  In other languages
  that can look a little weird - consider:

  ```
  foo().|bar().baz(); -> foo().bar|().baz(); -> foo().bar()|.baz();
  ```
  
  In a sense it could be considered "better", or at least distinct from
  forward-word to:

  ```
  foo().|bar().baz(); -> foo().bar()|.baz(); -> foo().bar().baz()|;
  ```

2. Should this new function be leveraged in transpose-sexps?

  IMO if the forward-sexp gets too close to forward-word, or
  forward-sentence we lose some nice properties with the current
  'treesit-transpose-sexps', namely (among others):

  ```
  f(String foo,| Integer bar) ->  void foo(Integer bar, String foo|)
  ```

  I know you Mickey have expressed some dissatisfaction with the current
  implementation - now is a good time to make some worthwhile
  improvements.


3. What are the "rules"?

  In c-mode, elisp-mode without paredit forward-sexp won't jump out of
  the current scope, however with paredit enabled it will.


If we simply want some code similar to this to live and slowly evolve I
guess we can install something like this patch after some tweaks and
iterate when we have more experience.

Anyway, I hope these questions and thoughs will spark some discussion,

Theo


[0001-Add-treesit-forward-sexp.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Tue, 17 Jan 2023 20:54:02 GMT) Full text and rfc822 format available.

Message #8 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Theodor Thornhill <theo <at> thornhill.no>, 60894 <at> debbugs.gnu.org
Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Tue, 17 Jan 2023 22:53:09 +0200
On 17/01/2023 22:44, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> 1. What should a sexp be?
> 
>    Is it basically "everything", or is there a distincition between
>    "word", "sexp" and "sentence"?  For lisp forward-sexp looks like a
>    "jump over words, or a balanced pair of parens".  In other languages
>    that can look a little weird - consider:
> 
>    ```
>    foo().|bar().baz(); -> foo().bar|().baz(); -> foo().bar()|.baz();
>    ```
>    
>    In a sense it could be considered "better", or at least distinct from
>    forward-word to:
> 
>    ```
>    foo().|bar().baz(); -> foo().bar()|.baz(); -> foo().bar().baz()|;

One of the key things for Ruby, I think, is to jump over expressions.

E.g. when the point is before 'def' in

  def foo
    ...
  end

forward-sexp jumps to after 'end'. And backward-sexp jumps back.

Same for

  if 2 == 3
    ...
  end

, parenthesized expressions and (less important) method calls and 
statements as well.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Tue, 17 Jan 2023 21:08:02 GMT) Full text and rfc822 format available.

Message #11 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Dmitry Gutov <dgutov <at> yandex.ru>, 60894 <at> debbugs.gnu.org
Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Tue, 17 Jan 2023 22:07:45 +0100
[Message part 1 (text/plain, inline)]
Dmitry Gutov <dgutov <at> yandex.ru> writes:

> On 17/01/2023 22:44, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>> 1. What should a sexp be?
>> 
>>    Is it basically "everything", or is there a distincition between
>>    "word", "sexp" and "sentence"?  For lisp forward-sexp looks like a
>>    "jump over words, or a balanced pair of parens".  In other languages
>>    that can look a little weird - consider:
>> 
>>    ```
>>    foo().|bar().baz(); -> foo().bar|().baz(); -> foo().bar()|.baz();
>>    ```
>>    
>>    In a sense it could be considered "better", or at least distinct from
>>    forward-word to:
>> 
>>    ```
>>    foo().|bar().baz(); -> foo().bar()|.baz(); -> foo().bar().baz()|;
>
> One of the key things for Ruby, I think, is to jump over expressions.
>
> E.g. when the point is before 'def' in
>
>    def foo
>      ...
>    end
>
> forward-sexp jumps to after 'end'. And backward-sexp jumps back.
>
> Same for
>
>    if 2 == 3
>      ...
>    end
>
> , parenthesized expressions and (less important) method calls and 
> statements as well.


Test this very untested addition to the patch (I know no ruby).  It
seems to do what you want.  I'd consider this sentence movement, though.
For M-e

Theo

[0001-Add-treesit-forward-sexp.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 01:51:02 GMT) Full text and rfc822 format available.

Message #14 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Theodor Thornhill <theo <at> thornhill.no>, 60894 <at> debbugs.gnu.org
Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 03:50:41 +0200
On 17/01/2023 23:07, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Test this very untested addition to the patch (I know no ruby).  It
> seems to do what you want.  I'd consider this sentence movement, though.
> For M-e

That seems to be working rather well, thanks. I just needed to extend 
the list of nodes:

  (setq-local treesit-sexp-type-regexp
              (regexp-opt '("class"
                            "module"
                            "method"
                            "argument_list"
                            "array"
                            "hash"
                            "parenthesized_statements"
                            "if"
                            "case"
                            "block"
                            "do_block"
                            "begin")))

With array, hash, etc, you see it's not exactly like a sentence.

Regarding your previous question -- whether forward-sexp should jump 
over the arglist together with the called method name -- ruby-mode's 
answer to that is:

- If point is before ".", jump over ".bar(...)".
- If point is after ".", jump over "bar" only.

But the difference is more subtle here, and different people might have 
different preferences. This also seems more difficult to express via 
node types since "." is in the middle of the (call) node.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 05:36:01 GMT) Full text and rfc822 format available.

Message #17 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Dmitry Gutov <dgutov <at> yandex.ru>, 60894 <at> debbugs.gnu.org
Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 06:35:36 +0100
[Message part 1 (text/plain, inline)]
Dmitry Gutov <dgutov <at> yandex.ru> writes:

> On 17/01/2023 23:07, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>> Test this very untested addition to the patch (I know no ruby).  It
>> seems to do what you want.  I'd consider this sentence movement, though.
>> For M-e
>
> That seems to be working rather well, thanks. I just needed to extend 
> the list of nodes:
>
>    (setq-local treesit-sexp-type-regexp
>                (regexp-opt '("class"
>                              "module"
>                              "method"
>                              "argument_list"
>                              "array"
>                              "hash"
>                              "parenthesized_statements"
>                              "if"
>                              "case"
>                              "block"
>                              "do_block"
>                              "begin")))
>
> With array, hash, etc, you see it's not exactly like a sentence.
>
> Regarding your previous question -- whether forward-sexp should jump 
> over the arglist together with the called method name -- ruby-mode's 
> answer to that is:
>
> - If point is before ".", jump over ".bar(...)".
> - If point is after ".", jump over "bar" only.
>
> But the difference is more subtle here, and different people might have 
> different preferences. This also seems more difficult to express via 
> node types since "." is in the middle of the (call) node.

Yeah, it's not the easiest problem, but we can wait until we get more
experience in other modes too to solidify this design.  It is not for
pemacs-29 anyway, so we have time.

Added some words to the manual and your node types to ruby-ts-mode

Theo

[0001-Add-treesit-forward-sexp.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 08:10:02 GMT) Full text and rfc822 format available.

Message #20 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Mickey Petersen <mickey <at> masteringemacs.org>
To: Theodor Thornhill <theo <at> thornhill.no>
Cc: 60894 <at> debbugs.gnu.org, debbugs-submit <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Tue, 17 Jan 2023 21:13:49 +0000
Theodor Thornhill <theo <at> thornhill.no> writes:

> Hi Emacs (also Juri and Mickey as you've expressed some interest for this)
>
> This is an example patch for sexp movement with tree sitter.  I want to
> put it out here to hopefully produce some discussion, sooner rather than
> later.
>
> Three initial questions:
>
> 1. What should a sexp be?
>

`forward-sexp' and friends behave exactly as they should behave. If
there are sexp-like things on the left and/or right-hand side of
point, notwithstanding certain punction in the direction of travel,
then it uses that to determine where to move; where to kill to; and
the boundaries for transposition.

Absent an sexp, it defaults to word-based movement. Exactly as it should.

With that out of the way. That does not mean that we cannot extend its
behaviour in ways that are both predictable but also useful.

It should know about things the default `syntax-ppss' machinery cannot
figure out for itself.

Consider HTML (or anything in SGML club of languages). I would expect
sexp movement to move over a matched pair of tags. It currently does
not; the reason why is understandable when you know how `syntax-ppss'
does (or does not, as it were) work. (You can equally make an argument
that it should simply go to the end of an opening/closing node and not
the pair, but that is personal preference.)

`nxml-mode' handles it properly; Combobulate handles it properly, too.
But Combobulate also falls back to the classic sexp behaviour if it
cannot find a suitable node in the direction of travel.

And that is the key thing here. It augments; it does not replace.


>   Is it basically "everything", or is there a distincition between
>   "word", "sexp" and "sentence"?  For lisp forward-sexp looks like a
>   "jump over words, or a balanced pair of parens".  In other languages
>   that can look a little weird - consider:
>

Yes, there are distinctions, and no, sexp-based movement is not the
only game in town. Certainly not pre-TS; and definitely not after.

Word movement is a subset of sexp-based movement. Sentence and
paragraph-based movement is rarely altered in coding major modes. But M-{
and M-} are still useful for moving through contiguous blocks of code
separated by blank lines. Not its intended use, but absent something
better, it's OK. Room for improvement for sure.


>   ```
>   foo().|bar().baz(); -> foo().bar|().baz(); -> foo().bar()|.baz();
>   ```
>

Subject to the vagaries of the syntax table, yes, that's right. It's
also useful. I can punch `C-M-k' twice and gobble up the caller and
whatever is in the brackets. That is not possible with word-based
killing alone.

>   In a sense it could be considered "better", or at least distinct from
>   forward-word to:
>
>   ```
>   foo().|bar().baz(); -> foo().bar()|.baz(); -> foo().bar().baz()|;
>   ```
>

That is not how word movement works on my machine:

    foo().bar().baz();
          |  |     |

That is the path of travel for `M-f' in C-mode. This is how `C-M-f'
moves:

    foo().bar().baz();
          |  | |   |

> 2. Should this new function be leveraged in transpose-sexps?
>
>   IMO if the forward-sexp gets too close to forward-word, or
>   forward-sentence we lose some nice properties with the current
>   'treesit-transpose-sexps', namely (among others):
>
>   ```
>   f(String foo,| Integer bar) ->  void foo(Integer bar, String foo|)
>   ```
>
>   I know you Mickey have expressed some dissatisfaction with the current
>   implementation - now is a good time to make some worthwhile
>   improvements.

My dissatisfaction was merely the approach. You cannot transpose
sibling nodes and expect that to behave in a manner that is logical to
anyone who is not fluent in the specifics of every TS grammar they
interact with.

Things that appear to be 'siblings' to the human eye may in fact not
be that at all.

And some grammars are rather more generous with their level of
granularity than others: both in how expansive they are, but also in
how deep the trees are. So when you go foraging for nodes to
transpose, you may end up finding sub-nodes to the ones you'd actually
want to transpose because they are smaller and closer units.

>
> 3. What are the "rules"?
>
>   In c-mode, elisp-mode without paredit forward-sexp won't jump out of
>   the current scope, however with paredit enabled it will.
>
>
> If we simply want some code similar to this to live and slowly evolve I
> guess we can install something like this patch after some tweaks and
> iterate when we have more experience.
>

The problem with that patch is -- and caveat, I do not use Java, so I
am not too versed in the details of the note types used -- but most of
the things I see are things that a normal sexp would work with anyway:

- true, false: they are just words, right?
- identifier: presumably a symbol or something that a decent syntax
  table would probably catch.
- parenthesized_expression: delimited by ( ), etc. ? Sexp handles that
- just fine.
- argument_list: presumably uses ( ) also


> Anyway, I hope these questions and thoughs will spark some discussion,
>
> Theo
>
>
> [2. text/x-patch; 0001-Add-treesit-forward-sexp.patch]...





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 12:22:01 GMT) Full text and rfc822 format available.

Message #23 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Theodor Thornhill <theo <at> thornhill.no>
Cc: 60894 <at> debbugs.gnu.org, juri <at> linkov.net, mickey <at> masteringemacs.org,
 monnier <at> iro.umontreal.ca, dgutov <at> yandex.ru
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 14:21:20 +0200
> Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
>  Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
> Date: Wed, 18 Jan 2023 06:35:36 +0100
> From:  Theodor Thornhill via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> diff --git a/doc/lispref/positions.texi b/doc/lispref/positions.texi
> index 8d95ecee7a..8b7431096f 100644
> --- a/doc/lispref/positions.texi
> +++ b/doc/lispref/positions.texi
> @@ -875,6 +875,23 @@ List Motion
>  @code{backward-sentence}(@pxref{Moving by Sentences,,, emacs, The
>  extensible self-documenting text editor}).
>  
> +@defvar treesit-sexp-type-regexp
> +The value of this variable is a regexp matching the node type of sexp
> +nodes.  (For ``node'' and ``node type'', @pxref{Parsing Program
> +Source}.)
> +@end defvar
> +
> +@findex treesit-forward-sexp
> +@findex forward-sexp
> +@findex backward-sexp
> +If Emacs is compiled with tree-sitter, it can use the tree-sitter
> +parser information to move across syntax constructs.  Since what
> +exactly is considered a sexp varies between languages, a major mode
> +should set @code{treesit-sentence-type-regexp} to determine that.
> +Then the mode can get navigation-by-sexp functionality for free, by
> +using @code{forward-sexp} and @code{backward-sexp}(@pxref{Moving by
> +Sentences,,, emacs, The extensible self-documenting text editor}).

The last 2 index entries should either not be there or be qualified:

  @findex forward-sexp <at> r{, and tree-sitter}

That's because these two commands are described in another place in
the manual, and the unqualified index entries should go there.
Otherwise, if the readers type in Info "i forward-s TAB", they will
see something like this:

  forward-sexp
  forward-sexp<1>

and will not know which one of these two they want.  And similarly in
the printed version of the manual: you will get "forward-sexp" with 2
page numbers -- which one is the one you are looking for?

> +*** New function 'treesit-forward-sexp'.
> +treesit.el conditionally sets 'forward-sexp-function` for major modes
> +that have defined 'treesit-sexp-type-regexp' to enable sexp related
> +commands.

"sexp-related motion commands" should be better.  (If "motion" is not
inclusive enough, we could use additional descriptions.)

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 13:32:02 GMT) Full text and rfc822 format available.

Message #26 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Mickey Petersen <mickey <at> masteringemacs.org>,
 Theodor Thornhill <theo <at> thornhill.no>
Cc: 60894 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 15:31:22 +0200
On 17/01/2023 23:13, Mickey Petersen wrote:
>> 3. What are the "rules"?
>>
>>    In c-mode, elisp-mode without paredit forward-sexp won't jump out of
>>    the current scope, however with paredit enabled it will.
>>
>>
>> If we simply want some code similar to this to live and slowly evolve I
>> guess we can install something like this patch after some tweaks and
>> iterate when we have more experience.
>>
> The problem with that patch is -- and caveat, I do not use Java, so I
> am not too versed in the details of the note types used -- but most of
> the things I see are things that a normal sexp would work with anyway:
> 
> - true, false: they are just words, right?
> - identifier: presumably a symbol or something that a decent syntax
>    table would probably catch.
> - parenthesized_expression: delimited by ( ), etc. ? Sexp handles that
> - just fine.
> - argument_list: presumably uses ( ) also

Yeah, it seems like in C-based languages (where any block of code is 
surrounded by curlier) the default forward-sexp should behave decently 
already.

Whether forward-sexp should jump over the whole if () { ... } 
expression, for example, is a matter of taste. Not so with e.g. Python, 
where an 'if' statement can have zero parens and no visible terminator.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 13:40:02 GMT) Full text and rfc822 format available.

Message #29 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Theodor Thornhill <theo <at> thornhill.no>, 60894 <at> debbugs.gnu.org
Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 15:39:07 +0200
On 18/01/2023 07:35, Theodor Thornhill wrote:
> Yeah, it's not the easiest problem, but we can wait until we get more
> experience in other modes too to solidify this design.  It is not for
> pemacs-29 anyway, so we have time.
> 
> Added some words to the manual and your node types to ruby-ts-mode

Thanks! As far as I'm concerned, this is good for master, thank you.

Consider looking into an additional couple of commands which (probably) 
could be implemented using the same starting data: backward-up-list and 
down-list. These have been pretty useful to me.

backward-up-list might even work automatically with a sufficiently 
conforming forward-sexp-function (it does with SMIE); not sure about 
down-list -- SMIE has a dedicated implementation of it.

And show-paren-mode support would be nice to have too -- though it'll 
probably require a separate defcustom with node types.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 17:42:01 GMT) Full text and rfc822 format available.

Message #32 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Mickey Petersen <mickey <at> masteringemacs.org>
Cc: 60894 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 19:09:29 +0200
> Consider HTML (or anything in SGML club of languages). I would expect
> sexp movement to move over a matched pair of tags. It currently does
> not; the reason why is understandable when you know how `syntax-ppss'
> does (or does not, as it were) work. (You can equally make an argument
> that it should simply go to the end of an opening/closing node and not
> the pair, but that is personal preference.)
>
> `nxml-mode' handles it properly; Combobulate handles it properly, too.
> But Combobulate also falls back to the classic sexp behaviour if it
> cannot find a suitable node in the direction of travel.

While ‘forward-sexp’ moves over the next tag, there is also ‘C-c C-f’
(‘sgml-skip-tag-forward’) that moves over the whole element to the end tag.
I'm not sure if sexp movement in nxml-mode is an improvement since
there is no way to move over the tag only.

To support both cases maybe ‘forward-sexp’ should move over the tag,
and ‘forward-sentence’ over the whole element?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 17:42:02 GMT) Full text and rfc822 format available.

Message #35 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Theodor Thornhill <theo <at> thornhill.no>
Cc: 60894 <at> debbugs.gnu.org, Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 19:16:06 +0200
>> That seems to be working rather well, thanks. I just needed to extend
>> the list of nodes:
>>
>>    (setq-local treesit-sexp-type-regexp
>>                (regexp-opt '("class"
>>                              "module"
>>                              "method"
>>                              "argument_list"
>>                              "array"
>>                              "hash"
>>                              "parenthesized_statements"
>>                              "if"
>>                              "case"
>>                              "block"
>>                              "do_block"
>>                              "begin")))
>
> Added some words to the manual and your node types to ruby-ts-mode

Thanks, I tried it out, and it works even better than in ruby-mode.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 18:28:02 GMT) Full text and rfc822 format available.

Message #38 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Juri Linkov <juri <at> linkov.net>, Mickey Petersen <mickey <at> masteringemacs.org>
Cc: 60894 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 19:27:34 +0100

On 18 January 2023 18:09:29 CET, Juri Linkov <juri <at> linkov.net> wrote:
>> Consider HTML (or anything in SGML club of languages). I would expect
>> sexp movement to move over a matched pair of tags. It currently does
>> not; the reason why is understandable when you know how `syntax-ppss'
>> does (or does not, as it were) work. (You can equally make an argument
>> that it should simply go to the end of an opening/closing node and not
>> the pair, but that is personal preference.)
>>
>> `nxml-mode' handles it properly; Combobulate handles it properly, too.
>> But Combobulate also falls back to the classic sexp behaviour if it
>> cannot find a suitable node in the direction of travel.
>
>While ‘forward-sexp’ moves over the next tag, there is also ‘C-c C-f’
>(‘sgml-skip-tag-forward’) that moves over the whole element to the end tag.
>I'm not sure if sexp movement in nxml-mode is an improvement since
>there is no way to move over the tag only.
>
>To support both cases maybe ‘forward-sexp’ should move over the tag,
>and ‘forward-sentence’ over the whole element?

Yes! This is what I find intuitive, and have tried to explain in my docstrings of the two.

But I'm new in town and want all to at least discuss:)

I can extract a forward-sexp-default-function to amend some of the damage?

Theo




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 18:29:01 GMT) Full text and rfc822 format available.

Message #41 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Juri Linkov <juri <at> linkov.net>
Cc: 60894 <at> debbugs.gnu.org, Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 19:27:59 +0100

On 18 January 2023 18:16:06 CET, Juri Linkov <juri <at> linkov.net> wrote:
>>> That seems to be working rather well, thanks. I just needed to extend
>>> the list of nodes:
>>>
>>>    (setq-local treesit-sexp-type-regexp
>>>                (regexp-opt '("class"
>>>                              "module"
>>>                              "method"
>>>                              "argument_list"
>>>                              "array"
>>>                              "hash"
>>>                              "parenthesized_statements"
>>>                              "if"
>>>                              "case"
>>>                              "block"
>>>                              "do_block"
>>>                              "begin")))
>>
>> Added some words to the manual and your node types to ruby-ts-mode
>
>Thanks, I tried it out, and it works even better than in ruby-mode.

I'm glad!

Theo




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 18:30:02 GMT) Full text and rfc822 format available.

Message #44 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Dmitry Gutov <dgutov <at> yandex.ru>, 60894 <at> debbugs.gnu.org
Cc: Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 19:28:48 +0100

On 18 January 2023 14:39:07 CET, Dmitry Gutov <dgutov <at> yandex.ru> wrote:
>On 18/01/2023 07:35, Theodor Thornhill wrote:
>> Yeah, it's not the easiest problem, but we can wait until we get more
>> experience in other modes too to solidify this design.  It is not for
>> pemacs-29 anyway, so we have time.
>> 
>> Added some words to the manual and your node types to ruby-ts-mode
>
>Thanks! As far as I'm concerned, this is good for master, thank you.
>
>Consider looking into an additional couple of commands which (probably) could be implemented using the same starting data: backward-up-list and down-list. These have been pretty useful to me.
>
>backward-up-list might even work automatically with a sufficiently conforming forward-sexp-function (it does with SMIE); not sure about down-list -- SMIE has a dedicated implementation of it.
>
>And show-paren-mode support would be nice to have too -- though it'll probably require a separate defcustom with node types.


Yep next on my list:)

Thanks for the feedback,
Theo




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 18:57:01 GMT) Full text and rfc822 format available.

Message #47 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Theodor Thornhill <theo <at> thornhill.no>
Cc: 60894 <at> debbugs.gnu.org, Mickey Petersen <mickey <at> masteringemacs.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 20:55:33 +0200
>>> Consider HTML (or anything in SGML club of languages). I would expect
>>> sexp movement to move over a matched pair of tags. It currently does
>>> not; the reason why is understandable when you know how `syntax-ppss'
>>> does (or does not, as it were) work. (You can equally make an argument
>>> that it should simply go to the end of an opening/closing node and not
>>> the pair, but that is personal preference.)
>>>
>>> `nxml-mode' handles it properly; Combobulate handles it properly, too.
>>> But Combobulate also falls back to the classic sexp behaviour if it
>>> cannot find a suitable node in the direction of travel.
>>
>>While ‘forward-sexp’ moves over the next tag, there is also ‘C-c C-f’
>>(‘sgml-skip-tag-forward’) that moves over the whole element to the end tag.
>>I'm not sure if sexp movement in nxml-mode is an improvement since
>>there is no way to move over the tag only.
>>
>>To support both cases maybe ‘forward-sexp’ should move over the tag,
>>and ‘forward-sentence’ over the whole element?
>
> Yes! This is what I find intuitive, and have tried to explain in my docstrings of the two.
>
> But I'm new in town and want all to at least discuss:)
>
> I can extract a forward-sexp-default-function to amend some of the damage?

Maybe in a new html-ts-mode?  I see there is libtree-sitter-html,
but can't find the corresponding ts mode.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 19:02:02 GMT) Full text and rfc822 format available.

Message #50 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Mickey Petersen <mickey <at> masteringemacs.org>, 60894 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 14:01:30 -0500
> backward-up-list might even work automatically with a sufficiently
>  conforming forward-sexp-function (it does with SMIE);

I'm happy to hear that opinion, but it's a bit too optimistic:
`backward-up-list` doesn't really work sufficiently well (IMO) just
using `forward-sexp-function`.

My SMIE experience with it is that we really need a specific hook
for that if we want to avoid wonky behaviors in "corner" cases.


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 22:00:02 GMT) Full text and rfc822 format available.

Message #53 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Mickey Petersen <mickey <at> masteringemacs.org>, 60894 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 23:59:44 +0200
On 18/01/2023 21:01, Stefan Monnier wrote:
>> backward-up-list might even work automatically with a sufficiently
>>   conforming forward-sexp-function (it does with SMIE);
> 
> I'm happy to hear that opinion, but it's a bit too optimistic:
> `backward-up-list` doesn't really work sufficiently well (IMO) just
> using `forward-sexp-function`.

It does work fairly well in my experience.

> My SMIE experience with it is that we really need a specific hook
> for that if we want to avoid wonky behaviors in "corner" cases.

A bug report with repro would probably help, I guess. Or you could start 
on that hook yourself.

Do you mean a hook like backward-up-list-function, or something smaller?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Wed, 18 Jan 2023 22:07:02 GMT) Full text and rfc822 format available.

Message #56 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Theodor Thornhill <theo <at> thornhill.no>, Juri Linkov <juri <at> linkov.net>,
 Mickey Petersen <mickey <at> masteringemacs.org>
Cc: 60894 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Thu, 19 Jan 2023 00:06:22 +0200
On 18/01/2023 20:27, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> 
> On 18 January 2023 18:09:29 CET, Juri Linkov<juri <at> linkov.net>  wrote:
>>> Consider HTML (or anything in SGML club of languages). I would expect
>>> sexp movement to move over a matched pair of tags. It currently does
>>> not; the reason why is understandable when you know how `syntax-ppss'
>>> does (or does not, as it were) work. (You can equally make an argument
>>> that it should simply go to the end of an opening/closing node and not
>>> the pair, but that is personal preference.)
>>>
>>> `nxml-mode' handles it properly; Combobulate handles it properly, too.
>>> But Combobulate also falls back to the classic sexp behaviour if it
>>> cannot find a suitable node in the direction of travel.
>> While ‘forward-sexp’ moves over the next tag, there is also ‘C-c C-f’
>> (‘sgml-skip-tag-forward’) that moves over the whole element to the end tag.
>> I'm not sure if sexp movement in nxml-mode is an improvement since
>> there is no way to move over the tag only.
>>
>> To support both cases maybe ‘forward-sexp’ should move over the tag,
>> and ‘forward-sentence’ over the whole element?
> Yes! This is what I find intuitive, and have tried to explain in my docstrings of the two.
> 
> But I'm new in town and want all to at least discuss:)

In my intuition, sexps are expressions which can be nested (a lot).

Sentences are "flat" expressions. I would probably say that in 
tree-sitter modes sentences should be equivalent to "statements".

So, sexps could be small, and they could be large. For them various list 
navigation operations make sense (like the previously mentioned ones).

Sentences stand somewhere in the middle, and they're more like 
sequential. A operation like backward-up-statement wouldn't make a lot 
of sense, however.

Note quite sure what would correspond to statement in html-mode, but I 
would put separate tag elements in the category of word, or symbols, or, 
okay, statements, rather than sexps. A sexp is the tag opener, plus tag 
contents, plus its closer.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 02:44:02 GMT) Full text and rfc822 format available.

Message #59 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Mickey Petersen <mickey <at> masteringemacs.org>, 60894 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 21:43:21 -0500
>> My SMIE experience with it is that we really need a specific hook
>> for that if we want to avoid wonky behaviors in "corner" cases.
> A bug report with repro would probably help, I guess. Or you could start on
> that hook yourself.

I think `C-M-u` from within a LaTeX environment was one of the cases
where it misbehaved (tho that one is not using SMIE).

> Do you mean a hook like backward-up-list-function, or something smaller?

Something like that.  Maybe it could also be used for `expand-region`
and `thing-at-point` kind of purposes maybe and could work even for
treesit nodes that aren't "matching begin..end thingies".


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 03:31:02 GMT) Full text and rfc822 format available.

Message #62 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Theodor Thornhill <theo <at> thornhill.no>, 60894 <at> debbugs.gnu.org,
 Mickey Petersen <mickey <at> masteringemacs.org>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Thu, 19 Jan 2023 05:30:43 +0200
On 19/01/2023 04:43, Stefan Monnier via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
>>> My SMIE experience with it is that we really need a specific hook
>>> for that if we want to avoid wonky behaviors in "corner" cases.
>> A bug report with repro would probably help, I guess. Or you could start on
>> that hook yourself.
> 
> I think `C-M-u` from within a LaTeX environment was one of the cases
> where it misbehaved (tho that one is not using SMIE).

Any chance SMIE is doing something different, or something particularly 
correct?

Because in my experience as well backward-up-list seems to work fine 
with SMIE, but not with the tree-sitter forward-sexp implementation 
under discussion.

>> Do you mean a hook like backward-up-list-function, or something smaller?
> 
> Something like that.  Maybe it could also be used for `expand-region`
> and `thing-at-point` kind of purposes maybe and could work even for
> treesit nodes that aren't "matching begin..end thingies".

A treesit node doesn't need an explicit "end" token, though.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 04:00:02 GMT) Full text and rfc822 format available.

Message #65 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Theodor Thornhill <theo <at> thornhill.no>, 60894 <at> debbugs.gnu.org,
 Mickey Petersen <mickey <at> masteringemacs.org>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Wed, 18 Jan 2023 22:58:55 -0500
>> I think `C-M-u` from within a LaTeX environment was one of the cases
>> where it misbehaved (tho that one is not using SMIE).
> Any chance SMIE is doing something different, or something
> particularly correct?

Could be.  Maybe its simplistic approach rules out the bad cases?

>>> Do you mean a hook like backward-up-list-function, or something smaller?
>> Something like that.  Maybe it could also be used for `expand-region`
>> and `thing-at-point` kind of purposes maybe and could work even for
>> treesit nodes that aren't "matching begin..end thingies".
> A treesit node doesn't need an explicit "end" token, though.

And that's what I want: I want to use successive `C-M-u` (or
`expand-region`) to consider ever greater subexpressions that include
the position from which I started and to do that at a fine grain.
E.g. if I start with point on `b` in:

       a + b * c

I'd like to first consider "b" then "b * c" then the whole thing.


        Stefan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 06:24:02 GMT) Full text and rfc822 format available.

Message #68 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: mickey <at> masteringemacs.org, 60894 <at> debbugs.gnu.org, theo <at> thornhill.no,
 monnier <at> iro.umontreal.ca, juri <at> linkov.net
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Thu, 19 Jan 2023 08:24:01 +0200
> Cc: 60894 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
> Date: Thu, 19 Jan 2023 00:06:22 +0200
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> In my intuition, sexps are expressions which can be nested (a lot).
> 
> Sentences are "flat" expressions. I would probably say that in 
> tree-sitter modes sentences should be equivalent to "statements".
> 
> So, sexps could be small, and they could be large. For them various list 
> navigation operations make sense (like the previously mentioned ones).
> 
> Sentences stand somewhere in the middle, and they're more like 
> sequential. A operation like backward-up-statement wouldn't make a lot 
> of sense, however.

Intuitively, the above SGTM wrt programming languages.

> Note quite sure what would correspond to statement in html-mode, but I 
> would put separate tag elements in the category of word, or symbols, or, 
> okay, statements, rather than sexps. A sexp is the tag opener, plus tag 
> contents, plus its closer.

If we could find HTML equivalents of the above notions, that would be
very good for consistent UX.  But if that turns out to be hard or
far-fetched or impossible, we could alternatively come up with
HTML-specific interpretation of these abstract notions.  As long as
the result makes sense to users who edit or view HTML files, it would
still be okay, I think, even if the relation to programming language
construct is more remote.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 08:24:01 GMT) Full text and rfc822 format available.

Message #71 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Mickey Petersen <mickey <at> masteringemacs.org>, 60894 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Thu, 19 Jan 2023 09:58:00 +0200
> Note quite sure what would correspond to statement in html-mode, but
> I would put separate tag elements in the category of word, or symbols, or,
> okay, statements, rather than sexps. A sexp is the tag opener, plus tag
> contents, plus its closer.

An HTML tag with attributes as a statement and an HTML element
with start/end tags as a sexp makes sense.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 18:45:02 GMT) Full text and rfc822 format available.

Message #74 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Mickey Petersen <mickey <at> masteringemacs.org>, 60894 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Thu, 19 Jan 2023 20:44:04 +0200
On 19/01/2023 05:58, Stefan Monnier via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
>>> I think `C-M-u` from within a LaTeX environment was one of the cases
>>> where it misbehaved (tho that one is not using SMIE).
>> Any chance SMIE is doing something different, or something
>> particularly correct?
> 
> Could be.  Maybe its simplistic approach rules out the bad cases?

Simplistic meaning one that uses a list of openers and closers?

>>>> Do you mean a hook like backward-up-list-function, or something smaller?
>>> Something like that.  Maybe it could also be used for `expand-region`
>>> and `thing-at-point` kind of purposes maybe and could work even for
>>> treesit nodes that aren't "matching begin..end thingies".
>> A treesit node doesn't need an explicit "end" token, though.
> 
> And that's what I want: I want to use successive `C-M-u` (or
> `expand-region`) to consider ever greater subexpressions that include
> the position from which I started and to do that at a fine grain.
> E.g. if I start with point on `b` in:
> 
>         a + b * c
> 
> I'd like to first consider "b" then "b * c" then the whole thing.

That should be easy enough to do using the provided tree-sitter 
framework, just by adding binary nodes to the list of types.

Whether this behavior is preferable is a matter of opinion, though. My 
guess is Ruby users will find it too fiddly, and the end result is that 
one will have to press 'C-M-u' more times to get to the same result 
(which would usually be to get to the beginning of a block, or a 
method). But people can customize it.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#60894; Package emacs. (Thu, 19 Jan 2023 19:04:01 GMT) Full text and rfc822 format available.

Message #77 received at 60894 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Dmitry Gutov <dgutov <at> yandex.ru>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 60894 <at> debbugs.gnu.org, Mickey Petersen <mickey <at> masteringemacs.org>,
 Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp
Date: Thu, 19 Jan 2023 20:03:11 +0100
Dmitry Gutov <dgutov <at> yandex.ru> writes:

> On 19/01/2023 05:58, Stefan Monnier via Bug reports for GNU Emacs, the 
> Swiss army knife of text editors wrote:
>>>> I think `C-M-u` from within a LaTeX environment was one of the cases
>>>> where it misbehaved (tho that one is not using SMIE).
>>> Any chance SMIE is doing something different, or something
>>> particularly correct?
>> 
>> Could be.  Maybe its simplistic approach rules out the bad cases?
>
> Simplistic meaning one that uses a list of openers and closers?
>
>>>>> Do you mean a hook like backward-up-list-function, or something smaller?
>>>> Something like that.  Maybe it could also be used for `expand-region`
>>>> and `thing-at-point` kind of purposes maybe and could work even for
>>>> treesit nodes that aren't "matching begin..end thingies".
>>> A treesit node doesn't need an explicit "end" token, though.
>> 
>> And that's what I want: I want to use successive `C-M-u` (or
>> `expand-region`) to consider ever greater subexpressions that include
>> the position from which I started and to do that at a fine grain.
>> E.g. if I start with point on `b` in:
>> 
>>         a + b * c
>> 
>> I'd like to first consider "b" then "b * c" then the whole thing.
>
> That should be easy enough to do using the provided tree-sitter 
> framework, just by adding binary nodes to the list of types.
>
> Whether this behavior is preferable is a matter of opinion, though. My 
> guess is Ruby users will find it too fiddly, and the end result is that 
> one will have to press 'C-M-u' more times to get to the same result 
> (which would usually be to get to the beginning of a block, or a 
> method). But people can customize it.

So I pushed the changes I've made so far, after addressing Elis
comments.  Let's try it for a while and see how we feel about it going
forward.


Theo




bug marked as fixed in version 30.1, send any further explanations to 60894 <at> debbugs.gnu.org and Theodor Thornhill <theo <at> thornhill.no> Request was from Theodor Thornhill <theo <at> thornhill.no> to control <at> debbugs.gnu.org. (Sat, 21 Jan 2023 20:49:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 19 Feb 2023 12:24:12 GMT) Full text and rfc822 format available.

This bug report was last modified 1 year and 38 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.