GNU bug report logs - #68993
treesitter support for forward-sexp-default-function

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Juri Linkov <juri@HIDDEN>; dated Thu, 8 Feb 2024 17:42:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 68993 <at> debbugs.gnu.org:


Received: (at 68993) by debbugs.gnu.org; 10 Feb 2024 17:49:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 10 12:49:41 2024
Received: from localhost ([127.0.0.1]:57738 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rYrTx-0006P0-8x
	for submit <at> debbugs.gnu.org; Sat, 10 Feb 2024 12:49:41 -0500
Received: from relay9-d.mail.gandi.net ([2001:4b98:dc4:8::229]:53363)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <juri@HIDDEN>) id 1rYrTu-0006OZ-Tn
 for 68993 <at> debbugs.gnu.org; Sat, 10 Feb 2024 12:49:39 -0500
Received: by mail.gandi.net (Postfix) with ESMTPSA id D8E9DFF803;
 Sat, 10 Feb 2024 17:49:15 +0000 (UTC)
From: Juri Linkov <juri@HIDDEN>
To: Yuan Fu <casouri@HIDDEN>
Subject: Re: bug#68993: treesitter support for forward-sexp-default-function
In-Reply-To: <86y1bv6jsm.fsf@HIDDEN> (Juri Linkov's message of "Thu, 
 08 Feb 2024 19:38:17 +0200")
Organization: LINKOV.NET
References: <86y1bv6jsm.fsf@HIDDEN>
Date: Sat, 10 Feb 2024 19:46:46 +0200
Message-ID: <86a5o86wl5.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain
X-GND-Sasl: juri@HIDDEN
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 68993
Cc: 68993 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Hi Yuan,

Do you think 'comment' is a suitable name for 'treesit-forward-sexp'?
I'm unsure even if 'text' is a good name for 'treesit-forward-sentence'.
But at least these two should be consistent with each another.

Maybe better to add the prefix 'default-' to both?
This will hint that the default function is used.

Then 'treesit-forward-sentence' will support types
'sentence' and 'default-sentence'.
And 'treesit-forward-sexp' will support types
'sexp' and 'default-sexp'.

> @@ -2137,7 +2137,10 @@ treesit-forward-sexp
>    (interactive "^p")
>    (let ((arg (or arg 1))
>          (pred (or treesit-sexp-type-regexp 'sexp)))
> -    (or (if (> arg 0)
> +    (or (when (treesit-node-match-p (treesit-node-at (point)) 'comment t)
> +          (funcall #'forward-sexp-default-function arg)
> +          t)
> +        (if (> arg 0)
>              (treesit-end-of-thing pred (abs arg) 'restricted)
>            (treesit-beginning-of-thing pred (abs arg) 'restricted))
>          ;; If we couldn't move, we should signal an error and report




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#68993; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 8 Feb 2024 17:41:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 08 12:41:10 2024
Received: from localhost ([127.0.0.1]:33557 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rY8Ob-0004Go-LN
	for submit <at> debbugs.gnu.org; Thu, 08 Feb 2024 12:41:10 -0500
Received: from lists.gnu.org ([2001:470:142::17]:33500)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <juri@HIDDEN>) id 1rY8OV-0004Fu-Vm
 for submit <at> debbugs.gnu.org; Thu, 08 Feb 2024 12:41:08 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <juri@HIDDEN>) id 1rY8OB-0002D8-P9
 for bug-gnu-emacs@HIDDEN; Thu, 08 Feb 2024 12:40:43 -0500
Received: from relay6-d.mail.gandi.net ([2001:4b98:dc4:8::226])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <juri@HIDDEN>) id 1rY8O8-0000Hd-TG
 for bug-gnu-emacs@HIDDEN; Thu, 08 Feb 2024 12:40:43 -0500
Received: by mail.gandi.net (Postfix) with ESMTPSA id AC334C0002
 for <bug-gnu-emacs@HIDDEN>; Thu,  8 Feb 2024 17:40:36 +0000 (UTC)
From: Juri Linkov <juri@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: treesitter support for forward-sexp-default-function
Organization: LINKOV.NET
X-Debbugs-Cc: Yuan Fu <casouri@HIDDEN>
Date: Thu, 08 Feb 2024 19:38:17 +0200
Message-ID: <86y1bv6jsm.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-GND-Sasl: juri@HIDDEN
Received-SPF: pass client-ip=2001:4b98:dc4:8::226;
 envelope-from=juri@HIDDEN; helo=relay6-d.mail.gandi.net
X-Spam_score_int: -25
X-Spam_score: -2.6
X-Spam_bar: --
X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001,
 T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.6 (/)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.4 (/)

--=-=-=
Content-Type: text/plain

'treesit-forward-sentence' supports the node type 'text',
and for matching nodes it uses the fallback to
'forward-sentence-default-function'.

This patch does exactly the same for 'treesit-forward-sexp':
for nodes that match a new node type 'comment',
it uses the fallback to the new function
'forward-sexp-default-function'.


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline; filename=forward-sexp-default-function.patch

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 82b2f97b4a5..284c4915f3a 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -2137,7 +2137,10 @@ treesit-forward-sexp
   (interactive "^p")
   (let ((arg (or arg 1))
         (pred (or treesit-sexp-type-regexp 'sexp)))
-    (or (if (> arg 0)
+    (or (when (treesit-node-match-p (treesit-node-at (point)) 'comment t)
+          (funcall #'forward-sexp-default-function arg)
+          t)
+        (if (> arg 0)
             (treesit-end-of-thing pred (abs arg) 'restricted)
           (treesit-beginning-of-thing pred (abs arg) 'restricted))
         ;; If we couldn't move, we should signal an error and report
diff --git a/lisp/emacs-lisp/lisp.el b/lisp/emacs-lisp/lisp.el
index 4b722b4e9a7..d3c3bf55de3 100644
--- a/lisp/emacs-lisp/lisp.el
+++ b/lisp/emacs-lisp/lisp.el
@@ -45,7 +45,12 @@ parens-require-spaces
   :type 'boolean
   :group 'lisp)
 
-(defvar forward-sexp-function nil
+(defun forward-sexp-default-function (&optional arg)
+  "Default function for `forward-sexp-function'."
+  (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
+  (if (< arg 0) (backward-prefix-chars)))
+
+(defvar forward-sexp-function #'forward-sexp-default-function
   ;; FIXME:
   ;; - for some uses, we may want a "sexp-only" version, which only
   ;;   jumps over a well-formed sexp, rather than some dwimish thing
@@ -74,10 +79,7 @@ forward-sexp
                                     "No next sexp"
                                   "No previous sexp"))))
     (or arg (setq arg 1))
-    (if forward-sexp-function
-        (funcall forward-sexp-function arg)
-      (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
-      (if (< arg 0) (backward-prefix-chars)))))
+    (funcall forward-sexp-function arg)))
 
 (defun backward-sexp (&optional arg interactive)
   "Move backward across one balanced expression (sexp).

--=-=-=
Content-Type: text/plain


Maybe the node type 'comment' is not the best name,
but it was intended to allow using the default function
to be able to move with 'M-C-f' in the comments and strings
there tree-sitter has no information.

It makes sense to support the default movement with 'M-C-f'
in the comments and strings of all ts modes.  The second patch
shows how this could be achieved by adding the default
'comment' match to 'treesit-thing-settings' of all modes.
Or maybe this should be customizable?


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline; filename=treesit-major-mode-setup.patch

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 82b2f97b4a5..284c4915f3a 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -3054,6 +3057,18 @@ treesit-major-mode-setup
     (setq-local outline-search-function #'treesit-outline-search
                 outline-level #'treesit-outline-level))
 
+  (dolist (parser (treesit-parser-list))
+    (let ((language (treesit-parser-language parser))
+          (comment (regexp-opt '("comment" "string" "string_content"))))
+      (unless (treesit-thing-defined-p 'comment language)
+        (if-let ((l (alist-get language treesit-thing-settings)))
+            (progn
+              (setf (alist-get 'comment l) (list comment))
+              (setf (alist-get language treesit-thing-settings) l))
+          (setq-local treesit-thing-settings
+                      (append `((,language (comment ,comment)))
+                              treesit-thing-settings))))))
+
   ;; Remove existing local parsers.
   (dolist (ov (overlays-in (point-min) (point-max)))
     (when-let ((parser (overlay-get ov 'treesit-parser)))

--=-=-=
Content-Type: text/plain


The third patch demonstrates how it's possible to close bug#67036
that was impossible to fix without more general changes in treesit.el.

The problem is that e.g. Ruby parser to such text:

  hash[:key]

produces such syntax tree:

  (element_reference object: (identifier) [ (simple_symbol) ])

so when point is on [ then 'M-C-f' can't move to ].

This is fixed now by the third patch:


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline; filename=ruby-ts-mode.patch

diff --git a/lisp/progmodes/ruby-ts-mode.el b/lisp/progmodes/ruby-ts-mode.el
index 598eaa461ff..4d0ae2e9303 100644
--- a/lisp/progmodes/ruby-ts-mode.el
+++ b/lisp/progmodes/ruby-ts-mode.el
@@ -1170,7 +1170,20 @@ ruby-ts-mode
                                 "global_variable"
                                 )
                                eol)
-                              #'ruby-ts--sexp-p)))))
+                              #'ruby-ts--sexp-p))
+                 (comment ,(lambda (node)
+                             (or (member (treesit-node-type node)
+                                         '("comment" "string_content"))
+                                 (and (member (treesit-node-text node)
+                                              '("[" "]"))
+                                      (equal (treesit-node-type
+                                              (treesit-node-parent node))
+                                             "element_reference"))
+                                 (and (member (treesit-node-text node)
+                                              '("#{" "}"))
+                                      (equal (treesit-node-type
+                                              (treesit-node-parent node))
+                                             "interpolation"))))))))
 
   ;; AFAIK, Ruby can not nest methods
   (setq-local treesit-defun-prefer-top-level nil)

--=-=-=--




Acknowledgement sent to Juri Linkov <juri@HIDDEN>:
New bug report received and forwarded. Copy sent to casouri@HIDDEN, bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to casouri@HIDDEN, bug-gnu-emacs@HIDDEN:
bug#68993; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sat, 10 Feb 2024 18:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.