GNU bug report logs -
#61913
java-ts-mode query error for string highlight, due to recent commit in tree-sitter-java.
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 61913 in the body.
You can then email your comments to 61913 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 02 Mar 2023 12:22:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Vincenzo Pupillo <v.pupillo <at> gmail.com>
:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org
.
(Thu, 02 Mar 2023 12:22:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi,
a recent commit in tree-sitter-java (https://github.com/tree-sitter/tree-sitter-java/commit/2b3b776b953a34ccffb647dd89732db79690d4c5)
removed "text_block" from grammar.js. This causes a bug in java-ts-mode when treesit-fon-lock-level is > 1
Error during redeployment: (jit-lock-function 8295) reported (treesit-query-error "Node type error at" 42 "(string_literal) @font-lock-string-face (text_block) @font-lock-string-face" "Debugging the query with `treesit-query-validate'")
The attached patch fixes the problem.
Thank you very much.
Vincenzo
[java_ts_mode_string_highlight.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 02 Mar 2023 13:03:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 61913 <at> debbugs.gnu.org (full text, mbox):
> From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> Date: Thu, 02 Mar 2023 13:21:05 +0100
>
> a recent commit in tree-sitter-java (https://github.com/tree-sitter/tree-sitter-java/commit/2b3b776b953a34ccffb647dd89732db79690d4c5)
> removed "text_block" from grammar.js. This causes a bug in java-ts-mode when treesit-fon-lock-level is > 1
>
> Error during redeployment: (jit-lock-function 8295) reported (treesit-query-error "Node type error at" 42 "(string_literal) @font-lock-string-face (text_block) @font-lock-string-face" "Debugging the query with `treesit-query-validate'")
>
>
> The attached patch fixes the problem.
Wouldn't removing text_block from our code cause problems if someone
uses tree-sitter-java from before the removal?
Is there a way to write code which handles text_block if it exists?
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 02 Mar 2023 14:46:03 GMT)
Full text and
rfc822 format available.
Message #11 received at 61913 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Sorry, I sent a broken patch.
my emacs version is:
GNU Emacs 30.0.50 (build 11, x86_64-pc-linux-gnu, GTK+ Version 3.24.36, cairo version 1.17.6) of 2023-03-02
Vincenzo
In data giovedì 2 marzo 2023 13:21:05 CET, hai scritto:
> Hi,
> a recent commit in tree-sitter-java (https://github.com/tree-sitter/tree-sitter-java/commit/2b3b776b953a34ccffb647dd89732db79690d4c5)
> removed "text_block" from grammar.js. This causes a bug in java-ts-mode when treesit-fon-lock-level is > 1
>
> Error during redeployment: (jit-lock-function 8295) reported (treesit-query-error "Node type error at" 42 "(string_literal) @font-lock-string-face (text_block) @font-lock-string-face" "Debugging the query with `treesit-query-validate'")
>
>
> The attached patch fixes the problem.
> Thank you very much.
>
> Vincenzo
>
>
>
>
>
--
Vincenzo Pupillo
Direzione ICT
Università degli Studi di Milano / University of Milan (Italy)
Via Giuseppe Colombo, 46, 20133 Milano
[java_ts_mode_string_highlight.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 02 Mar 2023 16:19:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 61913 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Yes, indeed "text_block" are not recognized by earlier versions of tree-sitter-java.
I have prepared a new patch that checks if "text_block" are recognized and adjusts the query accordingly.
Also java-mode seems not to recognize "text_block".
You can check the result in the attached images.
Thank you.
Vincenzo
In data gioved� 2 marzo 2023 14:01:56 CET, Eli Zaretskii ha scritto:
> > From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> > Date: Thu, 02 Mar 2023 13:21:05 +0100
> >
> > a recent commit in tree-sitter-java (https://github.com/tree-sitter/tree-sitter-java/commit/2b3b776b953a34ccffb647dd89732db79690d4c5)
> > removed "text_block" from grammar.js. This causes a bug in java-ts-mode when treesit-fon-lock-level is > 1
> >
> > Error during redeployment: (jit-lock-function 8295) reported (treesit-query-error "Node type error at" 42 "(string_literal) @font-lock-string-face (text_block) @font-lock-string-face" "Debugging the query with `treesit-query-validate'")
> >
> >
> > The attached patch fixes the problem.
>
> Wouldn't removing text_block from our code cause problems if someone
> uses tree-sitter-java from before the removal?
>
> Is there a way to write code which handles text_block if it exists?
>
[java_ts_mode_string_highlight.patch (text/x-patch, attachment)]
[java_text_block_new_ts_grammar.png (image/png, attachment)]
[java_text_block_regula_java_mode.png (image/png, attachment)]
[java_text_block_previous_ts_grammar.png (image/png, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Fri, 03 Mar 2023 22:33:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 61913 <at> debbugs.gnu.org (full text, mbox):
Vincenzo Pupillo <v.pupillo <at> gmail.com> writes:
> Yes, indeed "text_block" are not recognized by earlier versions of tree-sitter-java.
> I have prepared a new patch that checks if "text_block" are recognized and adjusts the query accordingly.
>
> Also java-mode seems not to recognize "text_block".
> You can check the result in the attached images.
>
> Thank you.
>
> Vincenzo
>
> In data gioved� 2 marzo 2023 14:01:56 CET, Eli Zaretskii ha scritto:
>> > From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
>> > Date: Thu, 02 Mar 2023 13:21:05 +0100
>> >
>> > a recent commit in tree-sitter-java (https://github.com/tree-sitter/tree-sitter-java/commit/2b3b776b953a34ccffb647dd89732db79690d4c5)
>> > removed "text_block" from grammar.js. This causes a bug in java-ts-mode when treesit-fon-lock-level is > 1
>> >
>> > Error during redeployment: (jit-lock-function 8295) reported
>> > (treesit-query-error "Node type error at" 42 "(string_literal)
>> > @font-lock-string-face (text_block) @font-lock-string-face"
>> > "Debugging the query with `treesit-query-validate'")
>> >
>> >
>> > The attached patch fixes the problem.
>>
>> Wouldn't removing text_block from our code cause problems if someone
>> uses tree-sitter-java from before the removal?
>>
>> Is there a way to write code which handles text_block if it exists?
>>
Tree-sitter language grammars really need some versioning system. I
wonder how do we propose such things to them tho...
Yuan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Sat, 04 Mar 2023 07:11:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 61913 <at> debbugs.gnu.org (full text, mbox):
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Fri, 3 Mar 2023 14:32:02 -0800
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
> 61913 <at> debbugs.gnu.org
>
> >> > Error during redeployment: (jit-lock-function 8295) reported
> >> > (treesit-query-error "Node type error at" 42 "(string_literal)
> >> > @font-lock-string-face (text_block) @font-lock-string-face"
> >> > "Debugging the query with `treesit-query-validate'")
> >> >
> >> >
> >> > The attached patch fixes the problem.
> >>
> >> Wouldn't removing text_block from our code cause problems if someone
> >> uses tree-sitter-java from before the removal?
> >>
> >> Is there a way to write code which handles text_block if it exists?
> >>
>
> Tree-sitter language grammars really need some versioning system. I
> wonder how do we propose such things to them tho...
That'd be hard, given that many of them don't even make releases.
It should be enough for our purposes to have a possibility of querying
the grammar about support for specific features. Then the version
will not matter, only the supported features will. Maybe we could do
that ourselves, on-the-fly, like we do with programs when we want to
know whether they support some command-line switch?
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Sat, 04 Mar 2023 09:28:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 61913 <at> debbugs.gnu.org (full text, mbox):
> On Mar 3, 2023, at 11:10 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Fri, 3 Mar 2023 14:32:02 -0800
>> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>> 61913 <at> debbugs.gnu.org
>>
>>>>> Error during redeployment: (jit-lock-function 8295) reported
>>>>> (treesit-query-error "Node type error at" 42 "(string_literal)
>>>>> @font-lock-string-face (text_block) @font-lock-string-face"
>>>>> "Debugging the query with `treesit-query-validate'")
>>>>>
>>>>>
>>>>> The attached patch fixes the problem.
>>>>
>>>> Wouldn't removing text_block from our code cause problems if someone
>>>> uses tree-sitter-java from before the removal?
>>>>
>>>> Is there a way to write code which handles text_block if it exists?
>>>>
>>
>> Tree-sitter language grammars really need some versioning system. I
>> wonder how do we propose such things to them tho...
>
> That'd be hard, given that many of them don't even make releases.
>
> It should be enough for our purposes to have a possibility of querying
> the grammar about support for specific features. Then the version
> will not matter, only the supported features will. Maybe we could do
> that ourselves, on-the-fly, like we do with programs when we want to
> know whether they support some command-line switch?
We can test whether a node type exists in the grammar, by trying to compile a query using that node type. If it returns successfully, then the node type exists. But there are other assumptions we make about a grammar, like “the 2nd child of a if node must be the condition”. Hopefully these more subtle things don’t change easily, because we can’t easily test them.
Yuan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Sun, 05 Mar 2023 13:52:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 61913 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
nvim-treesiter seems to use a "lockfile" in which they put the version of
the language parser to be installed (
https://github.com/nvim-treesitter/nvim-treesitter/blob/master/lockfile.json)
to allow some stability I suppose.
Il giorno sab 4 mar 2023 alle ore 10:27 Yuan Fu <casouri <at> gmail.com> ha
scritto:
>
>
> > On Mar 3, 2023, at 11:10 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> >> From: Yuan Fu <casouri <at> gmail.com>
> >> Date: Fri, 3 Mar 2023 14:32:02 -0800
> >> Cc: Eli Zaretskii <eliz <at> gnu.org>,
> >> 61913 <at> debbugs.gnu.org
> >>
> >>>>> Error during redeployment: (jit-lock-function 8295) reported
> >>>>> (treesit-query-error "Node type error at" 42 "(string_literal)
> >>>>> @font-lock-string-face (text_block) @font-lock-string-face"
> >>>>> "Debugging the query with `treesit-query-validate'")
> >>>>>
> >>>>>
> >>>>> The attached patch fixes the problem.
> >>>>
> >>>> Wouldn't removing text_block from our code cause problems if someone
> >>>> uses tree-sitter-java from before the removal?
> >>>>
> >>>> Is there a way to write code which handles text_block if it exists?
> >>>>
> >>
> >> Tree-sitter language grammars really need some versioning system. I
> >> wonder how do we propose such things to them tho...
> >
> > That'd be hard, given that many of them don't even make releases.
> >
> > It should be enough for our purposes to have a possibility of querying
> > the grammar about support for specific features. Then the version
> > will not matter, only the supported features will. Maybe we could do
> > that ourselves, on-the-fly, like we do with programs when we want to
> > know whether they support some command-line switch?
>
> We can test whether a node type exists in the grammar, by trying to
> compile a query using that node type. If it returns successfully, then the
> node type exists. But there are other assumptions we make about a grammar,
> like “the 2nd child of a if node must be the condition”. Hopefully these
> more subtle things don’t change easily, because we can’t easily test them.
>
> Yuan
>
>
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Sun, 05 Mar 2023 15:10:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 61913 <at> debbugs.gnu.org (full text, mbox):
> From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> Date: Sun, 5 Mar 2023 14:50:39 +0100
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 61913 <at> debbugs.gnu.org
>
> nvim-treesiter seems to use a "lockfile" in which they put the version of the language parser to be installed
> (https://github.com/nvim-treesitter/nvim-treesitter/blob/master/lockfile.json) to allow some stability I suppose.
Those seem to be Git SHA hashes, but how can Emacs verify that the
particular grammar shared library is of that revision or later? We
can only tell users to verify that, which is not very reliable, and
requires users to clone the repository.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Mon, 06 Mar 2023 06:13:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 61913 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
>> Date: Sun, 5 Mar 2023 14:50:39 +0100
>> Cc: Eli Zaretskii <eliz <at> gnu.org>, 61913 <at> debbugs.gnu.org
>>
>> nvim-treesiter seems to use a "lockfile" in which they put the version of the language parser to be installed
>> (https://github.com/nvim-treesitter/nvim-treesitter/blob/master/lockfile.json) to allow some stability I suppose.
>
> Those seem to be Git SHA hashes, but how can Emacs verify that the
> particular grammar shared library is of that revision or later? We
> can only tell users to verify that, which is not very reliable, and
> requires users to clone the repository.
We could keep a header in the *-ts-mode.el file maybe? Something like:
;;; Grammar version: 95e84b5493e8d21060f1a905d2e5e3c5590bae45
Anyway, I think until we have such a system in place, we should strive
to adhere to the most recent version of the grammar. The diff is ok,
Vicenzo. Would you like to add a proper patch or should I just do it?
I mean with a commit message and format like in CONTRIBUTE?
Theo
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Mon, 06 Mar 2023 14:04:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 61913 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Is this patch okay? Please feel free to make any changes you see fit.
Thank you.
Vincenzo
In data lunedì 6 marzo 2023 07:12:49 CET, Theodor Thornhill ha scritto:
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> >> Date: Sun, 5 Mar 2023 14:50:39 +0100
> >> Cc: Eli Zaretskii <eliz <at> gnu.org>, 61913 <at> debbugs.gnu.org
> >>
> >> nvim-treesiter seems to use a "lockfile" in which they put the version of the language parser to be installed
> >> (https://github.com/nvim-treesitter/nvim-treesitter/blob/master/lockfile.json) to allow some stability I suppose.
> >
> > Those seem to be Git SHA hashes, but how can Emacs verify that the
> > particular grammar shared library is of that revision or later? We
> > can only tell users to verify that, which is not very reliable, and
> > requires users to clone the repository.
>
> We could keep a header in the *-ts-mode.el file maybe? Something like:
>
>
> ;;; Grammar version: 95e84b5493e8d21060f1a905d2e5e3c5590bae45
>
>
> Anyway, I think until we have such a system in place, we should strive
> to adhere to the most recent version of the grammar. The diff is ok,
> Vicenzo. Would you like to add a proper patch or should I just do it?
> I mean with a commit message and format like in CONTRIBUTE?
>
> Theo
>
[0001-java-ts-mode-detect-if-text_block-is-supported.patch (text/x-patch, attachment)]
Reply sent
to
Eli Zaretskii <eliz <at> gnu.org>
:
You have taken responsibility.
(Thu, 09 Mar 2023 10:26:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Vincenzo Pupillo <v.pupillo <at> gmail.com>
:
bug acknowledged by developer.
(Thu, 09 Mar 2023 10:26:02 GMT)
Full text and
rfc822 format available.
Message #40 received at 61913-done <at> debbugs.gnu.org (full text, mbox):
> From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> Cc: casouri <at> gmail.com, 61913 <at> debbugs.gnu.org
> Date: Mon, 06 Mar 2023 15:03:46 +0100
>
> Is this patch okay? Please feel free to make any changes you see fit.
Thanks, installed on the emacs-29 branch.
Btw, this patch all but exhausts the amount of changes we can accept
from you without a copyright assignment. Would you like to start you
legal paperwork of assigning the copyright to the FSF? If yes, I will
send you the form to fill.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 09 Mar 2023 12:50:02 GMT)
Full text and
rfc822 format available.
Message #43 received at 61913-done <at> debbugs.gnu.org (full text, mbox):
Yes, Eli, please send me the the form. Thank you.
V.
In data gioved� 9 marzo 2023 11:24:55 CET, Eli Zaretskii ha scritto:
> > From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> > Cc: casouri <at> gmail.com, 61913 <at> debbugs.gnu.org
> > Date: Mon, 06 Mar 2023 15:03:46 +0100
> >
> > Is this patch okay? Please feel free to make any changes you see fit.
>
> Thanks, installed on the emacs-29 branch.
>
> Btw, this patch all but exhausts the amount of changes we can accept
> from you without a copyright assignment. Would you like to start you
> legal paperwork of assigning the copyright to the FSF? If yes, I will
> send you the form to fill.
>
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 09 Mar 2023 15:29:01 GMT)
Full text and
rfc822 format available.
Message #46 received at 61913-done <at> debbugs.gnu.org (full text, mbox):
> From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
> Cc: theo <at> thornhill.no, casouri <at> gmail.com, 61913-done <at> debbugs.gnu.org
> Date: Thu, 09 Mar 2023 13:49:44 +0100
>
> Yes, Eli, please send me the the form. Thank you.
Form sent off-list.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 09 Mar 2023 16:17:02 GMT)
Full text and
rfc822 format available.
Message #49 received at 61913 <at> debbugs.gnu.org (full text, mbox):
On 06/03/2023 16:03, Vincenzo Pupillo wrote:
> Is this patch okay? Please feel free to make any changes you see fit.
Does it also work if you replace treesit-query-capture with
treesit-query-validate? The latter should be faster in general.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 09 Mar 2023 16:50:03 GMT)
Full text and
rfc822 format available.
Message #52 received at 61913 <at> debbugs.gnu.org (full text, mbox):
treesit-query-validate calls treesit-query-capture which is defined in treesit.c.
It's a wrapper for interactive use, I guess it's not faster.
V.
In data giovedì 9 marzo 2023 17:16:38 CET, Dmitry Gutov ha scritto:
> On 06/03/2023 16:03, Vincenzo Pupillo wrote:
> > Is this patch okay? Please feel free to make any changes you see fit.
>
> Does it also work if you replace treesit-query-capture with
> treesit-query-validate? The latter should be faster in general.
>
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#61913
; Package
emacs
.
(Thu, 09 Mar 2023 17:09:02 GMT)
Full text and
rfc822 format available.
Message #55 received at 61913 <at> debbugs.gnu.org (full text, mbox):
On 09/03/2023 18:49, Vincenzo Pupillo wrote:
> treesit-query-validate calls treesit-query-capture which is defined in treesit.c.
> It's a wrapper for interactive use, I guess it's not faster.
Fair point, but there is a certain difference: treesit-query-validate
creates a temporary buffer to run treesit-query-capture in. Whereas, in
your patch, treesit-query-capture is called in the current buffer,
whichever buffer is current when the code is loaded, I suppose.
If the buffer is large, that might take a non-negligible amount of time.
So okay, reusing treesit-query-validate is not a good idea, but we could
use the same pattern here, or try treesit-query-string which also uses a
temp buffer:
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index fca00cdce4f..0db6c9822ed 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -148,13 +148,13 @@ java-ts-mode--operators
"Java operators for tree-sitter font-locking.")
(defun java-ts-mode--string-highlight-helper ()
-"Returns, for strings, a query based on what is supported by
+ "Returns, for strings, a query based on what is supported by
the available version of Tree-sitter for java."
(condition-case nil
- (progn (treesit-query-capture 'java '((text_block)
@font-lock-string-face))
+ (progn (treesit-query-string "" '((text_block)
@font-lock-string-face) 'java)
`((string_literal) @font-lock-string-face
(text_block) @font-lock-string-face))
- (error
+ (treesit-query-error
`((string_literal) @font-lock-string-face))))
(defvar java-ts-mode--font-lock-settings
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Fri, 07 Apr 2023 11:24:05 GMT)
Full text and
rfc822 format available.
This bug report was last modified 2 years and 36 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.