GNU bug report logs - #59633
29.0.50; tree-sitter-bash: Incorrect behaviour of heredocs with expansions

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 59633 in the body.
You can then email your comments to 59633 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: miha <at> kamnitnik.top
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; tree-sitter-bash: Incorrect behaviour of heredocs with
 expansions
Date: Sun, 27 Nov 2022 17:33:11 +0100

[Message part 1 (text/plain, inline)]

1. Visit empty buffer test in fundamental mode (C-x b test RET)
2. Insert

tee << EOF
text1 $var
text2 $(echo cmd)
text3
$var2
EOF

3. Put point before "text2"
4. M-x load-library RET treesit RET
5. M-: (treesit-node-children (treesit-node-at (point) 'bash))

    => (#<treesit-node (simple_expansion) in 18-22>
        #<treesit-node (command_substitution) in 29-40>)

   Note that this list is incorrect, "$var2" at the end of heredoc is
   missing.

6. M-: (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))

   => nil

   Expected return value here is #<treesit-node (command_substitution) in 29-40>

I'm not sure if this bug is on Emacs, tree-sitter-bash or even
tree-sitter itself and I don't know how to check.

tree-sitter-bash version: 0.19.0.r19.g77cf8a7-1
tree-sitter version: 0.20.7-1

Thanks and best regards.

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 59633 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: miha <at> kamnitnik.top
Cc: 59633 <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of 
 heredocs with expansions
Date: Thu, 1 Dec 2022 21:25:26 -0800

miha <at> kamnitnik.top writes:

> 1. Visit empty buffer test in fundamental mode (C-x b test RET)
> 2. Insert
>
> tee << EOF
> text1 $var
> text2 $(echo cmd)
> text3
> $var2
> EOF
>
> 3. Put point before "text2"
> 4. M-x load-library RET treesit RET
> 5. M-: (treesit-node-children (treesit-node-at (point) 'bash))
>
>     => (#<treesit-node (simple_expansion) in 18-22>
>         #<treesit-node (command_substitution) in 29-40>)
>
>    Note that this list is incorrect, "$var2" at the end of heredoc is
>    missing.
>
> 6. M-: (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
>
>    => nil
>
>    Expected return value here is #<treesit-node (command_substitution) in 29-40>
>
> I'm not sure if this bug is on Emacs, tree-sitter-bash or even
> tree-sitter itself and I don't know how to check.
>
> tree-sitter-bash version: 0.19.0.r19.g77cf8a7-1
> tree-sitter version: 0.20.7-1
>
> Thanks and best regards.

Thanks. This is the tree produced by tree-sitter, does it match your
expectation?

(heredoc_body
(simple_expansion $ (variable_name))
(command_substitution $(
  (command
   name: (command_name (word))
   argument: (word))
  ))
(simple_expansion $ (variable_name)))

treesit-node-at gives you the _smallest_ node at point, maybe that’s why
you didn’t see expected behavior?

Yuan

Message #11 received at 59633 <at> debbugs.gnu.org (full text, mbox):

From: <miha <at> kamnitnik.top>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 59633 <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of
 heredocs with expansions
Date: Fri, 02 Dec 2022 17:26:23 +0100

[Message part 1 (text/plain, inline)]

Yuan Fu <casouri <at> gmail.com> writes:

> miha <at> kamnitnik.top writes:
>
>> 1. Visit empty buffer test in fundamental mode (C-x b test RET)
>> 2. Insert
>>
>> tee << EOF
>> text1 $var
>> text2 $(echo cmd)
>> text3
>> $var2
>> EOF
>>
>> 3. Put point before "text2"
>> 4. M-x load-library RET treesit RET
>> 5. M-: (treesit-node-children (treesit-node-at (point) 'bash))
>>
>>     => (#<treesit-node (simple_expansion) in 18-22>
>>         #<treesit-node (command_substitution) in 29-40>)
>>
>>    Note that this list is incorrect, "$var2" at the end of heredoc is
>>    missing.
>>
>> 6. M-: (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
>>
>>    => nil
>>
>>    Expected return value here is #<treesit-node (command_substitution) in 29-40>
>>
>> I'm not sure if this bug is on Emacs, tree-sitter-bash or even
>> tree-sitter itself and I don't know how to check.
>>
>> tree-sitter-bash version: 0.19.0.r19.g77cf8a7-1
>> tree-sitter version: 0.20.7-1
>>
>> Thanks and best regards.
>
> Thanks. This is the tree produced by tree-sitter, does it match your
> expectation?
>
> (heredoc_body
> (simple_expansion $ (variable_name))
> (command_substitution $(
>   (command
>    name: (command_name (word))
>    argument: (word))
>   ))
> (simple_expansion $ (variable_name)))
>
> treesit-node-at gives you the _smallest_ node at point, maybe that’s why
> you didn’t see expected behavior?
>
> Yuan

The latest commit to tree-sitter-bash is

commit 4488aa41406547e478636a4fcfd24f5bbc3f2f74
Author: João P. L. Carvalho <jaopaulolc <at> gmail.com>
Date:   Sun Nov 27 20:39:41 2022 -0700

    Fix scanning of heredoc_body to allow empty bodies (#137).

Seems like it has fixed the problem described in the 5. step and the
tree produced by tree-sitter is now expected.

The problem in the 6. step is still present though. With point in front
of command_substitution, on would expect

    (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))

to return the command_substitution node instead of returning nil.

Thanks.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 59633 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: miha <at> kamnitnik.top
Cc: 59633 <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of
 heredocs with expansions
Date: Fri, 2 Dec 2022 16:50:20 -0800

> On Dec 2, 2022, at 8:26 AM, miha <at> kamnitnik.top wrote:
> 
> Yuan Fu <casouri <at> gmail.com> writes:
> 
>> miha <at> kamnitnik.top writes:
>> 
>>> 1. Visit empty buffer test in fundamental mode (C-x b test RET)
>>> 2. Insert
>>> 
>>> tee << EOF
>>> text1 $var
>>> text2 $(echo cmd)
>>> text3
>>> $var2
>>> EOF
>>> 
>>> 3. Put point before "text2"
>>> 4. M-x load-library RET treesit RET
>>> 5. M-: (treesit-node-children (treesit-node-at (point) 'bash))
>>> 
>>>    => (#<treesit-node (simple_expansion) in 18-22>
>>>        #<treesit-node (command_substitution) in 29-40>)
>>> 
>>>   Note that this list is incorrect, "$var2" at the end of heredoc is
>>>   missing.
>>> 
>>> 6. M-: (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
>>> 
>>>   => nil
>>> 
>>>   Expected return value here is #<treesit-node (command_substitution) in 29-40>
>>> 
>>> I'm not sure if this bug is on Emacs, tree-sitter-bash or even
>>> tree-sitter itself and I don't know how to check.
>>> 
>>> tree-sitter-bash version: 0.19.0.r19.g77cf8a7-1
>>> tree-sitter version: 0.20.7-1
>>> 
>>> Thanks and best regards.
>> 
>> Thanks. This is the tree produced by tree-sitter, does it match your
>> expectation?
>> 
>> (heredoc_body
>> (simple_expansion $ (variable_name))
>> (command_substitution $(
>>  (command
>>   name: (command_name (word))
>>   argument: (word))
>>  ))
>> (simple_expansion $ (variable_name)))
>> 
>> treesit-node-at gives you the _smallest_ node at point, maybe that’s why
>> you didn’t see expected behavior?
>> 
>> Yuan
> 
> The latest commit to tree-sitter-bash is
> 
> commit 4488aa41406547e478636a4fcfd24f5bbc3f2f74
> Author: João P. L. Carvalho <jaopaulolc <at> gmail.com>
> Date:   Sun Nov 27 20:39:41 2022 -0700
> 
>    Fix scanning of heredoc_body to allow empty bodies (#137).
> 
> Seems like it has fixed the problem described in the 5. step and the
> tree produced by tree-sitter is now expected.
> 
> The problem in the 6. step is still present though. With point in front
> of command_substitution, on would expect
> 
>    (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
> 
> to return the command_substitution node instead of returning nil.

What’s the return value of (treesit-node-at (point)) at that point? Probably the command_substitution node, and since it doesn’t have any children, treesit-node-first-child-for-pos would return nil.

Yuan

Message #17 received at 59633 <at> debbugs.gnu.org (full text, mbox):

From: <miha <at> kamnitnik.top>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 59633 <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of
 heredocs with expansions
Date: Sat, 03 Dec 2022 08:25:38 +0100

[Message part 1 (text/plain, inline)]

Yuan Fu <casouri <at> gmail.com> writes:

> What’s the return value of (treesit-node-at (point)) at that point?
> Probably the command_substitution node, and since it doesn’t have any
> children, treesit-node-first-child-for-pos would return nil.

(treesit-node-at (point) 'bash)
=> #<treesit-node (heredoc_body) in 12-56>

I forgot to say that point is before "text2", outside of the
command_substitution node.

> Yuan

tee << EOF
text1 $var
text2 $(echo cmd)
text3
$var2
EOF


Filling a test buffer with these contents and placing point before
"text2", we get

    (treesit-node-at (point) 'bash)
    => #<treesit-node (heredoc_body) in 12-56>

    (treesit-node-children (treesit-node-at (point) 'bash))
    => (#<treesit-node (simple_expansion) in 18-22>
        #<treesit-node (command_substitution) in 29-40>
        #<treesit-node (simple_expansion) in 47-52>)

This is now expected.

    (point)
    => 23

    (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
    => nil

This is not expected. One would expect this to return
command_substitution node, since it follows the position 23.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 59633 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: miha <at> kamnitnik.top
Cc: 59633 <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of 
 heredocs with expansions
Date: Fri, 9 Dec 2022 14:10:55 -0800

<miha <at> kamnitnik.top> writes:

> Yuan Fu <casouri <at> gmail.com> writes:
>
>> What’s the return value of (treesit-node-at (point)) at that point?
>> Probably the command_substitution node, and since it doesn’t have any
>> children, treesit-node-first-child-for-pos would return nil.
>
> (treesit-node-at (point) 'bash)
> => #<treesit-node (heredoc_body) in 12-56>
>
> I forgot to say that point is before "text2", outside of the
> command_substitution node.
>
>> Yuan
>
> tee << EOF
> text1 $var
> text2 $(echo cmd)
> text3
> $var2
> EOF
>
>
> Filling a test buffer with these contents and placing point before
> "text2", we get
>
>     (treesit-node-at (point) 'bash)
>     => #<treesit-node (heredoc_body) in 12-56>
>
>     (treesit-node-children (treesit-node-at (point) 'bash))
>     => (#<treesit-node (simple_expansion) in 18-22>
>         #<treesit-node (command_substitution) in 29-40>
>         #<treesit-node (simple_expansion) in 47-52>)
>
> This is now expected.
>
>     (point)
>     => 23
>
>     (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
>     => nil
>
> This is not expected. One would expect this to return
> command_substitution node, since it follows the position 23.

Ah, I (finally) see what you mean. I agree that it should return the
command_substitution node. It seems that tree-sitter returns nil
whenever POS is not in a child node. I need to see what’s the problem
here since treesit-node-first-child-for-pos is just a think wrapper
around the C counterpart.

Yuan

Message #23 received at 59633 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: miha <at> kamnitnik.top
Cc: 59633 <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of 
 heredocs with expansions
Date: Tue, 13 Dec 2022 20:01:31 -0800

Yuan Fu <casouri <at> gmail.com> writes:

> <miha <at> kamnitnik.top> writes:
>
>> Yuan Fu <casouri <at> gmail.com> writes:
>>
>>> What’s the return value of (treesit-node-at (point)) at that point?
>>> Probably the command_substitution node, and since it doesn’t have any
>>> children, treesit-node-first-child-for-pos would return nil.
>>
>> (treesit-node-at (point) 'bash)
>> => #<treesit-node (heredoc_body) in 12-56>
>>
>> I forgot to say that point is before "text2", outside of the
>> command_substitution node.
>>
>>> Yuan
>>
>> tee << EOF
>> text1 $var
>> text2 $(echo cmd)
>> text3
>> $var2
>> EOF
>>
>>
>> Filling a test buffer with these contents and placing point before
>> "text2", we get
>>
>>     (treesit-node-at (point) 'bash)
>>     => #<treesit-node (heredoc_body) in 12-56>
>>
>>     (treesit-node-children (treesit-node-at (point) 'bash))
>>     => (#<treesit-node (simple_expansion) in 18-22>
>>         #<treesit-node (command_substitution) in 29-40>
>>         #<treesit-node (simple_expansion) in 47-52>)
>>
>> This is now expected.
>>
>>     (point)
>>     => 23
>>
>>     (treesit-node-first-child-for-pos (treesit-node-at (point) 'bash) (point))
>>     => nil
>>
>> This is not expected. One would expect this to return
>> command_substitution node, since it follows the position 23.
>
> Ah, I (finally) see what you mean. I agree that it should return the
> command_substitution node. It seems that tree-sitter returns nil
> whenever POS is not in a child node. I need to see what’s the problem
> here since treesit-node-first-child-for-pos is just a think wrapper
> around the C counterpart.

Seems to be tree-sitter-bash problem, I submitted an issue at
https://github.com/tree-sitter/tree-sitter-bash/issues/139

Yuan

Message #28 received at 59633-done <at> debbugs.gnu.org (full text, mbox):

From: <miha <at> kamnitnik.top>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 59633-done <at> debbugs.gnu.org
Subject: Re: bug#59633: 29.0.50; tree-sitter-bash: Incorrect behaviour of
 heredocs with expansions
Date: Wed, 14 Dec 2022 16:07:10 +0100

[Message part 1 (text/plain, inline)]

Yuan Fu <casouri <at> gmail.com> writes:

> Seems to be tree-sitter-bash problem, I submitted an issue at
> https://github.com/tree-sitter/tree-sitter-bash/issues/139

Thanks. As this seems to be a tree-sitter-bash problem, I'm closing this
bug.

> Yuan

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.