GNU bug report logs - #69952
[PATCH] Support pdumping compiled queries by dumping their source

Previous Next

To reply to this bug, email your comments to 69952 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sergey Vinokurov <serg.foo <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH] Support pdumping compiled queries by dumping their source
Date: Sat, 23 Mar 2024 03:27:18 +0000

[Message part 1 (text/plain, inline)]

Hello,

This patch aims to allow more things to be dumped. In particular, 
compiled treesitter queries can be easily dumped by storing their source 
so that they will be recompiled on load.

I noticed that in my config compiled quires are created somewhere which 
prevents me dumping with standard Emacs build. But with this patch I can 
dump successfully and not bother finding out who produced the queries.

It seems like there should be no drawbacks in allowing more things to be 
dumped, please correct me if I'm wrong.

Regards,
Sergey

[pdump.patch (text/x-patch, attachment)]

Message #8 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sergey Vinokurov <serg.foo <at> gmail.com>
Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
 Yuan Fu <casouri <at> gmail.com>
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 23 Mar 2024 09:08:21 +0200

> Date: Sat, 23 Mar 2024 03:27:18 +0000
> From: Sergey Vinokurov <serg.foo <at> gmail.com>
> 
> This patch aims to allow more things to be dumped. In particular, 
> compiled treesitter queries can be easily dumped by storing their source 
> so that they will be recompiled on load.
> 
> I noticed that in my config compiled quires are created somewhere which 
> prevents me dumping with standard Emacs build. But with this patch I can 
> dump successfully and not bother finding out who produced the queries.
> 
> It seems like there should be no drawbacks in allowing more things to be 
> dumped, please correct me if I'm wrong.

Thanks.  I added Daniel and Yuan to the discussion, in case they have
comments.

And I have a question: what happens if the pdumper file dumped with
tree-sitter available is loaded by an Emacs session in which
tree-sitter is not available?  That can happen on Windows, for
example, if the tree-sitter library or the grammar library required
for recomputing the query is not available.  I think we need some code
to prevent Emacs from crashing on startup in that case.

Message #11 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sergey Vinokurov <serg.foo <at> gmail.com>
Cc: 69952 <at> debbugs.gnu.org, dancol <at> dancol.org, casouri <at> gmail.com
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 23 Mar 2024 15:09:30 +0200

> Date: Sat, 23 Mar 2024 12:53:37 +0000
> Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
>  Yuan Fu <casouri <at> gmail.com>
> From: Sergey Vinokurov <serg.foo <at> gmail.com>
> 
> On 23/03/2024 07:08, Eli Zaretskii wrote:
> >> Date: Sat, 23 Mar 2024 03:27:18 +0000
> >> From: Sergey Vinokurov <serg.foo <at> gmail.com>
> >>
> >> This patch aims to allow more things to be dumped. In particular,
> >> compiled treesitter queries can be easily dumped by storing their source
> >> so that they will be recompiled on load.
> >>
> >> I noticed that in my config compiled quires are created somewhere which
> >> prevents me dumping with standard Emacs build. But with this patch I can
> >> dump successfully and not bother finding out who produced the queries.
> >>
> >> It seems like there should be no drawbacks in allowing more things to be
> >> dumped, please correct me if I'm wrong.
> > 
> > Thanks.  I added Daniel and Yuan to the discussion, in case they have
> > comments.
> > 
> > And I have a question: what happens if the pdumper file dumped with
> > tree-sitter available is loaded by an Emacs session in which
> > tree-sitter is not available?  That can happen on Windows, for
> > example, if the tree-sitter library or the grammar library required
> > for recomputing the query is not available.  I think we need some code
> > to prevent Emacs from crashing on startup in that case.
> 
> According to c2ecb08775dc24618de507d2d1ce0f9b0debe17e, treesitter 
> queries are compiled lazily on fist use. When loading dump file in an 
> Emacs with missing grammar library the user should encounter an error on 
> first use of the query.
> 
> Quick testing shows that dump file can be loaded even with grammar 
> definitions removed. Compiled queries are still there in elisp and 
> there's no error during load.

OK, thanks.

Message #14 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Vinokurov <serg.foo <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
 Yuan Fu <casouri <at> gmail.com>
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 23 Mar 2024 12:53:37 +0000

On 23/03/2024 07:08, Eli Zaretskii wrote:
>> Date: Sat, 23 Mar 2024 03:27:18 +0000
>> From: Sergey Vinokurov <serg.foo <at> gmail.com>
>>
>> This patch aims to allow more things to be dumped. In particular,
>> compiled treesitter queries can be easily dumped by storing their source
>> so that they will be recompiled on load.
>>
>> I noticed that in my config compiled quires are created somewhere which
>> prevents me dumping with standard Emacs build. But with this patch I can
>> dump successfully and not bother finding out who produced the queries.
>>
>> It seems like there should be no drawbacks in allowing more things to be
>> dumped, please correct me if I'm wrong.
> 
> Thanks.  I added Daniel and Yuan to the discussion, in case they have
> comments.
> 
> And I have a question: what happens if the pdumper file dumped with
> tree-sitter available is loaded by an Emacs session in which
> tree-sitter is not available?  That can happen on Windows, for
> example, if the tree-sitter library or the grammar library required
> for recomputing the query is not available.  I think we need some code
> to prevent Emacs from crashing on startup in that case.

According to c2ecb08775dc24618de507d2d1ce0f9b0debe17e, treesitter 
queries are compiled lazily on fist use. When loading dump file in an 
Emacs with missing grammar library the user should encounter an error on 
first use of the query.

Quick testing shows that dump file can be loaded even with grammar 
definitions removed. Compiled queries are still there in elisp and 
there's no error during load.

Message #17 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dancol <at> dancol.org, casouri <at> gmail.com
Cc: serg.foo <at> gmail.com, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 13 Apr 2024 10:41:48 +0300

Ping!  Daniel and Yuan, any comments on this proposal?

> Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
>  Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 23 Mar 2024 09:08:21 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > Date: Sat, 23 Mar 2024 03:27:18 +0000
> > From: Sergey Vinokurov <serg.foo <at> gmail.com>
> > 
> > This patch aims to allow more things to be dumped. In particular, 
> > compiled treesitter queries can be easily dumped by storing their source 
> > so that they will be recompiled on load.
> > 
> > I noticed that in my config compiled quires are created somewhere which 
> > prevents me dumping with standard Emacs build. But with this patch I can 
> > dump successfully and not bother finding out who produced the queries.
> > 
> > It seems like there should be no drawbacks in allowing more things to be 
> > dumped, please correct me if I'm wrong.
> 
> Thanks.  I added Daniel and Yuan to the discussion, in case they have
> comments.
> 
> And I have a question: what happens if the pdumper file dumped with
> tree-sitter available is loaded by an Emacs session in which
> tree-sitter is not available?  That can happen on Windows, for
> example, if the tree-sitter library or the grammar library required
> for recomputing the query is not available.  I think we need some code
> to prevent Emacs from crashing on startup in that case.
> 
> 
> 
> 

Message #20 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Apr 2024 01:08:15 -0700

> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> Ping!  Daniel and Yuan, any comments on this proposal?
> 
>> Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
>> Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 23 Mar 2024 09:08:21 +0200
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> 
>>> Date: Sat, 23 Mar 2024 03:27:18 +0000
>>> From: Sergey Vinokurov <serg.foo <at> gmail.com>
>>> 
>>> This patch aims to allow more things to be dumped. In particular, 
>>> compiled treesitter queries can be easily dumped by storing their source 
>>> so that they will be recompiled on load.
>>> 
>>> I noticed that in my config compiled quires are created somewhere which 
>>> prevents me dumping with standard Emacs build. But with this patch I can 
>>> dump successfully and not bother finding out who produced the queries.
>>> 
>>> It seems like there should be no drawbacks in allowing more things to be 
>>> dumped, please correct me if I'm wrong.
>> 
>> Thanks.  I added Daniel and Yuan to the discussion, in case they have
>> comments.
>> 
>> And I have a question: what happens if the pdumper file dumped with
>> tree-sitter available is loaded by an Emacs session in which
>> tree-sitter is not available?  That can happen on Windows, for
>> example, if the tree-sitter library or the grammar library required
>> for recomputing the query is not available.  I think we need some code
>> to prevent Emacs from crashing on startup in that case.

Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.

Yuan

Message #23 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Apr 2024 12:32:57 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 20 Apr 2024 01:08:15 -0700
> Cc: dancol <at> dancol.org,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> 
> >> And I have a question: what happens if the pdumper file dumped with
> >> tree-sitter available is loaded by an Emacs session in which
> >> tree-sitter is not available?  That can happen on Windows, for
> >> example, if the tree-sitter library or the grammar library required
> >> for recomputing the query is not available.  I think we need some code
> >> to prevent Emacs from crashing on startup in that case.
> 
> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.

Giving up: no.  But I think the code which loads the queries from the
pdumper file should be protected from crashing in those cases.  Can
you suggest how to do that?

Message #26 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Apr 2024 15:50:36 -0700

> On Apr 20, 2024, at 2:32 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 20 Apr 2024 01:08:15 -0700
>> Cc: dancol <at> dancol.org,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>> 
>> 
>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>> 
>>>> And I have a question: what happens if the pdumper file dumped with
>>>> tree-sitter available is loaded by an Emacs session in which
>>>> tree-sitter is not available?  That can happen on Windows, for
>>>> example, if the tree-sitter library or the grammar library required
>>>> for recomputing the query is not available.  I think we need some code
>>>> to prevent Emacs from crashing on startup in that case.
>> 
>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
> 
> Giving up: no.  But I think the code which loads the queries from the
> pdumper file should be protected from crashing in those cases.  Can
> you suggest how to do that?

Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.

Yuan

Message #29 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 07:53:46 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 20 Apr 2024 15:50:36 -0700
> Cc: Daniel Colascione <dancol <at> dancol.org>,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> >>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>>> 
> >>>> And I have a question: what happens if the pdumper file dumped with
> >>>> tree-sitter available is loaded by an Emacs session in which
> >>>> tree-sitter is not available?  That can happen on Windows, for
> >>>> example, if the tree-sitter library or the grammar library required
> >>>> for recomputing the query is not available.  I think we need some code
> >>>> to prevent Emacs from crashing on startup in that case.
> >> 
> >> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
> > 
> > Giving up: no.  But I think the code which loads the queries from the
> > pdumper file should be protected from crashing in those cases.  Can
> > you suggest how to do that?
> 
> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.

Can Emacs compile a query if the tree-sitter shared library and/or the
grammar library is not available?

Message #32 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 16:41:54 -0700

> On Apr 20, 2024, at 9:53 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 20 Apr 2024 15:50:36 -0700
>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>>>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>>>> 
>>>>>> And I have a question: what happens if the pdumper file dumped with
>>>>>> tree-sitter available is loaded by an Emacs session in which
>>>>>> tree-sitter is not available?  That can happen on Windows, for
>>>>>> example, if the tree-sitter library or the grammar library required
>>>>>> for recomputing the query is not available.  I think we need some code
>>>>>> to prevent Emacs from crashing on startup in that case.
>>>> 
>>>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
>>> 
>>> Giving up: no.  But I think the code which loads the queries from the
>>> pdumper file should be protected from crashing in those cases.  Can
>>> you suggest how to do that?
>> 
>> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.
> 
> Can Emacs compile a query if the tree-sitter shared library and/or the
> grammar library is not available?

No. You need both tree-sitter library and the grammar library.

Yuan

Message #35 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Mon, 22 Apr 2024 08:47:31 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sun, 21 Apr 2024 16:41:54 -0700
> Cc: Daniel Colascione <dancol <at> dancol.org>,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On Apr 20, 2024, at 9:53 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri <at> gmail.com>
> >> Date: Sat, 20 Apr 2024 15:50:36 -0700
> >> Cc: Daniel Colascione <dancol <at> dancol.org>,
> >> serg.foo <at> gmail.com,
> >> 69952 <at> debbugs.gnu.org
> >> 
> >>>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>>>>> 
> >>>>>> And I have a question: what happens if the pdumper file dumped with
> >>>>>> tree-sitter available is loaded by an Emacs session in which
> >>>>>> tree-sitter is not available?  That can happen on Windows, for
> >>>>>> example, if the tree-sitter library or the grammar library required
> >>>>>> for recomputing the query is not available.  I think we need some code
> >>>>>> to prevent Emacs from crashing on startup in that case.
> >>>> 
> >>>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
> >>> 
> >>> Giving up: no.  But I think the code which loads the queries from the
> >>> pdumper file should be protected from crashing in those cases.  Can
> >>> you suggest how to do that?
> >> 
> >> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.
> > 
> > Can Emacs compile a query if the tree-sitter shared library and/or the
> > grammar library is not available?
> 
> No. You need both tree-sitter library and the grammar library.

That's what I thought.  So starting Emacs in that case will attempt to
call a function from the tree-sitter library, and will segfault,
right?  If so, we should have some protection in the code in
pdumper.c that loads queries, which tests that tree-sitter is
availabale, and if not does something to prevent the segfault, like
not loading the query, perhaps?

Message #38 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 23:04:47 -0700

> On Apr 21, 2024, at 10:47 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sun, 21 Apr 2024 16:41:54 -0700
>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>> 
>> 
>>> On Apr 20, 2024, at 9:53 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri <at> gmail.com>
>>>> Date: Sat, 20 Apr 2024 15:50:36 -0700
>>>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>>>> serg.foo <at> gmail.com,
>>>> 69952 <at> debbugs.gnu.org
>>>> 
>>>>>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>>>>>> 
>>>>>>>> And I have a question: what happens if the pdumper file dumped with
>>>>>>>> tree-sitter available is loaded by an Emacs session in which
>>>>>>>> tree-sitter is not available?  That can happen on Windows, for
>>>>>>>> example, if the tree-sitter library or the grammar library required
>>>>>>>> for recomputing the query is not available.  I think we need some code
>>>>>>>> to prevent Emacs from crashing on startup in that case.
>>>>>> 
>>>>>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
>>>>> 
>>>>> Giving up: no.  But I think the code which loads the queries from the
>>>>> pdumper file should be protected from crashing in those cases.  Can
>>>>> you suggest how to do that?
>>>> 
>>>> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.
>>> 
>>> Can Emacs compile a query if the tree-sitter shared library and/or the
>>> grammar library is not available?
>> 
>> No. You need both tree-sitter library and the grammar library.
> 
> That's what I thought.  So starting Emacs in that case will attempt to
> call a function from the tree-sitter library, and will segfault,
> right?  If so, we should have some protection in the code in
> pdumper.c that loads queries, which tests that tree-sitter is
> availabale, and if not does something to prevent the segfault, like
> not loading the query, perhaps?

Right, it can use Ftreesit_available_p (we can define a C equivalent if pdumper can’t call lisp), and decide whether to compile the query or just keep the query as a string, or signal an error?

Yuan

Message #41 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Mon, 22 Apr 2024 09:18:49 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sun, 21 Apr 2024 23:04:47 -0700
> Cc: Daniel Colascione <dancol <at> dancol.org>,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> >> No. You need both tree-sitter library and the grammar library.
> > 
> > That's what I thought.  So starting Emacs in that case will attempt to
> > call a function from the tree-sitter library, and will segfault,
> > right?  If so, we should have some protection in the code in
> > pdumper.c that loads queries, which tests that tree-sitter is
> > availabale, and if not does something to prevent the segfault, like
> > not loading the query, perhaps?
> 
> Right, it can use Ftreesit_available_p (we can define a C equivalent if pdumper can’t call lisp), and decide whether to compile the query or just keep the query as a string, or signal an error?

We cannot safely signal an error at that point, I think, so just
keeping the query as a string should do, as it will then signal an
error at run time when that query is used, is that right?

Message #44 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 23:25:10 -0700

> On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sun, 21 Apr 2024 23:04:47 -0700
>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>>>> No. You need both tree-sitter library and the grammar library.
>>> 
>>> That's what I thought.  So starting Emacs in that case will attempt to
>>> call a function from the tree-sitter library, and will segfault,
>>> right?  If so, we should have some protection in the code in
>>> pdumper.c that loads queries, which tests that tree-sitter is
>>> availabale, and if not does something to prevent the segfault, like
>>> not loading the query, perhaps?
>> 
>> Right, it can use Ftreesit_available_p (we can define a C equivalent if pdumper can’t call lisp), and decide whether to compile the query or just keep the query as a string, or signal an error?
> 
> We cannot safely signal an error at that point, I think, so just
> keeping the query as a string should do, as it will then signal an
> error at run time when that query is used, is that right?

Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.

Yuan

Message #47 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>, serg.foo <at> gmail.com
Cc: 69952 <at> debbugs.gnu.org, dancol <at> dancol.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 04 May 2024 12:39:05 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sun, 21 Apr 2024 23:25:10 -0700
> Cc: dancol <at> dancol.org,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> > On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> > 
> > We cannot safely signal an error at that point, I think, so just
> > keeping the query as a string should do, as it will then signal an
> > error at run time when that query is used, is that right?
> 
> Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.

Can you suggest such an addition to the patch?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.