GNU logs - #60650, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#60650: 30.0.50; tree-sitter: parsing (and font locking) occasionally breaks during editing
Resent-From: Mickey Petersen <mickey@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sun, 08 Jan 2023 10:40:01 +0000
Resent-Message-ID: <handler.60650.B.167317434714071 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 60650 <at> debbugs.gnu.org
Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org
 designates 178.79.136.144 as permitted sender)
 receiver=protection.outlook.com; client-ip=178.79.136.144;
 helo=semantical.co.uk; pr=C
From: Mickey Petersen <mickey@HIDDEN>
Date: Sun, 08 Jan 2023 10:38:54 +0000
Message-ID: <87358ll39d.fsf@HIDDEN>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2023 10:38:56.1793 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8d3964e8-2971-4200-78db-08daf1648b48
X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144];
 Helo=[semantical.co.uk]
X-MS-Exchange-CrossTenant-AuthSource: CWLGBR01FT039.eop-gbr01.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO2P265MB5728
Received-SPF: pass client-ip=40.107.11.133;
 envelope-from=mickey@HIDDEN;
 helo=GBR01-CWL-obe.outbound.protection.outlook.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)


It is occasionally possible to put the tree-sitter parser, I am guessing, into an erroneous state w.r.t. the contents of the buffer it is charged with parsing.

This rarely presents itself, and when it does, it only seems to happen when text is deleted and inserted in short order (`delete-region' + `insert') from an elisp command.

The resulting erroneous state then breaks font locking, resulting in mis-fontified code.

I must note that the parse tree is _correct_ and not invalid. There is no easy way to 'fix' this bug, as newlines and other manual edits rarely fixes the issue. There is also no easy way to insist that tree-sitter must re-do everything. Presumably deleting and recreating the parser could work as a workaround? I do not know.

The only way to truly and accurately fix the issue is to save and re-open the file. This then forces a complete re-parse of the buffer and fontification, fixing the issue.

I have zero means of reliably of reproducing this, unfortunately: it is sporadic, intermittent, and happens somewhat infrequently.

So I'd love some ideas on how to go about debugging this.



In GNU Emacs 30.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version
 3.24.20, cairo version 1.16.0) of 2023-01-02 built on mickey-work
Repository revision: c209802f7b3721a1b95113290934a23fee88f678
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12013000
System Description: Ubuntu 20.04.3 LTS

Configured using:
 'configure --with-native-compilation --with-json --with-mailutils
 --without-compress-install --with-imagemagick CC=gcc-10'

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Mickey Petersen <mickey@HIDDEN>
Subject: bug#60650: Acknowledgement (30.0.50; tree-sitter: parsing (and
 font locking) occasionally breaks during editing)
Message-ID: <handler.60650.B.167317434714071.ack <at> debbugs.gnu.org>
References: <87358ll39d.fsf@HIDDEN>
X-Gnu-PR-Message: ack 60650
X-Gnu-PR-Package: emacs
Reply-To: 60650 <at> debbugs.gnu.org
Date: Sun, 08 Jan 2023 10:40:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 60650 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
60650: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D60650
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#60650: 30.0.50; tree-sitter: parsing (and font locking)  occasionally breaks during editing
References: <87358ll39d.fsf@HIDDEN>
In-Reply-To: <87358ll39d.fsf@HIDDEN>
Resent-From: Yuan Fu <casouri@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Mon, 09 Jan 2023 03:23:02 +0000
Resent-Message-ID: <handler.60650.B60650.16732345802509 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Mickey Petersen <mickey@HIDDEN>
Cc: 60650 <at> debbugs.gnu.org
From: Yuan Fu <casouri@HIDDEN>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\))
Message-Id: <FDC429FD-DB4B-4325-AC94-7C5898E7BADC@HIDDEN>
Date: Sun, 8 Jan 2023 19:22:49 -0800
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>


Mickey Petersen <mickey@HIDDEN> writes:

> It is occasionally possible to put the tree-sitter parser, I am
> guessing, into an erroneous state w.r.t. the contents of the buffer it
> is charged with parsing.
>
> This rarely presents itself, and when it does, it only seems to happen
> when text is deleted and inserted in short order (`delete-region' +
> `insert') from an elisp command.
>
> The resulting erroneous state then breaks font locking, resulting in =
mis-fontified code.
>
> I must note that the parse tree is _correct_ and not invalid. There is
> no easy way to 'fix' this bug, as newlines and other manual edits
> rarely fixes the issue. There is also no easy way to insist that
> tree-sitter must re-do everything. Presumably deleting and recreating
> the parser could work as a workaround? I do not know.
>
> The only way to truly and accurately fix the issue is to save and
> re-open the file. This then forces a complete re-parse of the buffer
> and fontification, fixing the issue.

What do you mean by "correct and not invalid"? If a complete re-parse
fixes the issue, then the old parse tree must be incorrect, right? As
for mis-fontified code, is the position wrong (everything skew left or
right for a few character), or there is no fontification, or something
else? If the position is wrong, then it=E2=80=99s probably a bug in =
syncing the
parser=E2=80=99s position wrt the buffer content.

> I have zero means of reliably of reproducing this, unfortunately: it =
is sporadic, intermittent, and happens somewhat infrequently.
>
> So I'd love some ideas on how to go about debugging this.

I think the first step is to better understand what=E2=80=99s happening. =
Some
screenshots are also welcome. Thanks for your effort in advance!

Yuan

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#60650: 30.0.50; tree-sitter: parsing (and font locking) occasionally breaks during editing
Resent-From: Mickey Petersen <mickey@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Mon, 09 Jan 2023 09:07:01 +0000
Resent-Message-ID: <handler.60650.B60650.167325521724123 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Yuan Fu <casouri@HIDDEN>
Cc: 60650 <at> debbugs.gnu.org
Received-SPF: Pass (protection.outlook.com: domain of masteringemacs.org
 designates 178.79.136.144 as permitted sender)
 receiver=protection.outlook.com; client-ip=178.79.136.144;
 helo=semantical.co.uk; pr=C
References: <FDC429FD-DB4B-4325-AC94-7C5898E7BADC@HIDDEN>
User-agent: mu4e @VERSION@; emacs 30.0.50
From: Mickey Petersen <mickey@HIDDEN>
Date: Mon, 09 Jan 2023 09:02:41 +0000
Organization: Mastering Emacs
In-reply-to: <FDC429FD-DB4B-4325-AC94-7C5898E7BADC@HIDDEN>
Message-ID: <87ilhghyam.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 09:06:43.4870 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c2e39bbe-f6f9-45af-aa4e-08daf220d3f3
X-MS-Exchange-CrossTenant-Id: a4e27e3d-bab0-45e8-8942-e64cf9fbd34f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a4e27e3d-bab0-45e8-8942-e64cf9fbd34f; Ip=[178.79.136.144];
 Helo=[semantical.co.uk]
X-MS-Exchange-CrossTenant-AuthSource: LO2GBR01FT027.eop-gbr01.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO0P265MB6818
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


Yuan Fu <casouri@HIDDEN> writes:

> Mickey Petersen <mickey@HIDDEN> writes:
>
>> It is occasionally possible to put the tree-sitter parser, I am
>> guessing, into an erroneous state w.r.t. the contents of the buffer it
>> is charged with parsing.
>>
>> This rarely presents itself, and when it does, it only seems to happen
>> when text is deleted and inserted in short order (`delete-region' +
>> `insert') from an elisp command.
>>
>> The resulting erroneous state then breaks font locking, resulting in mis=
-fontified code.
>>
>> I must note that the parse tree is _correct_ and not invalid. There is
>> no easy way to 'fix' this bug, as newlines and other manual edits
>> rarely fixes the issue. There is also no easy way to insist that
>> tree-sitter must re-do everything. Presumably deleting and recreating
>> the parser could work as a workaround? I do not know.
>>
>> The only way to truly and accurately fix the issue is to save and
>> re-open the file. This then forces a complete re-parse of the buffer
>> and fontification, fixing the issue.
>
> What do you mean by "correct and not invalid"? If a complete re-parse
> fixes the issue, then the old parse tree must be incorrect, right? As
> for mis-fontified code, is the position wrong (everything skew left or
> right for a few character), or there is no fontification, or something
> else? If the position is wrong, then it=E2=80=99s probably a bug in synci=
ng the
> parser=E2=80=99s position wrt the buffer content.

I mean that the source code is syntactically correct. Given a clean
slate, the parser would not report any erroneous nodes.

But, somehow, both the font lock and indentation remains incorrect, even wh=
en I type text that would otherwise force a reparse. The only way to fix th=
is is to save the buffer (in a correct state), close and re-open the file. =
That fixes it.

That, and undoing the edited steps done by the elisp code. That snaps it ba=
ck to its original state, and the parse tree is back to normal.

It's... weird.

>
>> I have zero means of reliably of reproducing this, unfortunately: it is =
sporadic, intermittent, and happens somewhat infrequently.
>>
>> So I'd love some ideas on how to go about debugging this.
>
> I think the first step is to better understand what=E2=80=99s happening. =
Some
> screenshots are also welcome. Thanks for your effort in advance!
>

Yep. Will do!


> Yuan

Last modified: Mon, 9 Jan 2023 09:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.