Received: (at 60691) by debbugs.gnu.org; 14 Jan 2023 07:51:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 14 02:51:14 2023
Received: from localhost ([127.0.0.1]:53181 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pGbJq-0000OC-Ee
for submit <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:51:14 -0500
Received: from mail-pj1-f50.google.com ([209.85.216.50]:53092)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <casouri@HIDDEN>) id 1pGbJo-0000Nx-U6
for 60691 <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:51:13 -0500
Received: by mail-pj1-f50.google.com with SMTP id o13so21047464pjg.2
for <60691 <at> debbugs.gnu.org>; Fri, 13 Jan 2023 23:51:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=tlwGvZ6qBr3cwq3HB43bzz+uAXUoxC54NqVO7H8qBDQ=;
b=gC6QEerdtly6ThSbVLgPj300mYvlo6rNAJNTIZRaZFla71m1OQYvcjcE2c72MZnCKT
Dx4QOLjT5Gdd8nyxVCHXbQcMIoUd+WyFUEKbIMdMiNf0X7+w9AEVLoH8kxDOODlez35o
tmFkFD3MaP5Im9Q+ImN6qVNv0s2rq/V9e1ojbHDV9zG4gPCF+i2XoPrDsP27kYJkojT7
wY5QulssJfCZaKZ39tXHPb2M9TX06E2A7Cqfa73RASTtcfPDi60cDfKRrkxkp+YLaBk/
daS050u17I9jv4HH6qIK50a8YwP4uWkk51rAscL9fNI5wrqWAMJmnzFZZllbu7q5osm5
oJTg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=tlwGvZ6qBr3cwq3HB43bzz+uAXUoxC54NqVO7H8qBDQ=;
b=e3eq3VFAX/kJA6DW5xX+2VZWV2xq1oums5m3jwcz0yuzsfKa67Sa8cUQomGSoDl08/
VAEkzX2JyqLNuhFoCE/l1jvCmoO6Aup4gRISvx8O0flyeSZ6eQTKPDVjco32ZVOHPqdg
nVuYrJiW8zbSIkZp4fjvwyOrc/sp7gOydFhIqNsJ9uv+w4PZRVvh5ecfKkzIr57iu9Km
T3zTQIpP5KLhIbWmIl71/BI6aRQai9esZp2U984LP5wYUFlIv7Bq2LIVxLbHMfxyraqg
u8RwKIhunHMi1OU8fAlT8UvOfD2uzjnrSvlyQ2WVlB0oqqKlw4unXd+G11ynQ4LI/+65
xPAw==
X-Gm-Message-State: AFqh2kquch3cPEwJYz9QuUa5pb8c2Z05GGzmr44kRamZ9mBO74zIDMF0
ZZBIiszlqx5+5O3f5ChvQ1c=
X-Google-Smtp-Source: AMrXdXsOaG8BsiuvSK7mo9yUGxzMSefoZmlyNv0xOcOUYTk+r7C+6fXbRfZ7QGv18ksB0NvXI6Fvxw==
X-Received: by 2002:a17:903:2c5:b0:192:cf35:3ff8 with SMTP id
s5-20020a17090302c500b00192cf353ff8mr51947643plk.21.1673682666821;
Fri, 13 Jan 2023 23:51:06 -0800 (PST)
Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com.
[172.117.161.177]) by smtp.gmail.com with ESMTPSA id
f5-20020a170902ce8500b001870dc3b4c0sm15428772plg.74.2023.01.13.23.51.05
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 13 Jan 2023 23:51:06 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\))
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
From: Yuan Fu <casouri@HIDDEN>
In-Reply-To: <83mt6l8th7.fsf@HIDDEN>
Date: Fri, 13 Jan 2023 23:51:05 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <09B9914D-2B6D-4F73-89E7-1B9A02A1C44E@HIDDEN>
References: <867cxv3dnn.fsf@HIDDEN>
<6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN>
<0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN>
<E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> <83mt6mac0c.fsf@HIDDEN>
<3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN> <83mt6l8th7.fsf@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.1)
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 60691
Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN,
dgutov@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On Jan 13, 2023, at 11:29 PM, Eli Zaretskii <eliz@HIDDEN> wrote:
>=20
>> From: Yuan Fu <casouri@HIDDEN>
>> Date: Fri, 13 Jan 2023 19:48:40 -0800
>> Cc: dgutov@HIDDEN,
>> 60691 <at> debbugs.gnu.org,
>> juri@HIDDEN,
>> monnier@HIDDEN
>>=20
>> Thanks, I pushed a fix for it. I also used intern_c_string in some =
places like these:
>>=20
>> intern_c_string (":?=E2=80=9D)
>> intern_c_string (":*")
>>=20
>> I want to change them to use DEFSYM, but what should be the c name =
for them?
>=20
> Yes, DEFSYM is better in such cases. The C name can be QCquestion and
> QCasterix, for example.
My worry is that they will conflict with, eg, symbol `question=E2=80=99 =
and `asterix=E2=80=99, if someone ever defines them in the C codebase. =
Is that not possible?
Yuan=
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at 60691) by debbugs.gnu.org; 14 Jan 2023 07:29:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 14 02:29:15 2023
Received: from localhost ([127.0.0.1]:53121 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pGayZ-00088D-LB
for submit <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:29:15 -0500
Received: from eggs.gnu.org ([209.51.188.92]:59966)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1pGayX-000881-IC
for 60691 <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:29:13 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1pGayQ-0006yC-WA; Sat, 14 Jan 2023 02:29:07 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
Date; bh=1AlmdwoHsiVSqHOb9EH33puDmiL/W0ey+HQq6y/Jpyg=; b=j+2vNrHy0Fkk+1nIOnWg
fIO+/66MG703M3UWCwwpdTNHg8gU0HPS5qWmQYMHak4f3zg8R1nhnAYR25f6Zs3ttvjunXpbVyJWN
iqC8JyHmrnVxasay3lftfOpi0TYnWbqs/LdP0vAiAI/qZV0eRrT709sSEptU+rYE0j8gpDVCGMHdJ
nSdgydJK8i2AQq3aEZUIpSz9cweapAdT5h+s60UpACvrilvpAU7jBWwLikfygW830QNqpMNEQP5mm
SjDdr/Nc/yinR/v1roLiwGAFlHWInONZu/kY9WoPG2ANOh8HEuLVNUy78zPwpUF7KmywVljcSOObE
nF2mqARb2J2cog==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1pGayQ-0004QO-7m; Sat, 14 Jan 2023 02:29:06 -0500
Date: Sat, 14 Jan 2023 09:29:08 +0200
Message-Id: <83mt6l8th7.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Yuan Fu <casouri@HIDDEN>
In-Reply-To: <3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN> (message from
Yuan Fu on Fri, 13 Jan 2023 19:48:40 -0800)
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
References: <867cxv3dnn.fsf@HIDDEN>
<6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN>
<0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN>
<E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> <83mt6mac0c.fsf@HIDDEN>
<3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 60691
Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN,
dgutov@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Yuan Fu <casouri@HIDDEN>
> Date: Fri, 13 Jan 2023 19:48:40 -0800
> Cc: dgutov@HIDDEN,
> 60691 <at> debbugs.gnu.org,
> juri@HIDDEN,
> monnier@HIDDEN
>
> Thanks, I pushed a fix for it. I also used intern_c_string in some places like these:
>
> intern_c_string (":?”)
> intern_c_string (":*")
>
> I want to change them to use DEFSYM, but what should be the c name for them?
Yes, DEFSYM is better in such cases. The C name can be QCquestion and
QCasterix, for example.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at 60691) by debbugs.gnu.org; 14 Jan 2023 03:48:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 22:48:50 2023
Received: from localhost ([127.0.0.1]:52926 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pGXXG-000886-86
for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 22:48:50 -0500
Received: from mail-pj1-f54.google.com ([209.85.216.54]:37558)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <casouri@HIDDEN>) id 1pGXXE-00087t-1X
for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 22:48:49 -0500
Received: by mail-pj1-f54.google.com with SMTP id
o1-20020a17090a678100b00219cf69e5f0so28848528pjj.2
for <60691 <at> debbugs.gnu.org>; Fri, 13 Jan 2023 19:48:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=mJwoFOhxZip+RMp6eVcrydfjakDfE+zcJXQfnL9xL+g=;
b=bllNYqR/56URhI50lwJTXe6eKVKS2ElYzc5rrfDx2vGMMTLe1n/fiiitn9X0FvvaLR
7K1Ybw+pmxe8AK0fUlz43OHNIl7zvHb6aBqmSBAWnLTxcldLkw8RTkfjgnoYG9odVpgW
Mklkr+98W0lZbNpmSuatBVZotqPdcLqXWkqs2PnOJsYCVp5p5JykGa1tlHMftagCazZ9
QYeKAC5ze4pQsZCWzA0PEwEEwqK+NMeCG6RALFC7yboItpJICrP3XBi3/XovSRN8oaxu
9RJtIRzMCCSftQ2lgiajhpmL55SIhJwyAuO/xJ+sSqbYG9zmNcfBi5jMElESxcWVbvVK
FHog==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=mJwoFOhxZip+RMp6eVcrydfjakDfE+zcJXQfnL9xL+g=;
b=yJV44p43g5WwanplNpYTdEAKLSJHGeiiuEFxnfTqZlDIyrsC1vTUaK4IeR8HVsuiHN
YVY4HVB4vsX438u8cvc4V0wWYrWRxumC457wMZsxXO+yQVTQ8Y4IcPN9AJQEkmfNWwQN
tc1kDjBceafkD2WDIlaMP84egRRTMxFuRIxEtOcjKJU9REwX8HP7s4jI70wnDaxzCrNm
TATQ+ykh3UrjKBHPmFYu/VFRrt0BZah0wj8vADwxvuJ8MOIgGbhiqPm40aOyPfrarIDK
l2n1ninYLqCcrJe/g9OXSASVFgYbCfjOjGF4EztLzmknJDLYIYLnruf/PQlNcq18qz1c
P79Q==
X-Gm-Message-State: AFqh2konV5y19veQUnVCYRVFF32Ov0Awc4EQYJ29nLwtRN+k7kCBU/rb
o+s5PghJJgsmEmIcFKjyK5g=
X-Google-Smtp-Source: AMrXdXtpSk23kJnzTR+bw/wfRcdIt8T3jDpdSKU5es3l/FqspbfkVSE3CUGlUYNrbDd1t0kotkb7bA==
X-Received: by 2002:a17:902:6a87:b0:194:6d39:5911 with SMTP id
n7-20020a1709026a8700b001946d395911mr4941697plk.40.1673668122011;
Fri, 13 Jan 2023 19:48:42 -0800 (PST)
Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com.
[172.117.161.177]) by smtp.gmail.com with ESMTPSA id
h18-20020a170902f55200b001929dff50a9sm14950031plf.87.2023.01.13.19.48.40
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 13 Jan 2023 19:48:41 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\))
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
From: Yuan Fu <casouri@HIDDEN>
In-Reply-To: <83mt6mac0c.fsf@HIDDEN>
Date: Fri, 13 Jan 2023 19:48:40 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN>
References: <867cxv3dnn.fsf@HIDDEN>
<6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN>
<0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN>
<E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> <83mt6mac0c.fsf@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.1)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 60691
Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN,
dgutov@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On Jan 13, 2023, at 3:51 AM, Eli Zaretskii <eliz@HIDDEN> wrote:
>=20
>> From: Yuan Fu <casouri@HIDDEN>
>> Date: Fri, 13 Jan 2023 01:15:09 -0800
>> Cc: Dmitry Gutov <dgutov@HIDDEN>,
>> 60691 <at> debbugs.gnu.org,
>> Juri Linkov <juri@HIDDEN>,
>> Stefan Monnier <monnier@HIDDEN>
>>=20
>>> On Jan 12, 2023, at 11:57 PM, Eli Zaretskii <eliz@HIDDEN> wrote:
>>>=20
>>>> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN
>>>> Date: Fri, 13 Jan 2023 01:40:56 +0200
>>>> From: Dmitry Gutov <dgutov@HIDDEN>
>>>>=20
>>>> Managed to reproduce this after running the test in a couple of=20
>>>> different files.
>>>>=20
>>>> But 'M-x memory-usage' says no such command, and 'M-x =
memory-report'=20
>>>> ends up with this error:
>>>>=20
>>>> Debugger entered--Lisp error: (wrong-type-argument =
number-or-marker-p nil)
>>>> memory-report--gc-elem(nil strings)
>>>> memory-report--garbage-collect()
>>>> memory-report()
>>>=20
>>> This means GC is disabled in this session at the time you invoke
>>> memory-report. Which shouldn't happen, of course. It sounds like
>>> your pure Lisp storage overflowed, and that disabled GC.
>>>=20
>>> And I think I see the problem: we use build_pure_c_string in =
treesit.c
>>> in places that we shouldn't.
>>>=20
>>> Yuan, build_pure_c_string should only be used in places such as
>>> syms_of_treesit, which are called just once, during dumping. Look =
at
>>> all the other calls to this function in the sources, and you will =
see
>>> it. In all other cases, you should do one of the following:
>>>=20
>>> . for strings whose text is fixed, define a variable, give it the
>>> value in syms_of_treesit using build_pure_c_string, then use that
>>> variable elsewhere in the source
>>=20
>> Can I define a bunch of static C variables and initialize them in =
syms_of_treesit, or they have to be all Lisp variables? Eg,
>>=20
>> static Lisp_Object TREESIT_STAR;
>>=20
>> ...
>>=20
>> void
>> syms_of_treesit (void)
>> {
>> ...
>> TREESIT_STAR =3D build_pure_c_string ("*");
>> ...
>> }
>=20
> Yes, of course. Look, for example, how coding.c does that:
>=20
> /* A string that serves as name of the reusable work buffer, and as =
base
> name of temporary work buffers used for code-conversion =
operations. */
> static Lisp_Object Vcode_conversion_workbuf_name;
> [...]
> void
> syms_of_coding (void)
> {
> [...]
> staticpro (&Vcode_conversion_workbuf_name);
> Vcode_conversion_workbuf_name =3D build_pure_c_string (" =
*code-conversion-work*");
>=20
> But please keep the convention of naming such variables Vsome_thing,
> both regarding the "V" and the fact that the name is otherwise
> lower-case.
Thanks, I pushed a fix for it. I also used intern_c_string in some =
places like these:
intern_c_string (":?=E2=80=9D)
intern_c_string (":*")
I want to change them to use DEFSYM, but what should be the c name for =
them?
Yuan
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at 60691) by debbugs.gnu.org; 13 Jan 2023 11:51:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 06:51:27 2023
Received: from localhost ([127.0.0.1]:49768 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pGIal-0000kU-4m
for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 06:51:27 -0500
Received: from eggs.gnu.org ([209.51.188.92]:44674)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1pGIaj-0000kI-93
for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 06:51:25 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1pGIab-0004gp-Of; Fri, 13 Jan 2023 06:51:17 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=v8L5KErd7/OHhGFb/Qc5qHRtsMnHLsT41EQx85iBh5k=; b=lt3mc3XnkpMi
wKgin12cfjd39wdkwFMBf7/au6rEBHLG6Z2GNLyrBxIbnFe+TLAE6iMNuJ0VIXMEeAoljP6oX0cio
KS8iIebJl1fSH9Fyd2ze698oCq85Fg2NzCQNEWKQxuJ6jv6ulKQKKcVy4WRks7Pr/Sa0om6kqKP61
y7Pw8RSU1qkUTwezMoSUkQ9KOG7pJSRIpFUTfRBXcac0caVHsI+VBhIorKFlAFT5KzEY7ROnLb6SA
0i8l9NKpyfUFtIjI1GgIgCa6frZ7TJMXPPP2UPB9WNJeJY3cdnK4NbNnppMTZuWBLscWzfK36txLC
EWQqAJPDl98sULV/BPzd6Q==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1pGIaa-0007gG-Vg; Fri, 13 Jan 2023 06:51:17 -0500
Date: Fri, 13 Jan 2023 13:51:15 +0200
Message-Id: <83mt6mac0c.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Yuan Fu <casouri@HIDDEN>
In-Reply-To: <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> (message from
Yuan Fu on Fri, 13 Jan 2023 01:15:09 -0800)
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
References: <867cxv3dnn.fsf@HIDDEN>
<6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN>
<0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN>
<E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 60691
Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN,
dgutov@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Yuan Fu <casouri@HIDDEN>
> Date: Fri, 13 Jan 2023 01:15:09 -0800
> Cc: Dmitry Gutov <dgutov@HIDDEN>,
> 60691 <at> debbugs.gnu.org,
> Juri Linkov <juri@HIDDEN>,
> Stefan Monnier <monnier@HIDDEN>
>
> > On Jan 12, 2023, at 11:57 PM, Eli Zaretskii <eliz@HIDDEN> wrote:
> >
> >> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN
> >> Date: Fri, 13 Jan 2023 01:40:56 +0200
> >> From: Dmitry Gutov <dgutov@HIDDEN>
> >>
> >> Managed to reproduce this after running the test in a couple of
> >> different files.
> >>
> >> But 'M-x memory-usage' says no such command, and 'M-x memory-report'
> >> ends up with this error:
> >>
> >> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
> >> memory-report--gc-elem(nil strings)
> >> memory-report--garbage-collect()
> >> memory-report()
> >
> > This means GC is disabled in this session at the time you invoke
> > memory-report. Which shouldn't happen, of course. It sounds like
> > your pure Lisp storage overflowed, and that disabled GC.
> >
> > And I think I see the problem: we use build_pure_c_string in treesit.c
> > in places that we shouldn't.
> >
> > Yuan, build_pure_c_string should only be used in places such as
> > syms_of_treesit, which are called just once, during dumping. Look at
> > all the other calls to this function in the sources, and you will see
> > it. In all other cases, you should do one of the following:
> >
> > . for strings whose text is fixed, define a variable, give it the
> > value in syms_of_treesit using build_pure_c_string, then use that
> > variable elsewhere in the source
>
> Can I define a bunch of static C variables and initialize them in syms_of_treesit, or they have to be all Lisp variables? Eg,
>
> static Lisp_Object TREESIT_STAR;
>
> ...
>
> void
> syms_of_treesit (void)
> {
> ...
> TREESIT_STAR = build_pure_c_string ("*");
> ...
> }
Yes, of course. Look, for example, how coding.c does that:
/* A string that serves as name of the reusable work buffer, and as base
name of temporary work buffers used for code-conversion operations. */
static Lisp_Object Vcode_conversion_workbuf_name;
[...]
void
syms_of_coding (void)
{
[...]
staticpro (&Vcode_conversion_workbuf_name);
Vcode_conversion_workbuf_name = build_pure_c_string (" *code-conversion-work*");
But please keep the convention of naming such variables Vsome_thing,
both regarding the "V" and the fact that the name is otherwise
lower-case.
Thanks.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at 60691) by debbugs.gnu.org; 13 Jan 2023 09:15:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 04:15:22 2023
Received: from localhost ([127.0.0.1]:49570 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pGG9i-0004xB-6E
for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 04:15:22 -0500
Received: from mail-pl1-f181.google.com ([209.85.214.181]:46730)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <casouri@HIDDEN>) id 1pGG9g-0004wv-Db
for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 04:15:21 -0500
Received: by mail-pl1-f181.google.com with SMTP id jn22so22830061plb.13
for <60691 <at> debbugs.gnu.org>; Fri, 13 Jan 2023 01:15:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=grdT2JPcyMNP8d/LqeBAZEhUtyUNSyPdL5gl1vq0JwQ=;
b=bThSs0rp4hQV/OnYaq7tmwI6hjfM+5E/b3PJpDr3WBhRNi/viUR+Q4kfYQKWIjIwyU
EItgHVlGzCZKVgU3E8bl9Npl7/WlaFEUQOsGmcr9HiAWY6n4JqTZg5VilQ23esOxXmBZ
grBJW+kgWJSmEuhDW3BNDD452UiGwqev2PZkmpC33l7UCkBKVBWvJvBvovtYYM49xi2s
Qwka22Z2s7nMP4YZQEeK356+o15ydGsPtVZ6IADbbFs/Sc3ABUor+vGmtXEBJ7AnO61d
Yc2jDxqXwSQEagkdhbEtIV86tP7aV/XW6TNsPt8AAYphLwWI7kUN2SpPHkQLJiMgiqIK
VHKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=grdT2JPcyMNP8d/LqeBAZEhUtyUNSyPdL5gl1vq0JwQ=;
b=rmzSb2gpRZm3SqfJcWcHRTPdrznlxr7SQMx+CJTFLMk6TBMYy6G0QYdTCvXKmR9hcQ
TRbm/H9j6U7PTOazcjznFSweonydJYtRgEVsr9SHsIgCbsi+wcALT59f4DFimEQFPXhO
TnXWkOqZ914Rr8MqX74ds6Q9eJ8ESSfFubtaOi63s5+PAlZfo1jMiBwGRC5S47R9ulHR
PYYwtXwBdNBd3fP7NVxwqINj+Bhymb0epgjkWztERcx8v86SKB03ecXnSDYyCYBqNqWs
XaQrmXbV/vdMn7wG1WvnKLZ30emlIqhGYFSZtU188ym1M1bwntP3ZUPa2mt2snzaGlXY
4F/w==
X-Gm-Message-State: AFqh2kof7hcb5GlE+bYYSxi4pBFALRz0HeOp6pFS6C+qoB9p7VJCJtGx
KsrDtAVhY7YoeEuRTBHHzr8=
X-Google-Smtp-Source: AMrXdXs491aEfVqGOkzMleIhIQp8QY8tXpp+MsNSsDX8VGQ815yIxeLK+jJIW3RfH8xVZHwrN6axhw==
X-Received: by 2002:a17:902:edc5:b0:192:c882:703e with SMTP id
q5-20020a170902edc500b00192c882703emr39929936plk.43.1673601314502;
Fri, 13 Jan 2023 01:15:14 -0800 (PST)
Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com.
[172.117.161.177]) by smtp.gmail.com with ESMTPSA id
u11-20020a6540cb000000b0046ff3634a78sm11320328pgp.71.2023.01.13.01.15.13
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 13 Jan 2023 01:15:14 -0800 (PST)
Content-Type: text/plain;
charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\))
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
From: Yuan Fu <casouri@HIDDEN>
In-Reply-To: <83r0vyamtb.fsf@HIDDEN>
Date: Fri, 13 Jan 2023 01:15:09 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN>
References: <867cxv3dnn.fsf@HIDDEN>
<6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN>
<0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.1)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 60691
Cc: Juri Linkov <juri@HIDDEN>, 60691 <at> debbugs.gnu.org,
Stefan Monnier <monnier@HIDDEN>, Dmitry Gutov <dgutov@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On Jan 12, 2023, at 11:57 PM, Eli Zaretskii <eliz@HIDDEN> wrote:
>=20
>> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN
>> Date: Fri, 13 Jan 2023 01:40:56 +0200
>> From: Dmitry Gutov <dgutov@HIDDEN>
>>=20
>> Managed to reproduce this after running the test in a couple of=20
>> different files.
>>=20
>> But 'M-x memory-usage' says no such command, and 'M-x memory-report'=20=
>> ends up with this error:
>>=20
>> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p =
nil)
>> memory-report--gc-elem(nil strings)
>> memory-report--garbage-collect()
>> memory-report()
>=20
> This means GC is disabled in this session at the time you invoke
> memory-report. Which shouldn't happen, of course. It sounds like
> your pure Lisp storage overflowed, and that disabled GC.
>=20
> And I think I see the problem: we use build_pure_c_string in treesit.c
> in places that we shouldn't.
>=20
> Yuan, build_pure_c_string should only be used in places such as
> syms_of_treesit, which are called just once, during dumping. Look at
> all the other calls to this function in the sources, and you will see
> it. In all other cases, you should do one of the following:
>=20
> . for strings whose text is fixed, define a variable, give it the
> value in syms_of_treesit using build_pure_c_string, then use that
> variable elsewhere in the source
Can I define a bunch of static C variables and initialize them in =
syms_of_treesit, or they have to be all Lisp variables? Eg,
static Lisp_Object TREESIT_STAR;
...
void
syms_of_treesit (void)
{
...
TREESIT_STAR =3D build_pure_c_string ("*");
...
}
Yuan=
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at 60691) by debbugs.gnu.org; 13 Jan 2023 07:58:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 02:58:11 2023
Received: from localhost ([127.0.0.1]:49444 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pGEx1-0000bJ-52
for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 02:58:11 -0500
Received: from eggs.gnu.org ([209.51.188.92]:56654)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1pGEwz-0000b7-95
for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 02:58:09 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1pGEwl-00043c-Ue; Fri, 13 Jan 2023 02:58:03 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
Date; bh=7SqaeqM7TefYeYdaQoiAgZ8yRV2f2LhqjeZu/WeQesA=; b=TAYP8vHh4M1fFMkMZ+AJ
/W4Yij1VZlOeA29vsX13ksDTIHjcpvpNMxbhg1b0vv57CRo+gx/j8WPcnY44pKYTjifyNAcfql71r
OYHFJnKlmKNWbbp44gR6K/oJX4vNaogJ6gEBKek8YmCykOi279FIX1m1xgSnKMzz2w0fxaGOUc3bR
XhbQRDK/XWQ0jg15D6NcPhO3sEMhliQoPFQ4nT6YxwnIQFRV0mUnOMXfU7sbn884VVdOe8dugBeZS
W93u8ktK989JbDseVn7YN1YACG18soPus+wjp9l04halMOq932XHMVXMyleJfnf/GMSCh5s8CVh+Q
0lRSjgDqxXEE0g==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1pGEwk-0008Bi-4R; Fri, 13 Jan 2023 02:57:55 -0500
Date: Fri, 13 Jan 2023 09:57:52 +0200
Message-Id: <83r0vyamtb.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: casouri@HIDDEN, Dmitry Gutov <dgutov@HIDDEN>
In-Reply-To: <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> (message from
Dmitry Gutov on Fri, 13 Jan 2023 01:40:56 +0200)
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
References: <867cxv3dnn.fsf@HIDDEN>
<6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN>
<0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 60691
Cc: 60691 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>,
juri@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN
> Date: Fri, 13 Jan 2023 01:40:56 +0200
> From: Dmitry Gutov <dgutov@HIDDEN>
>
> Managed to reproduce this after running the test in a couple of
> different files.
>
> But 'M-x memory-usage' says no such command, and 'M-x memory-report'
> ends up with this error:
>
> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
> memory-report--gc-elem(nil strings)
> memory-report--garbage-collect()
> memory-report()
This means GC is disabled in this session at the time you invoke
memory-report. Which shouldn't happen, of course. It sounds like
your pure Lisp storage overflowed, and that disabled GC.
And I think I see the problem: we use build_pure_c_string in treesit.c
in places that we shouldn't.
Yuan, build_pure_c_string should only be used in places such as
syms_of_treesit, which are called just once, during dumping. Look at
all the other calls to this function in the sources, and you will see
it. In all other cases, you should do one of the following:
. for strings whose text is fixed, define a variable, give it the
value in syms_of_treesit using build_pure_c_string, then use that
variable elsewhere in the source
. for strings whose text depends on run-time information, use
AUTO_STRING or build_string
This is a serious problem, and we should fix it ASAP.
> garbage-collect's docstring says:
>
> However, if there was overflow in pure space, and Emacs was dumped
> using the "unexec" method, ‘garbage-collect’ returns nil, because
> real GC can’t be done.
>
> I don't know if my Emacs was dumped using "unexec", though. ./configure
> says I'm using pdumper.
The above text doesn't account for bugs ;-) Functions that produce
objects in pure space are supposed to be called only during the build,
a.k.a. "when dumping", and for that the text is correct.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 12 Jan 2023 23:41:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 12 18:41:07 2023 Received: from localhost ([127.0.0.1]:48861 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pG7Bz-0006rz-8u for submit <at> debbugs.gnu.org; Thu, 12 Jan 2023 18:41:07 -0500 Received: from mail-ej1-f47.google.com ([209.85.218.47]:43983) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pG7Bx-0006rO-7J for 60691 <at> debbugs.gnu.org; Thu, 12 Jan 2023 18:41:06 -0500 Received: by mail-ej1-f47.google.com with SMTP id hw16so36701651ejc.10 for <60691 <at> debbugs.gnu.org>; Thu, 12 Jan 2023 15:41:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=G9PWF+FHoTG5EJvb7lvyvAbPqIQaMs0RZYu1eiR7/s0=; b=aiQ7w+/uAGckaH2duHMDgKSvzwZ5C9XZ55W331sgtGM1UThoYa6nO+R2q0+KjQhjO3 Eq3a4zS3JFVf263Y+adKKDr6DCw8oBzlr2Qr+LIqqdLHhscy4HcCHNcrdfNVdhoG/OdK YgAZAJMmyDKldeKwQR4EjPrlMKHO7Ly4/YyvW0x26QoVRgtaDRslIaEgKxd7VwulmYq0 L5XoSBqC+IYeAT8ENhxe60k6kPXFr7Vq2DzNAPM1s/CA1Z0cozmdAVKEyu43+Qe5Jq8K 1kvHtN6WGGSA19yr14utPDPjCZqx9hYGSKltxQ6D4fWD7Z304NP4lncwdndteMXmoP9R 9G7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G9PWF+FHoTG5EJvb7lvyvAbPqIQaMs0RZYu1eiR7/s0=; b=v6MRcz4GjazDZKa8sl3ihcD+GG/iBhmvSU5uK2HGBUCrsvvWZ6qPfr6ECSnwhGw6qr fF3+VXIDXLhXYVWAyVyhu+YvTcn2CSzxTFpoC1I6cqjllJCkGEnvM7M6gbtyoyz9bGGH ndQSRIg3HpyM8OrKnKe3QvxQOi2Z50l84N0m50kxxBJ3jkjowv3DcwAPZ7myMUZ8nrY6 rRUVgGHEtjZMMHQ3Zd6uR3EINsCzyw+duDFf984QrcEwFdySye6enqbfx4X90Byxm09U h8Bf5XvRGSBInXlXdYPnOHK91dsuov0i1CBQDco1nRSsqReylxNJciVR2D1hBM/ok0G8 PU7Q== X-Gm-Message-State: AFqh2kpiwdNAdhhXlidj6kaBEjUvCoDKo2FIE5WfohFrpF7MWWYSXE6o l1MzT6uQWQfAMBZnAmUlhBc= X-Google-Smtp-Source: AMrXdXsYW9Hxb41UNyatlRxxpijViRNQ6fbwiVaFzCthONJEMsK+HVANLJhLuB8SC1IWjVd6H1SKgQ== X-Received: by 2002:a17:907:a70b:b0:7c1:98d:a8a3 with SMTP id vw11-20020a170907a70b00b007c1098da8a3mr60271563ejc.7.1673566859131; Thu, 12 Jan 2023 15:40:59 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id d9-20020a1709063ec900b0084c2065b388sm7853479ejj.128.2023.01.12.15.40.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Jan 2023 15:40:58 -0800 (PST) Message-ID: <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> Date: Fri, 13 Jan 2023 01:40:56 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Yuan Fu <casouri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 12/01/2023 23:58, Yuan Fu wrote: > > Dmitry Gutov <dgutov@HIDDEN> writes: > >> Yuan? Just making sure you got this message. > > Sorry for the delay :-) > >> On 10/01/2023 16:10, Dmitry Gutov wrote: >>> Perhaps Yuan has some further ideas. There are some strong oddities here: >>> - Some time into debugging and repeating the benchmark again and >>> again, I get the "Pure Lisp storage overflowed" message. Just once >>> per Emacs session. It doesn't seem to change much, so it might be >>> unimportant. > > That sounds like 60653. The next time you encounter it, could you record > the output of M-x memory-usage and M-x memory-report? Managed to reproduce this after running the test in a couple of different files. But 'M-x memory-usage' says no such command, and 'M-x memory-report' ends up with this error: Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil) memory-report--gc-elem(nil strings) memory-report--garbage-collect() memory-report() funcall-interactively(memory-report) #<subr call-interactively>(memory-report record nil) apply(#<subr call-interactively> memory-report (record nil)) call-interactively@ido-cr+-record-current-command(#<subr call-interactively> memory-report record nil) apply(call-interactively@ido-cr+-record-current-command #<subr call-interactively> (memory-report record nil)) call-interactively(memory-report record nil) command-execute(memory-report record) execute-extended-command(nil "memory-report" nil) funcall-interactively(execute-extended-command nil "memory-report" nil) #<subr call-interactively>(execute-extended-command nil nil) apply(#<subr call-interactively> execute-extended-command (nil nil)) call-interactively@ido-cr+-record-current-command(#<subr call-interactively> execute-extended-command nil nil) apply(call-interactively@ido-cr+-record-current-command #<subr call-interactively> (execute-extended-command nil nil)) call-interactively(execute-extended-command nil nil) command-execute(execute-extended-command) garbage-collect's docstring says: However, if there was overflow in pure space, and Emacs was dumped using the "unexec" method, ‘garbage-collect’ returns nil, because real GC can’t be done. I don't know if my Emacs was dumped using "unexec", though. ./configure says I'm using pdumper. In case that matters, I'm testing the emacs-29 branch. >>> - The profiler output looks like this: >>> 18050 75% - >>> font-lock-fontify-syntactically-region >>> 15686 65% - treesit-font-lock-fontify-region >>> 3738 15% treesit--children-covering-range-recurse >>> 188 0% treesit-fontify-with-override >>> - When running the benchmark for the first time in a buffer (such as >>> ruby.rb), the variable treesit--font-lock-fast-mode is usually >>> changed to t. In one Emacs session, after I changed it to nil and >>> re-ran the benchmark, the variable stayed nil, and the benchmark ran >>> much faster (like 10s vs 36s). >>> In the next session, after I restarted Emacs, that didn't happen: it >>> always stayed at t, even if I reset it to nil between runs. But if I >>> comment out the block in treesit-font-lock-fontify-region that uses >>> it >>> ;; (when treesit--font-lock-fast-mode >>> ;; (setq nodes (treesit--children-covering-range-recurse >>> ;; (car nodes) start end (* 4 jit-lock-chunk-size)))) >>> and evaluate the defun, the benchmark runs much faster again: 11s. >>> (But then I brought it all back, and re-ran the tests, and the >>> variable stayed nil that time around; to sum up: the way it's turned >>> on is unstable.) >>> Should treesit--font-lock-fast-mode be locally bound inside that >>> function, so that it's reset between chunks? Or maybe the condition >>> for its enabling should be tweaked? E.g. I don't think there are any >>> particularly large or deep nodes in ruby.rb's parse tree. It's a >>> very shallow file. > > Yeah that is a not-very-clever hack. I’ve got an idea: I can add a C > function that checks the maximum depth of a parse tree and the maximum > node span, and turn on the fast-mode if the depth is too large or a node > is too wide. And we do that check once before doing any fontification. > > I’ll report back once I add it. Thanks! And if the check can be fast enough, we could probably do it in the beginning of fontifying every chunk.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 12 Jan 2023 21:58:58 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 12 16:58:58 2023 Received: from localhost ([127.0.0.1]:48724 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pG5b8-0003mj-1I for submit <at> debbugs.gnu.org; Thu, 12 Jan 2023 16:58:58 -0500 Received: from mail-pj1-f43.google.com ([209.85.216.43]:45003) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <casouri@HIDDEN>) id 1pG5b5-0003mU-Kw for 60691 <at> debbugs.gnu.org; Thu, 12 Jan 2023 16:58:56 -0500 Received: by mail-pj1-f43.google.com with SMTP id o7-20020a17090a0a0700b00226c9b82c3aso22453651pjo.3 for <60691 <at> debbugs.gnu.org>; Thu, 12 Jan 2023 13:58:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=/9woX2rdgAbZJ8bomT/4I+Ne9ZTerZyswRjNWI7oGkE=; b=TGu/eHxMPdXqGQDlTUsZu7WQJLAba4FGFePSyKV6LC76RBcKAT5zdbAxXbHTCRUH46 CF6lDPMcINyjQ1upETykht8ptQ0Mtb0QZmasa1C5z7g10lZu1+ztA3bgqus7+ZkKuo4m snnsxr/guH8E7nH9EXCx5Z7uJ/D3gIBlqEPINjyiTzMS3mhxcOIzvrvgenos0uOCPCdC qW4+JzeDnwwY5+EWiShCKQ4+xeDLnmgDw7khpvgnHTxjsC4w0eDeZXuN1nYd0ZFF7e/t J21tBp67jZFNSfWpSVGMnVu2oTP3/Rqy2f3KpCnlu1wW5YBQFCo4QeKXssNBQ+T3I/dr GGWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/9woX2rdgAbZJ8bomT/4I+Ne9ZTerZyswRjNWI7oGkE=; b=FvcxUX2tl0f9wgaX4c9EPRFFC2YJzpS2WOUHLGedqHyeOUsRQ9EJZePO2wtEYpevGU 45uSKetyLIyUjxykEuRNjAZ6UAIrAN7SsW5mf4vPtZ3aYnRcWDTmF/vxEzaeFgu0ORnV m0+4I299oooEJlSZj03dOgDpxi5E+J6jb3Cabnjv79OjGKrWuz55kwhEU9LRPOTWyLkq JzhspLkDp6FvbGnystfZ/AjTYMW3gvrOOMOr6NgTTPPIsI8VBLtOPHIUrGYM9QMZ0qzn FMpkiVsbGph4kUS5dt+ZxxBb2FLZz+4dOImXBMLDv5kKaCiuaYR3gxIFnSQU53TfYSlk Pz4w== X-Gm-Message-State: AFqh2kpkWeqPt1Yl4HPsRkprfZVJH6h86gVKSg6vYwTcDpiUdbroaX3R 615TccQ+PFHOr0PRauDUogw= X-Google-Smtp-Source: AMrXdXuz6obcOM3zRLdOo2rr7B+Y3MayZlhGofsxOm3zsumwAPCbGueVNgdmvrnpF8mf14Xqo+Q/gA== X-Received: by 2002:a17:903:2093:b0:194:4285:dfef with SMTP id d19-20020a170903209300b001944285dfefmr9954462plc.49.1673560729810; Thu, 12 Jan 2023 13:58:49 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id a10-20020a170902ecca00b00189a7fbfd44sm12612249plh.211.2023.01.12.13.58.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jan 2023 13:58:49 -0800 (PST) From: Yuan Fu <casouri@HIDDEN> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Message-Id: <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> Date: Thu, 12 Jan 2023 13:58:48 -0800 To: Dmitry Gutov <dgutov@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Dmitry Gutov <dgutov@HIDDEN> writes: > Yuan? Just making sure you got this message. Sorry for the delay :-) > On 10/01/2023 16:10, Dmitry Gutov wrote: >> Perhaps Yuan has some further ideas. There are some strong oddities = here: >> - Some time into debugging and repeating the benchmark again and >> again, I get the "Pure Lisp storage overflowed" message. Just once >> per Emacs session. It doesn't seem to change much, so it might be >> unimportant. That sounds like 60653. The next time you encounter it, could you record the output of M-x memory-usage and M-x memory-report?=20 >> - The profiler output looks like this: >> 18050 75% - >> font-lock-fontify-syntactically-region >> 15686 65% - treesit-font-lock-fontify-region >> 3738 15% treesit--children-covering-range-recurse >> 188 0% treesit-fontify-with-override >> - When running the benchmark for the first time in a buffer (such as >> ruby.rb), the variable treesit--font-lock-fast-mode is usually >> changed to t. In one Emacs session, after I changed it to nil and >> re-ran the benchmark, the variable stayed nil, and the benchmark ran >> much faster (like 10s vs 36s). >> In the next session, after I restarted Emacs, that didn't happen: it >> always stayed at t, even if I reset it to nil between runs. But if I >> comment out the block in treesit-font-lock-fontify-region that uses >> it >> ;; (when treesit--font-lock-fast-mode >> ;; (setq nodes (treesit--children-covering-range-recurse >> ;; (car nodes) start end (* 4 = jit-lock-chunk-size)))) >> and evaluate the defun, the benchmark runs much faster again: 11s. >> (But then I brought it all back, and re-ran the tests, and the >> variable stayed nil that time around; to sum up: the way it's turned >> on is unstable.) >> Should treesit--font-lock-fast-mode be locally bound inside that >> function, so that it's reset between chunks? Or maybe the condition >> for its enabling should be tweaked? E.g. I don't think there are any >> particularly large or deep nodes in ruby.rb's parse tree. It's a >> very shallow file. Yeah that is a not-very-clever hack. I=E2=80=99ve got an idea: I can add = a C function that checks the maximum depth of a parse tree and the maximum node span, and turn on the fast-mode if the depth is too large or a node is too wide. And we do that check once before doing any fontification. I=E2=80=99ll report back once I add it. Yuan
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 11 Jan 2023 12:12:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 07:12:37 2023 Received: from localhost ([127.0.0.1]:41853 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFZy9-0002hq-8X for submit <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:37 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:56299) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pFZy7-0002gv-W1 for 60691 <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:36 -0500 Received: by mail-wm1-f47.google.com with SMTP id l26so10940815wme.5 for <60691 <at> debbugs.gnu.org>; Wed, 11 Jan 2023 04:12:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=vC6n3tsfqPm2aUlkwOCfZYpy83jGbKoAY0Ti3HvVIVw=; b=BLpKty+bgWmjtBIcDYATth9XVabHbrnCaZOaXujQG7sDLX8QWjBMmj3dwQrTqL4zCH 432aahievevUnNb4zuxrrIYnDiOT5eWwcwh5d7SZzLbzOroB7DPKhIfZ5b5LVWClhQAJ p5i/qK2SqLa/mmUCYM5iPdY28e6JxYQJno/GSleq/gNdWQJAWaufTZ9QE1HXhkH3F7i2 D1/S0QJrh/ej2mhm7iLQEhE2CK2+LzZlr6VW2/NduwhqeNOOeis3z6/9rpky/ADyr+Uq DzFRC/qSxBXPc6OzFzKjW1XaAOuMxhZXoHrbX7aPGJyMsiycUG17YAboYFUmJs44EIux tK2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vC6n3tsfqPm2aUlkwOCfZYpy83jGbKoAY0Ti3HvVIVw=; b=QWVqvzAMNmyZDRPePg1omhREAa15lUzQ315TZia7c5XCBBLcGf0vIywYi3n3dRVqOP Poy1Ad/tzhyI0txsgCQ92SOXJ0Rqp37anf3DTbEbs6Ou1BruyLg3SWbKnuZVr5bclyTc HrQMjVU+IfnsRh3JJsJgjM2SZMl5eD6DaVBgJjmVzuzn5E6drHHaflvrijLW/2jpnOqc 4Lij9pXYQ5O/NLf/5OuR8emKJxVSkhN9/rS5Bk0SmAfvDjbh2jfUS3e2Y69DBo89ZytS 7JIHWrxmeWP3/PF8rqKo4jigTGrTuomD1XQqk4DWbDFjfiDbiaAfsW+ZlJhrrEo091px cd+Q== X-Gm-Message-State: AFqh2kqCx3nI1vLUDCnJr7mgHiGP6b6zpe2QdYL1AlpVaYOXJYiAQ0uQ qu/b410y4MU5dTa/+ozWKpAMA5VTeCo= X-Google-Smtp-Source: AMrXdXsb/M8YjD+tYmzVQ7OvS3SwAPae2g8YhFgFJHUBEfHBYX7hXdHwo7QdG8kScjSbkU53MDvoUw== X-Received: by 2002:a05:600c:4e51:b0:3d1:e1f4:21d1 with SMTP id e17-20020a05600c4e5100b003d1e1f421d1mr63557064wmq.26.1673439155546; Wed, 11 Jan 2023 04:12:35 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id g12-20020a05600c310c00b003c70191f267sm24680009wmo.39.2023.01.11.04.12.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Jan 2023 04:12:35 -0800 (PST) Message-ID: <122d12c9-9b7f-d8ff-9679-f2af0e8e2a93@HIDDEN> Date: Wed, 11 Jan 2023 14:12:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US From: Dmitry Gutov <dgutov@HIDDEN> To: Juri Linkov <juri@HIDDEN>, Yuan Fu <casouri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> In-Reply-To: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) Yuan? Just making sure you got this message. On 10/01/2023 16:10, Dmitry Gutov wrote: > Perhaps Yuan has some further ideas. There are some strong oddities here: > > - Some time into debugging and repeating the benchmark again and again, > I get the "Pure Lisp storage overflowed" message. Just once per Emacs > session. It doesn't seem to change much, so it might be unimportant. > > - The profiler output looks like this: > > 18050 75% - font-lock-fontify-syntactically-region > 15686 65% - treesit-font-lock-fontify-region > 3738 15% treesit--children-covering-range-recurse > 188 0% treesit-fontify-with-override > > - When running the benchmark for the first time in a buffer (such as > ruby.rb), the variable treesit--font-lock-fast-mode is usually changed > to t. In one Emacs session, after I changed it to nil and re-ran the > benchmark, the variable stayed nil, and the benchmark ran much faster > (like 10s vs 36s). > > In the next session, after I restarted Emacs, that didn't happen: it > always stayed at t, even if I reset it to nil between runs. But if I > comment out the block in treesit-font-lock-fontify-region that uses it > > ;; (when treesit--font-lock-fast-mode > ;; (setq nodes (treesit--children-covering-range-recurse > ;; (car nodes) start end (* 4 jit-lock-chunk-size)))) > > and evaluate the defun, the benchmark runs much faster again: 11s. > > (But then I brought it all back, and re-ran the tests, and the variable > stayed nil that time around; to sum up: the way it's turned on is > unstable.) > > Should treesit--font-lock-fast-mode be locally bound inside that > function, so that it's reset between chunks? Or maybe the condition for > its enabling should be tweaked? E.g. I don't think there are any > particularly large or deep nodes in ruby.rb's parse tree. It's a very > shallow file.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 11 Jan 2023 12:12:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 07:12:20 2023 Received: from localhost ([127.0.0.1]:41850 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFZxr-0002hD-UR for submit <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:20 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:56299) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pFZxo-0002gv-6u for 60691 <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:18 -0500 Received: by mail-wm1-f47.google.com with SMTP id l26so10940024wme.5 for <60691 <at> debbugs.gnu.org>; Wed, 11 Jan 2023 04:12:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=r/dqXjaqPACUM8T6HzTmldIJgBC3gYahWG0LRqnobaE=; b=E1NuJ7q9CT91Z5wPrmAGiQYwFVMrXE3FCF9rH/ty3V8nBu3aaGPbd4slClsF2WLMxU JAnDJ/6YR/+W3DJ+1CMdWhq3+wl6yUd2SkDUGrT/oEisCM+JjGqr6Vje73vFpCO/nirx IvcqkRnSCjGPD4Tl5+Y/CADiBiscwHuM/mOtcnyixNjw7xxaOw+ax07xYH6h8S4075ey dGubsj4IsOSGGWQ1KKoKJkAHMw4s8WqY1dr4VN4PQ9858nkhpmjz7uCn+Fs7ld1r6YvG QrbBcUOQXNxVkbe+LVayPElKm54E/KN/x/kX6aGEYWnpS9JQpp+e1rZlFXFhmg7y1hwH 3P9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=r/dqXjaqPACUM8T6HzTmldIJgBC3gYahWG0LRqnobaE=; b=jVzrMFEbpLDp03pnJDZmfiA95j7zr+Clzc5ic15W4fZSFM0cMtDrKsb27Iu4OLG7Sl 9PJMcAbRap5vswhPjztES1BXta43ETlZT2kGOLhXSkUIqOBFfN9hX5a6je+Cd5x6JHUD frIaqMtLcow5fEvRLo/OyInoixpm7VW1/JcEci3U93lvb8WDC97DAFY6XOfYqLziBrNC Aujaz11KNCh4YR34EcL1fM/kzOG8molU61kD8t5FCB44FsGCLA0ULIB6NqqSRNwwuABd +p6BXeTQA0bCAGnelpXQ/bXXNJV2CFe17ivANYsPCahWO7JLz44Zgawgs6bNO+LSdda1 rKug== X-Gm-Message-State: AFqh2kq69iy+F0qzBM2mWHYKRhqFPtIKoW2d+N8mtmqgdl0HSGxLy8Vd qu0JrHEYgn+r1e3UT1pxEs4= X-Google-Smtp-Source: AMrXdXuO5Z0p/z3o7HMYwqaVdRD/nYqwEHixlgivPZSyu6BqB1biVzoSO7OxSjc/p2z6+KqzAtUyKg== X-Received: by 2002:a05:600c:4e48:b0:3cf:69f4:bfd4 with SMTP id e8-20020a05600c4e4800b003cf69f4bfd4mr53607359wmq.7.1673439129168; Wed, 11 Jan 2023 04:12:09 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id l36-20020a05600c1d2400b003d9fb59c16fsm5362760wms.11.2023.01.11.04.12.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Jan 2023 04:12:08 -0800 (PST) Message-ID: <dd585de9-5fc7-cd0e-5ef4-5cfd812f3df9@HIDDEN> Date: Wed, 11 Jan 2023 14:12:06 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Juri Linkov <juri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> <86tu0yqnwr.fsf@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <86tu0yqnwr.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: Yuan Fu <casouri@HIDDEN>, 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/01/2023 19:50, Juri Linkov wrote: >>>> Is it common to try to highlight 1000 or even 100 files in one diff? >>> 100 is rare, but tens is pretty common, so this problem affects >>> only this specific case. >> >> So it's a 0,8-3s delay in those cases? That's not ideal. > > The delay is noticeable, alas. Right. I'm somewhat worried for the processing speed xref--collect-matches too. But that's probably only going to be noticeable after we add syntax-propertize-function to ruby-ts-mode. >>> I noticed that while most library files are small, e.g. >>> libtree-sitter-c.so is 401,528 bytes, >>> libtree-sitter-ruby.so is 2,130,616 bytes >>> that means that it has more complex logic >>> that might explain its performance. >> >> ruby is indeed one of the larger ones. Among the ones I have here compiled, >> it's exceeded only by cpp. 2.29 MB vs 2.12 MB. > > The winner is libtree-sitter-julia.so with 7.25 MB. > But regarding libtree-sitter-cpp.so I confirm it's 2.3 MB. > And c++-ts-mode is even faster than c-ts-mode. Yep. > On the same admin/alloc-colors.c: > > c-mode > (33.378821569 1500 17.632000617) > > c-ts-mode > (2.1949608069999997 34 0.4119784769999981) > > c++-ts-mode > (2.0979403910000003 34 0.39749122499999956) > > So size doesn't matter.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 10 Jan 2023 17:57:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 10 12:57:53 2023 Received: from localhost ([127.0.0.1]:41068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFIsj-00073F-KI for submit <at> debbugs.gnu.org; Tue, 10 Jan 2023 12:57:53 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:42941) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <juri@HIDDEN>) id 1pFIsi-00072o-79 for 60691 <at> debbugs.gnu.org; Tue, 10 Jan 2023 12:57:52 -0500 Received: (Authenticated sender: juri@HIDDEN) by mail.gandi.net (Postfix) with ESMTPSA id E634B60008; Tue, 10 Jan 2023 17:57:44 +0000 (UTC) From: Juri Linkov <juri@HIDDEN> To: Dmitry Gutov <dgutov@HIDDEN> Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode In-Reply-To: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> (Dmitry Gutov's message of "Tue, 10 Jan 2023 16:10:49 +0200") Organization: LINKOV.NET References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> Date: Tue, 10 Jan 2023 19:50:44 +0200 Message-ID: <86tu0yqnwr.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 60691 Cc: Yuan Fu <casouri@HIDDEN>, 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) >>> Is it common to try to highlight 1000 or even 100 files in one diff? >> 100 is rare, but tens is pretty common, so this problem affects >> only this specific case. > > So it's a 0,8-3s delay in those cases? That's not ideal. The delay is noticeable, alas. >> I noticed that while most library files are small, e.g. >> libtree-sitter-c.so is 401,528 bytes, >> libtree-sitter-ruby.so is 2,130,616 bytes >> that means that it has more complex logic >> that might explain its performance. > > ruby is indeed one of the larger ones. Among the ones I have here compiled, > it's exceeded only by cpp. 2.29 MB vs 2.12 MB. The winner is libtree-sitter-julia.so with 7.25 MB. But regarding libtree-sitter-cpp.so I confirm it's 2.3 MB. And c++-ts-mode is even faster than c-ts-mode. On the same admin/alloc-colors.c: c-mode (33.378821569 1500 17.632000617) c-ts-mode (2.1949608069999997 34 0.4119784769999981) c++-ts-mode (2.0979403910000003 34 0.39749122499999956) So size doesn't matter.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at 60691) by debbugs.gnu.org; 10 Jan 2023 14:11:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 10 09:11:00 2023
Received: from localhost ([127.0.0.1]:39163 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pFFLA-0006c8-8e
for submit <at> debbugs.gnu.org; Tue, 10 Jan 2023 09:11:00 -0500
Received: from mail-wm1-f46.google.com ([209.85.128.46]:52084)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <raaahh@HIDDEN>) id 1pFFL8-0006bs-47
for 60691 <at> debbugs.gnu.org; Tue, 10 Jan 2023 09:10:59 -0500
Received: by mail-wm1-f46.google.com with SMTP id g10so8886810wmo.1
for <60691 <at> debbugs.gnu.org>; Tue, 10 Jan 2023 06:10:58 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:cc:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:from:to:cc:subject:date:message-id:reply-to;
bh=TjVO7Iu2PTtRuEs848MOngu4IUBVHaOl2a8o366V1ms=;
b=mrNBiJoScwiSRm/Lhc5+jcHxvT/Vdwmt6EVFVRtCfCxCxKCX4Iybb06RjBS+Ae87jl
mncde8aZGREUgKaagbs8TnpZ1HX9gBR5KWrWZ7Y2aBBMo6vnCpM44Sj//0ttVlwzMMCd
YF760PI+PIlkdhbKUGKao3uQuURe6rANQOnLDEc5+Yb8YkO+s9s/V7db/HfsFoz27Of/
QJzcy39gwJlgs9jeV4DiQ3bPo8vklzazgObzLybOhZ49zROZ/3Zs0nZs1jUoUbB6gOtr
ZjedvEu9g1VV4VwOzCXQcO8ybTFWXVhNSnH0G72xWgu4IZC35bzhe+8uuKuDnytTozQR
1Bdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:cc:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=TjVO7Iu2PTtRuEs848MOngu4IUBVHaOl2a8o366V1ms=;
b=2EtT6sutPxHqezg8hl7tHubZqrzA3mksm5tr/QxUU62nb4Aw9UTxWXmjNWoPA3DQ2y
wAeOAPDPqGWxH0oB1EfwlYOaObJ2cWkdJpTRGzWkSComeor8zCNmB+obVzRTL8JCUrQa
2u5aYN+IhNI6h0T0RICZDl46AF9D91dHqfpNFFiEKj39zw57wiPWsbZI38198a8ADf6A
rl6ZzmPfFxV2av+GzzMa0J209T2HARGMLsgGpNKM/3FF/l8prCkygBMKFVcDg9mPK64c
LHgjpSeikTDVkxK1ttK6OZlUnw7IAAZvureCgBh6Ig3Gtd8ZZJa+/1BRplbqenpcIXH/
kb2Q==
X-Gm-Message-State: AFqh2krljc7xaIfA0DSMbsL/HveqmiyqAUQOQDyGnixwDPtlcAh1tuYD
MrCFqy049GMnAeltKO6w/3k=
X-Google-Smtp-Source: AMrXdXvMhePPVrJP0Vk6j6S4D0bW5vHAxMAn40mHnz+tstN2UpV1JNYObp/GlBJMFWQkXFRIjfy2Ww==
X-Received: by 2002:a05:600c:4da2:b0:3d2:39dc:f50e with SMTP id
v34-20020a05600c4da200b003d239dcf50emr49123608wmp.7.1673359852028;
Tue, 10 Jan 2023 06:10:52 -0800 (PST)
Received: from [192.168.0.2] ([46.251.119.176])
by smtp.googlemail.com with ESMTPSA id
i14-20020a05600c354e00b003d1d5a83b2esm21678073wmq.35.2023.01.10.06.10.50
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Tue, 10 Jan 2023 06:10:51 -0800 (PST)
Message-ID: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN>
Date: Tue, 10 Jan 2023 16:10:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.4.2
Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
Content-Language: en-US
To: Juri Linkov <juri@HIDDEN>, Yuan Fu <casouri@HIDDEN>
References: <867cxv3dnn.fsf@HIDDEN>
<51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN>
<86leman71u.fsf@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
In-Reply-To: <86leman71u.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 60691
Cc: 60691 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
On 10/01/2023 10:10, Juri Linkov wrote:
>>> After more rules were added recently to ruby-ts--font-lock-settings,
>>> font-lock became slow even on very small files. Some measurements:
>>
>> If you saw a particular commit that made things slower, did you try
>> reverting it? What was the performance after?
>
> No particular commit, just adding more rules degrades performance
> gradually.
But I don't think I added that many rules recently. No more than a
quarter anyway.
>>> M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure)))
>>> M-x ruby-mode
>>> (1.3564674989999999 0 0.0)
>>> M-x ruby-ts-mode
>>> (8.349582391999999 2 6.489918534000001)
>>
>> I have tried this scenario (which, to be frank, is pretty artificial, given
>> that fontification is usually performed in chunks, not over the whole
>> buffer).
>>
>> Perhaps the results depend on a particular file. The ones I have tried
>> (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or
>> less). The difference was in favor of ruby-mode, but given the difference
>> in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed
>> overhead somewhere.
>
> On test/lisp/progmodes/ruby-mode-resources/ruby.rb I see these numbers:
>
> ruby-mode
> (8.701560543000001 95 1.045961102)
>
> ruby-ts-mode
> (34.653148898000005 1464 16.904981779)
Interesting. It's 12s vs 36s for me, as I've retested now.
>>> This is not a problem when files are visited infrequently, but
>>> becomes a problem for diff-syntax fontification that wants to
>>> highlight simultaneously many files from git logs.
>>> So a temporary measure would be not to enable ruby-ts-mode
>>> in internal buffers:
>>
>> Is it common to try to highlight 1000 or even 100 files in one diff?
>
> 100 is rare, but tens is pretty common, so this problem affects
> only this specific case.
So it's a 0,8-3s delay in those cases? That's not ideal.
>>> (add-hook 'find-file-hook
>>> (lambda ()
>>> (when (and (eq major-mode 'ruby-mode)
>>> ;; Only when not internal as from diff-syntax
>>> (not (string-prefix-p " " (buffer-name))))
>>> (ruby-ts-mode))))
>>
>> Have you tried similar tests with other -ts- modes? Ones with complex
>> font-lock rules in particular.
>
> I tried with c-ts-mode, and it's very fast.
Just how fast is it? The number of font-lock features is has is
comparable (though a little smaller).
I've tried the same benchmark for it in admin/alloc-colors.c, and it
comes out to
(3.2004193190000003 30 0.9609690980000067)
Which seems comparable.
Not sure how to directly test the modes against each other, but if I
enable ruby-ts-mode in the same file, the benchmark comes to 1s.
Or if I enable c-ts-mode in ruby.rb -- 16s.
>> I've tried commenting out different rules in ruby-ts--font-lock-settings,
>> but none of them seem to have particularly outsides impact. Performance
>> seems, roughly, inversely proportional to the number of separate
>> "features".
>
> Indeed, this is what I see - no particular rule, only their number
> affects performance.
>
>> And if all ts modes turn out to have this problem, perhaps the place to
>> improve this is inside some common code.
>
> I noticed that while most library files are small, e.g.
> libtree-sitter-c.so is 401,528 bytes,
> libtree-sitter-ruby.so is 2,130,616 bytes
> that means that it has more complex logic
> that might explain its performance.
ruby is indeed one of the larger ones. Among the ones I have here
compiled, it's exceeded only by cpp. 2.29 MB vs 2.12 MB.
But testing admin/alloc-colors.c with c++-ts-mode vs c-ts-mode gives
very similar performance, so it's unlikely that the complexity of the
grammar is directly responsible.
> In this case, when nothing could be done to improve performance,
> please close this request.
Perhaps Yuan has some further ideas. There are some strong oddities here:
- Some time into debugging and repeating the benchmark again and again,
I get the "Pure Lisp storage overflowed" message. Just once per Emacs
session. It doesn't seem to change much, so it might be unimportant.
- The profiler output looks like this:
18050 75% - font-lock-fontify-syntactically-region
15686 65% - treesit-font-lock-fontify-region
3738 15%
treesit--children-covering-range-recurse
188 0% treesit-fontify-with-override
- When running the benchmark for the first time in a buffer (such as
ruby.rb), the variable treesit--font-lock-fast-mode is usually changed
to t. In one Emacs session, after I changed it to nil and re-ran the
benchmark, the variable stayed nil, and the benchmark ran much faster
(like 10s vs 36s).
In the next session, after I restarted Emacs, that didn't happen: it
always stayed at t, even if I reset it to nil between runs. But if I
comment out the block in treesit-font-lock-fontify-region that uses it
;; (when treesit--font-lock-fast-mode
;; (setq nodes (treesit--children-covering-range-recurse
;; (car nodes) start end (* 4 jit-lock-chunk-size))))
and evaluate the defun, the benchmark runs much faster again: 11s.
(But then I brought it all back, and re-ran the tests, and the variable
stayed nil that time around; to sum up: the way it's turned on is unstable.)
Should treesit--font-lock-fast-mode be locally bound inside that
function, so that it's reset between chunks? Or maybe the condition for
its enabling should be tweaked? E.g. I don't think there are any
particularly large or deep nodes in ruby.rb's parse tree. It's a very
shallow file.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 10 Jan 2023 08:25:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 10 03:25:43 2023 Received: from localhost ([127.0.0.1]:38704 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pF9x1-0004dY-6d for submit <at> debbugs.gnu.org; Tue, 10 Jan 2023 03:25:43 -0500 Received: from relay10.mail.gandi.net ([217.70.178.230]:54479) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <juri@HIDDEN>) id 1pF9wz-0004dI-3d for 60691 <at> debbugs.gnu.org; Tue, 10 Jan 2023 03:25:42 -0500 Received: (Authenticated sender: juri@HIDDEN) by mail.gandi.net (Postfix) with ESMTPSA id B02AD240004; Tue, 10 Jan 2023 08:25:31 +0000 (UTC) From: Juri Linkov <juri@HIDDEN> To: Dmitry Gutov <dgutov@HIDDEN> Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode In-Reply-To: <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> (Dmitry Gutov's message of "Tue, 10 Jan 2023 00:33:12 +0200") Organization: LINKOV.NET References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> Date: Tue, 10 Jan 2023 10:10:53 +0200 Message-ID: <86leman71u.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) >> After more rules were added recently to ruby-ts--font-lock-settings, >> font-lock became slow even on very small files. Some measurements: > > If you saw a particular commit that made things slower, did you try > reverting it? What was the performance after? No particular commit, just adding more rules degrades performance gradually. >> M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure))) >> M-x ruby-mode >> (1.3564674989999999 0 0.0) >> M-x ruby-ts-mode >> (8.349582391999999 2 6.489918534000001) > > I have tried this scenario (which, to be frank, is pretty artificial, given > that fontification is usually performed in chunks, not over the whole > buffer). > > Perhaps the results depend on a particular file. The ones I have tried > (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or > less). The difference was in favor of ruby-mode, but given the difference > in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed > overhead somewhere. On test/lisp/progmodes/ruby-mode-resources/ruby.rb I see these numbers: ruby-mode (8.701560543000001 95 1.045961102) ruby-ts-mode (34.653148898000005 1464 16.904981779) >> This is not a problem when files are visited infrequently, but >> becomes a problem for diff-syntax fontification that wants to >> highlight simultaneously many files from git logs. >> So a temporary measure would be not to enable ruby-ts-mode >> in internal buffers: > > Is it common to try to highlight 1000 or even 100 files in one diff? 100 is rare, but tens is pretty common, so this problem affects only this specific case. >> (add-hook 'find-file-hook >> (lambda () >> (when (and (eq major-mode 'ruby-mode) >> ;; Only when not internal as from diff-syntax >> (not (string-prefix-p " " (buffer-name)))) >> (ruby-ts-mode)))) > > Have you tried similar tests with other -ts- modes? Ones with complex > font-lock rules in particular. I tried with c-ts-mode, and it's very fast. > I've tried commenting out different rules in ruby-ts--font-lock-settings, > but none of them seem to have particularly outsides impact. Performance > seems, roughly, inversely proportional to the number of separate > "features". Indeed, this is what I see - no particular rule, only their number affects performance. > And if all ts modes turn out to have this problem, perhaps the place to > improve this is inside some common code. I noticed that while most library files are small, e.g. libtree-sitter-c.so is 401,528 bytes, libtree-sitter-ruby.so is 2,130,616 bytes that means that it has more complex logic that might explain its performance. In this case, when nothing could be done to improve performance, please close this request.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.Received: (at 60691) by debbugs.gnu.org; 9 Jan 2023 22:33:23 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 09 17:33:22 2023 Received: from localhost ([127.0.0.1]:38348 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pF0hm-0003tH-KA for submit <at> debbugs.gnu.org; Mon, 09 Jan 2023 17:33:22 -0500 Received: from mail-wm1-f51.google.com ([209.85.128.51]:43611) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pF0hk-0003sx-HX for 60691 <at> debbugs.gnu.org; Mon, 09 Jan 2023 17:33:21 -0500 Received: by mail-wm1-f51.google.com with SMTP id k22-20020a05600c1c9600b003d1ee3a6289so8425690wms.2 for <60691 <at> debbugs.gnu.org>; Mon, 09 Jan 2023 14:33:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=WYRubfQD14nka8tVqf3xnXSCiwbxr3yjYi5GLCe7sTw=; b=ij17FLxTXlpXUnY5J8+Sioh9IjSFeYAPR+atDsUeR0BWYw2F6qpjQWBBrQkUgi3t/3 Pfz3DRzIMdRe6PcEuBE6wbWrpKd+uk7/08O318TDKfhAEeXW4xtNLi6d5gvICpsTWeB3 8Hh/3WF+uJ7UULXKbfbRsIOwV8ijYkXWUFfe9RcUXJ2Q0S8FNYvwV2g1+SwKG3woi21S ohrOqHTrmegct/uqT7PiDHLoD9Hkwg9/ZHOR9jPT5yGlpYMa/S9NlhH0vYuUc2hn5xJS DklzrdabSToE2HZtG+8FiJaUBMbX1tIDsbrZ8wH44QbI8EXu9oVUSYQAaK1AZVjCBh5H WgXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WYRubfQD14nka8tVqf3xnXSCiwbxr3yjYi5GLCe7sTw=; b=Fq8eL8l3M92G7CjY+LI2jQLzMd992jmnmKE0FigHz0VKEN/lKjh3bTBmmT1rIO36cX 9LvFbymgFu8Na5pdPm/eWUfqr9xmXbNhh94yQqQNLNlrTTh5GBNvuskandvKcGd39ePy yB5vlkYQiqUL1aB92OUAIvOLBDe0D7mheHXE8aqEuvXkuwyDKjRWR64eCgKKbh7hhUVO 5gklnbYOSduEVRsf+KN37hmU6O0BrPSkg73f2+Igq1qCV7mUMH09kGCILm5RnGlu4cm/ SnVKzaf1R8hMtb5JoVzUecRWPYhalRwGSmGg1e1BkyL52BPIkQGsAk4VzPhzSH4WNRfR bBpw== X-Gm-Message-State: AFqh2koRyLKs5cRcQIy+OeYgWS6T/74fF+9YO+Q7VJRprxxBFSuPLgjb kgsrPIXpPJdQJE8SWM/R+I0= X-Google-Smtp-Source: AMrXdXsGYKRrlHNV0Y6gPFwFPASJC3AIAEVFKgNP5cK/yVbpSCpOnZjAn472t/hlRDEJrg1IoCrywQ== X-Received: by 2002:a7b:ca51:0:b0:3d2:7a7:5cc6 with SMTP id m17-20020a7bca51000000b003d207a75cc6mr51171463wml.18.1673303594410; Mon, 09 Jan 2023 14:33:14 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id o5-20020a05600c510500b003b4ff30e566sm45901wms.3.2023.01.09.14.33.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Jan 2023 14:33:13 -0800 (PST) Message-ID: <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> Date: Tue, 10 Jan 2023 00:33:12 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Juri Linkov <juri@HIDDEN>, 60691 <at> debbugs.gnu.org References: <867cxv3dnn.fsf@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <867cxv3dnn.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) Hi! On 09/01/2023 19:16, Juri Linkov wrote: > X-Debbugs-Cc: Dmitry Gutov <dgutov@HIDDEN> > > After more rules were added recently to ruby-ts--font-lock-settings, > font-lock became slow even on very small files. Some measurements: If you saw a particular commit that made things slower, did you try reverting it? What was the performance after? > M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure))) > > M-x ruby-mode > (1.3564674989999999 0 0.0) > > M-x ruby-ts-mode > (8.349582391999999 2 6.489918534000001) I have tried this scenario (which, to be frank, is pretty artificial, given that fontification is usually performed in chunks, not over the whole buffer). Perhaps the results depend on a particular file. The ones I have tried (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or less). The difference was in favor of ruby-mode, but given the difference in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed overhead somewhere. > This is not a problem when files are visited infrequently, but > becomes a problem for diff-syntax fontification that wants to > highlight simultaneously many files from git logs. > So a temporary measure would be not to enable ruby-ts-mode > in internal buffers: Is it common to try to highlight 1000 or even 100 files in one diff? > (add-hook 'find-file-hook > (lambda () > (when (and (eq major-mode 'ruby-mode) > ;; Only when not internal as from diff-syntax > (not (string-prefix-p " " (buffer-name)))) > (ruby-ts-mode)))) Have you tried similar tests with other -ts- modes? Ones with complex font-lock rules in particular. I've tried commenting out different rules in ruby-ts--font-lock-settings, but none of them seem to have particularly outsides impact. Performance seems, roughly, inversely proportional to the number of separate "features". And if all ts modes turn out to have this problem, perhaps the place to improve this is inside some common code.
bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 9 Jan 2023 17:35:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 09 12:35:20 2023
Received: from localhost ([127.0.0.1]:38091 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pEw3M-0004DE-CP
for submit <at> debbugs.gnu.org; Mon, 09 Jan 2023 12:35:20 -0500
Received: from lists.gnu.org ([209.51.188.17]:33332)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <juri@HIDDEN>) id 1pEw3K-0004D6-TW
for submit <at> debbugs.gnu.org; Mon, 09 Jan 2023 12:35:19 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <juri@HIDDEN>) id 1pEw3J-0007G0-8U
for bug-gnu-emacs@HIDDEN; Mon, 09 Jan 2023 12:35:17 -0500
Received: from relay6-d.mail.gandi.net ([217.70.183.198])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <juri@HIDDEN>) id 1pEw3H-0005Eh-OE
for bug-gnu-emacs@HIDDEN; Mon, 09 Jan 2023 12:35:17 -0500
Received: (Authenticated sender: juri@HIDDEN)
by mail.gandi.net (Postfix) with ESMTPSA id 3DADEC0009
for <bug-gnu-emacs@HIDDEN>; Mon, 9 Jan 2023 17:35:09 +0000 (UTC)
From: Juri Linkov <juri@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
Organization: LINKOV.NET
Date: Mon, 09 Jan 2023 19:16:12 +0200
Message-ID: <867cxv3dnn.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=217.70.183.198; envelope-from=juri@HIDDEN;
helo=relay6-d.mail.gandi.net
X-Spam_score_int: -25
X-Spam_score: -2.6
X-Spam_bar: --
X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)
X-Debbugs-Cc: Dmitry Gutov <dgutov@HIDDEN>
After more rules were added recently to ruby-ts--font-lock-settings,
font-lock became slow even on very small files. Some measurements:
M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure)))
M-x ruby-mode
(1.3564674989999999 0 0.0)
M-x ruby-ts-mode
(8.349582391999999 2 6.489918534000001)
This is not a problem when files are visited infrequently, but
becomes a problem for diff-syntax fontification that wants to
highlight simultaneously many files from git logs.
So a temporary measure would be not to enable ruby-ts-mode
in internal buffers:
(add-hook 'find-file-hook
(lambda ()
(when (and (eq major-mode 'ruby-mode)
;; Only when not internal as from diff-syntax
(not (string-prefix-p " " (buffer-name))))
(ruby-ts-mode))))
Juri Linkov <juri@HIDDEN>:dgutov@HIDDEN, bug-gnu-emacs@HIDDEN.
Full text available.dgutov@HIDDEN, bug-gnu-emacs@HIDDEN:bug#60691; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.