Received: (at 60691) by debbugs.gnu.org; 14 Jan 2023 07:51:15 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 14 02:51:14 2023 Received: from localhost ([127.0.0.1]:53181 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pGbJq-0000OC-Ee for submit <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:51:14 -0500 Received: from mail-pj1-f50.google.com ([209.85.216.50]:53092) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <casouri@HIDDEN>) id 1pGbJo-0000Nx-U6 for 60691 <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:51:13 -0500 Received: by mail-pj1-f50.google.com with SMTP id o13so21047464pjg.2 for <60691 <at> debbugs.gnu.org>; Fri, 13 Jan 2023 23:51:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=tlwGvZ6qBr3cwq3HB43bzz+uAXUoxC54NqVO7H8qBDQ=; b=gC6QEerdtly6ThSbVLgPj300mYvlo6rNAJNTIZRaZFla71m1OQYvcjcE2c72MZnCKT Dx4QOLjT5Gdd8nyxVCHXbQcMIoUd+WyFUEKbIMdMiNf0X7+w9AEVLoH8kxDOODlez35o tmFkFD3MaP5Im9Q+ImN6qVNv0s2rq/V9e1ojbHDV9zG4gPCF+i2XoPrDsP27kYJkojT7 wY5QulssJfCZaKZ39tXHPb2M9TX06E2A7Cqfa73RASTtcfPDi60cDfKRrkxkp+YLaBk/ daS050u17I9jv4HH6qIK50a8YwP4uWkk51rAscL9fNI5wrqWAMJmnzFZZllbu7q5osm5 oJTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tlwGvZ6qBr3cwq3HB43bzz+uAXUoxC54NqVO7H8qBDQ=; b=e3eq3VFAX/kJA6DW5xX+2VZWV2xq1oums5m3jwcz0yuzsfKa67Sa8cUQomGSoDl08/ VAEkzX2JyqLNuhFoCE/l1jvCmoO6Aup4gRISvx8O0flyeSZ6eQTKPDVjco32ZVOHPqdg nVuYrJiW8zbSIkZp4fjvwyOrc/sp7gOydFhIqNsJ9uv+w4PZRVvh5ecfKkzIr57iu9Km T3zTQIpP5KLhIbWmIl71/BI6aRQai9esZp2U984LP5wYUFlIv7Bq2LIVxLbHMfxyraqg u8RwKIhunHMi1OU8fAlT8UvOfD2uzjnrSvlyQ2WVlB0oqqKlw4unXd+G11ynQ4LI/+65 xPAw== X-Gm-Message-State: AFqh2kquch3cPEwJYz9QuUa5pb8c2Z05GGzmr44kRamZ9mBO74zIDMF0 ZZBIiszlqx5+5O3f5ChvQ1c= X-Google-Smtp-Source: AMrXdXsOaG8BsiuvSK7mo9yUGxzMSefoZmlyNv0xOcOUYTk+r7C+6fXbRfZ7QGv18ksB0NvXI6Fvxw== X-Received: by 2002:a17:903:2c5:b0:192:cf35:3ff8 with SMTP id s5-20020a17090302c500b00192cf353ff8mr51947643plk.21.1673682666821; Fri, 13 Jan 2023 23:51:06 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id f5-20020a170902ce8500b001870dc3b4c0sm15428772plg.74.2023.01.13.23.51.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Jan 2023 23:51:06 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode From: Yuan Fu <casouri@HIDDEN> In-Reply-To: <83mt6l8th7.fsf@HIDDEN> Date: Fri, 13 Jan 2023 23:51:05 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <09B9914D-2B6D-4F73-89E7-1B9A02A1C44E@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN> <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> <83mt6mac0c.fsf@HIDDEN> <3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN> <83mt6l8th7.fsf@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 60691 Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN, dgutov@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On Jan 13, 2023, at 11:29 PM, Eli Zaretskii <eliz@HIDDEN> wrote: >=20 >> From: Yuan Fu <casouri@HIDDEN> >> Date: Fri, 13 Jan 2023 19:48:40 -0800 >> Cc: dgutov@HIDDEN, >> 60691 <at> debbugs.gnu.org, >> juri@HIDDEN, >> monnier@HIDDEN >>=20 >> Thanks, I pushed a fix for it. I also used intern_c_string in some = places like these: >>=20 >> intern_c_string (":?=E2=80=9D) >> intern_c_string (":*") >>=20 >> I want to change them to use DEFSYM, but what should be the c name = for them? >=20 > Yes, DEFSYM is better in such cases. The C name can be QCquestion and > QCasterix, for example. My worry is that they will conflict with, eg, symbol `question=E2=80=99 = and `asterix=E2=80=99, if someone ever defines them in the C codebase. = Is that not possible? Yuan=
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 14 Jan 2023 07:29:16 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 14 02:29:15 2023 Received: from localhost ([127.0.0.1]:53121 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pGayZ-00088D-LB for submit <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:29:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1pGayX-000881-IC for 60691 <at> debbugs.gnu.org; Sat, 14 Jan 2023 02:29:13 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1pGayQ-0006yC-WA; Sat, 14 Jan 2023 02:29:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=1AlmdwoHsiVSqHOb9EH33puDmiL/W0ey+HQq6y/Jpyg=; b=j+2vNrHy0Fkk+1nIOnWg fIO+/66MG703M3UWCwwpdTNHg8gU0HPS5qWmQYMHak4f3zg8R1nhnAYR25f6Zs3ttvjunXpbVyJWN iqC8JyHmrnVxasay3lftfOpi0TYnWbqs/LdP0vAiAI/qZV0eRrT709sSEptU+rYE0j8gpDVCGMHdJ nSdgydJK8i2AQq3aEZUIpSz9cweapAdT5h+s60UpACvrilvpAU7jBWwLikfygW830QNqpMNEQP5mm SjDdr/Nc/yinR/v1roLiwGAFlHWInONZu/kY9WoPG2ANOh8HEuLVNUy78zPwpUF7KmywVljcSOObE nF2mqARb2J2cog==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1pGayQ-0004QO-7m; Sat, 14 Jan 2023 02:29:06 -0500 Date: Sat, 14 Jan 2023 09:29:08 +0200 Message-Id: <83mt6l8th7.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Yuan Fu <casouri@HIDDEN> In-Reply-To: <3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN> (message from Yuan Fu on Fri, 13 Jan 2023 19:48:40 -0800) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN> <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> <83mt6mac0c.fsf@HIDDEN> <3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 60691 Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN, dgutov@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Yuan Fu <casouri@HIDDEN> > Date: Fri, 13 Jan 2023 19:48:40 -0800 > Cc: dgutov@HIDDEN, > 60691 <at> debbugs.gnu.org, > juri@HIDDEN, > monnier@HIDDEN > > Thanks, I pushed a fix for it. I also used intern_c_string in some places like these: > > intern_c_string (":?”) > intern_c_string (":*") > > I want to change them to use DEFSYM, but what should be the c name for them? Yes, DEFSYM is better in such cases. The C name can be QCquestion and QCasterix, for example.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 14 Jan 2023 03:48:50 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 22:48:50 2023 Received: from localhost ([127.0.0.1]:52926 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pGXXG-000886-86 for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 22:48:50 -0500 Received: from mail-pj1-f54.google.com ([209.85.216.54]:37558) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <casouri@HIDDEN>) id 1pGXXE-00087t-1X for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 22:48:49 -0500 Received: by mail-pj1-f54.google.com with SMTP id o1-20020a17090a678100b00219cf69e5f0so28848528pjj.2 for <60691 <at> debbugs.gnu.org>; Fri, 13 Jan 2023 19:48:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=mJwoFOhxZip+RMp6eVcrydfjakDfE+zcJXQfnL9xL+g=; b=bllNYqR/56URhI50lwJTXe6eKVKS2ElYzc5rrfDx2vGMMTLe1n/fiiitn9X0FvvaLR 7K1Ybw+pmxe8AK0fUlz43OHNIl7zvHb6aBqmSBAWnLTxcldLkw8RTkfjgnoYG9odVpgW Mklkr+98W0lZbNpmSuatBVZotqPdcLqXWkqs2PnOJsYCVp5p5JykGa1tlHMftagCazZ9 QYeKAC5ze4pQsZCWzA0PEwEEwqK+NMeCG6RALFC7yboItpJICrP3XBi3/XovSRN8oaxu 9RJtIRzMCCSftQ2lgiajhpmL55SIhJwyAuO/xJ+sSqbYG9zmNcfBi5jMElESxcWVbvVK FHog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mJwoFOhxZip+RMp6eVcrydfjakDfE+zcJXQfnL9xL+g=; b=yJV44p43g5WwanplNpYTdEAKLSJHGeiiuEFxnfTqZlDIyrsC1vTUaK4IeR8HVsuiHN YVY4HVB4vsX438u8cvc4V0wWYrWRxumC457wMZsxXO+yQVTQ8Y4IcPN9AJQEkmfNWwQN tc1kDjBceafkD2WDIlaMP84egRRTMxFuRIxEtOcjKJU9REwX8HP7s4jI70wnDaxzCrNm TATQ+ykh3UrjKBHPmFYu/VFRrt0BZah0wj8vADwxvuJ8MOIgGbhiqPm40aOyPfrarIDK l2n1ninYLqCcrJe/g9OXSASVFgYbCfjOjGF4EztLzmknJDLYIYLnruf/PQlNcq18qz1c P79Q== X-Gm-Message-State: AFqh2konV5y19veQUnVCYRVFF32Ov0Awc4EQYJ29nLwtRN+k7kCBU/rb o+s5PghJJgsmEmIcFKjyK5g= X-Google-Smtp-Source: AMrXdXtpSk23kJnzTR+bw/wfRcdIt8T3jDpdSKU5es3l/FqspbfkVSE3CUGlUYNrbDd1t0kotkb7bA== X-Received: by 2002:a17:902:6a87:b0:194:6d39:5911 with SMTP id n7-20020a1709026a8700b001946d395911mr4941697plk.40.1673668122011; Fri, 13 Jan 2023 19:48:42 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id h18-20020a170902f55200b001929dff50a9sm14950031plf.87.2023.01.13.19.48.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Jan 2023 19:48:41 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode From: Yuan Fu <casouri@HIDDEN> In-Reply-To: <83mt6mac0c.fsf@HIDDEN> Date: Fri, 13 Jan 2023 19:48:40 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <3FBD8C40-2C2B-4B10-BDB3-8CDD93B495A9@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN> <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> <83mt6mac0c.fsf@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 60691 Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN, dgutov@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On Jan 13, 2023, at 3:51 AM, Eli Zaretskii <eliz@HIDDEN> wrote: >=20 >> From: Yuan Fu <casouri@HIDDEN> >> Date: Fri, 13 Jan 2023 01:15:09 -0800 >> Cc: Dmitry Gutov <dgutov@HIDDEN>, >> 60691 <at> debbugs.gnu.org, >> Juri Linkov <juri@HIDDEN>, >> Stefan Monnier <monnier@HIDDEN> >>=20 >>> On Jan 12, 2023, at 11:57 PM, Eli Zaretskii <eliz@HIDDEN> wrote: >>>=20 >>>> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN >>>> Date: Fri, 13 Jan 2023 01:40:56 +0200 >>>> From: Dmitry Gutov <dgutov@HIDDEN> >>>>=20 >>>> Managed to reproduce this after running the test in a couple of=20 >>>> different files. >>>>=20 >>>> But 'M-x memory-usage' says no such command, and 'M-x = memory-report'=20 >>>> ends up with this error: >>>>=20 >>>> Debugger entered--Lisp error: (wrong-type-argument = number-or-marker-p nil) >>>> memory-report--gc-elem(nil strings) >>>> memory-report--garbage-collect() >>>> memory-report() >>>=20 >>> This means GC is disabled in this session at the time you invoke >>> memory-report. Which shouldn't happen, of course. It sounds like >>> your pure Lisp storage overflowed, and that disabled GC. >>>=20 >>> And I think I see the problem: we use build_pure_c_string in = treesit.c >>> in places that we shouldn't. >>>=20 >>> Yuan, build_pure_c_string should only be used in places such as >>> syms_of_treesit, which are called just once, during dumping. Look = at >>> all the other calls to this function in the sources, and you will = see >>> it. In all other cases, you should do one of the following: >>>=20 >>> . for strings whose text is fixed, define a variable, give it the >>> value in syms_of_treesit using build_pure_c_string, then use that >>> variable elsewhere in the source >>=20 >> Can I define a bunch of static C variables and initialize them in = syms_of_treesit, or they have to be all Lisp variables? Eg, >>=20 >> static Lisp_Object TREESIT_STAR; >>=20 >> ... >>=20 >> void >> syms_of_treesit (void) >> { >> ... >> TREESIT_STAR =3D build_pure_c_string ("*"); >> ... >> } >=20 > Yes, of course. Look, for example, how coding.c does that: >=20 > /* A string that serves as name of the reusable work buffer, and as = base > name of temporary work buffers used for code-conversion = operations. */ > static Lisp_Object Vcode_conversion_workbuf_name; > [...] > void > syms_of_coding (void) > { > [...] > staticpro (&Vcode_conversion_workbuf_name); > Vcode_conversion_workbuf_name =3D build_pure_c_string (" = *code-conversion-work*"); >=20 > But please keep the convention of naming such variables Vsome_thing, > both regarding the "V" and the fact that the name is otherwise > lower-case. Thanks, I pushed a fix for it. I also used intern_c_string in some = places like these: intern_c_string (":?=E2=80=9D) intern_c_string (":*") I want to change them to use DEFSYM, but what should be the c name for = them? Yuan
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 13 Jan 2023 11:51:27 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 06:51:27 2023 Received: from localhost ([127.0.0.1]:49768 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pGIal-0000kU-4m for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 06:51:27 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44674) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1pGIaj-0000kI-93 for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 06:51:25 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1pGIab-0004gp-Of; Fri, 13 Jan 2023 06:51:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=v8L5KErd7/OHhGFb/Qc5qHRtsMnHLsT41EQx85iBh5k=; b=lt3mc3XnkpMi wKgin12cfjd39wdkwFMBf7/au6rEBHLG6Z2GNLyrBxIbnFe+TLAE6iMNuJ0VIXMEeAoljP6oX0cio KS8iIebJl1fSH9Fyd2ze698oCq85Fg2NzCQNEWKQxuJ6jv6ulKQKKcVy4WRks7Pr/Sa0om6kqKP61 y7Pw8RSU1qkUTwezMoSUkQ9KOG7pJSRIpFUTfRBXcac0caVHsI+VBhIorKFlAFT5KzEY7ROnLb6SA 0i8l9NKpyfUFtIjI1GgIgCa6frZ7TJMXPPP2UPB9WNJeJY3cdnK4NbNnppMTZuWBLscWzfK36txLC EWQqAJPDl98sULV/BPzd6Q==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1pGIaa-0007gG-Vg; Fri, 13 Jan 2023 06:51:17 -0500 Date: Fri, 13 Jan 2023 13:51:15 +0200 Message-Id: <83mt6mac0c.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Yuan Fu <casouri@HIDDEN> In-Reply-To: <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> (message from Yuan Fu on Fri, 13 Jan 2023 01:15:09 -0800) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN> <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 60691 Cc: juri@HIDDEN, 60691 <at> debbugs.gnu.org, monnier@HIDDEN, dgutov@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Yuan Fu <casouri@HIDDEN> > Date: Fri, 13 Jan 2023 01:15:09 -0800 > Cc: Dmitry Gutov <dgutov@HIDDEN>, > 60691 <at> debbugs.gnu.org, > Juri Linkov <juri@HIDDEN>, > Stefan Monnier <monnier@HIDDEN> > > > On Jan 12, 2023, at 11:57 PM, Eli Zaretskii <eliz@HIDDEN> wrote: > > > >> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN > >> Date: Fri, 13 Jan 2023 01:40:56 +0200 > >> From: Dmitry Gutov <dgutov@HIDDEN> > >> > >> Managed to reproduce this after running the test in a couple of > >> different files. > >> > >> But 'M-x memory-usage' says no such command, and 'M-x memory-report' > >> ends up with this error: > >> > >> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil) > >> memory-report--gc-elem(nil strings) > >> memory-report--garbage-collect() > >> memory-report() > > > > This means GC is disabled in this session at the time you invoke > > memory-report. Which shouldn't happen, of course. It sounds like > > your pure Lisp storage overflowed, and that disabled GC. > > > > And I think I see the problem: we use build_pure_c_string in treesit.c > > in places that we shouldn't. > > > > Yuan, build_pure_c_string should only be used in places such as > > syms_of_treesit, which are called just once, during dumping. Look at > > all the other calls to this function in the sources, and you will see > > it. In all other cases, you should do one of the following: > > > > . for strings whose text is fixed, define a variable, give it the > > value in syms_of_treesit using build_pure_c_string, then use that > > variable elsewhere in the source > > Can I define a bunch of static C variables and initialize them in syms_of_treesit, or they have to be all Lisp variables? Eg, > > static Lisp_Object TREESIT_STAR; > > ... > > void > syms_of_treesit (void) > { > ... > TREESIT_STAR = build_pure_c_string ("*"); > ... > } Yes, of course. Look, for example, how coding.c does that: /* A string that serves as name of the reusable work buffer, and as base name of temporary work buffers used for code-conversion operations. */ static Lisp_Object Vcode_conversion_workbuf_name; [...] void syms_of_coding (void) { [...] staticpro (&Vcode_conversion_workbuf_name); Vcode_conversion_workbuf_name = build_pure_c_string (" *code-conversion-work*"); But please keep the convention of naming such variables Vsome_thing, both regarding the "V" and the fact that the name is otherwise lower-case. Thanks.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 13 Jan 2023 09:15:22 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 04:15:22 2023 Received: from localhost ([127.0.0.1]:49570 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pGG9i-0004xB-6E for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 04:15:22 -0500 Received: from mail-pl1-f181.google.com ([209.85.214.181]:46730) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <casouri@HIDDEN>) id 1pGG9g-0004wv-Db for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 04:15:21 -0500 Received: by mail-pl1-f181.google.com with SMTP id jn22so22830061plb.13 for <60691 <at> debbugs.gnu.org>; Fri, 13 Jan 2023 01:15:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=grdT2JPcyMNP8d/LqeBAZEhUtyUNSyPdL5gl1vq0JwQ=; b=bThSs0rp4hQV/OnYaq7tmwI6hjfM+5E/b3PJpDr3WBhRNi/viUR+Q4kfYQKWIjIwyU EItgHVlGzCZKVgU3E8bl9Npl7/WlaFEUQOsGmcr9HiAWY6n4JqTZg5VilQ23esOxXmBZ grBJW+kgWJSmEuhDW3BNDD452UiGwqev2PZkmpC33l7UCkBKVBWvJvBvovtYYM49xi2s Qwka22Z2s7nMP4YZQEeK356+o15ydGsPtVZ6IADbbFs/Sc3ABUor+vGmtXEBJ7AnO61d Yc2jDxqXwSQEagkdhbEtIV86tP7aV/XW6TNsPt8AAYphLwWI7kUN2SpPHkQLJiMgiqIK VHKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=grdT2JPcyMNP8d/LqeBAZEhUtyUNSyPdL5gl1vq0JwQ=; b=rmzSb2gpRZm3SqfJcWcHRTPdrznlxr7SQMx+CJTFLMk6TBMYy6G0QYdTCvXKmR9hcQ TRbm/H9j6U7PTOazcjznFSweonydJYtRgEVsr9SHsIgCbsi+wcALT59f4DFimEQFPXhO TnXWkOqZ914Rr8MqX74ds6Q9eJ8ESSfFubtaOi63s5+PAlZfo1jMiBwGRC5S47R9ulHR PYYwtXwBdNBd3fP7NVxwqINj+Bhymb0epgjkWztERcx8v86SKB03ecXnSDYyCYBqNqWs XaQrmXbV/vdMn7wG1WvnKLZ30emlIqhGYFSZtU188ym1M1bwntP3ZUPa2mt2snzaGlXY 4F/w== X-Gm-Message-State: AFqh2kof7hcb5GlE+bYYSxi4pBFALRz0HeOp6pFS6C+qoB9p7VJCJtGx KsrDtAVhY7YoeEuRTBHHzr8= X-Google-Smtp-Source: AMrXdXs491aEfVqGOkzMleIhIQp8QY8tXpp+MsNSsDX8VGQ815yIxeLK+jJIW3RfH8xVZHwrN6axhw== X-Received: by 2002:a17:902:edc5:b0:192:c882:703e with SMTP id q5-20020a170902edc500b00192c882703emr39929936plk.43.1673601314502; Fri, 13 Jan 2023 01:15:14 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id u11-20020a6540cb000000b0046ff3634a78sm11320328pgp.71.2023.01.13.01.15.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Jan 2023 01:15:14 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode From: Yuan Fu <casouri@HIDDEN> In-Reply-To: <83r0vyamtb.fsf@HIDDEN> Date: Fri, 13 Jan 2023 01:15:09 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <E6EC8A50-8CA9-40B5-B827-022FB7311D97@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> <83r0vyamtb.fsf@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 60691 Cc: Juri Linkov <juri@HIDDEN>, 60691 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>, Dmitry Gutov <dgutov@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On Jan 12, 2023, at 11:57 PM, Eli Zaretskii <eliz@HIDDEN> wrote: >=20 >> Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN >> Date: Fri, 13 Jan 2023 01:40:56 +0200 >> From: Dmitry Gutov <dgutov@HIDDEN> >>=20 >> Managed to reproduce this after running the test in a couple of=20 >> different files. >>=20 >> But 'M-x memory-usage' says no such command, and 'M-x memory-report'=20= >> ends up with this error: >>=20 >> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p = nil) >> memory-report--gc-elem(nil strings) >> memory-report--garbage-collect() >> memory-report() >=20 > This means GC is disabled in this session at the time you invoke > memory-report. Which shouldn't happen, of course. It sounds like > your pure Lisp storage overflowed, and that disabled GC. >=20 > And I think I see the problem: we use build_pure_c_string in treesit.c > in places that we shouldn't. >=20 > Yuan, build_pure_c_string should only be used in places such as > syms_of_treesit, which are called just once, during dumping. Look at > all the other calls to this function in the sources, and you will see > it. In all other cases, you should do one of the following: >=20 > . for strings whose text is fixed, define a variable, give it the > value in syms_of_treesit using build_pure_c_string, then use that > variable elsewhere in the source Can I define a bunch of static C variables and initialize them in = syms_of_treesit, or they have to be all Lisp variables? Eg, static Lisp_Object TREESIT_STAR; ... void syms_of_treesit (void) { ... TREESIT_STAR =3D build_pure_c_string ("*"); ... } Yuan=
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 13 Jan 2023 07:58:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 13 02:58:11 2023 Received: from localhost ([127.0.0.1]:49444 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pGEx1-0000bJ-52 for submit <at> debbugs.gnu.org; Fri, 13 Jan 2023 02:58:11 -0500 Received: from eggs.gnu.org ([209.51.188.92]:56654) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1pGEwz-0000b7-95 for 60691 <at> debbugs.gnu.org; Fri, 13 Jan 2023 02:58:09 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1pGEwl-00043c-Ue; Fri, 13 Jan 2023 02:58:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=7SqaeqM7TefYeYdaQoiAgZ8yRV2f2LhqjeZu/WeQesA=; b=TAYP8vHh4M1fFMkMZ+AJ /W4Yij1VZlOeA29vsX13ksDTIHjcpvpNMxbhg1b0vv57CRo+gx/j8WPcnY44pKYTjifyNAcfql71r OYHFJnKlmKNWbbp44gR6K/oJX4vNaogJ6gEBKek8YmCykOi279FIX1m1xgSnKMzz2w0fxaGOUc3bR XhbQRDK/XWQ0jg15D6NcPhO3sEMhliQoPFQ4nT6YxwnIQFRV0mUnOMXfU7sbn884VVdOe8dugBeZS W93u8ktK989JbDseVn7YN1YACG18soPus+wjp9l04halMOq932XHMVXMyleJfnf/GMSCh5s8CVh+Q 0lRSjgDqxXEE0g==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1pGEwk-0008Bi-4R; Fri, 13 Jan 2023 02:57:55 -0500 Date: Fri, 13 Jan 2023 09:57:52 +0200 Message-Id: <83r0vyamtb.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: casouri@HIDDEN, Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> (message from Dmitry Gutov on Fri, 13 Jan 2023 01:40:56 +0200) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>, juri@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN > Date: Fri, 13 Jan 2023 01:40:56 +0200 > From: Dmitry Gutov <dgutov@HIDDEN> > > Managed to reproduce this after running the test in a couple of > different files. > > But 'M-x memory-usage' says no such command, and 'M-x memory-report' > ends up with this error: > > Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil) > memory-report--gc-elem(nil strings) > memory-report--garbage-collect() > memory-report() This means GC is disabled in this session at the time you invoke memory-report. Which shouldn't happen, of course. It sounds like your pure Lisp storage overflowed, and that disabled GC. And I think I see the problem: we use build_pure_c_string in treesit.c in places that we shouldn't. Yuan, build_pure_c_string should only be used in places such as syms_of_treesit, which are called just once, during dumping. Look at all the other calls to this function in the sources, and you will see it. In all other cases, you should do one of the following: . for strings whose text is fixed, define a variable, give it the value in syms_of_treesit using build_pure_c_string, then use that variable elsewhere in the source . for strings whose text depends on run-time information, use AUTO_STRING or build_string This is a serious problem, and we should fix it ASAP. > garbage-collect's docstring says: > > However, if there was overflow in pure space, and Emacs was dumped > using the "unexec" method, ‘garbage-collect’ returns nil, because > real GC can’t be done. > > I don't know if my Emacs was dumped using "unexec", though. ./configure > says I'm using pdumper. The above text doesn't account for bugs ;-) Functions that produce objects in pure space are supposed to be called only during the build, a.k.a. "when dumping", and for that the text is correct.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 12 Jan 2023 23:41:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 12 18:41:07 2023 Received: from localhost ([127.0.0.1]:48861 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pG7Bz-0006rz-8u for submit <at> debbugs.gnu.org; Thu, 12 Jan 2023 18:41:07 -0500 Received: from mail-ej1-f47.google.com ([209.85.218.47]:43983) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pG7Bx-0006rO-7J for 60691 <at> debbugs.gnu.org; Thu, 12 Jan 2023 18:41:06 -0500 Received: by mail-ej1-f47.google.com with SMTP id hw16so36701651ejc.10 for <60691 <at> debbugs.gnu.org>; Thu, 12 Jan 2023 15:41:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=G9PWF+FHoTG5EJvb7lvyvAbPqIQaMs0RZYu1eiR7/s0=; b=aiQ7w+/uAGckaH2duHMDgKSvzwZ5C9XZ55W331sgtGM1UThoYa6nO+R2q0+KjQhjO3 Eq3a4zS3JFVf263Y+adKKDr6DCw8oBzlr2Qr+LIqqdLHhscy4HcCHNcrdfNVdhoG/OdK YgAZAJMmyDKldeKwQR4EjPrlMKHO7Ly4/YyvW0x26QoVRgtaDRslIaEgKxd7VwulmYq0 L5XoSBqC+IYeAT8ENhxe60k6kPXFr7Vq2DzNAPM1s/CA1Z0cozmdAVKEyu43+Qe5Jq8K 1kvHtN6WGGSA19yr14utPDPjCZqx9hYGSKltxQ6D4fWD7Z304NP4lncwdndteMXmoP9R 9G7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G9PWF+FHoTG5EJvb7lvyvAbPqIQaMs0RZYu1eiR7/s0=; b=v6MRcz4GjazDZKa8sl3ihcD+GG/iBhmvSU5uK2HGBUCrsvvWZ6qPfr6ECSnwhGw6qr fF3+VXIDXLhXYVWAyVyhu+YvTcn2CSzxTFpoC1I6cqjllJCkGEnvM7M6gbtyoyz9bGGH ndQSRIg3HpyM8OrKnKe3QvxQOi2Z50l84N0m50kxxBJ3jkjowv3DcwAPZ7myMUZ8nrY6 rRUVgGHEtjZMMHQ3Zd6uR3EINsCzyw+duDFf984QrcEwFdySye6enqbfx4X90Byxm09U h8Bf5XvRGSBInXlXdYPnOHK91dsuov0i1CBQDco1nRSsqReylxNJciVR2D1hBM/ok0G8 PU7Q== X-Gm-Message-State: AFqh2kpiwdNAdhhXlidj6kaBEjUvCoDKo2FIE5WfohFrpF7MWWYSXE6o l1MzT6uQWQfAMBZnAmUlhBc= X-Google-Smtp-Source: AMrXdXsYW9Hxb41UNyatlRxxpijViRNQ6fbwiVaFzCthONJEMsK+HVANLJhLuB8SC1IWjVd6H1SKgQ== X-Received: by 2002:a17:907:a70b:b0:7c1:98d:a8a3 with SMTP id vw11-20020a170907a70b00b007c1098da8a3mr60271563ejc.7.1673566859131; Thu, 12 Jan 2023 15:40:59 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id d9-20020a1709063ec900b0084c2065b388sm7853479ejj.128.2023.01.12.15.40.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Jan 2023 15:40:58 -0800 (PST) Message-ID: <0ba1ca9c-78e3-f961-787e-4758beaa3c5b@HIDDEN> Date: Fri, 13 Jan 2023 01:40:56 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Yuan Fu <casouri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 12/01/2023 23:58, Yuan Fu wrote: > > Dmitry Gutov <dgutov@HIDDEN> writes: > >> Yuan? Just making sure you got this message. > > Sorry for the delay :-) > >> On 10/01/2023 16:10, Dmitry Gutov wrote: >>> Perhaps Yuan has some further ideas. There are some strong oddities here: >>> - Some time into debugging and repeating the benchmark again and >>> again, I get the "Pure Lisp storage overflowed" message. Just once >>> per Emacs session. It doesn't seem to change much, so it might be >>> unimportant. > > That sounds like 60653. The next time you encounter it, could you record > the output of M-x memory-usage and M-x memory-report? Managed to reproduce this after running the test in a couple of different files. But 'M-x memory-usage' says no such command, and 'M-x memory-report' ends up with this error: Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil) memory-report--gc-elem(nil strings) memory-report--garbage-collect() memory-report() funcall-interactively(memory-report) #<subr call-interactively>(memory-report record nil) apply(#<subr call-interactively> memory-report (record nil)) call-interactively@ido-cr+-record-current-command(#<subr call-interactively> memory-report record nil) apply(call-interactively@ido-cr+-record-current-command #<subr call-interactively> (memory-report record nil)) call-interactively(memory-report record nil) command-execute(memory-report record) execute-extended-command(nil "memory-report" nil) funcall-interactively(execute-extended-command nil "memory-report" nil) #<subr call-interactively>(execute-extended-command nil nil) apply(#<subr call-interactively> execute-extended-command (nil nil)) call-interactively@ido-cr+-record-current-command(#<subr call-interactively> execute-extended-command nil nil) apply(call-interactively@ido-cr+-record-current-command #<subr call-interactively> (execute-extended-command nil nil)) call-interactively(execute-extended-command nil nil) command-execute(execute-extended-command) garbage-collect's docstring says: However, if there was overflow in pure space, and Emacs was dumped using the "unexec" method, ‘garbage-collect’ returns nil, because real GC can’t be done. I don't know if my Emacs was dumped using "unexec", though. ./configure says I'm using pdumper. In case that matters, I'm testing the emacs-29 branch. >>> - The profiler output looks like this: >>> 18050 75% - >>> font-lock-fontify-syntactically-region >>> 15686 65% - treesit-font-lock-fontify-region >>> 3738 15% treesit--children-covering-range-recurse >>> 188 0% treesit-fontify-with-override >>> - When running the benchmark for the first time in a buffer (such as >>> ruby.rb), the variable treesit--font-lock-fast-mode is usually >>> changed to t. In one Emacs session, after I changed it to nil and >>> re-ran the benchmark, the variable stayed nil, and the benchmark ran >>> much faster (like 10s vs 36s). >>> In the next session, after I restarted Emacs, that didn't happen: it >>> always stayed at t, even if I reset it to nil between runs. But if I >>> comment out the block in treesit-font-lock-fontify-region that uses >>> it >>> ;; (when treesit--font-lock-fast-mode >>> ;; (setq nodes (treesit--children-covering-range-recurse >>> ;; (car nodes) start end (* 4 jit-lock-chunk-size)))) >>> and evaluate the defun, the benchmark runs much faster again: 11s. >>> (But then I brought it all back, and re-ran the tests, and the >>> variable stayed nil that time around; to sum up: the way it's turned >>> on is unstable.) >>> Should treesit--font-lock-fast-mode be locally bound inside that >>> function, so that it's reset between chunks? Or maybe the condition >>> for its enabling should be tweaked? E.g. I don't think there are any >>> particularly large or deep nodes in ruby.rb's parse tree. It's a >>> very shallow file. > > Yeah that is a not-very-clever hack. I’ve got an idea: I can add a C > function that checks the maximum depth of a parse tree and the maximum > node span, and turn on the fast-mode if the depth is too large or a node > is too wide. And we do that check once before doing any fontification. > > I’ll report back once I add it. Thanks! And if the check can be fast enough, we could probably do it in the beginning of fontifying every chunk.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 12 Jan 2023 21:58:58 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 12 16:58:58 2023 Received: from localhost ([127.0.0.1]:48724 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pG5b8-0003mj-1I for submit <at> debbugs.gnu.org; Thu, 12 Jan 2023 16:58:58 -0500 Received: from mail-pj1-f43.google.com ([209.85.216.43]:45003) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <casouri@HIDDEN>) id 1pG5b5-0003mU-Kw for 60691 <at> debbugs.gnu.org; Thu, 12 Jan 2023 16:58:56 -0500 Received: by mail-pj1-f43.google.com with SMTP id o7-20020a17090a0a0700b00226c9b82c3aso22453651pjo.3 for <60691 <at> debbugs.gnu.org>; Thu, 12 Jan 2023 13:58:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=/9woX2rdgAbZJ8bomT/4I+Ne9ZTerZyswRjNWI7oGkE=; b=TGu/eHxMPdXqGQDlTUsZu7WQJLAba4FGFePSyKV6LC76RBcKAT5zdbAxXbHTCRUH46 CF6lDPMcINyjQ1upETykht8ptQ0Mtb0QZmasa1C5z7g10lZu1+ztA3bgqus7+ZkKuo4m snnsxr/guH8E7nH9EXCx5Z7uJ/D3gIBlqEPINjyiTzMS3mhxcOIzvrvgenos0uOCPCdC qW4+JzeDnwwY5+EWiShCKQ4+xeDLnmgDw7khpvgnHTxjsC4w0eDeZXuN1nYd0ZFF7e/t J21tBp67jZFNSfWpSVGMnVu2oTP3/Rqy2f3KpCnlu1wW5YBQFCo4QeKXssNBQ+T3I/dr GGWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/9woX2rdgAbZJ8bomT/4I+Ne9ZTerZyswRjNWI7oGkE=; b=FvcxUX2tl0f9wgaX4c9EPRFFC2YJzpS2WOUHLGedqHyeOUsRQ9EJZePO2wtEYpevGU 45uSKetyLIyUjxykEuRNjAZ6UAIrAN7SsW5mf4vPtZ3aYnRcWDTmF/vxEzaeFgu0ORnV m0+4I299oooEJlSZj03dOgDpxi5E+J6jb3Cabnjv79OjGKrWuz55kwhEU9LRPOTWyLkq JzhspLkDp6FvbGnystfZ/AjTYMW3gvrOOMOr6NgTTPPIsI8VBLtOPHIUrGYM9QMZ0qzn FMpkiVsbGph4kUS5dt+ZxxBb2FLZz+4dOImXBMLDv5kKaCiuaYR3gxIFnSQU53TfYSlk Pz4w== X-Gm-Message-State: AFqh2kpkWeqPt1Yl4HPsRkprfZVJH6h86gVKSg6vYwTcDpiUdbroaX3R 615TccQ+PFHOr0PRauDUogw= X-Google-Smtp-Source: AMrXdXuz6obcOM3zRLdOo2rr7B+Y3MayZlhGofsxOm3zsumwAPCbGueVNgdmvrnpF8mf14Xqo+Q/gA== X-Received: by 2002:a17:903:2093:b0:194:4285:dfef with SMTP id d19-20020a170903209300b001944285dfefmr9954462plc.49.1673560729810; Thu, 12 Jan 2023 13:58:49 -0800 (PST) Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id a10-20020a170902ecca00b00189a7fbfd44sm12612249plh.211.2023.01.12.13.58.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jan 2023 13:58:49 -0800 (PST) From: Yuan Fu <casouri@HIDDEN> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Message-Id: <6F1CC7E3-E5B2-4E51-93F6-455A2D0C771E@HIDDEN> Date: Thu, 12 Jan 2023 13:58:48 -0800 To: Dmitry Gutov <dgutov@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org, juri@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Dmitry Gutov <dgutov@HIDDEN> writes: > Yuan? Just making sure you got this message. Sorry for the delay :-) > On 10/01/2023 16:10, Dmitry Gutov wrote: >> Perhaps Yuan has some further ideas. There are some strong oddities = here: >> - Some time into debugging and repeating the benchmark again and >> again, I get the "Pure Lisp storage overflowed" message. Just once >> per Emacs session. It doesn't seem to change much, so it might be >> unimportant. That sounds like 60653. The next time you encounter it, could you record the output of M-x memory-usage and M-x memory-report?=20 >> - The profiler output looks like this: >> 18050 75% - >> font-lock-fontify-syntactically-region >> 15686 65% - treesit-font-lock-fontify-region >> 3738 15% treesit--children-covering-range-recurse >> 188 0% treesit-fontify-with-override >> - When running the benchmark for the first time in a buffer (such as >> ruby.rb), the variable treesit--font-lock-fast-mode is usually >> changed to t. In one Emacs session, after I changed it to nil and >> re-ran the benchmark, the variable stayed nil, and the benchmark ran >> much faster (like 10s vs 36s). >> In the next session, after I restarted Emacs, that didn't happen: it >> always stayed at t, even if I reset it to nil between runs. But if I >> comment out the block in treesit-font-lock-fontify-region that uses >> it >> ;; (when treesit--font-lock-fast-mode >> ;; (setq nodes (treesit--children-covering-range-recurse >> ;; (car nodes) start end (* 4 = jit-lock-chunk-size)))) >> and evaluate the defun, the benchmark runs much faster again: 11s. >> (But then I brought it all back, and re-ran the tests, and the >> variable stayed nil that time around; to sum up: the way it's turned >> on is unstable.) >> Should treesit--font-lock-fast-mode be locally bound inside that >> function, so that it's reset between chunks? Or maybe the condition >> for its enabling should be tweaked? E.g. I don't think there are any >> particularly large or deep nodes in ruby.rb's parse tree. It's a >> very shallow file. Yeah that is a not-very-clever hack. I=E2=80=99ve got an idea: I can add = a C function that checks the maximum depth of a parse tree and the maximum node span, and turn on the fast-mode if the depth is too large or a node is too wide. And we do that check once before doing any fontification. I=E2=80=99ll report back once I add it. Yuan
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 11 Jan 2023 12:12:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 07:12:37 2023 Received: from localhost ([127.0.0.1]:41853 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFZy9-0002hq-8X for submit <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:37 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:56299) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pFZy7-0002gv-W1 for 60691 <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:36 -0500 Received: by mail-wm1-f47.google.com with SMTP id l26so10940815wme.5 for <60691 <at> debbugs.gnu.org>; Wed, 11 Jan 2023 04:12:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=vC6n3tsfqPm2aUlkwOCfZYpy83jGbKoAY0Ti3HvVIVw=; b=BLpKty+bgWmjtBIcDYATth9XVabHbrnCaZOaXujQG7sDLX8QWjBMmj3dwQrTqL4zCH 432aahievevUnNb4zuxrrIYnDiOT5eWwcwh5d7SZzLbzOroB7DPKhIfZ5b5LVWClhQAJ p5i/qK2SqLa/mmUCYM5iPdY28e6JxYQJno/GSleq/gNdWQJAWaufTZ9QE1HXhkH3F7i2 D1/S0QJrh/ej2mhm7iLQEhE2CK2+LzZlr6VW2/NduwhqeNOOeis3z6/9rpky/ADyr+Uq DzFRC/qSxBXPc6OzFzKjW1XaAOuMxhZXoHrbX7aPGJyMsiycUG17YAboYFUmJs44EIux tK2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vC6n3tsfqPm2aUlkwOCfZYpy83jGbKoAY0Ti3HvVIVw=; b=QWVqvzAMNmyZDRPePg1omhREAa15lUzQ315TZia7c5XCBBLcGf0vIywYi3n3dRVqOP Poy1Ad/tzhyI0txsgCQ92SOXJ0Rqp37anf3DTbEbs6Ou1BruyLg3SWbKnuZVr5bclyTc HrQMjVU+IfnsRh3JJsJgjM2SZMl5eD6DaVBgJjmVzuzn5E6drHHaflvrijLW/2jpnOqc 4Lij9pXYQ5O/NLf/5OuR8emKJxVSkhN9/rS5Bk0SmAfvDjbh2jfUS3e2Y69DBo89ZytS 7JIHWrxmeWP3/PF8rqKo4jigTGrTuomD1XQqk4DWbDFjfiDbiaAfsW+ZlJhrrEo091px cd+Q== X-Gm-Message-State: AFqh2kqCx3nI1vLUDCnJr7mgHiGP6b6zpe2QdYL1AlpVaYOXJYiAQ0uQ qu/b410y4MU5dTa/+ozWKpAMA5VTeCo= X-Google-Smtp-Source: AMrXdXsb/M8YjD+tYmzVQ7OvS3SwAPae2g8YhFgFJHUBEfHBYX7hXdHwo7QdG8kScjSbkU53MDvoUw== X-Received: by 2002:a05:600c:4e51:b0:3d1:e1f4:21d1 with SMTP id e17-20020a05600c4e5100b003d1e1f421d1mr63557064wmq.26.1673439155546; Wed, 11 Jan 2023 04:12:35 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id g12-20020a05600c310c00b003c70191f267sm24680009wmo.39.2023.01.11.04.12.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Jan 2023 04:12:35 -0800 (PST) Message-ID: <122d12c9-9b7f-d8ff-9679-f2af0e8e2a93@HIDDEN> Date: Wed, 11 Jan 2023 14:12:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US From: Dmitry Gutov <dgutov@HIDDEN> To: Juri Linkov <juri@HIDDEN>, Yuan Fu <casouri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> In-Reply-To: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) Yuan? Just making sure you got this message. On 10/01/2023 16:10, Dmitry Gutov wrote: > Perhaps Yuan has some further ideas. There are some strong oddities here: > > - Some time into debugging and repeating the benchmark again and again, > I get the "Pure Lisp storage overflowed" message. Just once per Emacs > session. It doesn't seem to change much, so it might be unimportant. > > - The profiler output looks like this: > > 18050 75% - font-lock-fontify-syntactically-region > 15686 65% - treesit-font-lock-fontify-region > 3738 15% treesit--children-covering-range-recurse > 188 0% treesit-fontify-with-override > > - When running the benchmark for the first time in a buffer (such as > ruby.rb), the variable treesit--font-lock-fast-mode is usually changed > to t. In one Emacs session, after I changed it to nil and re-ran the > benchmark, the variable stayed nil, and the benchmark ran much faster > (like 10s vs 36s). > > In the next session, after I restarted Emacs, that didn't happen: it > always stayed at t, even if I reset it to nil between runs. But if I > comment out the block in treesit-font-lock-fontify-region that uses it > > ;; (when treesit--font-lock-fast-mode > ;; (setq nodes (treesit--children-covering-range-recurse > ;; (car nodes) start end (* 4 jit-lock-chunk-size)))) > > and evaluate the defun, the benchmark runs much faster again: 11s. > > (But then I brought it all back, and re-ran the tests, and the variable > stayed nil that time around; to sum up: the way it's turned on is > unstable.) > > Should treesit--font-lock-fast-mode be locally bound inside that > function, so that it's reset between chunks? Or maybe the condition for > its enabling should be tweaked? E.g. I don't think there are any > particularly large or deep nodes in ruby.rb's parse tree. It's a very > shallow file.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 11 Jan 2023 12:12:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 11 07:12:20 2023 Received: from localhost ([127.0.0.1]:41850 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFZxr-0002hD-UR for submit <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:20 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:56299) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pFZxo-0002gv-6u for 60691 <at> debbugs.gnu.org; Wed, 11 Jan 2023 07:12:18 -0500 Received: by mail-wm1-f47.google.com with SMTP id l26so10940024wme.5 for <60691 <at> debbugs.gnu.org>; Wed, 11 Jan 2023 04:12:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=r/dqXjaqPACUM8T6HzTmldIJgBC3gYahWG0LRqnobaE=; b=E1NuJ7q9CT91Z5wPrmAGiQYwFVMrXE3FCF9rH/ty3V8nBu3aaGPbd4slClsF2WLMxU JAnDJ/6YR/+W3DJ+1CMdWhq3+wl6yUd2SkDUGrT/oEisCM+JjGqr6Vje73vFpCO/nirx IvcqkRnSCjGPD4Tl5+Y/CADiBiscwHuM/mOtcnyixNjw7xxaOw+ax07xYH6h8S4075ey dGubsj4IsOSGGWQ1KKoKJkAHMw4s8WqY1dr4VN4PQ9858nkhpmjz7uCn+Fs7ld1r6YvG QrbBcUOQXNxVkbe+LVayPElKm54E/KN/x/kX6aGEYWnpS9JQpp+e1rZlFXFhmg7y1hwH 3P9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=r/dqXjaqPACUM8T6HzTmldIJgBC3gYahWG0LRqnobaE=; b=jVzrMFEbpLDp03pnJDZmfiA95j7zr+Clzc5ic15W4fZSFM0cMtDrKsb27Iu4OLG7Sl 9PJMcAbRap5vswhPjztES1BXta43ETlZT2kGOLhXSkUIqOBFfN9hX5a6je+Cd5x6JHUD frIaqMtLcow5fEvRLo/OyInoixpm7VW1/JcEci3U93lvb8WDC97DAFY6XOfYqLziBrNC Aujaz11KNCh4YR34EcL1fM/kzOG8molU61kD8t5FCB44FsGCLA0ULIB6NqqSRNwwuABd +p6BXeTQA0bCAGnelpXQ/bXXNJV2CFe17ivANYsPCahWO7JLz44Zgawgs6bNO+LSdda1 rKug== X-Gm-Message-State: AFqh2kq69iy+F0qzBM2mWHYKRhqFPtIKoW2d+N8mtmqgdl0HSGxLy8Vd qu0JrHEYgn+r1e3UT1pxEs4= X-Google-Smtp-Source: AMrXdXuO5Z0p/z3o7HMYwqaVdRD/nYqwEHixlgivPZSyu6BqB1biVzoSO7OxSjc/p2z6+KqzAtUyKg== X-Received: by 2002:a05:600c:4e48:b0:3cf:69f4:bfd4 with SMTP id e8-20020a05600c4e4800b003cf69f4bfd4mr53607359wmq.7.1673439129168; Wed, 11 Jan 2023 04:12:09 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id l36-20020a05600c1d2400b003d9fb59c16fsm5362760wms.11.2023.01.11.04.12.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Jan 2023 04:12:08 -0800 (PST) Message-ID: <dd585de9-5fc7-cd0e-5ef4-5cfd812f3df9@HIDDEN> Date: Wed, 11 Jan 2023 14:12:06 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Juri Linkov <juri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> <86tu0yqnwr.fsf@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <86tu0yqnwr.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: Yuan Fu <casouri@HIDDEN>, 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/01/2023 19:50, Juri Linkov wrote: >>>> Is it common to try to highlight 1000 or even 100 files in one diff? >>> 100 is rare, but tens is pretty common, so this problem affects >>> only this specific case. >> >> So it's a 0,8-3s delay in those cases? That's not ideal. > > The delay is noticeable, alas. Right. I'm somewhat worried for the processing speed xref--collect-matches too. But that's probably only going to be noticeable after we add syntax-propertize-function to ruby-ts-mode. >>> I noticed that while most library files are small, e.g. >>> libtree-sitter-c.so is 401,528 bytes, >>> libtree-sitter-ruby.so is 2,130,616 bytes >>> that means that it has more complex logic >>> that might explain its performance. >> >> ruby is indeed one of the larger ones. Among the ones I have here compiled, >> it's exceeded only by cpp. 2.29 MB vs 2.12 MB. > > The winner is libtree-sitter-julia.so with 7.25 MB. > But regarding libtree-sitter-cpp.so I confirm it's 2.3 MB. > And c++-ts-mode is even faster than c-ts-mode. Yep. > On the same admin/alloc-colors.c: > > c-mode > (33.378821569 1500 17.632000617) > > c-ts-mode > (2.1949608069999997 34 0.4119784769999981) > > c++-ts-mode > (2.0979403910000003 34 0.39749122499999956) > > So size doesn't matter.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 10 Jan 2023 17:57:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 10 12:57:53 2023 Received: from localhost ([127.0.0.1]:41068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFIsj-00073F-KI for submit <at> debbugs.gnu.org; Tue, 10 Jan 2023 12:57:53 -0500 Received: from relay3-d.mail.gandi.net ([217.70.183.195]:42941) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <juri@HIDDEN>) id 1pFIsi-00072o-79 for 60691 <at> debbugs.gnu.org; Tue, 10 Jan 2023 12:57:52 -0500 Received: (Authenticated sender: juri@HIDDEN) by mail.gandi.net (Postfix) with ESMTPSA id E634B60008; Tue, 10 Jan 2023 17:57:44 +0000 (UTC) From: Juri Linkov <juri@HIDDEN> To: Dmitry Gutov <dgutov@HIDDEN> Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode In-Reply-To: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> (Dmitry Gutov's message of "Tue, 10 Jan 2023 16:10:49 +0200") Organization: LINKOV.NET References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> Date: Tue, 10 Jan 2023 19:50:44 +0200 Message-ID: <86tu0yqnwr.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 60691 Cc: Yuan Fu <casouri@HIDDEN>, 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) >>> Is it common to try to highlight 1000 or even 100 files in one diff? >> 100 is rare, but tens is pretty common, so this problem affects >> only this specific case. > > So it's a 0,8-3s delay in those cases? That's not ideal. The delay is noticeable, alas. >> I noticed that while most library files are small, e.g. >> libtree-sitter-c.so is 401,528 bytes, >> libtree-sitter-ruby.so is 2,130,616 bytes >> that means that it has more complex logic >> that might explain its performance. > > ruby is indeed one of the larger ones. Among the ones I have here compiled, > it's exceeded only by cpp. 2.29 MB vs 2.12 MB. The winner is libtree-sitter-julia.so with 7.25 MB. But regarding libtree-sitter-cpp.so I confirm it's 2.3 MB. And c++-ts-mode is even faster than c-ts-mode. On the same admin/alloc-colors.c: c-mode (33.378821569 1500 17.632000617) c-ts-mode (2.1949608069999997 34 0.4119784769999981) c++-ts-mode (2.0979403910000003 34 0.39749122499999956) So size doesn't matter.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 10 Jan 2023 14:11:00 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 10 09:11:00 2023 Received: from localhost ([127.0.0.1]:39163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pFFLA-0006c8-8e for submit <at> debbugs.gnu.org; Tue, 10 Jan 2023 09:11:00 -0500 Received: from mail-wm1-f46.google.com ([209.85.128.46]:52084) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pFFL8-0006bs-47 for 60691 <at> debbugs.gnu.org; Tue, 10 Jan 2023 09:10:59 -0500 Received: by mail-wm1-f46.google.com with SMTP id g10so8886810wmo.1 for <60691 <at> debbugs.gnu.org>; Tue, 10 Jan 2023 06:10:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=TjVO7Iu2PTtRuEs848MOngu4IUBVHaOl2a8o366V1ms=; b=mrNBiJoScwiSRm/Lhc5+jcHxvT/Vdwmt6EVFVRtCfCxCxKCX4Iybb06RjBS+Ae87jl mncde8aZGREUgKaagbs8TnpZ1HX9gBR5KWrWZ7Y2aBBMo6vnCpM44Sj//0ttVlwzMMCd YF760PI+PIlkdhbKUGKao3uQuURe6rANQOnLDEc5+Yb8YkO+s9s/V7db/HfsFoz27Of/ QJzcy39gwJlgs9jeV4DiQ3bPo8vklzazgObzLybOhZ49zROZ/3Zs0nZs1jUoUbB6gOtr ZjedvEu9g1VV4VwOzCXQcO8ybTFWXVhNSnH0G72xWgu4IZC35bzhe+8uuKuDnytTozQR 1Bdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TjVO7Iu2PTtRuEs848MOngu4IUBVHaOl2a8o366V1ms=; b=2EtT6sutPxHqezg8hl7tHubZqrzA3mksm5tr/QxUU62nb4Aw9UTxWXmjNWoPA3DQ2y wAeOAPDPqGWxH0oB1EfwlYOaObJ2cWkdJpTRGzWkSComeor8zCNmB+obVzRTL8JCUrQa 2u5aYN+IhNI6h0T0RICZDl46AF9D91dHqfpNFFiEKj39zw57wiPWsbZI38198a8ADf6A rl6ZzmPfFxV2av+GzzMa0J209T2HARGMLsgGpNKM/3FF/l8prCkygBMKFVcDg9mPK64c LHgjpSeikTDVkxK1ttK6OZlUnw7IAAZvureCgBh6Ig3Gtd8ZZJa+/1BRplbqenpcIXH/ kb2Q== X-Gm-Message-State: AFqh2krljc7xaIfA0DSMbsL/HveqmiyqAUQOQDyGnixwDPtlcAh1tuYD MrCFqy049GMnAeltKO6w/3k= X-Google-Smtp-Source: AMrXdXvMhePPVrJP0Vk6j6S4D0bW5vHAxMAn40mHnz+tstN2UpV1JNYObp/GlBJMFWQkXFRIjfy2Ww== X-Received: by 2002:a05:600c:4da2:b0:3d2:39dc:f50e with SMTP id v34-20020a05600c4da200b003d239dcf50emr49123608wmp.7.1673359852028; Tue, 10 Jan 2023 06:10:52 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id i14-20020a05600c354e00b003d1d5a83b2esm21678073wmq.35.2023.01.10.06.10.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 10 Jan 2023 06:10:51 -0800 (PST) Message-ID: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@HIDDEN> Date: Tue, 10 Jan 2023 16:10:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Juri Linkov <juri@HIDDEN>, Yuan Fu <casouri@HIDDEN> References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> <86leman71u.fsf@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <86leman71u.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/01/2023 10:10, Juri Linkov wrote: >>> After more rules were added recently to ruby-ts--font-lock-settings, >>> font-lock became slow even on very small files. Some measurements: >> >> If you saw a particular commit that made things slower, did you try >> reverting it? What was the performance after? > > No particular commit, just adding more rules degrades performance > gradually. But I don't think I added that many rules recently. No more than a quarter anyway. >>> M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure))) >>> M-x ruby-mode >>> (1.3564674989999999 0 0.0) >>> M-x ruby-ts-mode >>> (8.349582391999999 2 6.489918534000001) >> >> I have tried this scenario (which, to be frank, is pretty artificial, given >> that fontification is usually performed in chunks, not over the whole >> buffer). >> >> Perhaps the results depend on a particular file. The ones I have tried >> (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or >> less). The difference was in favor of ruby-mode, but given the difference >> in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed >> overhead somewhere. > > On test/lisp/progmodes/ruby-mode-resources/ruby.rb I see these numbers: > > ruby-mode > (8.701560543000001 95 1.045961102) > > ruby-ts-mode > (34.653148898000005 1464 16.904981779) Interesting. It's 12s vs 36s for me, as I've retested now. >>> This is not a problem when files are visited infrequently, but >>> becomes a problem for diff-syntax fontification that wants to >>> highlight simultaneously many files from git logs. >>> So a temporary measure would be not to enable ruby-ts-mode >>> in internal buffers: >> >> Is it common to try to highlight 1000 or even 100 files in one diff? > > 100 is rare, but tens is pretty common, so this problem affects > only this specific case. So it's a 0,8-3s delay in those cases? That's not ideal. >>> (add-hook 'find-file-hook >>> (lambda () >>> (when (and (eq major-mode 'ruby-mode) >>> ;; Only when not internal as from diff-syntax >>> (not (string-prefix-p " " (buffer-name)))) >>> (ruby-ts-mode)))) >> >> Have you tried similar tests with other -ts- modes? Ones with complex >> font-lock rules in particular. > > I tried with c-ts-mode, and it's very fast. Just how fast is it? The number of font-lock features is has is comparable (though a little smaller). I've tried the same benchmark for it in admin/alloc-colors.c, and it comes out to (3.2004193190000003 30 0.9609690980000067) Which seems comparable. Not sure how to directly test the modes against each other, but if I enable ruby-ts-mode in the same file, the benchmark comes to 1s. Or if I enable c-ts-mode in ruby.rb -- 16s. >> I've tried commenting out different rules in ruby-ts--font-lock-settings, >> but none of them seem to have particularly outsides impact. Performance >> seems, roughly, inversely proportional to the number of separate >> "features". > > Indeed, this is what I see - no particular rule, only their number > affects performance. > >> And if all ts modes turn out to have this problem, perhaps the place to >> improve this is inside some common code. > > I noticed that while most library files are small, e.g. > libtree-sitter-c.so is 401,528 bytes, > libtree-sitter-ruby.so is 2,130,616 bytes > that means that it has more complex logic > that might explain its performance. ruby is indeed one of the larger ones. Among the ones I have here compiled, it's exceeded only by cpp. 2.29 MB vs 2.12 MB. But testing admin/alloc-colors.c with c++-ts-mode vs c-ts-mode gives very similar performance, so it's unlikely that the complexity of the grammar is directly responsible. > In this case, when nothing could be done to improve performance, > please close this request. Perhaps Yuan has some further ideas. There are some strong oddities here: - Some time into debugging and repeating the benchmark again and again, I get the "Pure Lisp storage overflowed" message. Just once per Emacs session. It doesn't seem to change much, so it might be unimportant. - The profiler output looks like this: 18050 75% - font-lock-fontify-syntactically-region 15686 65% - treesit-font-lock-fontify-region 3738 15% treesit--children-covering-range-recurse 188 0% treesit-fontify-with-override - When running the benchmark for the first time in a buffer (such as ruby.rb), the variable treesit--font-lock-fast-mode is usually changed to t. In one Emacs session, after I changed it to nil and re-ran the benchmark, the variable stayed nil, and the benchmark ran much faster (like 10s vs 36s). In the next session, after I restarted Emacs, that didn't happen: it always stayed at t, even if I reset it to nil between runs. But if I comment out the block in treesit-font-lock-fontify-region that uses it ;; (when treesit--font-lock-fast-mode ;; (setq nodes (treesit--children-covering-range-recurse ;; (car nodes) start end (* 4 jit-lock-chunk-size)))) and evaluate the defun, the benchmark runs much faster again: 11s. (But then I brought it all back, and re-ran the tests, and the variable stayed nil that time around; to sum up: the way it's turned on is unstable.) Should treesit--font-lock-fast-mode be locally bound inside that function, so that it's reset between chunks? Or maybe the condition for its enabling should be tweaked? E.g. I don't think there are any particularly large or deep nodes in ruby.rb's parse tree. It's a very shallow file.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 10 Jan 2023 08:25:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 10 03:25:43 2023 Received: from localhost ([127.0.0.1]:38704 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pF9x1-0004dY-6d for submit <at> debbugs.gnu.org; Tue, 10 Jan 2023 03:25:43 -0500 Received: from relay10.mail.gandi.net ([217.70.178.230]:54479) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <juri@HIDDEN>) id 1pF9wz-0004dI-3d for 60691 <at> debbugs.gnu.org; Tue, 10 Jan 2023 03:25:42 -0500 Received: (Authenticated sender: juri@HIDDEN) by mail.gandi.net (Postfix) with ESMTPSA id B02AD240004; Tue, 10 Jan 2023 08:25:31 +0000 (UTC) From: Juri Linkov <juri@HIDDEN> To: Dmitry Gutov <dgutov@HIDDEN> Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode In-Reply-To: <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> (Dmitry Gutov's message of "Tue, 10 Jan 2023 00:33:12 +0200") Organization: LINKOV.NET References: <867cxv3dnn.fsf@HIDDEN> <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> Date: Tue, 10 Jan 2023 10:10:53 +0200 Message-ID: <86leman71u.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 60691 Cc: 60691 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) >> After more rules were added recently to ruby-ts--font-lock-settings, >> font-lock became slow even on very small files. Some measurements: > > If you saw a particular commit that made things slower, did you try > reverting it? What was the performance after? No particular commit, just adding more rules degrades performance gradually. >> M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure))) >> M-x ruby-mode >> (1.3564674989999999 0 0.0) >> M-x ruby-ts-mode >> (8.349582391999999 2 6.489918534000001) > > I have tried this scenario (which, to be frank, is pretty artificial, given > that fontification is usually performed in chunks, not over the whole > buffer). > > Perhaps the results depend on a particular file. The ones I have tried > (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or > less). The difference was in favor of ruby-mode, but given the difference > in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed > overhead somewhere. On test/lisp/progmodes/ruby-mode-resources/ruby.rb I see these numbers: ruby-mode (8.701560543000001 95 1.045961102) ruby-ts-mode (34.653148898000005 1464 16.904981779) >> This is not a problem when files are visited infrequently, but >> becomes a problem for diff-syntax fontification that wants to >> highlight simultaneously many files from git logs. >> So a temporary measure would be not to enable ruby-ts-mode >> in internal buffers: > > Is it common to try to highlight 1000 or even 100 files in one diff? 100 is rare, but tens is pretty common, so this problem affects only this specific case. >> (add-hook 'find-file-hook >> (lambda () >> (when (and (eq major-mode 'ruby-mode) >> ;; Only when not internal as from diff-syntax >> (not (string-prefix-p " " (buffer-name)))) >> (ruby-ts-mode)))) > > Have you tried similar tests with other -ts- modes? Ones with complex > font-lock rules in particular. I tried with c-ts-mode, and it's very fast. > I've tried commenting out different rules in ruby-ts--font-lock-settings, > but none of them seem to have particularly outsides impact. Performance > seems, roughly, inversely proportional to the number of separate > "features". Indeed, this is what I see - no particular rule, only their number affects performance. > And if all ts modes turn out to have this problem, perhaps the place to > improve this is inside some common code. I noticed that while most library files are small, e.g. libtree-sitter-c.so is 401,528 bytes, libtree-sitter-ruby.so is 2,130,616 bytes that means that it has more complex logic that might explain its performance. In this case, when nothing could be done to improve performance, please close this request.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at 60691) by debbugs.gnu.org; 9 Jan 2023 22:33:23 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 09 17:33:22 2023 Received: from localhost ([127.0.0.1]:38348 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pF0hm-0003tH-KA for submit <at> debbugs.gnu.org; Mon, 09 Jan 2023 17:33:22 -0500 Received: from mail-wm1-f51.google.com ([209.85.128.51]:43611) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1pF0hk-0003sx-HX for 60691 <at> debbugs.gnu.org; Mon, 09 Jan 2023 17:33:21 -0500 Received: by mail-wm1-f51.google.com with SMTP id k22-20020a05600c1c9600b003d1ee3a6289so8425690wms.2 for <60691 <at> debbugs.gnu.org>; Mon, 09 Jan 2023 14:33:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=WYRubfQD14nka8tVqf3xnXSCiwbxr3yjYi5GLCe7sTw=; b=ij17FLxTXlpXUnY5J8+Sioh9IjSFeYAPR+atDsUeR0BWYw2F6qpjQWBBrQkUgi3t/3 Pfz3DRzIMdRe6PcEuBE6wbWrpKd+uk7/08O318TDKfhAEeXW4xtNLi6d5gvICpsTWeB3 8Hh/3WF+uJ7UULXKbfbRsIOwV8ijYkXWUFfe9RcUXJ2Q0S8FNYvwV2g1+SwKG3woi21S ohrOqHTrmegct/uqT7PiDHLoD9Hkwg9/ZHOR9jPT5yGlpYMa/S9NlhH0vYuUc2hn5xJS DklzrdabSToE2HZtG+8FiJaUBMbX1tIDsbrZ8wH44QbI8EXu9oVUSYQAaK1AZVjCBh5H WgXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WYRubfQD14nka8tVqf3xnXSCiwbxr3yjYi5GLCe7sTw=; b=Fq8eL8l3M92G7CjY+LI2jQLzMd992jmnmKE0FigHz0VKEN/lKjh3bTBmmT1rIO36cX 9LvFbymgFu8Na5pdPm/eWUfqr9xmXbNhh94yQqQNLNlrTTh5GBNvuskandvKcGd39ePy yB5vlkYQiqUL1aB92OUAIvOLBDe0D7mheHXE8aqEuvXkuwyDKjRWR64eCgKKbh7hhUVO 5gklnbYOSduEVRsf+KN37hmU6O0BrPSkg73f2+Igq1qCV7mUMH09kGCILm5RnGlu4cm/ SnVKzaf1R8hMtb5JoVzUecRWPYhalRwGSmGg1e1BkyL52BPIkQGsAk4VzPhzSH4WNRfR bBpw== X-Gm-Message-State: AFqh2koRyLKs5cRcQIy+OeYgWS6T/74fF+9YO+Q7VJRprxxBFSuPLgjb kgsrPIXpPJdQJE8SWM/R+I0= X-Google-Smtp-Source: AMrXdXsGYKRrlHNV0Y6gPFwFPASJC3AIAEVFKgNP5cK/yVbpSCpOnZjAn472t/hlRDEJrg1IoCrywQ== X-Received: by 2002:a7b:ca51:0:b0:3d2:7a7:5cc6 with SMTP id m17-20020a7bca51000000b003d207a75cc6mr51171463wml.18.1673303594410; Mon, 09 Jan 2023 14:33:14 -0800 (PST) Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id o5-20020a05600c510500b003b4ff30e566sm45901wms.3.2023.01.09.14.33.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Jan 2023 14:33:13 -0800 (PST) Message-ID: <51ee2f6f-6e1d-eccd-f536-461d916cc94d@HIDDEN> Date: Tue, 10 Jan 2023 00:33:12 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Content-Language: en-US To: Juri Linkov <juri@HIDDEN>, 60691 <at> debbugs.gnu.org References: <867cxv3dnn.fsf@HIDDEN> From: Dmitry Gutov <dgutov@HIDDEN> In-Reply-To: <867cxv3dnn.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 60691 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) Hi! On 09/01/2023 19:16, Juri Linkov wrote: > X-Debbugs-Cc: Dmitry Gutov <dgutov@HIDDEN> > > After more rules were added recently to ruby-ts--font-lock-settings, > font-lock became slow even on very small files. Some measurements: If you saw a particular commit that made things slower, did you try reverting it? What was the performance after? > M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure))) > > M-x ruby-mode > (1.3564674989999999 0 0.0) > > M-x ruby-ts-mode > (8.349582391999999 2 6.489918534000001) I have tried this scenario (which, to be frank, is pretty artificial, given that fontification is usually performed in chunks, not over the whole buffer). Perhaps the results depend on a particular file. The ones I have tried (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or less). The difference was in favor of ruby-mode, but given the difference in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed overhead somewhere. > This is not a problem when files are visited infrequently, but > becomes a problem for diff-syntax fontification that wants to > highlight simultaneously many files from git logs. > So a temporary measure would be not to enable ruby-ts-mode > in internal buffers: Is it common to try to highlight 1000 or even 100 files in one diff? > (add-hook 'find-file-hook > (lambda () > (when (and (eq major-mode 'ruby-mode) > ;; Only when not internal as from diff-syntax > (not (string-prefix-p " " (buffer-name)))) > (ruby-ts-mode)))) Have you tried similar tests with other -ts- modes? Ones with complex font-lock rules in particular. I've tried commenting out different rules in ruby-ts--font-lock-settings, but none of them seem to have particularly outsides impact. Performance seems, roughly, inversely proportional to the number of separate "features". And if all ts modes turn out to have this problem, perhaps the place to improve this is inside some common code.
bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.Received: (at submit) by debbugs.gnu.org; 9 Jan 2023 17:35:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 09 12:35:20 2023 Received: from localhost ([127.0.0.1]:38091 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pEw3M-0004DE-CP for submit <at> debbugs.gnu.org; Mon, 09 Jan 2023 12:35:20 -0500 Received: from lists.gnu.org ([209.51.188.17]:33332) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <juri@HIDDEN>) id 1pEw3K-0004D6-TW for submit <at> debbugs.gnu.org; Mon, 09 Jan 2023 12:35:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <juri@HIDDEN>) id 1pEw3J-0007G0-8U for bug-gnu-emacs@HIDDEN; Mon, 09 Jan 2023 12:35:17 -0500 Received: from relay6-d.mail.gandi.net ([217.70.183.198]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <juri@HIDDEN>) id 1pEw3H-0005Eh-OE for bug-gnu-emacs@HIDDEN; Mon, 09 Jan 2023 12:35:17 -0500 Received: (Authenticated sender: juri@HIDDEN) by mail.gandi.net (Postfix) with ESMTPSA id 3DADEC0009 for <bug-gnu-emacs@HIDDEN>; Mon, 9 Jan 2023 17:35:09 +0000 (UTC) From: Juri Linkov <juri@HIDDEN> To: bug-gnu-emacs@HIDDEN Subject: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Organization: LINKOV.NET Date: Mon, 09 Jan 2023 19:16:12 +0200 Message-ID: <867cxv3dnn.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/30.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=217.70.183.198; envelope-from=juri@HIDDEN; helo=relay6-d.mail.gandi.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.6 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.6 (--) X-Debbugs-Cc: Dmitry Gutov <dgutov@HIDDEN> After more rules were added recently to ruby-ts--font-lock-settings, font-lock became slow even on very small files. Some measurements: M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure))) M-x ruby-mode (1.3564674989999999 0 0.0) M-x ruby-ts-mode (8.349582391999999 2 6.489918534000001) This is not a problem when files are visited infrequently, but becomes a problem for diff-syntax fontification that wants to highlight simultaneously many files from git logs. So a temporary measure would be not to enable ruby-ts-mode in internal buffers: (add-hook 'find-file-hook (lambda () (when (and (eq major-mode 'ruby-mode) ;; Only when not internal as from diff-syntax (not (string-prefix-p " " (buffer-name)))) (ruby-ts-mode))))
Juri Linkov <juri@HIDDEN>
:dgutov@HIDDEN, bug-gnu-emacs@HIDDEN
.
Full text available.dgutov@HIDDEN, bug-gnu-emacs@HIDDEN
:bug#60691
; Package emacs
.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.