Received: (at 77588) by debbugs.gnu.org; 7 Apr 2025 11:21:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Apr 07 07:21:53 2025 Received: from localhost ([127.0.0.1]:53015 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1u1kY5-0006kL-5r for submit <at> debbugs.gnu.org; Mon, 07 Apr 2025 07:21:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36236) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1u1kY2-0006jn-DZ for 77588 <at> debbugs.gnu.org; Mon, 07 Apr 2025 07:21:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1u1kXu-00014l-Kd; Mon, 07 Apr 2025 07:21:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=/1Lx7Zs2uPy1q2aLAMUKvchmRhEbCRsJXFnD5ycL7P0=; b=iC6o0OU6ivhe9a1DmbB9 h35dXXKGRXFe0dTNKmQfmCRKY8MS0m91geitK2fbRSousYF3VxTFvx5AyR3jzdsJuyDHofMWtadJZ pykBJsOcMOF19VufyjYqnJvmmM0tC2Sbx5cg9SsnjisNBaTJEvtFghCBzywM0ka6dRtmWI6qV9F4g 34SYhhxDmrvXzh4FPRgDt2RMeuVRgk9MNbHxYcUWt4z6B7V7rtOqUujxkD0VZw94oXf3kdAr4GqCU 6g7djSb6H44OgrKjF1/nduPx0J8AhmY9KVaUq/6i+Sa+ok397b/qUu5GmpchCivnr9Fo+0wI+H5dU PrqrOp7iJkpfSg==; Date: Mon, 07 Apr 2025 14:21:38 +0300 Message-Id: <86v7rgtdx9.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: JD Smith <jdtsmith@HIDDEN>, =?utf-8?B?Sm/Do28gVMOhdm9yYQ==?= <joaotavora@HIDDEN>, Spencer Baugh <sbaugh@HIDDEN> In-Reply-To: <CEA8A47E-2C5F-44A3-9909-DD342A7598A7@HIDDEN> (message from JD Smith on Sun, 6 Apr 2025 18:50:26 -0400) Subject: Re: bug#77588: Catastrophic slowdown in eglot via `flymake-diag-region' References: <CEA8A47E-2C5F-44A3-9909-DD342A7598A7@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 77588 Cc: 77588 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: JD Smith <jdtsmith@HIDDEN> > Date: Sun, 6 Apr 2025 18:50:26 -0400 > > I was diagnosing a painful eglot pause of about 10 seconds that would reliably occur every few edits when > completing candidates and working in a long python file (~8k lines). I am using the basedpyright LSP server. > I narrowed this down to eglot's `textDocument/publishDiagnostics' notification handler. > > The file I was working on has several hundred warnings and errors, so that represents a substantial message > from the LSP server. After a fair bit of timing and sleuthing, I narrowed it down further. A few interrelated > misbehaviors combined to produce the huge intermittent pauses: > > 1 When completing text directly after entering/removing a newline, the Diagnostics ranges received by eglot > are out of date and "one line off". > 2 Some of these off-by-one-line diagnostic message ranges now land on blank lines in the file, so the > effective range there is empty. > 3 Seeing an empty range, eglot declares the server has "botched" it, so it tries to make a by-hand > recalculation of the range using `flymake-diag-region'. > 4 `flymake-diag-region' for some reason calls (end-of-thing 'sexp) and (end-of-thing 'symbol). > 5 In some positions in a long python file these commands are VERY SLOW. Rather than the typical call time > of ~1ms, at these certain positions `flymake-diag-region' takes ~400ms. > > It doesn't take too many "botched eglot ranges" interacting with slow `thingatpt' misbehavior to add up to a > 10s delay. > > I'm not certain of the best solution. A few ideas, from hardest to easiest: > > Teach eglot textDocument/diagnostic > ++++++++++++++++++++++++++++ > > textDocument/publishDiagnostics messages arrive way too frequently IMO, after every buffer change of any > kind. They are pushed to eglot from the LSP server, and if they contain hundreds of errors, this becomes > very inefficient (re-painting with flymake the same hundreds of regions over and over after each keystroke). > > The best solution here would probably be to adopt "pull" diagnostics using textDocument/diagnostic, perhaps > in an idle-timer whose duration the user can configure. I don't believe EGLOT can do diagnostic pulls at the > moment. > > Don't use thingatpt in `flymake-diag-region' > +++++++++++++++++++++++++++++++++ > > `flymake-diag-region' should perhaps not use thingapt, which is subject to the performance vagaries of the > major-mode and underlying file. I am uncertain why it relies on that. Perhaps the performance of those will be > improved with treesitter variants. > > Eglot could detect off-by-one diagnostics > +++++++++++++++++++++++++++++++ > > Hard to know the best heuristic, but lots of null effective ranges is a good one. > > Eglot can simply ignore null range diagnostics > +++++++++++++++++++++++++++++++++++ > > Eglot doesn't need to use `flymake-diag-region' to try to calculate an update range if it encounters a null > diagnostic range. It could simply drop those, as they are probably wrong anyway, and will shortly be updated. Adding João and Spencer to the discussion.
bug-gnu-emacs@HIDDEN:bug#77588; Package emacs.
Full text available.Received: (at submit) by debbugs.gnu.org; 6 Apr 2025 22:50:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 06 18:50:53 2025 Received: from localhost ([127.0.0.1]:50058 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1u1YpJ-00028Q-3E for submit <at> debbugs.gnu.org; Sun, 06 Apr 2025 18:50:53 -0400 Received: from lists.gnu.org ([2001:470:142::17]:60528) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <jdtsmith@HIDDEN>) id 1u1YpG-00027l-LB for submit <at> debbugs.gnu.org; Sun, 06 Apr 2025 18:50:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <jdtsmith@HIDDEN>) id 1u1Yp7-0001f1-NO for bug-gnu-emacs@HIDDEN; Sun, 06 Apr 2025 18:50:41 -0400 Received: from mail-il1-x131.google.com ([2607:f8b0:4864:20::131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <jdtsmith@HIDDEN>) id 1u1Yp5-0005L4-IY for bug-gnu-emacs@HIDDEN; Sun, 06 Apr 2025 18:50:41 -0400 Received: by mail-il1-x131.google.com with SMTP id e9e14a558f8ab-3d5ebc2b725so10027675ab.3 for <bug-gnu-emacs@HIDDEN>; Sun, 06 Apr 2025 15:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743979837; x=1744584637; darn=gnu.org; h=to:date:message-id:subject:mime-version:from:from:to:cc:subject :date:message-id:reply-to; bh=qIoQNB8cTOkQmTILeCBQFnu0znz5jXn2sF2XvvaAeaw=; b=fIdZl7cCkJ4isdm/Wlv+GmQdruKzzH5fcf4UbP09ISHlDDdRVRiVeOZVDuN8tHQUga d2/nGsHZ+0SVcfzO/wq/Y2RSWNIz0+6QoEavkiNlEGobhg5ecFBlvQtlUD5UoBHOFXAG BdcdwYF7tbHuQx0AyY1uUtPKYxHrLZ4KMHGfv7nN9oeO+8bdhxDyfvz/XZl9Y4depEPE xmhAA29/JLEGg0wLgb6ptXB0Pj8IsIzf8w9GUgHQYEfa2aMIcEuMi1L1Dz17biP1ykYy A+CvkIggelmPhGxSRe4obFMFz4xXs0vfy/xzaWEKJVInB1IFAB9aVmT0hHNZrc996OWM ltrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743979837; x=1744584637; h=to:date:message-id:subject:mime-version:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=qIoQNB8cTOkQmTILeCBQFnu0znz5jXn2sF2XvvaAeaw=; b=JpUPjd1kihHog9JQ4l2RubszFpLNPBBBuTElgIuQGmDl9wO7uvq2tKkPd+8FSzqkjg oOGV8QgRbIbW5yhfZaDvfXiKNiXj9+siPa8h6Dx2HcjlSwT8iMOjeVn69Cz1UTiwpWaf UweR53qOX/fIIlarSyJ8qp8OIYeMwncLxII//KYXrhZMrCR4+6Xy8g7yBG299hU4zd89 AYllaesz565v9Fl2CMnPGHey7m5Ca0avM1glCpVfSDkvVR+ypyHGLneD9XRksP2g/s4d HBuSDGDV2+J58Gzjrs6S0zgCKBo72UT7Tum+ldtBfR4kK3PH7PJ0M/+s9MBB0RjIjqEy NOcA== X-Gm-Message-State: AOJu0Ywb2xD8X4BPS+uW2Sy8ELs5/1LhPi0IyG2y2Lg4bTvPFIXoC4og eP4dI61h7oTif4cncXk4ps7L6iG4/sWUIaos7DQoauSpWWewAteXawFCMg== X-Gm-Gg: ASbGncuBDVl0cvgMWI4Y4kVqyrrDSo+9udINbDFdwMX00tRhc1FCh0xZVwkadI3bglN dD1KRNauOUrR6WmnCwHkzuesoaUGq2D/7WBVJoq32F3vY6Gj2O2zNE5brsuCkYbzeHXyxpoDAsj BELYKl3W7D/IodP66ZNJmERw9segNEr5XnYI1pcEdOijhHeHZPOASDWoVRx61+6AJL9y+X7XDge 5vDo1WhqfIZ4SUgrVY6n99SegsJ1aI0/RzjbwWIRdV0okmDMJ1C06Hwl6JBr4TQ89plzyyOuoTL oPgsRV60LK0+04MadDvPeBdnqSczFjqhmZLWioMtpWnp9YACxjjE+YF+ZcrzupFclvtSoOUEFr/ nv7jFdfVu9iD3Ib3C9dPyJwEf178= X-Google-Smtp-Source: AGHT+IFPlXrLgiBwcKg5AtdYXuqGurSdXSn4OCxIZZk3A00fH3UKmIT6AfquLnIU7SL95T6OA3qigw== X-Received: by 2002:a05:6e02:2218:b0:3d5:dec0:7c03 with SMTP id e9e14a558f8ab-3d6e534ad64mr114040475ab.12.1743979837132; Sun, 06 Apr 2025 15:50:37 -0700 (PDT) Received: from smtpclient.apple (cm-24-53-185-196.buckeyecom.net. [24.53.185.196]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4f4b5d247c7sm1997251173.89.2025.04.06.15.50.36 for <bug-gnu-emacs@HIDDEN> (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Apr 2025 15:50:36 -0700 (PDT) From: JD Smith <jdtsmith@HIDDEN> Content-Type: multipart/alternative; boundary="Apple-Mail=_B8DC673C-8435-40E0-93A5-DE6AD84F4615" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.400.131.1.6\)) Subject: Catastrophic slowdown in eglot via `flymake-diag-region' Message-Id: <CEA8A47E-2C5F-44A3-9909-DD342A7598A7@HIDDEN> Date: Sun, 6 Apr 2025 18:50:26 -0400 To: bug-gnu-emacs@HIDDEN X-Mailer: Apple Mail (2.3826.400.131.1.6) Received-SPF: pass client-ip=2607:f8b0:4864:20::131; envelope-from=jdtsmith@HIDDEN; helo=mail-il1-x131.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.0 (/) --Apple-Mail=_B8DC673C-8435-40E0-93A5-DE6AD84F4615 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I was diagnosing a painful eglot pause of about 10 seconds that would = reliably occur every few edits when completing candidates and working in = a long python file (~8k lines). I am using the basedpyright LSP server. = I narrowed this down to eglot's `textDocument/publishDiagnostics' = notification handler. The file I was working on has several hundred warnings and errors, so = that represents a substantial message from the LSP server. After a fair = bit of timing and sleuthing, I narrowed it down further. A few = interrelated misbehaviors combined to produce the huge intermittent = pauses: When completing text directly after entering/removing a newline, the = Diagnostics ranges received by eglot are out of date and "one line off". Some of these off-by-one-line diagnostic message ranges now land on = blank lines in the file, so the effective range there is empty. Seeing an empty range, eglot declares the server has "botched" it, so it = tries to make a by-hand recalculation of the range using = `flymake-diag-region'. `flymake-diag-region' for some reason calls (end-of-thing 'sexp) and = (end-of-thing 'symbol). =20 In some positions in a long python file these commands are VERY SLOW. = Rather than the typical call time of ~1ms, at these certain positions = `flymake-diag-region' takes ~400ms. It doesn't take too many "botched eglot ranges" interacting with slow = `thingatpt' misbehavior to add up to a 10s delay. I'm not certain of the best solution. A few ideas, from hardest to = easiest: Teach eglot textDocument/diagnostic ++++++++++++++++++++++++++++ textDocument/publishDiagnostics messages arrive way too frequently IMO, = after every buffer change of any kind. They are pushed to eglot from = the LSP server, and if they contain hundreds of errors, this becomes = very inefficient (re-painting with flymake the same hundreds of regions = over and over after each keystroke). The best solution here would probably be to adopt "pull" diagnostics = using textDocument/diagnostic, perhaps in an idle-timer whose duration = the user can configure. I don't believe EGLOT can do diagnostic pulls = at the moment. Don't use thingatpt in `flymake-diag-region' +++++++++++++++++++++++++++++++++ `flymake-diag-region' should perhaps not use thingapt, which is subject = to the performance vagaries of the major-mode and underlying file. I am = uncertain why it relies on that. Perhaps the performance of those will = be improved with treesitter variants. Eglot could detect off-by-one diagnostics +++++++++++++++++++++++++++++++ Hard to know the best heuristic, but lots of null effective ranges is a = good one. Eglot can simply ignore null range diagnostics +++++++++++++++++++++++++++++++++++ Eglot doesn't need to use `flymake-diag-region' to try to calculate an = update range if it encounters a null diagnostic range. It could simply = drop those, as they are probably wrong anyway, and will shortly be = updated. --Apple-Mail=_B8DC673C-8435-40E0-93A5-DE6AD84F4615 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: = after-white-space;"><div><br></div>I was diagnosing a painful eglot = pause of about 10 seconds that would reliably occur every few edits when = completing candidates and working in a long python file (~8k lines). = I am using the basedpyright LSP server. I narrowed this down = to eglot's `textDocument/publishDiagnostics' notification = handler.<div><br></div><div>The file I was working on has several = hundred warnings and errors, so that represents a substantial message = from the LSP server. After a fair bit of timing and sleuthing, I = narrowed it down further. A few interrelated misbehaviors combined = to produce the<span style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, = 0);"> </span><span style=3D"caret-color: rgb(0, 0, 0); color: = rgb(0, 0, 0);">huge </span>intermittent = pauses:<div><br></div><div><ol class=3D"MailOutline"><li>When completing = text directly after entering/removing a newline, the Diagnostics ranges = received by eglot are out of date and "one line off".</li><li>Some of = these off-by-one-line diagnostic message ranges now land on blank lines = in the file, so the effective range there is empty.</li><li>Seeing an = empty range, eglot declares the server has "botched" it, so it tries to = make a by-hand recalculation of the range using = `flymake-diag-region'.</li><li>`flymake-diag-region' for some reason = calls (end-of-thing 'sexp) and (end-of-thing 'symbol). = </li><li>In some positions in a long python file these commands = are VERY SLOW. Rather than the typical call time of ~1ms, at these = certain positions `flymake-diag-region' takes = ~400ms.</li></ol><div><br></div><div>It doesn't take too many "botched = eglot ranges" interacting with slow `thingatpt' misbehavior to add up to = a 10s delay.</div><div><br></div><div>I'm not certain of the best = solution. A few ideas, from hardest to = easiest:</div><div><br></div><div><span style=3D"caret-color: rgb(0, 0, = 0); color: rgb(0, 0, 0);">Teach eglot = textDocument/diagnostic</span></div><div>++++++++++++++++++++++++++++</div= ><div><br></div><div> <font = color=3D"#000000">textDocument/publishDiagnostics messages arrive way = too frequently IMO, after every buffer change of any kind. They = are pushed to eglot from the LSP server, and if they contain hundreds of = errors, this becomes very inefficient (re-painting with flymake the same = hundreds of regions over and over after each = keystroke).</font></div><div><font = color=3D"#000000"><br></font></div><div><font color=3D"#000000">The best = solution here would probably be to adopt "pull" diagnostics = using textDocument/diagnostic, perhaps in an idle-timer whose = duration the user can configure. I don't believe EGLOT can do = diagnostic pulls at the moment.</font></div><div><br></div><div>Don't = use thingatpt in = `flymake-diag-region'</div><div>+++++++++++++++++++++++++++++++++</div><di= v><br></div><div><span style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, = 0, 0);">`</span><font color=3D"#000000"><span style=3D"caret-color: = rgb(0, 0, 0);">flymake-diag-region' should perhaps not use thingapt, = which is subject to the performance vagaries of the major-mode and = underlying file. I am uncertain why it relies on that. = Perhaps the performance of those will be improved with treesitter = variants.</span></font></div><div><br></div><div>Eglot could detect = off-by-one = diagnostics</div><div>+++++++++++++++++++++++++++++++</div><div><br></div>= <div>Hard to know the best heuristic, but lots of null effective ranges = is a good one.</div><div><br></div><div>Eglot can simply ignore null = range = diagnostics</div><div>+++++++++++++++++++++++++++++++++++</div><div><br></= div><div><div><font color=3D"#000000">Eglot doesn't need to use `<span = style=3D"caret-color: rgb(0, 0, 0);">flymake-diag-region' to try to = calculate an update range if it encounters a null diagnostic range. = It could simply drop those, as they are probably wrong anyway, and = will shortly be = updated.</span></font></div></div><div><br></div></div></div></body></html= >= --Apple-Mail=_B8DC673C-8435-40E0-93A5-DE6AD84F4615--
JD Smith <jdtsmith@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#77588; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.