GNU bug report logs - #47589
[PATCH] 27.1; Freeze on long lines in compilation-shell-minor-mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 47589 in the body.
You can then email your comments to 47589 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Anton Tayanovskyy <anton.tayanovskyy <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH] 27.1; Freeze on long lines in compilation-shell-minor-mode
Date: Sat, 3 Apr 2021 22:52:36 -0400

To reproduce the issue of Emacs freezing on long lines, do the
following:

```
M-x shell
M-x compilation-shell-minor-mode
$ python3 -c 'print("x"*1024*8)'
```

Depending on the machine configuration, make your lines longer. 1024*8
is enough to freeze my MacBook Pro 2019 visibly but I need more
characters to slow down an XPS-13 running Ubuntu.

In the real world the issue comes up when I'm enjoying my
compilation-shell-minor-mode buffer but accidentally cat or print some
data with very long lines. Then Emacs freezes and I experience one of my
very rare moments of unhappiness with the editor.

Using `M-x profiler-start, profiler-stop, profiler-report` the culprit
is `compilation-parse-errors`. The function loops over many (about 50)
patterns of potential error output and scans the buffer for them
repeatedly.

The patch edits the function to use a limited version of
`re-search-forward`. Instead of finding the pattern everywhere, it only
looks for the pattern in the first 1024 chars of every line. It seems
plausible that real-world compiler warnings would sit close to the
beginning of lines.

With the patch the experience (in combination with global-so-long-mode)
becomes tolerable; while not lightning fast, Emacs is a lot more
responsive in this situation and I as a user retain control to fix the
problem, such as comint-clear-buffer etc.

FWIW see also the PR https://github.com/emacs-mirror/emacs/pull/24/files


---
 lisp/progmodes/compile.el | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index 7a02c3a896..45c69f330c 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1535,7 +1535,7 @@ to `compilation-error-regexp-alist' if RULES is nil."
           (error "HYPERLINK should be an integer: %s" (nth 5 item)))

         (goto-char start)
-        (while (re-search-forward pat end t)
+        (while (compilation--re-search-forward-limited pat end 1024)
           (when (setq props (compilation-error-properties
                              file line end-line col end-col
                              (or type 2) fmt rule))
@@ -1597,6 +1597,28 @@ to `compilation-error-regexp-alist' if RULES is nil."
                (match-beginning mn) (match-end mn)
                'font-lock-face (cadr props)))))))))

+(defun compilation--re-search-forward-limited (regexp bound n)
+  "Like 're-search-forward limited to the first N chars per line.
+This avoids Emacs performance degradation on scanning excessively
+long lines of text.  It is reasonable when scanning for compiler
+warnings to expect to find them early on each line.  REGEXP and
+BOUND are as in 're-search-forward."
+  (let ((inhibit-field-text-motion t)
+        (found nil)
+        (orig-point (point)))
+    (while (and (null found)
+                (< (point) bound)
+                (not (eobp)))
+      (setq found (re-search-forward regexp
+                                     (max (point)
+                                          (min bound (+ (point-at-bol) n)))
+                                     t))
+      (when (null found)
+        (forward-line 1)))
+    (when (null found)
+      (goto-char orig-point))
+    found))
+
 (defvar-local compilation--parsed -1)

 (defun compilation--ensure-parse (limit)
-- 
2.28.0


--------------------------------------------------------------------------------

In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.21,
cairo version 1.16.0)
Windowing system distributor 'The X.Org Foundation', version 11.0.12009000
System Description: Ubuntu 20.04.2 LTS

Configured using:
 'configure
 --prefix=/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-emacs-27.1
 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft
 --with-cairo CFLAGS=-DMAC_OS_X_VERSION_MAX_ALLOWED=101200'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY
INOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER GMP

Message #8 received at 47589 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Anton Tayanovskyy <anton.tayanovskyy <at> gmail.com>
Cc: 47589 <at> debbugs.gnu.org
Subject: Re: bug#47589: [PATCH] 27.1; Freeze on long lines in
 compilation-shell-minor-mode
Date: Thu, 06 May 2021 12:51:53 +0200

Anton Tayanovskyy <anton.tayanovskyy <at> gmail.com> writes:

> The patch edits the function to use a limited version of
> `re-search-forward`. Instead of finding the pattern everywhere, it only
> looks for the pattern in the first 1024 chars of every line. It seems
> plausible that real-world compiler warnings would sit close to the
> beginning of lines.
>
> With the patch the experience (in combination with global-so-long-mode)
> becomes tolerable; while not lightning fast, Emacs is a lot more
> responsive in this situation and I as a user retain control to fix the
> problem, such as comint-clear-buffer etc.

I think that's an interesting approach to this problem.  I seem to
recall there being some work in this area, though, and I'm not able to
see the slowdowns in the example given here (even if I increase the
number hugely).

Would it be possible for you to test Emacs 28 and see whether you still
see the problem there?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #13 received at 47589 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Anton Tayanovskyy <anton.tayanovskyy <at> gmail.com>
Cc: 47589 <at> debbugs.gnu.org
Subject: Re: bug#47589: [PATCH] 27.1; Freeze on long lines in
 compilation-shell-minor-mode
Date: Sat, 05 Jun 2021 22:19:47 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> I think that's an interesting approach to this problem.  I seem to
> recall there being some work in this area, though, and I'm not able to
> see the slowdowns in the example given here (even if I increase the
> number hugely).
>
> Would it be possible for you to test Emacs 28 and see whether you still
> see the problem there?

More information was requested, but no response was given within a
month, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #18 received at 47589 <at> debbugs.gnu.org (full text, mbox):

From: Anton Tayanovskyy <anton.tayanovskyy <at> gmail.com>
To: 47589 <at> debbugs.gnu.org
Subject: not reproducing on emacs 28
Date: Fri, 11 Jun 2021 23:05:39 -0400

[Message part 1 (text/plain, inline)]

Thanks for looking into it.

I have tried to reproduce the slowdown on Ubuntu with Emacs 28 and 27.2 and
compared against my current patched 27.1 Emacs. The conclusion is that
something improved in a recent version of Emacs and the patch is not moving
the needle on this.

There is still a super-liner cost somewhere. But as a user I only hit this
when accidentally displaying a large file, and current behavior is
interactive enough that I can kill the process running in the shell.

Once I get a chance I would also like to try on Mac OS X where the problem
was more pronounced.

Thanks,

Anton

|--------------------+---------------------------------------+----------------|
| GNU Emacs 27.2     | time python3 -c 'print("x"*1024*256)' | real
0m20.805s |
| GNU Emacs 28.0.50  | time python3 -c 'print("x"*1024*32)'  | real
0m0.315s  |
| GNU Emacs 28.0.50  | time python3 -c 'print("x"*1024*64)'  | real
0m1.151s  |
| GNU Emacs 28.0.50  | time python3 -c 'print("x"*1024*128)' | real
0m6.168s  |
| GNU Emacs 28.0.50  | time python3 -c 'print("x"*1024*256)' | real
0m22.153s |
| GNU Emacs 27.1*    | time python3 -c 'print("x"*1024*32)'  | real
0m0.237s  |
| GNU Emacs 27.1*    | time python3 -c 'print("x"*1024*64)'  | real
0m1.108s  |
| GNU Emacs 27.1*    | time python3 -c 'print("x"*1024*128)' | real
0m5.172s  |
| GNU Emacs 27.1*    | time python3 -c 'print("x"*1024*256)' | real
0m24.369s |
| GNU Emacs 28.0.50* | time python3 -c 'print("x"*1024*32)'  | real
0m0.234s  |
| GNU Emacs 28.0.50* | time python3 -c 'print("x"*1024*64)'  | real
0m0.978s  |
| GNU Emacs 28.0.50* | time python3 -c 'print("x"*1024*128)' | real
0m4.716s  |
| GNU Emacs 28.0.50* | time python3 -c 'print("x"*1024*256)' | real
0m19.047s |

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.