GNU bug report logs - #29854
25.3; Eshell buffer editing gets slower as colored output grows

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29854 in the body.
You can then email your comments to 29854 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <ambrevar <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.3; Eshell buffer editing gets slower as colored output grows
Date: Tue, 26 Dec 2017 11:43:34 +0100

I originally reported this bug on Evil's issue tracker:

	https://github.com/emacs-evil/evil/issues/1006

The recipe with Evil is as follows:

- Start Emacs.

- `M-x eshell'.

- Insert lots of colored text. On Linux, you can call `dmesg -L=always' a few times.

- Go to normal state and press `x' wherever you can delete a character.

It should be possible to reproduce without Evil, I just could not figure
out a slow operation to replace the last step in the recipe.

According to the Evil maintainer, the issue comes from markers left
behind.

In fact, if I run `clone-buffer', the cloned buffer does not suffer from
this issue.  I think `clone-buffer' does not copy markers.

By the way, is there a way to scan for markers other than
`buffer-has-markers-at'?
Is there a way to remove them?


In GNU Emacs 25.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.26)
 of 2017-12-04 built on arojas
Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
System Description:	Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
<#secure method=pgpmime mode=sign>

-- 
Pierre Neidhardt

You will gain money by an immoral action.

Message #8 received at 29854 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Pierre Neidhardt <ambrevar <at> gmail.com>
Cc: 29854 <at> debbugs.gnu.org
Subject: Re: bug#29854: 25.3;
 Eshell buffer editing gets slower as colored output grows
Date: Sun, 31 Dec 2017 21:07:55 -0500

Pierre Neidhardt <ambrevar <at> gmail.com> writes:

> The recipe with Evil is as follows:
>
> - Start Emacs.
>
> - `M-x eshell'.
>
> - Insert lots of colored text. On Linux, you can call `dmesg -L=always' a few times.
>
> - Go to normal state and press `x' wherever you can delete a character.
>
> It should be possible to reproduce without Evil, I just could not figure
> out a slow operation to replace the last step in the recipe.
>
> According to the Evil maintainer, the issue comes from markers left
> behind.

So does this fix it?

--- i/lisp/ansi-color.el
+++ w/lisp/ansi-color.el
@@ -417,3 +417,7 @@ ansi-color-apply-on-region
 		 start-marker end-marker (ansi-color--find-face codes))
-	(setq ansi-color-context-region (if codes (list codes)))))))
+	(setq ansi-color-context-region (if codes (list codes)))))
+    ;; Clean up our temporary markers.
+    (unless (eq start-marker (cadr ansi-color-context-region))
+      (set-marker start-marker nil))
+    (set-marker end-marker nil)))

Also, I wonder if doing this would help also?

    (setq
     ansi-color-apply-face-function
     (lambda (beg end face)
       (when face
         (put-text-property beg end 'face face))))

That should make Emacs use text properties instead of overlays for the
colored text.

Message #11 received at 29854 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <ambrevar <at> gmail.com>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 29854 <at> debbugs.gnu.org
Subject: Re: bug#29854: 25.3; Eshell buffer editing gets slower as colored
 output grows
Date: Tue, 2 Jan 2018 13:40:00 +0100

[Message part 1 (text/plain, inline)]

The patch does not seem to do it.  Unless load order matters?

The redifinition of `ansi-color-apply-face-function` works however, plus it
comes at the bonus of making Eshell an order of magnitude faster!
I haven't tested estensively yet, but it seems to be a much better default.

On Mon, Jan 1, 2018 at 3:07 AM, Noam Postavsky <
npostavs <at> users.sourceforge.net> wrote:

> Pierre Neidhardt <ambrevar <at> gmail.com> writes:
>
> > The recipe with Evil is as follows:
> >
> > - Start Emacs.
> >
> > - `M-x eshell'.
> >
> > - Insert lots of colored text. On Linux, you can call `dmesg -L=always'
> a few times.
> >
> > - Go to normal state and press `x' wherever you can delete a character.
> >
> > It should be possible to reproduce without Evil, I just could not figure
> > out a slow operation to replace the last step in the recipe.
> >
> > According to the Evil maintainer, the issue comes from markers left
> > behind.
>
> So does this fix it?
>
> --- i/lisp/ansi-color.el
> +++ w/lisp/ansi-color.el
> @@ -417,3 +417,7 @@ ansi-color-apply-on-region
>                  start-marker end-marker (ansi-color--find-face codes))
> -       (setq ansi-color-context-region (if codes (list codes)))))))
> +       (setq ansi-color-context-region (if codes (list codes)))))
> +    ;; Clean up our temporary markers.
> +    (unless (eq start-marker (cadr ansi-color-context-region))
> +      (set-marker start-marker nil))
> +    (set-marker end-marker nil)))
>
> Also, I wonder if doing this would help also?
>
>     (setq
>      ansi-color-apply-face-function
>      (lambda (beg end face)
>        (when face
>          (put-text-property beg end 'face face))))
>
> That should make Emacs use text properties instead of overlays for the
> colored text.
>

[Message part 2 (text/html, inline)]

Message #14 received at 29854 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Pierre Neidhardt <ambrevar <at> gmail.com>
Cc: 29854 <at> debbugs.gnu.org
Subject: Re: bug#29854: 25.3;
 Eshell buffer editing gets slower as colored output grows
Date: Tue, 02 Jan 2018 08:50:51 -0500

Pierre Neidhardt <ambrevar <at> gmail.com> writes:

> The patch does not seem to do it.  Unless load order matters?

Load order should not matter.

> The redifinition of `ansi-color-apply-face-function` works however,
> plus it comes at the bonus of making Eshell an order of magnitude
> faster!
> I haven't tested estensively yet, but it seems to be a much better
> default.

Aha, maybe it's not a question of markers, but rather another case of
Bug#26051 "overlays may make emacs very slow".

(anyway, ansi-color-apply-to-region doesn't set markers per colour
boundary; it's just 1 or 2 markers per call (eshell calls it once per
~4096 bytes))

The question is then whether there is some case where using overlays is
necessary, or is it just a pessimization?  For reference, I took the
text property using definition from man.el (so it at least works for
that case):

    (defun Man-fontify-manpage ()
      [...]
      (let ((ansi-color-apply-face-function
             (lambda (beg end face)
               (when face
                 (put-text-property beg end 'face face))))

Message #17 received at 29854 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <ambrevar <at> gmail.com>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 29854 <at> debbugs.gnu.org
Subject: Re: bug#29854: 25.3;
 Eshell buffer editing gets slower as colored output grows
Date: Wed, 03 Jan 2018 12:46:26 +0100

[Message part 1 (text/plain, inline)]

> The question is then whether there is some case where using overlays is
> necessary, or is it just a pessimization?  For reference, I took the
> text property using definition from man.el (so it at least works for
> that case):

Any idea what the original intent was?
I can't seem to see why an overlay would be needed for Eshell's output.

-- 
Pierre Neidhardt

Abstainer, n.:
	A weak person who yields to the temptation of denying himself a
	pleasure.
		-- Ambrose Bierce, "The Devil's Dictionary"

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 29854 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Pierre Neidhardt <ambrevar <at> gmail.com>
Cc: 29854 <at> debbugs.gnu.org
Subject: Re: bug#29854: 25.3;
 Eshell buffer editing gets slower as colored output grows
Date: Wed, 03 Jan 2018 08:33:53 -0500

Pierre Neidhardt <ambrevar <at> gmail.com> writes:

>> The question is then whether there is some case where using overlays is
>> necessary, or is it just a pessimization?  For reference, I took the
>> text property using definition from man.el (so it at least works for
>> that case):
>
> Any idea what the original intent was?
> I can't seem to see why an overlay would be needed for Eshell's output.

Hard to say.  It might just be a case of going with ansi-color.el's
default.  ansi-color-apply-on-region was changed to use overlays in [1:
0e3c1e3ea6], but it's unclear why.  The commit message just says

    (ansi-color-apply-on-region): Use extents or overlays instead of
    text-properties.

[1: 0e3c1e3ea6]: 2001-01-09 11:38:28 +0000
  (ansi-color-process-output): Use markers instead of positions for start and end of region.[...]
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=0e3c1e3ea6a27e20d0252661336afe9fc84b21f5

Message #23 received at 29854 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 29854 <at> debbugs.gnu.org, Pierre Neidhardt <ambrevar <at> gmail.com>
Subject: Re: bug#29854: 25.3;
 Eshell buffer editing gets slower as colored output grows
Date: Sat, 04 May 2019 15:49:19 -0400

tags 29854 fixed
close 29854 27.1
quit

Noam Postavsky <npostavs <at> users.sourceforge.net> writes:

> Pierre Neidhardt <ambrevar <at> gmail.com> writes:
>
>>> The question is then whether there is some case where using overlays is
>>> necessary, or is it just a pessimization?  For reference, I took the
>>> text property using definition from man.el (so it at least works for
>>> that case):
>>
>> Any idea what the original intent was?
>> I can't seem to see why an overlay would be needed for Eshell's output.
>
> Hard to say.  It might just be a case of going with ansi-color.el's
> default.  ansi-color-apply-on-region was changed to use overlays in [1:
> 0e3c1e3ea6], but it's unclear why.  The commit message just says
>
>     (ansi-color-apply-on-region): Use extents or overlays instead of
>     text-properties.

I've changed to use text properties (but font-lock-face rather than
face) in master.  We'll see if it causes trouble.

4fd9048e94 2019-05-04T15:33:20-04:00 "Avoid slow overlay ansi coloring in eshell (Bug#29854)"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=4fd9048e940d38364caf4abe9b209f9288c78544

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.