GNU bug report logs - #22143
24.5; Emacs blocked on long lines.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22143 in the body.
You can then email your comments to 22143 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Oleksandr Gavenko <gavenkoa <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.5; Emacs blocked on long lines.
Date: Fri, 11 Dec 2015 17:54:31 +0200

It is usual for me to open files with large lines in Emacs.

Some times this was log, or executable, or dump.

But usually this was minified .js or .json file.

And only thing I can do is to live with slowness or to kill Emacs and may be
clean corresponding desktop-mode entries for *bad* file.

I contribute piece to:

  http://emacs.stackexchange.com/questions/598/how-do-i-handle-files-with-extremely-long-lines

Here it is:

  (setq line-number-display-limit large-file-warning-threshold)
  (setq line-number-display-limit-width 200)

  (defun my--is-file-large ()
    "If buffer too large and my cause performance issue."
    (< large-file-warning-threshold (buffer-size)))

  (define-derived-mode my-large-file-mode fundamental-mode "LargeFile"
    "Fixes performance issues in Emacs for large files."
    ;; (setq buffer-read-only t)
    (setq bidi-display-reordering nil)
    (jit-lock-mode nil)
    (buffer-disable-undo)
    (set (make-variable-buffer-local 'global-hl-line-mode) nil)
    (set (make-variable-buffer-local 'line-number-mode) nil)
    (set (make-variable-buffer-local 'column-number-mode) nil) )

  (add-to-list 'magic-mode-alist (cons #'my--is-file-large #'my-large-file-mode))

But it is for a large files (but help me in many case because files with long
lines usually also themselves big). I didn't investigate how to make test for long
lines jet.

Even with those settings some files (usually minified .js libraries or .json
files) froze Emacs. I think due to js-mode, but...

I would like to hear solution for long line issue or a way to detect long
lines and prevent file from opening (or decide to risk if lines is not toooo
long).

My point instead of restarting Emacs daemon - safely warn about long lines so
I have a chance to preserve my Emacs session before Emacs die.

================================================================

(info "(emacs) Visiting") say about large files:

     If you try to visit a file larger than ‘large-file-warning-threshold’
  (the default is 10000000, which is about 10 megabytes), Emacs asks you
  for confirmation first.  You can answer ‘y’ to proceed with visiting the
  file.  Note, however, that Emacs cannot visit files that are larger than
  the maximum Emacs buffer size, which is limited by the amount of memory
  Emacs can allocate and by the integers that Emacs can represent (*note
  Buffers::).  If you try, Emacs displays an error message saying that the
  maximum buffer size has been exceeded.

User warned about large files. But there are nothing about long lines in
manual!

I would like to see corresponding entry in manual about long line handling:

  If you try to scroll to line longer then `long-line-warning-threshold' (the
  default is 10000, which is about 100 lines long), Emacs asks you for
  confirmation first.

In GNU Emacs 24.5.1 (x86_64-pc-linux-gnu, GTK+ Version 3.18.2)
 of 2015-10-24 on trouble, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11702000
System Description:	Debian GNU/Linux testing (stretch)

Configured using:
 `configure --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
 --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fstack-protector-strong -Wformat -Werror=format-security -Wall'
 CPPFLAGS=-D_FORTIFY_SOURCE=2 LDFLAGS=-Wl,-z,relro'

Important settings:
  value of $LC_TIME: en_DK.utf8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

-- 
Best regards!

Message #8 received at 22143 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Oleksandr Gavenko <gavenkoa <at> gmail.com>
Cc: 22143 <at> debbugs.gnu.org
Subject: Re: bug#22143: 24.5; Emacs blocked on long lines.
Date: Thu, 05 Apr 2018 16:41:41 +0200

Hello,

(First, I don't think this is wishlist.  Isn't it a bug when an editor
can't open a file or freezes when doing so?)

In the last weeks I had to look at my .gnus.registry.eieio.  It has a
line with ~ 400000 chars in my case. Opening it with Emacs is not fun.
Searching and scrolling is all extremely sluggish.  I then realized that
something simple like

#+begin_src emacs-lisp
(add-hook 'find-file-hook
          (defun my-find-file-care-about-long-lines ()
            (save-excursion
              (when (and (search-forward-regexp ".\\{2000\\}" 50000 t)
                         (y-or-n-p "Very long lines detected - enable longlines-mode? "))
                (require 'longlines)
                (longlines-mode +1)))))
#+end_src

seems to work around the problem quite nicely (though the rest is only a
heuristic).  But

  - longlines is obsolete, and
  - this requires user configuration.

When I tried to open the .gnus.registry.eieio file with other editors in
my Debian, most froze.  "kate" OTOH warned about the extremely long line
and added line breaks automatically - AFAIU this is quite the same as
what longlines does.

If we can't solve the issue with long lines in the display code, maybe
we should do something like this by default?  Just freezing or
presenting a buffer where most actions are very sluggish is not a good
choice.  Needing to kill an editor after opening the "wrong file" is not
good.


Thanks,

Michael.

Message #11 received at 22143 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 22143 <at> debbugs.gnu.org, gavenkoa <at> gmail.com
Subject: Re: bug#22143: 24.5; Emacs blocked on long lines.
Date: Thu, 05 Apr 2018 18:00:12 +0300

> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Date: Thu, 05 Apr 2018 16:41:41 +0200
> Cc: 22143 <at> debbugs.gnu.org
> 
> (First, I don't think this is wishlist.  Isn't it a bug when an editor
> can't open a file or freezes when doing so?)

This bug was marked wishlist because it asks for a new feature: a
warning about long lines when visiting a file.

The bug report about slow redisplay in this case and in similar cases
is bug#13675, and it is not marked wishlist.

> If we can't solve the issue with long lines in the display code, maybe
> we should do something like this by default?

Practical and detailed proposals, let alone patches, in that direction
will be most welcome.

But please don't forget that long lines could materialize not just
when a file is first visited, but also as result of its editing, or
more generally when some text is generated by editing, even if the
buffer doesn't visit any file.  Bonus points for displaying such
warnings in those cases as well.

TIA

Message #14 received at 22143 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 22143 <at> debbugs.gnu.org, gavenkoa <at> gmail.com
Subject: Re: bug#22143: 24.5; Emacs blocked on long lines.
Date: Thu, 05 Apr 2018 17:20:48 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> The bug report about slow redisplay in this case and in similar cases
> is bug#13675, and it is not marked wishlist.

Ah, thanks.  Didn't expect multiple reports about this.

> > If we can't solve the issue with long lines in the display code, maybe
> > we should do something like this by default?
>
> Practical and detailed proposals, let alone patches, in that direction
> will be most welcome.

Sorry about my ignorance, but - are there reliable means of handling
such buffers other than longlines?  Does longlines always behave sane
with such files?

Michael.

Message #17 received at 22143 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 22143 <at> debbugs.gnu.org, gavenkoa <at> gmail.com
Subject: Re: bug#22143: 24.5; Emacs blocked on long lines.
Date: Thu, 05 Apr 2018 18:34:40 +0300

> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: gavenkoa <at> gmail.com,  22143 <at> debbugs.gnu.org
> Date: Thu, 05 Apr 2018 17:20:48 +0200
> 
> > Practical and detailed proposals, let alone patches, in that direction
> > will be most welcome.
> 
> Sorry about my ignorance, but - are there reliable means of handling
> such buffers other than longlines?  Does longlines always behave sane
> with such files?

I don't know; it's possible.  I know that turning on truncate-lines
sometimes helps, but not always.  Some research is probably needed to
see if longlines-like solution is "good enough".

Also, if we decide that something like longlines is a reasonably good
solution, or the best we can come up with, I think we need a
stripped-down version of it, we don't need all the bells and whistles
of a full-fledged customizable minor mode.

Message #20 received at 22143 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Oleksandr Gavenko <gavenkoa <at> gmail.com>
Cc: 22143 <at> debbugs.gnu.org
Subject: Re: bug#22143: 24.5; Emacs blocked on long lines.
Date: Fri, 21 Aug 2020 06:53:24 -0700

Oleksandr Gavenko <gavenkoa <at> gmail.com> writes:

> It is usual for me to open files with large lines in Emacs.
>
> Some times this was log, or executable, or dump.
>
> But usually this was minified .js or .json file.
>
> And only thing I can do is to live with slowness or to kill Emacs and may be
> clean corresponding desktop-mode entries for *bad* file.
>
> I contribute piece to:
>
>   http://emacs.stackexchange.com/questions/598/how-do-i-handle-files-with-extremely-long-lines
>
> Here it is:
>
>   (setq line-number-display-limit large-file-warning-threshold)
>   (setq line-number-display-limit-width 200)
>
>   (defun my--is-file-large ()
>     "If buffer too large and my cause performance issue."
>     (< large-file-warning-threshold (buffer-size)))
>
>   (define-derived-mode my-large-file-mode fundamental-mode "LargeFile"
>     "Fixes performance issues in Emacs for large files."
>     ;; (setq buffer-read-only t)
>     (setq bidi-display-reordering nil)
>     (jit-lock-mode nil)
>     (buffer-disable-undo)
>     (set (make-variable-buffer-local 'global-hl-line-mode) nil)
>     (set (make-variable-buffer-local 'line-number-mode) nil)
>     (set (make-variable-buffer-local 'column-number-mode) nil) )
>
>   (add-to-list 'magic-mode-alist (cons #'my--is-file-large #'my-large-file-mode))
>
> But it is for a large files (but help me in many case because files with long
> lines usually also themselves big). I didn't investigate how to make test for long
> lines jet.
>
> Even with those settings some files (usually minified .js libraries or .json
> files) froze Emacs. I think due to js-mode, but...
>
> I would like to hear solution for long line issue or a way to detect long
> lines and prevent file from opening (or decide to risk if lines is not toooo
> long).
>
> My point instead of restarting Emacs daemon - safely warn about long lines so
> I have a chance to preserve my Emacs session before Emacs die.

Is this better with so-long.el?

etc/PROBLEMS says:

*** Editing files with very long lines is slow.

For example, simply moving through a file that contains hundreds of
thousands of characters per line is slow, and consumes a lot of CPU.
This is a known limitation of Emacs with no solution at this time.

Is there anything more we can/should do in this case short of rewriting
the display engine?  Does it make sense to track this limitation in a
bug report?

Best regards,
Stefan Kangas

Message #23 received at 22143 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 22143 <at> debbugs.gnu.org, gavenkoa <at> gmail.com
Subject: Re: bug#22143: 24.5; Emacs blocked on long lines.
Date: Fri, 21 Aug 2020 17:01:43 +0300

> From: Stefan Kangas <stefan <at> marxist.se>
> Date: Fri, 21 Aug 2020 06:53:24 -0700
> Cc: 22143 <at> debbugs.gnu.org
> 
> Is this better with so-long.el?

No, I think so-long.el is better, and in any case so-long.el is our
canonical solution (or, rather, band-aid) for this annoying problem.

> etc/PROBLEMS says:
> 
> *** Editing files with very long lines is slow.
> 
> For example, simply moving through a file that contains hundreds of
> thousands of characters per line is slow, and consumes a lot of CPU.
> This is a known limitation of Emacs with no solution at this time.
> 
> Is there anything more we can/should do in this case short of rewriting
> the display engine?

Maybe there is, but I have yet to hear ideas for how to do that.
Ideas are still welcome.

> Does it make sense to track this limitation in a bug report?

We already have one: bug#13675.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.