GNU bug report logs - #18530
24.3; Emacs unable to handle very large lines

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18530 in the body.
You can then email your comments to 18530 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Amit Barve <mtbar131 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3; Emacs unable to handle very large lines
Date: Mon, 22 Sep 2014 10:57:58 +0530

[Message part 1 (text/plain, inline)]

Hi,

I am a daily user of emacs. I am a big fan and supporter of emacs.
First of all I would like thank all developers of emacs for such a
wonderful editor!

I would like to report a bug in emacs.

Here goes the description of the bug:
Whenever I open a text file which has a very big line(a continuous line)
without a newline inside it (let's say first 1000000 space
separated numbers) then emacs is just unable to handle it.
Emacs is unable to show line count of buffer, instead it shows 'L??'
and general commands like 'move to end of file', 'move to start of line'
take a lot of time (something like a complete minute) to work.I tried by
removing
everything from my '.emacs' file but that didn't help!

I really want to use emacs and I can't even think of using some
other editor!

I am on 64 bit ubuntu and using emacs 24.3.1

Is there any way so that I can make emacs to handle such big files?

P.S: As I don't have a proper mail server configured on my system
I am just copying the content from 'M-x report-emacs-bug' here!

Thank you,
Amit Barve


In GNU Emacs 24.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.10.7)
 of 2014-03-07 on lamiak, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
System Description:    Ubuntu 14.04.1 LTS

Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--build' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/x86_64-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk3' '--with-toolkit-scroll-bars'
 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'
 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LANG: en_IN
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: iso-latin-1-unix
  default enable-multibyte-characters: t

Major mode: C/l

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Recent input:
C-x 1 M-x e <backspace> r e p <tab> o <tab> r t <tab>
<return>

Recent messages:
Loading 00debian-vars...done
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...
Loading /var/cache/dictionaries-common/emacsen-ispell-dicts.el (source)...
Error while loading 50dictionaries-common: Symbol's value as variable is
void: debian-aspell-only-dictionary-alist
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading cc-langs...done
Loading vc-git...done
Making completion list... [2 times]

Load-path shadows:
/usr/share/emacs/24.3/site-lisp/debian-startup hides
/usr/share/emacs/site-lisp/debian-startup

Features:
(shadow sort gnus-util mail-extr emacsbug message cl-macs gv format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils help-mode vc-git cc-langs cl cl-lib
cc-mode cc-fonts easymenu cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

[Message part 2 (text/html, inline)]

Message #10 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Alexis <flexibeast <at> gmail.com>
To: 18530 <at> debbugs.gnu.org
Cc: Amit Barve <mtbar131 <at> gmail.com>
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Sun, 28 Sep 2014 17:16:06 +1000

Amit Barve writes:

> Here goes the description of the bug: Whenever I open a text file
> which has a very big line(a continuous line) without a newline inside
> it (let's say first 1000000 space separated numbers) then emacs is
> just unable to handle it.  Emacs is unable to show line count of
> buffer, instead it shows 'L??'  and general commands like 'move to end
> of file', 'move to start of line' take a lot of time (something like a
> complete minute) to work.I tried by removing everything from my
> '.emacs' file but that didn't help!

i've been able to reproduce this on both 24.3.3 and 24.3.93.2 (both
manually compiled on 64-bit Debian Wheezy running on an i5), having
created the utf-8 file test.txt with:

    for i in `seq 1 1000000`; do echo -n "$i " >> test.txt; done

then loading that file into an `emacs -Q` session. Moving from one end
of the buffer takes 15-20 seconds.

If i `toggle-truncate-lines`, the line count is correctly listed as 1,
but trying to move to the end of the line only results in displaying
complete numbers up to and including '194455', even though the column
number on the mode-line says that point is at the line's end. The
highest column number at which point is still visible is 1250082; after
that, moving point further towards the end of the line makes the cursor
no longer visible.

If i call `fill-paragraph` on that line, i get a line count of 98882,
and moving from one end of the buffer to the other only takes a second
or so.

Message #13 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Alexis <flexibeast <at> gmail.com>
To: 18530 <at> debbugs.gnu.org
Cc: Amit Barve <mtbar131 <at> gmail.com>
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Sun, 28 Sep 2014 20:10:45 +1000

Amit Barve writes:

> Thanks a lot Alexis for your reply!

You're welcome. :-)

> "toggle-truncate-lines" and "fil-paragraph" worked!!
> Though execution of both commands take a long time
> but after that I can move and edit very quickly!

Oh, well, i wasn't suggesting those commands as any kind of solution; i
just wanted to indicate what behaviours i observed when using those
commands.

> These commands work when emacs is opened with -nw option
> if I open emacs in GUI mode then line count is still shown as
> 'L??'

Hm, okay. In my own tests, above i was using Emacs in an X window, not
in a terminal ....

> Anyway thanks a lot! Now I can work with such big files!
> but I think this can still be considered as bug!
> because vim opens the file quite easily and without any
> modifications!

*nod* Yes, when i open test.txt in Vim, moving to the end of the line
with $, or to the beginning of the line with ^, both take less than a
second. And cursor movement with j, k etc. is basically
responsive. Whereas in Emacs, trying to move point up or down results in
20+ seconds of lag before point is actually moved.

As another data point: i tried disabling `visual-line-mode`, instead
loading longlines.el from EmacsWiki and enabling
`longlines-mode`. `longlines-mode` was /much/ more responsive - moving
point up, down, to beginning of line, or to end of line, all only had a
lag of 1-2 seconds.

Message #16 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alexis <flexibeast <at> gmail.com>
Cc: 18530 <at> debbugs.gnu.org, mtbar131 <at> gmail.com
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Sun, 28 Sep 2014 19:19:18 +0300

> From: Alexis <flexibeast <at> gmail.com>
> Date: Sun, 28 Sep 2014 17:16:06 +1000
> Cc: Amit Barve <mtbar131 <at> gmail.com>
> 
> 
> Amit Barve writes:
> 
> > Here goes the description of the bug: Whenever I open a text file
> > which has a very big line(a continuous line) without a newline inside
> > it (let's say first 1000000 space separated numbers) then emacs is
> > just unable to handle it.  Emacs is unable to show line count of
> > buffer, instead it shows 'L??'  and general commands like 'move to end
> > of file', 'move to start of line' take a lot of time (something like a
> > complete minute) to work.I tried by removing everything from my
> > '.emacs' file but that didn't help!
> 
> i've been able to reproduce this on both 24.3.3 and 24.3.93.2 (both
> manually compiled on 64-bit Debian Wheezy running on an i5), having
> created the utf-8 file test.txt with:
> 
>     for i in `seq 1 1000000`; do echo -n "$i " >> test.txt; done
> 
> then loading that file into an `emacs -Q` session. Moving from one end
> of the buffer takes 15-20 seconds.

This is bug #13675, a well-known imitation of the current Emacs
display engine.

Message #19 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alexis <flexibeast <at> gmail.com>
Cc: 18530 <at> debbugs.gnu.org, mtbar131 <at> gmail.com
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Sun, 28 Sep 2014 19:21:13 +0300

> From: Alexis <flexibeast <at> gmail.com>
> Date: Sun, 28 Sep 2014 20:10:45 +1000
> Cc: Amit Barve <mtbar131 <at> gmail.com>
> 
> *nod* Yes, when i open test.txt in Vim, moving to the end of the line
> with $, or to the beginning of the line with ^, both take less than a
> second. And cursor movement with j, k etc. is basically
> responsive. Whereas in Emacs, trying to move point up or down results in
> 20+ seconds of lag before point is actually moved.

Does Vim support different font sizes on the same line, inline images,
display properties, overlays, invisible text, etc.?  These features is
what makes this particular case hard; other editors don't need to
handle all that complexity.

Message #22 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Amit Barve <mtbar131 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 18530 <at> debbugs.gnu.org, Alexis <flexibeast <at> gmail.com>
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Mon, 29 Sep 2014 06:56:53 +0530

[Message part 1 (text/plain, inline)]

Eil I accept that emacs is way powerful than vim and other
editors. Even I am also a strong supporter of emacs.

I filed this bug because for me it was very difficult to work
with "long lines" and I don't want to use some other editor
just for this purpose.

I think if we can get over this limitation of emacs it will
make emacs a little more powerful!


On Sun, Sep 28, 2014 at 9:51 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Alexis <flexibeast <at> gmail.com>
> > Date: Sun, 28 Sep 2014 20:10:45 +1000
> > Cc: Amit Barve <mtbar131 <at> gmail.com>
> >
> > *nod* Yes, when i open test.txt in Vim, moving to the end of the line
> > with $, or to the beginning of the line with ^, both take less than a
> > second. And cursor movement with j, k etc. is basically
> > responsive. Whereas in Emacs, trying to move point up or down results in
> > 20+ seconds of lag before point is actually moved.
>
> Does Vim support different font sizes on the same line, inline images,
> display properties, overlays, invisible text, etc.?  These features is
> what makes this particular case hard; other editors don't need to
> handle all that complexity.
>

[Message part 2 (text/html, inline)]

Message #25 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Amit Barve <mtbar131 <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Alexis <flexibeast <at> gmail.com>,
 18530 <at> debbugs.gnu.org
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Sun, 28 Sep 2014 22:30:53 -0400

> I think if we can get over this limitation of Emacs it will
> make Emacs a little more powerful!

Yes, we all agree.  Eli is probably the most knowledgeable about fixing
this bug.  Maybe some kind of caching could get us there.


        Stefan

Message #28 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 18530 <at> debbugs.gnu.org, flexibeast <at> gmail.com, mtbar131 <at> gmail.com
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Mon, 29 Sep 2014 05:42:37 +0300

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  18530 <at> debbugs.gnu.org,  Alexis <flexibeast <at> gmail.com>
> Date: Sun, 28 Sep 2014 22:30:53 -0400
> 
> > I think if we can get over this limitation of Emacs it will
> > make Emacs a little more powerful!
> 
> Yes, we all agree.  Eli is probably the most knowledgeable about fixing
> this bug.

Eli has actually filed the original bug about this.

> Maybe some kind of caching could get us there.

Not sure how that could help.  Anyway, this issue is constantly on my
mind, so "some day" I might have an idea.  Or maybe someone else will.

Message #31 received at 18530 <at> debbugs.gnu.org (full text, mbox):

From: Amit Barve <mtbar131 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 18530 <at> debbugs.gnu.org, Alexis <flexibeast <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#18530: 24.3; Emacs unable to handle very large lines
Date: Mon, 29 Sep 2014 08:21:08 +0530

[Message part 1 (text/plain, inline)]

Thanks a lot!
I hope that this will get fixed soon
Till then we can use "long-line mode!"

On Mon, Sep 29, 2014 at 8:12 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> > Cc: Eli Zaretskii <eliz <at> gnu.org>,  18530 <at> debbugs.gnu.org,  Alexis <
> flexibeast <at> gmail.com>
> > Date: Sun, 28 Sep 2014 22:30:53 -0400
> >
> > > I think if we can get over this limitation of Emacs it will
> > > make Emacs a little more powerful!
> >
> > Yes, we all agree.  Eli is probably the most knowledgeable about fixing
> > this bug.
>
> Eli has actually filed the original bug about this.
>
> > Maybe some kind of caching could get us there.
>
> Not sure how that could help.  Anyway, this issue is constantly on my
> mind, so "some day" I might have an idea.  Or maybe someone else will.
>

Thanks
Amit Barve

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.