GNU bug report logs - #12745
crash in bidi_pop_it during (idle) redisplay

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 12745 in the body.
You can then email your comments to 12745 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: bug-gnu-emacs <at> gnu.org
Cc: ami <at> fischman.org
Subject: crash in bidi_pop_it during (idle) redisplay
Date: Sat, 27 Oct 2012 20:31:08 -0700

[I'm forwarding this from emacs-devel; the original is at
 <http://lists.gnu.org/archive/html/emacs-devel/2012-10/msg00743.html>.

 The rest of this message is from Ami Fischman.]

Emacs 24.2 regularly crashes on me with the backtrace below.  I usually
start it with "emacs --daemon" and then run one emacsclient on each of two
X servers (a local one on :0 and a remote one through
NX<http://www.nomachine.com/>).
 This setup will be up/stable for anywhere between an hour and a few days,
and then the server aborts dumping core like this.  FWIW, emacs-version
says:
GNU Emacs 24.2.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
and I built it using this this configuration:
--with-x-toolkit=lucid --with-xpm --with-jpeg --with-tiff --with-gif
--with-png --with-x --without-dbug --without-gconf

Possibly of note is that this happens when I'm not doing anything with
emacs (usually I'll come back to one of my workstations and find the emacs
process is gone and a corefile has shown up).  Both of the X servers in
question are much longer lived than the emacs processes involved.

Cheers,
-a



(gdb) bt
#0  0x00007fa43b6f1707 in kill () at ../sysdeps/unix/syscall-template.S:82
#1  0x00000000004e93d2 in fatal_error_signal (sig=<optimized out>) at
emacs.c:366
#2  <signal handler called>
#3  0x00007fa43b6f1707 in kill () at ../sysdeps/unix/syscall-template.S:82
#4  0x00000000004e7452 in abort () at emacs.c:394
#5  0x00000000004923b5 in bidi_pop_it (bidi_it=<optimized out>) at
bidi.c:636
#6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769
#7  0x0000000000451f23 in next_overlay_string (it=0x7fff2251f1e0) at
xdisp.c:5223
#8  0x0000000000426ff7 in set_iterator_to_next (it=0x7fff2251f1e0,
reseat_p=<optimized out>) at xdisp.c:7188
#9  0x00000000004311ed in display_line (it=0x7fff2251f1e0) at xdisp.c:19519
#10 0x00000000004302d8 in try_window (window=<optimized out>, flags=1,
pos=...) at xdisp.c:16137
#11 0x000000000044ae4a in redisplay_window (window=82406597,
just_this_one_p=0) at xdisp.c:15662
#12 0x00000000004514e1 in redisplay_window_0 (window=9054) at xdisp.c:13748
#13 0x000000000055a046 in internal_condition_case_1 (bfun=0x4514c0
<redisplay_window_0>, arg=82406597, handlers=9957798, hfun=<optimized out>)
at eval.c:1552
#14 0x0000000000448376 in redisplay_windows (window=<optimized out>) at
xdisp.c:13728
#15 0x0000000000448385 in redisplay_windows (window=<optimized out>) at
xdisp.c:13720
#16 0x000000000042d3d0 in redisplay_internal () at xdisp.c:13305
#17 0x000000000042fb64 in redisplay_preserve_echo_area (from_where=9054) at
xdisp.c:13556
#18 0x00000000004f1a8a in detect_input_pending_run_timers
(do_display=<optimized out>) at keyboard.c:10512
#19 0x0000000000596414 in wait_reading_process_output
(time_limit=<optimized out>, microsecs=0, read_kbd=-1, do_display=1,
wait_for_cell=9755602, wait_proc=0x0, just_wait_proc=<optimized out>) at
process.c:4738
#20 0x00000000004f0954 in kbd_buffer_get_event (kbp=<optimized out>,
end_time=<optimized out>, used_mouse_menu=0x7fff22528354, kbp=<optimized
out>, end_time=<optimized out>) at keyboard.c:3855
#21 read_char (commandflag=<optimized out>, nmaps=9, maps=<optimized out>,
prev_event=9755602, used_mouse_menu=0x7fff22528354, end_time=<optimized
out>) at keyboard.c:2801
#22 0x00000000004ec170 in read_key_sequence (bufsize=30, keybuf=<optimized
out>, prompt=<optimized out>, dont_downcase_last=<optimized out>,
can_return_switch_frame=<optimized out>, fix_current_buffer=<optimized
out>) at keyboard.c:9328
#23 0x00000000004ea909 in command_loop_1 () at keyboard.c:1449
#24 0x000000000055b680 in internal_condition_case (bfun=0x4ea490
<command_loop_1>, handlers=9807890, hfun=<optimized out>) at eval.c:1514
#25 0x00000000004fbf06 in command_loop_2 (ignore=<optimized out>) at
keyboard.c:1160
#26 0x000000000055b0dc in internal_catch (tag=<optimized out>,
func=0x4fbee0 <command_loop_2>, arg=9755602) at eval.c:1271
#27 0x00000000004e9c39 in command_loop () at keyboard.c:1139
#28 recursive_edit_1 () at keyboard.c:759
#29 0x00000000004e9d55 in Frecursive_edit () at keyboard.c:823
#30 0x00000000004e8acd in main (argc=<error reading variable: Cannot access
memory at address 0x0>, argv=<optimized out>) at emacs.c:1715

Message #8 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 12745 <at> debbugs.gnu.org
Subject: Re:  crash in bidi_pop_it during (idle) redisplay
Date: Sun, 28 Oct 2012 00:49:23 -0700

[This is a copy of <http://lists.gnu.org/archive/html/emacs-devel/2012-10/msg00744.html>.]

Date: Sun, 28 Oct 2012 05:58:18 +0200
From: Eli Zaretskii <eliz <at> gnu.org>

> Date: Sat, 27 Oct 2012 19:25:49 -0700
> From: Ami Fischman <ami <at> fischman.org>
>
> Emacs 24.2 regularly crashes on me with the backtrace below.

Thank you for your report.

First, please file a bug about this via "M-x report-emacs-bug RET".

Second, please show the value of it->current in frame #7 (inside
next_overlay_string).

Third, could you please try the latest development code, and possibly
compile without optimizations?  Several bugs in this area were fixed
since 24.2 was released, so perhaps this is one of them.

Message #11 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 12745 <at> debbugs.gnu.org
Subject: Re:  crash in bidi_pop_it during (idle) redisplay
Date: Sun, 28 Oct 2012 00:50:08 -0700

[This is a copy of
 <http://lists.gnu.org/archive/html/emacs-devel/2012-10/msg00745.html>.]

From: Ami Fischman <ami <at> fischman.org>
Date: Sat, 27 Oct 2012 21:09:43 -0700

Thanks for your reply, Eli.


> First, please file a bug about this via "M-x report-emacs-bug RET".
>

When? :)
(after the crash, the process is gone, so can't M-x anything; before the
crash, there's nothing interesting to report :))
In case it's interesting, appended the info from r-e-b (from an emacs -Q
instance) to the bottom of this email.


> Second, please show the value of it->current in frame #7 (inside
> next_overlay_string).
>

(gdb) p it->current
$3 = {
  pos = {
    charpos = 1295,
    bytepos = 1295
  },
  overlay_string_index = 0,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}


> Third, could you please try the latest development code, and possibly
> compile without optimizations?


Will do & report back in a few days (or sooner, if the bug recurs in HEAD).

Cheers,
-a


In GNU Emacs 24.2.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
 of 2012-09-11 on <REDACTED>
Windowing system distributor `The X.Org Foundation', version 11.0.70000000
Configured using:
 `configure '--prefix=/usr/gmacs-24.2' '--mandir=${prefix}/share/man'
 '--infodir=${prefix}/share/info' '--with-x-toolkit=lucid' '--with-xpm'
 '--with-jpeg' '--with-tiff' '--with-gif' '--with-png' '--with-x'
 '--program-transform-name=s/emacs/gmacs/g' '--without-dbug'
 '--without-gconf' 'CC=clang
 -B/home/fischman/src/chromium/src/third_party/gold/' 'CFLAGS=-Wall -g
 -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LC_ALL:
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x r e p o r <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail regexp-opt rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils time-date tooltip ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer loaddefs button faces cus-face files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting x-toolkit x
multi-tty emacs)

Message #14 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sun, 28 Oct 2012 19:23:38 +0200

> Date: Sun, 28 Oct 2012 00:50:08 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> > First, please file a bug about this via "M-x report-emacs-bug RET".
> >
> 
> When? :)
> (after the crash, the process is gone, so can't M-x anything; before the
> crash, there's nothing interesting to report :))

That's not true, the important information is about your system,
environment, and packages loaded into Emacs, so a report from the same
kind of session will do.

> In case it's interesting, appended the info from r-e-b (from an emacs -Q
> instance) to the bottom of this email.

Thanks.  But I'd like to see the report from your normal session,
invoked just as you invoke those that crash.

> (gdb) p it->current
> $3 = {
>   pos = {
>     charpos = 1295,
>     bytepos = 1295
>   },
>   overlay_string_index = 0,
>   string_pos = {
>     charpos = -1,
>     bytepos = -1
>   },
>   dpvec_index = -1
> }

So this seems to say that there's at least one overlay string at
buffer position 1295.  Is that reasonable?  What was the current
buffer when this crashed?  You can find that out by typing this at GDB
prompt:

  (gdb) pp current_buffer->name_

(If "pp" doesn't work, type "source /path/to/emacs/src/.gdbinit" and
try again.)

You can display the relevant part of the text of that buffer like
this:

  (gdb) p current_buffer->text->beg[1200]@100

What does this show?

Also, what do the following commands produce?

  (gdb) frame 6
  (gdb) pgrowx it->glyph_row

> > Third, could you please try the latest development code, and possibly
> > compile without optimizations?
> 
> 
> Will do & report back in a few days (or sooner, if the bug recurs in HEAD).

Thanks, but please also show the above information for the version
that did crash.

Message #17 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sun, 28 Oct 2012 12:00:49 -0700

[Message part 1 (text/plain, inline)]

>
> Thanks.  But I'd like to see the report from your normal session,
> invoked just as you invoke those that crash.
>

Of course now I don't have such a session b/c I'm running w/ git HEAD :)

So this seems to say that there's at least one overlay string at
> buffer position 1295.  Is that reasonable?  What was the current
> buffer when this crashed?  You can find that out by typing this at GDB
> prompt:
>   (gdb) pp current_buffer->name_
>

(gdb) pp current_buffer->name_
Cannot access memory at address 0x8b6a00


>   (gdb) p current_buffer->text->beg[1200]@100
>

 (gdb) p current_buffer->text->beg[1200]@100
$1 = "num to avoid later static_cast in\n// PluginInstance.\nenum
MediaKeyError {\n  kUnknownError = 1,\n  kCl"
which tells me the current buffer was an edited version of
http://src.chromium.org/viewvc/chrome/trunk/src/webkit/media/crypto/ppapi/cdm_wrapper.cc?view=markup(which
I can't share in its entirety).  FWIW, there's nothing
non-7-bit-ascii in this file, and nothing that should have triggered any
bidi-specific logic.  It's just a cc-mode C++ file.

Possibly interestingly, if I print p current_buffer->text->beg[0]@100000 to
emit the entire buffer, I see this text starting at char 1675:
http://go", '\000' <repeats 2000 times>, "/b
Those 2000 NULs are definitely out of place (the URL should have started
with http://go/b) but I don't know if that's a debugging artifact, or what.

If I load the modified buffer into my HEAD session (overlays-at 1295)
returns nil.

Also, what do the following commands produce?
>   (gdb) frame 6
>   (gdb) pgrowx it->glyph_row
>
>>

> (gdb) frame 6

> #6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769

> 5769          bidi_pop_it (&it->bidi_it);

> (gdb) pgrowx it->glyph_row
You can't do that without a process to debug.

Cheers,
-a

[Message part 2 (text/html, inline)]

Message #20 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sun, 28 Oct 2012 21:50:26 +0200

> Date: Sun, 28 Oct 2012 12:00:49 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: 12745 <at> debbugs.gnu.org
> 
> > Thanks.  But I'd like to see the report from your normal session,
> > invoked just as you invoke those that crash.
> >
> 
> Of course now I don't have such a session b/c I'm running w/ git HEAD :)

If it loads the same .emacs, that is good enough.

> So this seems to say that there's at least one overlay string at
> > buffer position 1295.  Is that reasonable?  What was the current
> > buffer when this crashed?  You can find that out by typing this at GDB
> > prompt:
> >   (gdb) pp current_buffer->name_
> >
> 
> (gdb) pp current_buffer->name_
> Cannot access memory at address 0x8b6a00

How about this:

  (gdb) p current_buffer->name_
  (gdb) xtype

(Note: "p", not "pp".)

If the last command says it's a Lisp string, display the contents of
'struct Lisp_String' whose address it shows.

> >   (gdb) p current_buffer->text->beg[1200]@100
> >
> 
>  (gdb) p current_buffer->text->beg[1200]@100
> $1 = "num to avoid later static_cast in\n// PluginInstance.\nenum
> MediaKeyError {\n  kUnknownError = 1,\n  kCl"
> which tells me the current buffer was an edited version of
> http://src.chromium.org/viewvc/chrome/trunk/src/webkit/media/crypto/ppapi/cdm_wrapper.cc?view=markup(which

Did that buffer have any minor mode or some other optional feature
turned on, in addition to C++ Mode?

> FWIW, there's nothing non-7-bit-ascii in this file, and nothing that
> should have triggered any bidi-specific logic.  It's just a cc-mode
> C++ file.

This crash is not about bidi.  It is about handling overlays.  That it
barfs inside a function whose name begins with "bidi" is just sheer
luck--or lack thereof.

> Possibly interestingly, if I print p current_buffer->text->beg[0]@100000 to
> emit the entire buffer, I see this text starting at char 1675:
> http://go", '\000' <repeats 2000 times>, "/b
> Those 2000 NULs are definitely out of place (the URL should have started
> with http://go/b) but I don't know if that's a debugging artifact, or what.

This could be the gap, you should see its position and size like this:

  (gdb) p current_buffer->text->gpt
  (gdb) p current_buffer->text->gap_size

> If I load the modified buffer into my HEAD session (overlays-at 1295)
> returns nil.

That's what I suspected, this is the reason for the abort: Emacs
thinks there's an overlay string at that position, when there is none,
actually.

> > (gdb) frame 6
> 
> > #6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769
> 
> > 5769          bidi_pop_it (&it->bidi_it);
> 
> > (gdb) pgrowx it->glyph_row
> You can't do that without a process to debug.

So are you debugging a core dump?

Message #23 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sun, 28 Oct 2012 21:26:55 -0700

[Message part 1 (text/plain, inline)]

> If it loads the same .emacs, that is good enough.
>

Ok, attached.

> So this seems to say that there's at least one overlay string at
> > > buffer position 1295.  Is that reasonable?  What was the current
> > > buffer when this crashed?  You can find that out by typing this at GDB
> > > prompt:
> > >   (gdb) pp current_buffer->name_
> > (gdb) pp current_buffer->name_
> > Cannot access memory at address 0x8b6a00
> How about this:
>   (gdb) p current_buffer->name_
>   (gdb) xtype
> (Note: "p", not "pp".)
> If the last command says it's a Lisp string, display the contents of
> 'struct Lisp_String' whose address it shows.
>

(gdb) p current_buffer->name_
$22 = 101548849
(gdb) xtype
Lisp_String
(gdb) xstring current_buffer->name_
$23 = (struct Lisp_String *) 0x60d8330
  "cdm_wrapper.cc"


> > >   (gdb) p current_buffer->text->beg[1200]@100
> >  (gdb) p current_buffer->text->beg[1200]@100
> > $1 = "num to avoid later static_cast in\n// PluginInstance.\nenum
> > MediaKeyError {\n  kUnknownError = 1,\n  kCl"
> > which tells me the current buffer was an edited version of
> >
> http://src.chromium.org/viewvc/chrome/trunk/src/webkit/media/crypto/ppapi/cdm_wrapper.cc?view=markup(which
> Did that buffer have any minor mode or some other optional feature
> turned on, in addition to C++ Mode?


See attached b-g-e.txt, in which the current buffer is the same .cc file in
my HEAD session loading the same .emacs as the crashed one.

> Possibly interestingly, if I print p current_buffer->text->beg[0]@100000
> to
> > emit the entire buffer, I see this text starting at char 1675:
> > http://go", '\000' <repeats 2000 times>, "/b
> > Those 2000 NULs are definitely out of place (the URL should have started
> > with http://go/b) but I don't know if that's a debugging artifact, or
> what.
>
> This could be the gap, you should see its position and size like this:
>
>   (gdb) p current_buffer->text->gpt
>   (gdb) p current_buffer->text->gap_size


Yep, looks like it:
(gdb) p current_buffer->text->gpt
$24 = 1684
(gdb) p current_buffer->text->gap_size
$25 = 2000

> > (gdb) frame 6
> > > #6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769
> > > 5769          bidi_pop_it (&it->bidi_it);
> > > (gdb) pgrowx it->glyph_row
> > You can't do that without a process to debug.
> So are you debugging a core dump?
>

Yes.

-a

[Message part 2 (text/html, inline)]

[b-g-e.txt (text/plain, attachment)]

Message #26 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 19:11:11 +0200

> Date: Sun, 28 Oct 2012 21:26:55 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: 12745 <at> debbugs.gnu.org
> 
> > If it loads the same .emacs, that is good enough.
> >
> 
> Ok, attached.

Thanks.

> (gdb) p current_buffer->name_
> $22 = 101548849
> (gdb) xtype
> Lisp_String
> (gdb) xstring current_buffer->name_
> $23 = (struct Lisp_String *) 0x60d8330
>   "cdm_wrapper.cc"

OK, so we were looking at the correct buffer.

> > Did that buffer have any minor mode or some other optional feature
> > turned on, in addition to C++ Mode?
> 
> 
> See attached b-g-e.txt, in which the current buffer is the same .cc file in
> my HEAD session loading the same .emacs as the crashed one.

Fwew!  There are quite a few minor modes you have turned on that use
overlays to do their job.  Coupled with the fact that the problem
happens after hours of work, I don't think a reproducible recipe will
be possible any time soon.  Hmmm...

> > So are you debugging a core dump?
> >
> 
> Yes.

Well, I hope that now you are running under GDB to begin with, because
debugging a live Emacs process is so much easier than debugging a core
dump.

Thanks.

Message #29 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 10:56:48 -0700

[Message part 1 (text/plain, inline)]

In case it's useful, replacing
printf "%s: %d glyphs\n", ($area == 0 ? "LEFT" : $area == 2 ? "RIGHT" :
"TEXT"), $used
in pgrowx with:
printf "%d glyphs\n", $used
gives me:

(gdb) pgrowx (it->glyph_row)
4 glyphs
  0    0: CHAR[ ] str=3b06db1[0] blev=0,btyp=L w=6 a+d=10+2 face=22
  1    6: CHAR[ ] str=3b06db1[1] blev=0,btyp=L w=6 a+d=10+2 face=22
  2   12: CHAR[3] str=3b06db1[2] blev=0,btyp=L w=6 a+d=10+2 face=22
  3   18: CHAR[9] str=3b06db1[3] blev=0,btyp=L w=6 a+d=10+2 face=22
23 glyphs
  0   24: CHAR[ ] pos=1275 blev=0,btyp=L w=6 a+d=10+2 MB
  1   30: CHAR[ ] pos=1276 blev=0,btyp=L w=6 a+d=10+2 MB
  2   36: CHAR[k] pos=1277 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  3   42: CHAR[U] pos=1278 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  4   48: CHAR[n] pos=1279 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  5   54: CHAR[k] pos=1280 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  6   60: CHAR[n] pos=1281 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  7   66: CHAR[o] pos=1282 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  8   72: CHAR[w] pos=1283 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  9   78: CHAR[n] pos=1284 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 10   84: CHAR[E] pos=1285 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 11   90: CHAR[r] pos=1286 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 12   96: CHAR[r] pos=1287 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 13  102: CHAR[o] pos=1288 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 14  108: CHAR[r] pos=1289 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 15  114: CHAR[ ] pos=1290 blev=0,btyp=L w=6 a+d=10+2 MB
 16  120: CHAR[=] pos=1291 blev=0,btyp=L w=6 a+d=10+2 MB
 17  126: CHAR[ ] pos=1292 blev=0,btyp=L w=6 a+d=10+2 MB
 18  132: CHAR[1] pos=1293 blev=0,btyp=L w=6 a+d=10+2 MB
 19  138: CHAR[,] pos=1294 blev=0,btyp=L w=6 a+d=10+2 MB
 20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
 21  150: STRETCH[12+10] str=61335d1[1] w=354 a+d=10+2 MB
 22  504: IMAGE[0] str=61335d1[2] w=6 a+d=10+2 MB slice=0,0,6,12

(I imagine the IMAGE at the end there is
fci-mode<https://github.com/alpaker/Fill-Column-Indicator>
)

Cheers,
-a

[Message part 2 (text/html, inline)]

Message #32 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 20:10:48 +0200

> Date: Mon, 29 Oct 2012 10:56:48 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: 12745 <at> debbugs.gnu.org
> 
> In case it's useful, replacing
> printf "%s: %d glyphs\n", ($area == 0 ? "LEFT" : $area == 2 ? "RIGHT" :
> "TEXT"), $used
> in pgrowx with:
> printf "%d glyphs\n", $used
> gives me:

Yes, it's useful, thanks.

>  19  138: CHAR[,] pos=1294 blev=0,btyp=L w=6 a+d=10+2 MB
>  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
>  21  150: STRETCH[12+10] str=61335d1[1] w=354 a+d=10+2 MB
>  22  504: IMAGE[0] str=61335d1[2] w=6 a+d=10+2 MB slice=0,0,6,12
> 
> (I imagine the IMAGE at the end there is
> fci-mode<https://github.com/alpaker/Fill-Column-Indicator>
> )

Yes, and so is the STRETCH glyph.  Can you figure out what is the
string before that, viz.:

>  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB

This is a display string or an overlay string, but where does it come
from, and which mode puts it there?

Message #35 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 11:29:28 -0700

[Message part 1 (text/plain, inline)]

On Mon, Oct 29, 2012 at 11:10 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> Yes, and so is the STRETCH glyph.  Can you figure out what is the
> string before that, viz.:
> >  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
> This is a display string or an overlay string, but where does it come
> from, and which mode puts it there?
>

I'm groping in the dark, but AFAICT it's a null str?
(gdb) p/x (it->glyph_row->glyphs[1][20].object)
$39 = 0x61334f1
(gdb) p *(it->glyph_row->glyphs[1][20].object)
$40 = 0
I'm probably Doing It Wrong (tm), though.

The only guess I have is that I had entered a trailing space after that
comma, which would highlight it as trailing whitespace because of this
snippet from my .emacs:
(highlight-regexp "[ \t]+$" 'hi-blue)
using hi-lock-mode.

[Message part 2 (text/html, inline)]

Message #38 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 20:55:48 +0200

> Date: Mon, 29 Oct 2012 11:29:28 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: 12745 <at> debbugs.gnu.org
> 
> I'm groping in the dark, but AFAICT it's a null str?

No, it looks like a string with a blank character.

> (gdb) p/x (it->glyph_row->glyphs[1][20].object)
> $39 = 0x61334f1
> (gdb) p *(it->glyph_row->glyphs[1][20].object)
> $40 = 0

To be sure, try this:

  (gdb) p it->glyph_row->glyphs[1][20].object
  (gdb) xtype

If I'm right, it should tell that the object is a Lisp string, and you
can then display it like you did with the name of the buffer.

> The only guess I have is that I had entered a trailing space after that
> comma, which would highlight it as trailing whitespace because of this
> snippet from my .emacs:
> (highlight-regexp "[ \t]+$" 'hi-blue)
> using hi-lock-mode.

Probably.  But then why isn't there a "face=" part there?  hi-blue is
not the default face, is it?

Message #41 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Alp Aker <alptekin.aker <at> gmail.com>
To: Ami Fischman <ami <at> fischman.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 14:56:49 -0400

>> Yes, and so is the STRETCH glyph.  Can you figure out what is the
>> string before that, viz.:
>> >  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
>> This is a display string or an overlay string, but where does it come
>> from, and which mode puts it there?
>
>
> I'm groping in the dark, but AFAICT it's a null str?
> (gdb) p/x (it->glyph_row->glyphs[1][20].object)
> $39 = 0x61334f1
> (gdb) p *(it->glyph_row->glyphs[1][20].object)
> $40 = 0

That string is likely also from fci-mode.  The overlay string that
mode puts at the end of each line has the structure <character> +
<stretch glyph> + <image>, where by default <character> is U+E000.  It
shouldn't be null.

Message #44 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Alp Aker <alptekin.aker <at> gmail.com>
To: Ami Fischman <ami <at> fischman.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 15:09:22 -0400

> That string is likely also from fci-mode.  The overlay string that
> mode puts at the end of each line has the structure <character> +
> <stretch glyph> + <image>, where by default <character> is U+E000.  It
> shouldn't be null.

And if I'm reading things right, the display table entry for U+E000
should be [32] in this situation.

Message #47 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Alp Aker <alptekin.aker <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 12:23:25 -0700

[Message part 1 (text/plain, inline)]

More poking:

(gdb) p it->glyph_row->glyphs[1][20].object
$44 = 101922033
(gdb) xtype
Lisp_String
(gdb) xstring it->glyph_row->glyphs[1][20].object
$45 = (struct Lisp_String *) 0x61334f0
  ""

(gdb) p ((struct Lisp_String *)0x61334f0)->data
$48 = (unsigned char *) 0x4fe9388 ""
(gdb) p ((struct Lisp_String *)0x61334f0)->data[0]
$49 = 238 '\356'
(gdb) p ((struct Lisp_String *)0x61334f0)->data[1]
$50 = 128 '\200'
(gdb) p ((struct Lisp_String *)0x61334f0)->data[2]
$51 = 128 '\200'
(gdb) p ((struct Lisp_String *)0x61334f0)->data[3]
$52 = 0 '\000'

[Message part 2 (text/html, inline)]

Message #50 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 22:24:12 +0200

> Date: Mon, 29 Oct 2012 12:23:25 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 12745 <at> debbugs.gnu.org
> 
> (gdb) p it->glyph_row->glyphs[1][20].object
> $44 = 101922033
> (gdb) xtype
> Lisp_String
> (gdb) xstring it->glyph_row->glyphs[1][20].object
> $45 = (struct Lisp_String *) 0x61334f0
>   ""
> 
> (gdb) p ((struct Lisp_String *)0x61334f0)->data
> $48 = (unsigned char *) 0x4fe9388 ""

Thanks, this is clear.

Anyway, the problem happened at position 1295, which I believe is the
newline that ends this line.  I need to think...

Message #53 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alp Aker <alptekin.aker <at> gmail.com>
Cc: 12745 <at> debbugs.gnu.org, ami <at> fischman.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 22:25:22 +0200

> Date: Mon, 29 Oct 2012 15:09:22 -0400
> From: Alp Aker <alptekin.aker <at> gmail.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 12745 <at> debbugs.gnu.org
> 
> > That string is likely also from fci-mode.  The overlay string that
> > mode puts at the end of each line has the structure <character> +
> > <stretch glyph> + <image>, where by default <character> is U+E000.  It
> > shouldn't be null.
> 
> And if I'm reading things right, the display table entry for U+E000
> should be [32] in this situation.

Out of curiosity: what is the purpose of having in an overlay this
weird character and then to display it as a space?

Message #56 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Alp Aker <alptekin.aker <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org, ami <at> fischman.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 16:42:23 -0400

> Out of curiosity: what is the purpose of having in an overlay this
> weird character and then to display it as a space?

It's a hack to achieve compatibility with packages that change the
display table entry for the newline character.  When fci-mode is
active it puts, at the end of each line, a stretch glyph and then an
image with the line segment indicating the fill column.  When, e.g.,
whitespace-mode changes the display table entry for newlines to
display a '$' at the end of the line, this would show up *after* the
stretch glyph and image unless we do something to handle that case.
So what fci-mode tries to do is intercept changes to the display table
entry for newlines.  If something like whitespace mode changes newline
display, fci-mode changes the display table back to showing newlines
as blanks, and then updates the display table entry for U+E000 to (in
this case) [?$], so that the end-of-line indication appears where the
user expects it to be.

So the weird character is just a hook on which we can hang display
vectors intended for newlines.  But when the display settings for
newlines are normal it just gets displayed as a space. This is
admittedly a gross manoeuvre, but I couldn't think of any other way to
handle the issue.

Message #59 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: ami <at> fischman.org
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 22:57:30 +0200

> Date: Mon, 29 Oct 2012 22:24:12 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
> 
> Anyway, the problem happened at position 1295, which I believe is the
> newline that ends this line.  I need to think...

Ami, can you put a breakpoint in the function init_from_display_pos
and see if it ever breaks in your usage of C++ buffers?  There's
something suspicious going on in this function, but since I cannot
trigger its entry here (I even tried several of the minor modes you
have on, including fci-mode, linum-mode, and hi-lock), I'm not sure if
looking into this isn't a wild goose chase.

If you succeed in triggering the breaks inside that function, please
show the backtrace and the values of it->current and of
it->n_overlay_strings when the function returns.

Message #62 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alp Aker <alptekin.aker <at> gmail.com>
Cc: 12745 <at> debbugs.gnu.org, ami <at> fischman.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 22:59:51 +0200

> Date: Mon, 29 Oct 2012 16:42:23 -0400
> From: Alp Aker <alptekin.aker <at> gmail.com>
> Cc: ami <at> fischman.org, 12745 <at> debbugs.gnu.org
> 
> So the weird character is just a hook on which we can hang display
> vectors intended for newlines.  But when the display settings for
> newlines are normal it just gets displayed as a space. This is
> admittedly a gross manoeuvre, but I couldn't think of any other way to
> handle the issue.

Couldn't you simply put the $ (or whatever) in the display string,
where you now put the u+e000 character?

Message #65 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 14:08:11 -0700

[Message part 1 (text/plain, inline)]

On Mon, Oct 29, 2012 at 1:57 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> Ami, can you put a breakpoint in the function init_from_display_pos
> and see if it ever breaks in your usage of C++ buffers?


It hasn't in a few minutes of usage.  Will follow up if it does.

-a

[Message part 2 (text/html, inline)]

Message #68 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Alp Aker <alptekin.aker <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org, ami <at> fischman.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 17:25:08 -0400

> Couldn't you simply put the $ (or whatever) in the display string,
> where you now put the u+e000 character?

Probably.  IIRC I found that (a) I needed to put a cursor property on
the overlay string to get sensible cursor movement at the end of
lines, but (b) putting the cursor property on a string that gets
displayed as a stretch glyph via its display property didn't work.  So
I needed to have a blank character there to put a cursor property on,
even when the newline has a nil display table entry.  At that point it
made the code simpler to use that char for visible newline display as
well, via display table manipulation.

But if this turns out to cause too many headaches for the redisplay
routine I can rewrite it along the lines you describe.

Message #71 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Alp Aker <alptekin.aker <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Mon, 29 Oct 2012 17:53:06 -0400

>> Couldn't you simply put the $ (or whatever) in the display string,
>> where you now put the u+e000 character?
>
> I needed to have a blank character there to put a cursor property on,
> even when the newline has a nil display table entry.  At that point it
> made the code simpler to use that char for visible newline display as
> well, via display table manipulation.

In other words, my earlier explanation wasn't accurate (it's been a
while since I thought about this code).  The blank character in the
overlay string is there for the sake of the cursor property.  That
it's a weird Private Use Area character instead of a space is just to
simplify the whitespace-mode compatibility hack.

Message #74 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: ami <at> fischman.org
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Tue, 30 Oct 2012 19:52:18 +0200

> Date: Mon, 29 Oct 2012 22:24:12 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
> 
> > Date: Mon, 29 Oct 2012 12:23:25 -0700
> > From: Ami Fischman <ami <at> fischman.org>
> > Cc: Eli Zaretskii <eliz <at> gnu.org>, 12745 <at> debbugs.gnu.org
> > 
> > (gdb) p it->glyph_row->glyphs[1][20].object
> > $44 = 101922033
> > (gdb) xtype
> > Lisp_String
> > (gdb) xstring it->glyph_row->glyphs[1][20].object
> > $45 = (struct Lisp_String *) 0x61334f0
> >   ""
> > 
> > (gdb) p ((struct Lisp_String *)0x61334f0)->data
> > $48 = (unsigned char *) 0x4fe9388 ""
> 
> Thanks, this is clear.
> 
> Anyway, the problem happened at position 1295, which I believe is the
> newline that ends this line.  I need to think...

After some thinking, can you show the values of these members of
'struct it' in frame #7 of the crashed 24.2 session (using the core
file):

   it->n_overlay_strings
   it->overlay_strings_charpos
   it->sp

Message #77 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Tue, 30 Oct 2012 11:29:31 -0700

[Message part 1 (text/plain, inline)]

Sure:

(gdb) frame 7
#7  0x0000000000451f23 in next_overlay_string (it=0x7fff2251f1e0) at
xdisp.c:5223
5223          pop_it (it);
(gdb) p    it->n_overlay_strings
$60 = 1
(gdb) p   it->overlay_strings_charpos
$61 = 1295
(gdb) p   it->sp
$62 = 0

[Message part 2 (text/html, inline)]

Message #80 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Tue, 30 Oct 2012 23:00:36 +0200

> Date: Tue, 30 Oct 2012 11:29:31 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
> 
> (gdb) frame 7
> #7  0x0000000000451f23 in next_overlay_string (it=0x7fff2251f1e0) at
> xdisp.c:5223
> 5223          pop_it (it);
> (gdb) p    it->n_overlay_strings
> $60 = 1
> (gdb) p   it->overlay_strings_charpos
> $61 = 1295
> (gdb) p   it->sp
> $62 = 0

That's what I thought, but this is normal and doesn't tell anything
about why the problem happened.  Hmm...

Is it correct that the character at buffer position 1295 is a newline?

Also, what are the values of the following?

  it->stack[0].current
  it->stack[0].string
  it->stack[0].method
  it->method

(The second one I expect to be a Lisp string or nil.)

Message #83 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Tue, 30 Oct 2012 14:08:49 -0700

[Message part 1 (text/plain, inline)]

>
> Is it correct that the character at buffer position 1295 is a newline?
>

Yes.


> Also, what are the values of the following?
>   it->stack[0].current
>

(gdb) p  it->stack[0].current
$63 = {
  pos = {
    charpos = 1295,
    bytepos = 1295
  },
  overlay_string_index = 0,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}

  it->stack[0].string
>

 (gdb) p   it->stack[0].string
$66 = 9755602
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol  it->stack[0].string
$67 = (struct Lisp_Symbol *) 0x94dbd0
  "nil"

  it->stack[0].method
>

(gdb) p   it->stack[0].method
$68 = GET_FROM_BUFFER


>   it->method
>

(gdb) p it->method
$69 = GET_FROM_BUFFER

[Message part 2 (text/html, inline)]

Message #86 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Wed, 31 Oct 2012 17:46:48 +0200

> Date: Tue, 30 Oct 2012 14:08:49 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
> 
> > Is it correct that the character at buffer position 1295 is a newline?
> >
> 
> Yes.
> 
> 
> > Also, what are the values of the following?
> >   it->stack[0].current
> >
> 
> (gdb) p  it->stack[0].current
> $63 = {
>   pos = {
>     charpos = 1295,
>     bytepos = 1295
>   },
>   overlay_string_index = 0,
>   string_pos = {
>     charpos = -1,
>     bytepos = -1
>   },
>   dpvec_index = -1
> }
> 
>   it->stack[0].string
> >
> 
>  (gdb) p   it->stack[0].string
> $66 = 9755602
> (gdb) xtype
> Lisp_Symbol
> (gdb) xsymbol  it->stack[0].string
> $67 = (struct Lisp_Symbol *) 0x94dbd0
>   "nil"
> 
>   it->stack[0].method
> >
> 
> (gdb) p   it->stack[0].method
> $68 = GET_FROM_BUFFER
> 
> 
> >   it->method
> >
> 
> (gdb) p it->method
> $69 = GET_FROM_BUFFER

As expected.

I guess we need to wait for another crash, this time in the trunk
code, and take it from there.  I will meanwhile fix
init_from_display_pos, because it's certainly looks broken, and could
well be the root cause for this.

Thanks.

Message #89 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Thu, 1 Nov 2012 15:25:36 -0700

[Message part 1 (text/plain, inline)]

On Wed, Oct 31, 2012 at 8:46 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> I guess we need to wait for another crash, this time in the trunk code,
> and take it from there.
>

I haven't seen the crash recur in the trunk build so I think we can
probably call this bug fixed in trunk.  I'll definitely follow up if this
re-triggers.

[Message part 2 (text/html, inline)]

Message #92 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Fri, 02 Nov 2012 09:17:28 +0200

> Date: Thu, 1 Nov 2012 15:25:36 -0700
> From: Ami Fischman <ami <at> fischman.org>
> Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
> 
> On Wed, Oct 31, 2012 at 8:46 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> > I guess we need to wait for another crash, this time in the trunk code,
> > and take it from there.
> >
> 
> I haven't seen the crash recur in the trunk build so I think we can
> probably call this bug fixed in trunk.  I'll definitely follow up if this
> re-triggers.

OK, that's good news.  Are you running an optimized build or an
unoptimized one?  If the latter, and the problem doesn't show, I
suggest to try the optimized build of the trunk, and see if the crash
recurs.

Thanks.

Message #95 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Fri, 2 Nov 2012 00:43:26 -0700

[Message part 1 (text/plain, inline)]

Running an -O0 build.  Will rebuild & restart tomorrow and see where I end
up.

[Message part 2 (text/html, inline)]

Message #98 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: ami <at> fischman.org
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sat, 03 Nov 2012 11:31:54 +0200

> Date: Wed, 31 Oct 2012 17:46:48 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
> 
> I will meanwhile fix init_from_display_pos, because it's certainly
> looks broken, and could well be the root cause for this.

Now done in revision 110767 on the emacs-24 branch.  Will appear on
the trunk on the next merge.  The patch is below, FYI.

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2012-10-20 21:30:51 +0000
+++ src/xdisp.c	2012-11-03 09:25:34 +0000
@@ -928,6 +928,7 @@ static enum move_it_result
        move_it_in_display_line_to (struct it *, ptrdiff_t, int,
 				   enum move_operation_enum);
 void move_it_vertically_backward (struct it *, int);
+static void get_visually_first_element (struct it *);
 static void init_to_row_start (struct it *, struct window *,
                                struct glyph_row *);
 static int init_to_row_end (struct it *, struct window *,
@@ -3113,6 +3114,40 @@ init_from_display_pos (struct it *it, st
       eassert (STRINGP (it->string));
       it->current.string_pos = pos->string_pos;
       it->method = GET_FROM_STRING;
+      it->end_charpos = SCHARS (it->string);
+      /* Set up the bidi iterator for this overlay string.  */
+      if (it->bidi_p)
+	{
+	  it->bidi_it.string.lstring = it->string;
+	  it->bidi_it.string.s = NULL;
+	  it->bidi_it.string.schars = SCHARS (it->string);
+	  it->bidi_it.string.bufpos = it->overlay_strings_charpos;
+	  it->bidi_it.string.from_disp_str = it->string_from_display_prop_p;
+	  it->bidi_it.string.unibyte = !it->multibyte_p;
+	  bidi_init_it (IT_STRING_CHARPOS (*it), IT_STRING_BYTEPOS (*it),
+			FRAME_WINDOW_P (it->f), &it->bidi_it);
+
+	  /* Synchronize the state of the bidi iterator with
+	     pos->string_pos.  For any string position other than
+	     zero, this will be done automagically when we resume
+	     iteration over the string and get_visually_first_element
+	     is called.  But if string_pos is zero, and the string is
+	     to be reordered for display, we need to resync manually,
+	     since it could be that the iteration state recorded in
+	     pos ended at string_pos of 0 moving backwards in string.  */
+	  if (CHARPOS (pos->string_pos) == 0)
+	    {
+	      get_visually_first_element (it);
+	      if (IT_STRING_CHARPOS (*it) != 0)
+		do {
+		  /* Paranoia.  */
+		  eassert (it->bidi_it.charpos < it->bidi_it.string.schars);
+		  bidi_move_to_visually_next (&it->bidi_it);
+		} while (it->bidi_it.charpos != 0);
+	    }
+	  eassert (IT_STRING_CHARPOS (*it) == it->bidi_it.charpos
+		   && IT_STRING_BYTEPOS (*it) == it->bidi_it.bytepos);
+	}
     }
 
   if (CHARPOS (pos->string_pos) >= 0)
@@ -3122,6 +3157,9 @@ init_from_display_pos (struct it *it, st
 	 IT should already be filled with that string.  */
       it->current.string_pos = pos->string_pos;
       eassert (STRINGP (it->string));
+      if (it->bidi_p)
+	bidi_init_it (IT_STRING_CHARPOS (*it), IT_STRING_BYTEPOS (*it),
+		      FRAME_WINDOW_P (it->f), &it->bidi_it);
     }
 
   /* Restore position in display vector translations, control

Message #101 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Fri, 23 Nov 2012 12:14:27 -0800

[Message part 1 (text/plain, inline)]

This morning I had the first crash since updating to
d7ef9678754509d426df5f6f2086ca03f7d68b1c on trunk (which doesn't include
110767's fix to init_from_display_pos), in an emacs that's been running for
10 days.

#0  0x00007f958a006b7b in raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
#1  0x00000000004cf767 in terminate_due_to_signal (sig=6,
backtrace_limit=<optimized out>) at emacs.c:344
#2  0x00000000004e98c0 in emacs_abort () at sysdep.c:2061
#3  0x0000000000492515 in bidi_pop_it (bidi_it=<optimized out>) at
bidi.c:638
#4  0x00000000004492a1 in pop_it (it=0x7fff2ab33b18) at xdisp.c:5860
#5  0x0000000000453558 in next_overlay_string (it=0x7fff2ab33b18) at
xdisp.c:5309
#6  0x0000000000427a74 in set_iterator_to_next (it=0x7fff2ab33b18,
reseat_p=<optimized out>) at xdisp.c:7279
#7  0x00000000004315df in display_line (it=0x7fff2ab33b18) at xdisp.c:19790
#8  0x0000000000431008 in try_window (window=<optimized out>, flags=1,
pos=...) at xdisp.c:16300
#9  0x000000000044ce04 in redisplay_window (window=161353781,
just_this_one_p=0) at xdisp.c:15826
#10 0x0000000000452ba3 in redisplay_window_0 (window=28326) at xdisp.c:13894
#11 0x0000000000541aeb in internal_condition_case_1 (bfun=0x452b80
<redisplay_window_0>, arg=161353781, handlers=10062150, hfun=<optimized
out>) at eval.c:1326
#12 0x0000000000449b2f in redisplay_windows (window=<optimized out>) at
xdisp.c:13874
#13 0x000000000042e39b in redisplay_internal () at xdisp.c:13453
#14 0x00000000004308c4 in redisplay_preserve_echo_area (from_where=28326)
at xdisp.c:13710
#15 0x00000000004d9105 in detect_input_pending_run_timers
(do_display=<optimized out>) at keyboard.c:10276
#16 0x000000000057bce3 in wait_reading_process_output
(time_limit=<optimized out>, nsecs=<optimized out>, read_kbd=-1,
do_display=true, wait_for_cell=9858002, wait_proc=0x0,
    just_wait_proc=<optimized out>) at process.c:4749
#17 0x00000000004d7d30 in kbd_buffer_get_event (end_time=<optimized out>,
kbp=<optimized out>, used_mouse_menu=0x7fff2ab3cac7) at keyboard.c:3802
#18 read_char (commandflag=<optimized out>, nmaps=8, maps=0x7fff2ab3c930,
prev_event=9858002, used_mouse_menu=0x7fff2ab3cac7, end_time=<optimized
out>) at keyboard.c:2768
#19 0x00000000004d423d in read_key_sequence (bufsize=30, keybuf=<optimized
out>, prompt=<optimized out>, dont_downcase_last=<optimized out>,
can_return_switch_frame=<optimized out>,
    fix_current_buffer=<optimized out>) at keyboard.c:9230
#20 0x00000000004d381a in command_loop_1 () at keyboard.c:1458
#21 0x00000000005419b1 in internal_condition_case (bfun=0x4d2590
<command_loop_1>, handlers=9909682, hfun=<optimized out>) at eval.c:1288
#22 0x00000000004e2946 in command_loop_2 (ignore=<optimized out>) at
keyboard.c:1167
#23 0x0000000000541486 in internal_catch (tag=<optimized out>,
func=0x4e2920 <command_loop_2>, arg=9858002) at eval.c:1059
#24 0x00000000004d1d09 in command_loop () at keyboard.c:1146
#25 recursive_edit_1 () at keyboard.c:778
#26 0x00000000004d1e26 in Frecursive_edit () at keyboard.c:842
#27 0x00000000004d0d69 in main (argc=<error reading variable: Cannot access
memory at address 0x0>, argv=<optimized out>) at emacs.c:1552


Frame #3 aborted b/c bidi_cache_sp is 0.
Frame #4 has:
(gdb) p it->current
$2 = {
  pos = {
    charpos = 99256,
    bytepos = 99256
  },
  overlay_string_index = 0,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}

(gdb) p current_buffer->name_
$3 = 65944961
(gdb) pp current_buffer->name_
Cannot access memory at address 0x8ce6b8

(gdb) p current_buffer->text->beg[99256]@100
$5 = ' ' <repeats 24 times>, "],\n", ' ' <repeats 22 times>, "}],\n", ' '
<repeats 20 times>, "],\n", ' ' <repeats 20 times>, "'tar"

which tells me this is
common.gypi<http://src.chromium.org/viewvc/chrome/trunk/src/build/common.gypi?view=markup>,
running in gyp-mode (as opposed to the previous report which was in
cc-mode).

Let me know if you think this is worth debugging further or if I should
first sync & rebuild before a further crash will be interesting.

[Message part 2 (text/html, inline)]

Message #104 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Fri, 23 Nov 2012 23:56:24 +0200

> Date: Fri, 23 Nov 2012 12:14:27 -0800
> From: Ami Fischman <ami <at> fischman.org>
> Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
> 
> This morning I had the first crash since updating to
> d7ef9678754509d426df5f6f2086ca03f7d68b1c on trunk (which doesn't include
> 110767's fix to init_from_display_pos)

If you don't have 110767, then why is it interesting to report this
crash?  That revision was supposed to fix precisely this crash.

The current revision numbers on the emacs-24 branch and on the trunk
are 110941 and 110993, respectively.  So why would you use such an old
version, from more than 3 weeks ago?

> Let me know if you think this is worth debugging further or if I should
> first sync & rebuild before a further crash will be interesting.

I certainly recommend to update to the latest code.

Thanks.

Message #107 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Ami Fischman <ami <at> fischman.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sat, 24 Nov 2012 20:52:19 -0800

[Message part 1 (text/plain, inline)]

>
> If you don't have 110767, then why is it interesting to report this

crash?  That revision was supposed to fix precisely this crash.
>

I didn't get the impression from our previous exchange that you had such a
high confidence in the connection (just that you were sure it was fixing a
bug, not that it was necessarily the bug I was tripping over).


> The current revision numbers on the emacs-24 branch and on the trunk
> are 110941 and 110993, respectively.  So why would you use such an old
> version, from more than 3 weeks ago?
>

This bug took hours/days to manifest; I wanted to see whether trunk already
had it fixed.  It seems that it was at least partially fixed since this
time it took a lot longer to manifest.

I'll sync & rebuild and will report back if I get another one of these.

[Message part 2 (text/html, inline)]

Message #110 received at 12745 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ami Fischman <ami <at> fischman.org>
Cc: alptekin.aker <at> gmail.com, 12745 <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sun, 25 Nov 2012 18:51:56 +0200

> Date: Sat, 24 Nov 2012 20:52:19 -0800
> From: Ami Fischman <ami <at> fischman.org>
> Cc: Alp Aker <alptekin.aker <at> gmail.com>, 12745 <at> debbugs.gnu.org
> 
> > If you don't have 110767, then why is it interesting to report this
> > crash?  That revision was supposed to fix precisely this crash.
> 
> I didn't get the impression from our previous exchange that you had such a
> high confidence in the connection (just that you were sure it was fixing a
> bug, not that it was necessarily the bug I was tripping over).

I wasn't 100% sure because I could not reproduce the original problem.
But from looking at the code that caused assertion violation, I can
see no other reasons than what was fixed in 110767.

> I'll sync & rebuild and will report back if I get another one of these.

Thanks.

Message #115 received at 12745-done <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 12745-done <at> debbugs.gnu.org
Subject: Re: bug#12745: crash in bidi_pop_it during (idle) redisplay
Date: Sat, 09 Feb 2013 22:07:31 -0500

Ami Fischman wrote:

> I'll sync & rebuild and will report back if I get another one of these.

Assuming that no news is good news.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.