GNU bug report logs - #40661
Crash in regex search during redisplay

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40661 in the body.
You can then email your comments to 40661 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Richard Copley <rcopley <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org, Daniel Colascione <dancol <at> dancol.org>
Subject: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 15:35:20 +0100

Recipe from emacs -Q:

    Save the text below in a file with extension ".pl".
    Repeatedly: kill the buffer and visit the file again. (You can use
C-x C-v for this.)

Emacs eventually encounters a segfault. Backtrace below.

The text is reduced from a real program that exhibited the problem.
Repeating up to about 20 times is usually enough. You can use a
keyboard macro, [C-x ( C-x C-v RET C-x e e e e e], holding down the
'e' key until you get the crash.

This affects both the master branch and the release branch. Bisected
to this commit:

    938d252d1c6c5e2027aa250c649deb024154f936
    Commit:     Daniel Colascione <dancol <at> dancol.org>
    CommitDate: Sat Jun 16 13:46:38 2018 -0700

    Make regex matching reentrant; update syntax during match

BEGIN TEXT
use strict;

000000000000000000000000000000000000000000000000000000; # x

sub x { }

000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
0000000000000000000000000000000;

"", @x;
"";

eval {
    use autodie qw(:all);
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000000000000000000000000000000000000000000;
000000000000000000000;
};
END TEXT

Thread 1 received signal SIGSEGV, Segmentation fault.
rpl_re_search_2 (bufp=<optimized out>, bufp <at> entry=0x4005f3238
<searchbufs+5432>, str1=str1 <at> entry=0x90307fd <error: Cannot access
memory at address 0x90307fd>, size1=<optimized out>, size1 <at> entry=0,
str2=str2 <at> entry=0x90307fd <error: Cannot access memory at address
0x90307fd>, size2=size2 <at> entry=2051, startpos=502, startpos <at> entry=10,
range=1, regs=0x400534598 <main_thread+152>, stop=503) at
regex-emacs.c:3394
3394                  int len = BYTES_BY_CHAR_HEAD (*p);

(gdb) bt
#0  rpl_re_search_2 (bufp=<optimized out>, bufp <at> entry=0x4005f3238
<searchbufs+5432>, str1=str1 <at> entry=0x90307fd <error: Cannot access
memory at address 0x90307fd>, size1=<optimized out>, size1 <at> entry=0,
str2=str2 <at> entry=0x90307fd <error: Cannot access memory at address
0x90307fd>, size2=size2 <at> entry=2051, startpos=502, startpos <at> entry=10,
range=1, regs=0x400534598 <main_thread+152>, stop=503) at
regex-emacs.c:3394
#1  0x00000004000ea2c2 in search_buffer_re
(string=string <at> entry=XIL(0x48dc3e4), pos=pos <at> entry=11,
pos_byte=<optimized out>, pos_byte <at> entry=11, lim=lim <at> entry=504,
lim_byte=lim_byte <at> entry=504, n=n <at> entry=1, trt=trt <at> entry=XIL(0),
inverse_trt=inverse_trt <at> entry=XIL(0), posix=posix <at> entry=false) at
search.c:1233
#2  0x00000004000ee0b1 in search_buffer
(string=string <at> entry=XIL(0x48dc3e4), pos=11, pos_byte=11,
lim=lim <at> entry=504, lim_byte=lim_byte <at> entry=504, n=n <at> entry=1,
RE=RE <at> entry=1, trt=XIL(0), inverse_trt=XIL(0),
posix=posix <at> entry=false) at search.c:1505
#3  0x00000004000ee2a8 in search_command (string=XIL(0x48dc3e4),
bound=<optimized out>, noerror=XIL(0x30), count=<optimized out>,
direction=direction <at> entry=1, RE=RE <at> entry=1, posix=posix <at> entry=false)
at lisp.h:1409
#4  0x00000004000ee425 in Fre_search_forward (regexp=<optimized out>,
bound=<optimized out>, noerror=<optimized out>, count=<optimized out>)
at search.c:2276
#5  0x0000000400120277 in funcall_subr (subr=0x400540540
<Sre_search_forward>, numargs=numargs <at> entry=3,
args=args <at> entry=0xbf8938) at eval.c:2875
#6  0x000000040011f052 in Ffuncall (nargs=4, args=args <at> entry=0xbf8930)
at lisp.h:2113
#7  0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(770),
nargs=nargs <at> entry=3, args=<optimized out>, args <at> entry=0xbf8d00) at
bytecode.c:633
#8  0x00000004001214f3 in funcall_lambda (fun=XIL(0x3f61f25),
nargs=nargs <at> entry=3, arg_vector=arg_vector <at> entry=0xbf8d00) at
lisp.h:1862
#9  0x000000040011f062 in Ffuncall (nargs=4, args=args <at> entry=0xbf8cf8)
at eval.c:2796
#10 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(771),
nargs=nargs <at> entry=3, args=<optimized out>, args <at> entry=0xbf8fc8) at
bytecode.c:633
#11 0x00000004001214f3 in funcall_lambda (fun=XIL(0x3f61a65),
nargs=nargs <at> entry=3, arg_vector=arg_vector <at> entry=0xbf8fc8) at
lisp.h:1862
#12 0x000000040011f062 in Ffuncall (nargs=4, args=args <at> entry=0xbf8fc0)
at eval.c:2796
#13 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(770),
nargs=nargs <at> entry=2, args=<optimized out>, args <at> entry=0xbf91c8) at
bytecode.c:633
#14 0x00000004001214f3 in funcall_lambda (fun=XIL(0x3f61995),
nargs=nargs <at> entry=2, arg_vector=arg_vector <at> entry=0xbf91c8) at
lisp.h:1862
#15 0x000000040011f062 in Ffuncall (nargs=3, args=args <at> entry=0xbf91c0)
at eval.c:2796
#16 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(257),
nargs=nargs <at> entry=1, args=<optimized out>, args <at> entry=0xbf9618) at
bytecode.c:633
#17 0x00000004001214f3 in funcall_lambda (fun=XIL(0x8c7d7d5),
nargs=nargs <at> entry=1, arg_vector=arg_vector <at> entry=0xbf9618) at
lisp.h:1862
#18 0x000000040011f062 in Ffuncall (nargs=2, args=args <at> entry=0xbf9610)
at eval.c:2796
#19 0x000000040011f16a in run_hook_wrapped_funcall (nargs=<optimized
out>, args=0xbf9610) at eval.c:2531
#20 0x000000040011e89c in run_hook_with_args (nargs=2, args=0xbf9610,
funcall=funcall <at> entry=0x40011f14e <run_hook_wrapped_funcall>) at
eval.c:2612
#21 0x000000040011e9d6 in Frun_hook_wrapped (nargs=<optimized out>,
args=<optimized out>) at eval.c:2546
#22 0x00000004001201f2 in funcall_subr (subr=0x400543f00
<Srun_hook_wrapped>, numargs=numargs <at> entry=2,
args=args <at> entry=0xbf9610) at eval.c:2847
#23 0x000000040011f052 in Ffuncall (nargs=3, args=args <at> entry=0xbf9608)
at lisp.h:2113
#24 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(514),
nargs=nargs <at> entry=2, args=<optimized out>, args <at> entry=0xbf98a0) at
bytecode.c:633
#25 0x00000004001214f3 in funcall_lambda (fun=XIL(0x400ce1d),
nargs=nargs <at> entry=2, arg_vector=arg_vector <at> entry=0xbf98a0) at
lisp.h:1862
#26 0x000000040011f062 in Ffuncall (nargs=3, args=args <at> entry=0xbf9898)
at eval.c:2796
#27 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(512),
nargs=nargs <at> entry=2, args=<optimized out>, args <at> entry=0xbf9c08) at
bytecode.c:633
#28 0x00000004001214f3 in funcall_lambda (fun=XIL(0x400cb2d),
nargs=nargs <at> entry=2, arg_vector=arg_vector <at> entry=0xbf9c08) at
lisp.h:1862
#29 0x000000040011f062 in Ffuncall (nargs=3, args=args <at> entry=0xbf9c00)
at eval.c:2796
#30 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template <at> entry=make_fixnum(257),
nargs=nargs <at> entry=1, args=<optimized out>, args <at> entry=0xbf9eb8) at
bytecode.c:633
#31 0x00000004001214f3 in funcall_lambda (fun=XIL(0x400c775),
nargs=nargs <at> entry=1, arg_vector=arg_vector <at> entry=0xbf9eb8) at
lisp.h:1862
#32 0x000000040011f062 in Ffuncall (nargs=2, args=0xbf9eb0) at eval.c:2796
#33 0x000000040011e5eb in internal_condition_case_n (bfun=0x40011ee70
<Ffuncall>, nargs=nargs <at> entry=2, args=args <at> entry=0xbf9eb0,
handlers=handlers <at> entry=XIL(0x30), hfun=hfun <at> entry=0x40002c8ba
<safe_eval_handler>) at eval.c:1435
#34 0x000000040001a09b in safe__call
(inhibit_quit=inhibit_quit <at> entry=false, nargs=nargs <at> entry=2,
func=XIL(0xfffffffc03a118a0), ap=<optimized out>, ap <at> entry=0xbf9f50
"\006") at lisp.h:1042
#35 0x0000000400028a8a in safe_call (nargs=nargs <at> entry=2,
func=<optimized out>) at xdisp.c:2841
#36 0x0000000400028aa3 in safe_call1 (fn=<optimized out>,
arg=arg <at> entry=make_fixnum(1)) at xdisp.c:2852
#37 0x0000000400028ccf in handle_fontified_prop (it=0xbfa1b0) at xdisp.c:4158
#38 0x000000040002e4d5 in handle_stop (it=0xbfa1b0) at xdisp.c:3686
#39 0x000000040002e5b5 in reseat (it=0xbfa1b0, pos=...,
force_p=<optimized out>) at xdisp.c:6934
#40 0x000000040002efff in init_iterator (it=it <at> entry=0xbfa1b0,
w=w <at> entry=0x56eddd0, charpos=1, bytepos=1, row=<optimized out>,
base_face_id=<optimized out>, base_face_id <at> entry=DEFAULT_FACE_ID) at
xdisp.c:3287
#41 0x000000040003606c in start_display (it=it <at> entry=0xbfa1b0,
w=w <at> entry=0x56eddd0, pos=...) at xdisp.c:3303
#42 0x000000040003ea6f in try_window
(window=window <at> entry=XIL(0x56eddd5), pos=..., flags=flags <at> entry=1) at
xdisp.c:19077
#43 0x0000000400051dea in redisplay_window (window=XIL(0x56eddd5),
just_this_one_p=just_this_one_p <at> entry=false) at xdisp.c:18501
#44 0x00000004000538f9 in redisplay_window_0 (window=<optimized out>)
at xdisp.c:16215
#45 0x000000040011e4ed in internal_condition_case_1
(bfun=bfun <at> entry=0x4000538c6 <redisplay_window_0>,
arg=arg <at> entry=XIL(0x56eddd5), handlers=<optimized out>,
hfun=hfun <at> entry=0x400015902 <redisplay_window_error>) at eval.c:1379
#46 0x000000040001c945 in redisplay_windows (window=XIL(0x56eddd5)) at
xdisp.c:16195
#47 0x0000000400044416 in redisplay_internal () at xdisp.c:15663
#48 0x00000004000454a3 in redisplay () at xdisp.c:14891
#49 0x00000004000b49aa in read_char (commandflag=0, map=XIL(0),
map <at> entry=XIL(0x8cbfda3), prev_event=XIL(0), used_mouse_menu=0x0,
used_mouse_menu <at> entry=0xbff4cb, end_time=end_time <at> entry=0x0) at
keyboard.c:2493
#50 0x00000004000b644b in read_key_sequence
(keybuf=keybuf <at> entry=0xbff5d0, prompt=prompt <at> entry=XIL(0),
dont_downcase_last=dont_downcase_last <at> entry=false,
can_return_switch_frame=can_return_switch_frame <at> entry=true,
fix_current_buffer=fix_current_buffer <at> entry=true,
prevent_redisplay=prevent_redisplay <at> entry=false) at keyboard.c:9534
#51 0x00000004000b7785 in command_loop_1 () at lisp.h:1042
#52 0x000000040011e476 in internal_condition_case
(bfun=bfun <at> entry=0x4000b7552 <command_loop_1>,
handlers=handlers <at> entry=XIL(0x90), hfun=hfun <at> entry=0x4000ae1e4
<cmd_error>) at eval.c:1355
#53 0x00000004000a95d4 in command_loop_2 (ignore=<optimized out>) at lisp.h:1042
#54 0x000000040011e3e7 in internal_catch (tag=tag <at> entry=XIL(0xe070),
func=func <at> entry=0x4000a95b8 <command_loop_2>, arg=arg <at> entry=XIL(0)) at
eval.c:1116
#55 0x00000004000a9571 in command_loop () at lisp.h:1042
#56 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Lisp Backtrace:
"re-search-forward" (0xbf8938)
"font-lock-fontify-keywords-region" (0xbf8d00)
"font-lock-default-fontify-region" (0xbf8fc8)
"font-lock-fontify-region" (0xbf91c8)
0x8c7d7d0 PVEC_COMPILED
"run-hook-wrapped" (0xbf9610)
"jit-lock--run-functions" (0xbf98a0)
"jit-lock-fontify-now" (0xbf9c08)
"jit-lock-function" (0xbf9eb8)
"redisplay_internal (C function)" (0x0)

In GNU Emacs 28.0.50 (build 1, x86_64-w64-mingw32)
 of 2020-04-16 built on MACHINE
Repository revision: d5a7df8c02f04102d50a5cd2290262f59f2b1415
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 10.0.19041
System Description: Microsoft Windows 10 Pro (v10.0.2004.19041.153)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --config-cache --with-modules --without-pop
 --without-compress-install --without-dbus --without-gconf
 --without-gsettings 'CFLAGS=-Og -g -ggdb -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY W32NOTIFY ACL GNUTLS LIBXML2
HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: ENG
  locale-coding-system: cp1252

Major mode: Perl

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
perl-mode tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads w32notify w32 lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 47504 14219)
 (symbols 48 6132 1)
 (strings 32 17070 1654)
 (string-bytes 1 523301)
 (vectors 16 9436)
 (vector-slots 8 132460 12454)
 (floats 8 21 229)
 (intervals 56 238 0)
 (buffers 992 11))

Message #8 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Richard Copley <rcopley <at> gmail.com>
Cc: dancol <at> dancol.org, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 18:36:36 +0300

> From: Richard Copley <rcopley <at> gmail.com>
> Date: Thu, 16 Apr 2020 15:35:20 +0100
> 
> Recipe from emacs -Q:
> 
>     Save the text below in a file with extension ".pl".
>     Repeatedly: kill the buffer and visit the file again. (You can use
> C-x C-v for this.)
> 
> Emacs eventually encounters a segfault. Backtrace below.
> 
> The text is reduced from a real program that exhibited the problem.
> Repeating up to about 20 times is usually enough. You can use a
> keyboard macro, [C-x ( C-x C-v RET C-x e e e e e], holding down the
> 'e' key until you get the crash.

Looks like GC sometimes kicks in while we are inside re_search_2 and
have pointers to buffer text lying around.  This seems to fix the
crash for me:

diff --git a/src/search.c b/src/search.c
index 818bb4a..79423be 100644
--- a/src/search.c
+++ b/src/search.c
@@ -1188,6 +1188,7 @@ search_buffer_re (Lisp_Object string, ptrdiff_t pos, ptrdiff_t pos_byte,
   ptrdiff_t count = SPECPDL_INDEX ();
   freeze_buffer_relocation ();
   freeze_pattern (cache_entry);
+  inhibit_garbage_collection ();
 
   while (n < 0)
     {

Message #11 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rcopley <at> gmail.com
Cc: dancol <at> dancol.org, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 19:33:16 +0300

> Date: Thu, 16 Apr 2020 18:36:36 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 40661 <at> debbugs.gnu.org
> 
> Looks like GC sometimes kicks in while we are inside re_search_2

Or not.  I cannot get a breakpoint inside GC to fire while we are in
search_buffer_re, so maybe my hypothesis was wrong.  Although the
symptoms are all there: when the segfault hits, the pointers passed to
re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
they are supposed to be computed, are valid (and different).  And the
patch does seem to avoid the segfaults.  But maybe it's just a
coincidence or a side effect...

Message #14 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>,rcopley <at> gmail.com
Cc: 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 09:42:10 -0700

On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Thu, 16 Apr 2020 18:36:36 +0300
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> Cc: 40661 <at> debbugs.gnu.org
>> 
>> Looks like GC sometimes kicks in while we are inside re_search_2
>
>Or not.  I cannot get a breakpoint inside GC to fire while we are in
>search_buffer_re, so maybe my hypothesis was wrong.  Although the
>symptoms are all there: when the segfault hits, the pointers passed to
>re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
>they are supposed to be computed, are valid (and different).  And the
>patch does seem to avoid the segfaults.  But maybe it's just a
>coincidence or a side effect...

Try using rr and see where those pointers came from

Message #17 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: rcopley <at> gmail.com, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 19:57:00 +0300

> Date: Thu, 16 Apr 2020 09:42:10 -0700
> CC: 40661 <at> debbugs.gnu.org
> From: Daniel Colascione <dancol <at> dancol.org>
> 
> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Try using rr and see where those pointers came from

rr doesn't work on MS-Windows.

Message #20 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Richard Copley <rcopley <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 17:56:51 +0100

On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol <at> dancol.org> wrote:
>
> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> Date: Thu, 16 Apr 2020 18:36:36 +0300
> >> From: Eli Zaretskii <eliz <at> gnu.org>
> >> Cc: 40661 <at> debbugs.gnu.org
> >>
> >> Looks like GC sometimes kicks in while we are inside re_search_2
> >
> >Or not.  I cannot get a breakpoint inside GC to fire while we are in
> >search_buffer_re, so maybe my hypothesis was wrong.  Although the
> >symptoms are all there: when the segfault hits, the pointers passed to
> >re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
> >they are supposed to be computed, are valid (and different).  And the
> >patch does seem to avoid the segfaults.  But maybe it's just a
> >coincidence or a side effect...
>
> Try using rr and see where those pointers came from

It seems clear from "str1=str1 <at> entry=0xc607fd", etc., that they come
from the caller, search_buffer_re. The question is, why are they no
longer valid after updating syntax?

Message #23 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Richard Copley <rcopley <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 10:24:36 -0700

On 4/16/20 9:56 AM, Richard Copley wrote:
> On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol <at> dancol.org> wrote:
>>
>> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>> Date: Thu, 16 Apr 2020 18:36:36 +0300
>>>> From: Eli Zaretskii <eliz <at> gnu.org>
>>>> Cc: 40661 <at> debbugs.gnu.org
>>>>
>>>> Looks like GC sometimes kicks in while we are inside re_search_2
>>>
>>> Or not.  I cannot get a breakpoint inside GC to fire while we are in
>>> search_buffer_re, so maybe my hypothesis was wrong.  Although the
>>> symptoms are all there: when the segfault hits, the pointers passed to
>>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
>>> they are supposed to be computed, are valid (and different).  And the
>>> patch does seem to avoid the segfaults.  But maybe it's just a
>>> coincidence or a side effect...
>>
>> Try using rr and see where those pointers came from
> 
> It seems clear from "str1=str1 <at> entry=0xc607fd", etc., that they come
> from the caller, search_buffer_re. The question is, why are they no
> longer valid after updating syntax?

Right. So let's see what updated the valid pointers and invalidated the 
invalid ones.

Message #26 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Richard Copley <rcopley <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Thu, 16 Apr 2020 20:35:19 +0100

On Thu, 16 Apr 2020 at 18:24, Daniel Colascione <dancol <at> dancol.org> wrote:
>
> On 4/16/20 9:56 AM, Richard Copley wrote:
> > On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol <at> dancol.org> wrote:
> >>
> >> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>>> Date: Thu, 16 Apr 2020 18:36:36 +0300
> >>>> From: Eli Zaretskii <eliz <at> gnu.org>
> >>>> Cc: 40661 <at> debbugs.gnu.org
> >>>>
> >>>> Looks like GC sometimes kicks in while we are inside re_search_2
> >>>
> >>> Or not.  I cannot get a breakpoint inside GC to fire while we are in
> >>> search_buffer_re, so maybe my hypothesis was wrong.  Although the
> >>> symptoms are all there: when the segfault hits, the pointers passed to
> >>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
> >>> they are supposed to be computed, are valid (and different).  And the
> >>> patch does seem to avoid the segfaults.  But maybe it's just a
> >>> coincidence or a side effect...
> >>
> >> Try using rr and see where those pointers came from
> >
> > It seems clear from "str1=str1 <at> entry=0xc607fd", etc., that they come
> > from the caller, search_buffer_re. The question is, why are they no
> > longer valid after updating syntax?
>
> Right. So let's see what updated the valid pointers and invalidated the
> invalid ones.

Right, I see. Anyway, I wasn't able to reproduce the bug under
GNU/Linux (in order to use rr), or make much progress with GDB on
Windows.

Message #29 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Richard Copley <rcopley <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca> 
Cc: dancol <at> dancol.org, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Fri, 17 Apr 2020 14:22:00 +0300

> From: Richard Copley <rcopley <at> gmail.com>
> Date: Thu, 16 Apr 2020 20:35:19 +0100
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 40661 <at> debbugs.gnu.org
> 
> On Thu, 16 Apr 2020 at 18:24, Daniel Colascione <dancol <at> dancol.org> wrote:
> >
> > On 4/16/20 9:56 AM, Richard Copley wrote:
> > > On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol <at> dancol.org> wrote:
> > >>
> > >> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > >>>> Date: Thu, 16 Apr 2020 18:36:36 +0300
> > >>>> From: Eli Zaretskii <eliz <at> gnu.org>
> > >>>> Cc: 40661 <at> debbugs.gnu.org
> > >>>>
> > >>>> Looks like GC sometimes kicks in while we are inside re_search_2
> > >>>
> > >>> Or not.  I cannot get a breakpoint inside GC to fire while we are in
> > >>> search_buffer_re, so maybe my hypothesis was wrong.  Although the
> > >>> symptoms are all there: when the segfault hits, the pointers passed to
> > >>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which
> > >>> they are supposed to be computed, are valid (and different).  And the
> > >>> patch does seem to avoid the segfaults.  But maybe it's just a
> > >>> coincidence or a side effect...
> > >>
> > >> Try using rr and see where those pointers came from
> > >
> > > It seems clear from "str1=str1 <at> entry=0xc607fd", etc., that they come
> > > from the caller, search_buffer_re. The question is, why are they no
> > > longer valid after updating syntax?
> >
> > Right. So let's see what updated the valid pointers and invalidated the
> > invalid ones.
> 
> Right, I see. Anyway, I wasn't able to reproduce the bug under
> GNU/Linux (in order to use rr), or make much progress with GDB on
> Windows.

I finally succeeded to build a sophisticated enough trap to catch the
culprit.  It's GC allright.  Which is not surprising: the commit
pointed out by Richard changed re_match_2_internal to call
UPDATE_SYNTAX_TABLE_FORWARD, which calls Lisp, and thus can trigger
GC.  As seen from the backtrace, GC then calls compact_buffer, which
calls enlarge_buffer_text (to shrink buffer text), and on MS-Windows
-- and this is the w32-specific part -- we return some memory to the
OS and relocate buffer text.

Obviously, we cannot allow GC to run while regex routines do their
work, because they are passed C pointers to buffer text.  The question
is, where to disable GC?  We could do it inside
update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
from many places that evidently have no problems with GC.  So my
suggestion would be to disable GC inside re_match_2_internal instead.

Comments?

Here's the full backtrace I caught:

  mmap_alloc (var=0x6843168, nbytes=2257) at w32heap.c:676
  676           if (*var == NULL)
  #0  mmap_alloc (var=0x6843168, nbytes=2257) at w32heap.c:676
  #1  0x015a7934 in mmap_realloc (var=0x6843168, nbytes=2257) at w32heap.c:784
  #2  0x0124d431 in enlarge_buffer_text (b=0x6842f10, delta=-1840)
      at buffer.c:5049
  #3  0x01262107 in make_gap_smaller (nbytes_removed=1840) at insdel.c:549
  #4  0x0126221c in make_gap (nbytes_added=-1840) at insdel.c:589
  #5  0x01262246 in make_gap_1 (b=0x6842f10, nbytes=-1840) at insdel.c:602
  #6  0x012427e8 in compact_buffer (buffer=0x6842f10) at buffer.c:1672
  #7  0x01314c2e in garbage_collect () at alloc.c:5877
  #8  0x01314b9a in maybe_garbage_collect () at alloc.c:5853
  #9  0x0137696d in maybe_gc () at lisp.h:5065
  #10 0x013848c2 in Ffuncall (nargs=4, args=0x824360) at eval.c:2778
  #11 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865a98),
      vector=XIL(0xa000000006889d58), maxdepth=make_fixnum(7),
      args_template=make_fixnum(257), nargs=1, args=0x824918) at bytecode.c:633
  #12 0x01385af1 in funcall_lambda (fun=XIL(0xa0000000071f2c90), nargs=1,
      arg_vector=0x824910) at eval.c:2989
  #13 0x01384a33 in Ffuncall (nargs=2, args=0x824908) at eval.c:2796
  #14 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865ad8),
      vector=XIL(0xa0000000072005a0), maxdepth=make_fixnum(17),
      args_template=make_fixnum(257), nargs=1, args=0x825058) at bytecode.c:633
  #15 0x01385af1 in funcall_lambda (fun=XIL(0xa0000000071e4b40), nargs=1,
      arg_vector=0x825050) at eval.c:2989
  #16 0x01384a33 in Ffuncall (nargs=2, args=0x825048) at eval.c:2796
  #17 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865a18),
      vector=XIL(0xa000000006828648), maxdepth=make_fixnum(15),
      args_template=make_fixnum(514), nargs=2, args=0x825868) at bytecode.c:633
  #18 0x01385af1 in funcall_lambda (fun=XIL(0xa000000007167768), nargs=2,
      arg_vector=0x825858) at eval.c:2989
  #19 0x01384a33 in Ffuncall (nargs=3, args=0x825850) at eval.c:2796
  #20 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e847e0),
      vector=XIL(0xa000000005e84438), maxdepth=make_fixnum(12),
      args_template=make_fixnum(257), nargs=1, args=0x825e98) at bytecode.c:633
  #21 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e84408), nargs=1,
      arg_vector=0x825e90) at eval.c:2989
  #22 0x01384a33 in Ffuncall (nargs=2, args=0x825e88) at eval.c:2796
  #23 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005ecf1d8),
      vector=XIL(0xa000000005ecf0e0), maxdepth=make_fixnum(8),
      args_template=make_fixnum(257), nargs=1, args=0x826460) at bytecode.c:633
  #24 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005ecf0b0), nargs=1,
      arg_vector=0x826458) at eval.c:2989
  #25 0x01384a33 in Ffuncall (nargs=2, args=0x826450) at eval.c:2796
  #26 0x0137f3e7 in internal_condition_case_n (bfun=0x13847ec <Ffuncall>,
      nargs=2, args=0x826450, handlers=XIL(0x30),
      hfun=0x105a91d <safe_eval_handler>) at eval.c:1435
  #27 0x0105ab54 in safe__call (inhibit_quit=false, nargs=2, func=XIL(0x7fb0),
      ap=0x826514 "") at xdisp.c:2807
  #28 0x0105abcd in safe_call (nargs=2, func=XIL(0x7fb0)) at xdisp.c:2822
  #29 0x0105ac21 in safe_call1 (fn=XIL(0x7fb0), arg=make_fixnum(505))
      at xdisp.c:2833
  #30 0x014128bb in parse_sexp_propertize (charpos=504) at syntax.c:480
  #31 0x01412aa3 in update_syntax_table_forward (charpos=504, init=false,
      object=XIL(0)) at syntax.c:513
  #32 0x012e9cb5 in UPDATE_SYNTAX_TABLE_FORWARD (charpos=504) at syntax.h:185
  #33 0x012e9d39 in UPDATE_SYNTAX_TABLE (charpos=504) at syntax.h:205
  #34 0x012f81f9 in re_match_2_internal (bufp=0x1bb43a0 <searchbufs+4064>,
      string1=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\
  n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n",
      size1=0,
      string2=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\
  n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n",
      size2=2051, pos=502, regs=0x1958074 <main_thread+116>, stop=503)
      at regex-emacs.c:4780
  #35 0x012f39f3 in rpl_re_search_2 (bufp=0x1bb43a0 <searchbufs+4064>,
      str1=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\n",
   '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n", size1=0,
      str2=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\n",
   '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n",
      size2=2051, startpos=502, range=1, regs=0x1958074 <main_thread+116>,
      stop=503) at regex-emacs.c:3373
  #36 0x012dc6e0 in search_buffer_re (string=XIL(0x8000000006865028), pos=11,
      pos_byte=11, lim=504, lim_byte=504, n=1, trt=XIL(0), inverse_trt=XIL(0),
      posix=false) at search.c:1244
  #37 0x012dd74d in search_buffer (string=XIL(0x8000000006865028), pos=11,
      pos_byte=11, lim=504, lim_byte=504, n=1, RE=1, trt=XIL(0),
      inverse_trt=XIL(0), posix=false) at search.c:1506
  #38 0x012dbefa in search_command (string=XIL(0x8000000006865028),
      bound=make_fixnum(504), noerror=XIL(0x30), count=XIL(0), direction=1,
      RE=1, posix=false) at search.c:1048
  #39 0x012df7bc in Fre_search_forward (regexp=XIL(0x8000000006865028),
      bound=make_fixnum(504), noerror=XIL(0x30), count=XIL(0)) at search.c:2277
  #40 0x01385067 in funcall_subr (subr=0x195ebc0 <Sre_search_forward>,
      numargs=3, args=0x8274d8) at eval.c:2875
  #41 0x013849d9 in Ffuncall (nargs=4, args=0x8274d0) at eval.c:2794
  #42 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eed790),
      vector=XIL(0xa000000005e25f58), maxdepth=make_fixnum(25),
      args_template=make_fixnum(770), nargs=3, args=0x827c38) at bytecode.c:633
  #43 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e25f28), nargs=3,
      arg_vector=0x827c20) at eval.c:2989
  #44 0x01384a33 in Ffuncall (nargs=4, args=0x827c18) at eval.c:2796
  #45 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eee538),
      vector=XIL(0xa000000005e25ac0), maxdepth=make_fixnum(14),
      args_template=make_fixnum(771), nargs=3, args=0x828280) at bytecode.c:633
  #46 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e25a90), nargs=3,
      arg_vector=0x828268) at eval.c:2989
  #47 0x01384a33 in Ffuncall (nargs=4, args=0x828260) at eval.c:2796
  #48 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eee748),
      vector=XIL(0xa000000005e259f0), maxdepth=make_fixnum(7),
      args_template=make_fixnum(770), nargs=2, args=0x8287e8) at bytecode.c:633
  #49 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e259c0), nargs=2,
      arg_vector=0x8287d8) at eval.c:2989
  #50 0x01384a33 in Ffuncall (nargs=3, args=0x8287d0) at eval.c:2796
  #51 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8ab40),
      vector=XIL(0xa000000007255b70), maxdepth=make_fixnum(10),
      args_template=make_fixnum(257), nargs=1, args=0x829060) at bytecode.c:633
  #52 0x01385af1 in funcall_lambda (fun=XIL(0xa000000007255bc0), nargs=1,
      arg_vector=0x829058) at eval.c:2989
  #53 0x01384a33 in Ffuncall (nargs=2, args=0x829050) at eval.c:2796
  #54 0x013837c6 in run_hook_wrapped_funcall (nargs=2, args=0x829050)
      at eval.c:2531
  #55 0x01383ca9 in run_hook_with_args (nargs=2, args=0x829050,
      funcall=0x138377e <run_hook_wrapped_funcall>) at eval.c:2612
  #56 0x01383815 in Frun_hook_wrapped (nargs=2, args=0x829050) at eval.c:2546
  #57 0x01384e99 in funcall_subr (subr=0x1960bc0 <Srun_hook_wrapped>,
      numargs=2, args=0x829050) at eval.c:2847
  #58 0x013849d9 in Ffuncall (nargs=3, args=0x829048) at eval.c:2794
  #59 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8abf0),
      vector=XIL(0xa000000005e8aac8), maxdepth=make_fixnum(19),
      args_template=make_fixnum(514), nargs=2, args=0x829670) at bytecode.c:633
  #60 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8aa98), nargs=2,
      arg_vector=0x829660) at eval.c:2989
  #61 0x01384a33 in Ffuncall (nargs=3, args=0x829658) at eval.c:2796
  #62 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8b048),
      vector=XIL(0xa000000005e8a818), maxdepth=make_fixnum(27),
      args_template=make_fixnum(512), nargs=2, args=0x829d58) at bytecode.c:633
  #63 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8a7e8), nargs=2,
      arg_vector=0x829d48) at eval.c:2989
  #64 0x01384a33 in Ffuncall (nargs=3, args=0x829d40) at eval.c:2796
  #65 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8d5d0),
      vector=XIL(0xa000000005e8d040), maxdepth=make_fixnum(12),
      args_template=make_fixnum(257), nargs=1, args=0x82a380) at bytecode.c:633
  #66 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8d010), nargs=1,
      arg_vector=0x82a378) at eval.c:2989
  #67 0x01384a33 in Ffuncall (nargs=2, args=0x82a370) at eval.c:2796
  #68 0x0137f3e7 in internal_condition_case_n (bfun=0x13847ec <Ffuncall>,
      nargs=2, args=0x82a370, handlers=XIL(0x30),
      hfun=0x105a91d <safe_eval_handler>) at eval.c:1435
  #69 0x0105ab54 in safe__call (inhibit_quit=false, nargs=2,
      func=XIL(0x42cb120), ap=0x82a434 "") at xdisp.c:2807
  #70 0x0105abcd in safe_call (nargs=2, func=XIL(0x42cb120)) at xdisp.c:2822
  #71 0x0105ac21 in safe_call1 (fn=XIL(0x42cb120), arg=make_fixnum(1))
      at xdisp.c:2833
  #72 0x0105e9c7 in handle_fontified_prop (it=0x82a6c0) at xdisp.c:4136
  #73 0x0105d271 in handle_stop (it=0x82a6c0) at xdisp.c:3664
  #74 0x01069335 in reseat (it=0x82a6c0, pos=..., force_p=true) at xdisp.c:6900
  #75 0x0105c4c3 in init_iterator (it=0x82a6c0, w=0x70c0b88, charpos=1,
      bytepos=1, row=0x68432b0, base_face_id=DEFAULT_FACE_ID) at xdisp.c:3265
  #76 0x0105c55b in start_display (it=0x82a6c0, w=0x70c0b88, pos=...)
      at xdisp.c:3281
  #77 0x01091a71 in try_window (window=XIL(0xa0000000070c0b88), pos=...,
      flags=1) at xdisp.c:19056
  #78 0x0108e687 in redisplay_window (window=XIL(0xa0000000070c0b88),
      just_this_one_p=false) at xdisp.c:18480
  #79 0x01086178 in redisplay_window_0 (window=XIL(0xa0000000070c0b88))
      at xdisp.c:16194
  #80 0x0137f1f7 in internal_condition_case_1 (
      bfun=0x108611d <redisplay_window_0>, arg=XIL(0xa0000000070c0b88),
      handlers=XIL(0xc000000005fd6d40), hfun=0x10860da <redisplay_window_error>)
      at eval.c:1379
  #81 0x0108609d in redisplay_windows (window=XIL(0xa0000000070c0b88))
      at xdisp.c:16174
  #82 0x010848b0 in redisplay_internal () at xdisp.c:15642
  #83 0x0108577c in redisplay_preserve_echo_area (from_where=2) at xdisp.c:15995
  #84 0x01019565 in Fredisplay (force=XIL(0)) at dispnew.c:6085
  #85 0x01384f7d in funcall_subr (subr=0x195a0e0 <Sredisplay>, numargs=0,
      args=0x82e128) at eval.c:2867
  #86 0x013849d9 in Ffuncall (nargs=1, args=0x82e120) at eval.c:2794
  #87 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005d746e0),
      vector=XIL(0xa000000005d744f0), maxdepth=make_fixnum(7),
      args_template=make_fixnum(769), nargs=1, args=0x82e638) at bytecode.c:633
  #88 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005d744c0), nargs=1,
      arg_vector=0x82e630) at eval.c:2989
  #89 0x013855f9 in apply_lambda (fun=XIL(0xa000000005d744c0),
      args=XIL(0xc000000000fc65e0), count=10) at eval.c:2926
  #90 0x01382c31 in eval_sub (form=XIL(0xc000000000fc65f0)) at eval.c:2318
  #91 0x0137afa2 in Fprogn (body=XIL(0xc000000000fc6580)) at eval.c:462
  #92 0x013860de in funcall_lambda (fun=XIL(0xc000000000fc55b0), nargs=0,
      arg_vector=0x82ecd8) at eval.c:3060
  #93 0x01384bef in Ffuncall (nargs=1, args=0x82ecd0) at eval.c:2808
  #94 0x0136e970 in Ffuncall_interactively (nargs=1, args=0x82ecd0)
      at callint.c:254
  #95 0x01384e99 in funcall_subr (subr=0x1960720 <Sfuncall_interactively>,
      numargs=1, args=0x82ecd0) at eval.c:2847
  #96 0x013849d9 in Ffuncall (nargs=2, args=0x82ecc8) at eval.c:2794
  #97 0x013830ff in Fapply (nargs=3, args=0x82ecc8) at eval.c:2377
  #98 0x0136f077 in Fcall_interactively (function=XIL(0x4cc4f70),
      record_flag=XIL(0), keys=XIL(0xa000000006803768)) at callint.c:342
  #99 0x01384ff4 in funcall_subr (subr=0x1960740 <Scall_interactively>,
      numargs=3, args=0x82f010) at eval.c:2872
  #100 0x013849d9 in Ffuncall (nargs=4, args=0x82f008) at eval.c:2794
  #101 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005ecd2a8),
      vector=XIL(0xa000000005ecd050), maxdepth=make_fixnum(13),
      args_template=make_fixnum(1025), nargs=1, args=0x82f610) at bytecode.c:633
  #102 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005ecd020), nargs=1,
      arg_vector=0x82f608) at eval.c:2989
  #103 0x01384a33 in Ffuncall (nargs=2, args=0x82f600) at eval.c:2796
  #104 0x01383eec in call1 (fn=XIL(0x3f30), arg1=XIL(0x4cc4f70)) at eval.c:2654
  #105 0x011e09da in command_loop_1 () at keyboard.c:1463
  #106 0x0137f10d in internal_condition_case (bfun=0x11dfd8f <command_loop_1>,
      handlers=XIL(0x90), hfun=0x11df003 <cmd_error>) at eval.c:1355
  #107 0x011df806 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1091
  #108 0x0137e2d8 in internal_catch (tag=XIL(0xdfb0),
      func=0x11df7ca <command_loop_2>, arg=XIL(0)) at eval.c:1116
  #109 0x011df785 in command_loop () at keyboard.c:1070
  #110 0x011dea8b in recursive_edit_1 () at keyboard.c:714
  #111 0x011ded01 in Frecursive_edit () at keyboard.c:786
  #112 0x011d361b in main (argc=2, argv=0xa42848) at emacs.c:2054

  Lisp Backtrace:
  "Automatic GC" (0x0)
  "modify-syntax-entry" (0x824368)
  "perl-quote-syntax-table" (0x824910)
  "perl-syntax-propertize-special-constructs" (0x825050)
  "perl-syntax-propertize-function" (0x825858)
  "syntax-propertize" (0x825e90)
  "internal--syntax-propertize" (0x826458)
  "re-search-forward" (0x8274d8)
  "font-lock-fontify-keywords-region" (0x827c20)
  "font-lock-default-fontify-region" (0x828268)
  "font-lock-fontify-region" (0x8287d8)
  0x7255bc0 PVEC_COMPILED
  "run-hook-wrapped" (0x829050)
  "jit-lock--run-functions" (0x829660)
  "jit-lock-fontify-now" (0x829d48)
  "jit-lock-function" (0x82a378)
  "redisplay_internal (C function)" (0x0)
  "redisplay" (0x82e128)
  "sit-for" (0x82e630)
  "foo" (0x82ecd8)
  "funcall-interactively" (0x82ecd0)
  "call-interactively" (0x82f010)
  "command-execute" (0x82f608)

Message #32 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rcopley <at> gmail.com
Cc: dancol <at> dancol.org, monnier <at> iro.umontreal.ca, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Fri, 17 Apr 2020 16:55:44 +0300

> Date: Fri, 17 Apr 2020 14:22:00 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 40661 <at> debbugs.gnu.org
> 
> Obviously, we cannot allow GC to run while regex routines do their
> work, because they are passed C pointers to buffer text.  The question
> is, where to disable GC?  We could do it inside
> update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
> from many places that evidently have no problems with GC.  So my
> suggestion would be to disable GC inside re_match_2_internal instead.

Alternatively, we could set the buffer's inhibit_shrinking flag while
in re_match_2_internal.  Although that flag was introduced for a
different purpose: for when we have stuff inside the gap that we don't
want to lose.  The name of the flag notwithstanding, I'm not sure we
want to conflate these two purposes.  But maybe it's better than
preventing the GC entirely.

Message #35 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>, <rcopley <at> gmail.com>
Cc: monnier <at> iro.umontreal.ca, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Fri, 17 Apr 2020 07:01:04 -0700

On April 17, 2020 6:56:00 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

>> Date: Fri, 17 Apr 2020 14:22:00 +0300
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> Cc: 40661 <at> debbugs.gnu.org
>>
>> Obviously, we cannot allow GC to run while regex routines do their
>> work, because they are passed C pointers to buffer text.  The question
>> is, where to disable GC?  We could do it inside
>> update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
>> from many places that evidently have no problems with GC.  So my
>> suggestion would be to disable GC inside re_match_2_internal instead.
>
> Alternatively, we could set the buffer's inhibit_shrinking flag while
> in re_match_2_internal.  Although that flag was introduced for a
> different purpose: for when we have stuff inside the gap that we don't
> want to lose.  The name of the flag notwithstanding, I'm not sure we
> want to conflate these two purposes.  But maybe it's better than
> preventing the GC entirely.

I think I'd prefer this approach to inhibiting GC entirely. I can imagine 
code allocating enough garbage that we really want to get rid of it, and 
inhibiting shrinking is more conservative.

Message #38 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Richard Copley <rcopley <at> gmail.com>, dancol <at> dancol.org,
 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Fri, 17 Apr 2020 11:28:49 -0400

> I finally succeeded to build a sophisticated enough trap to catch the
> culprit.  It's GC allright.  Which is not surprising: the commit
> pointed out by Richard changed re_match_2_internal to call
> UPDATE_SYNTAX_TABLE_FORWARD, which calls Lisp, and thus can trigger
> GC.  As seen from the backtrace, GC then calls compact_buffer, which
> calls enlarge_buffer_text (to shrink buffer text), and on MS-Windows
> -- and this is the w32-specific part -- we return some memory to the
> OS and relocate buffer text.
>
> Obviously, we cannot allow GC to run while regex routines do their
> work, because they are passed C pointers to buffer text.  The question
> is, where to disable GC?  We could do it inside
> update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
> from many places that evidently have no problems with GC.  So my
> suggestion would be to disable GC inside re_match_2_internal instead.
>
> Comments?

Looks fine to me.  I think a better fix is to move the execution of
compact_buffer: there's no reason it has to be done during GC, we just
need to do it "every once in a while" and the GC was a convenient point
for that.  But we could avoid several such problems if we were to run
such background tasks elsewhere.  It could still be linked to GC,
e.g. we could start a timer during the GC so its run at the next
opportunity.


        Stefan

Message #41 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: rcopley <at> gmail.com, dancol <at> dancol.org, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Fri, 17 Apr 2020 18:50:27 +0300

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Richard Copley <rcopley <at> gmail.com>,  dancol <at> dancol.org,
>   40661 <at> debbugs.gnu.org
> Date: Fri, 17 Apr 2020 11:28:49 -0400
> 
> > Obviously, we cannot allow GC to run while regex routines do their
> > work, because they are passed C pointers to buffer text.  The question
> > is, where to disable GC?  We could do it inside
> > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
> > from many places that evidently have no problems with GC.  So my
> > suggestion would be to disable GC inside re_match_2_internal instead.
> >
> > Comments?
> 
> Looks fine to me.

So you prefer disabling GC to setting the inhibit_shrinking flag?
I tend to agree with Daniel here, FWIW.

> I think a better fix is to move the execution of compact_buffer:
> there's no reason it has to be done during GC, we just need to do it
> "every once in a while" and the GC was a convenient point for that.
> But we could avoid several such problems if we were to run such
> background tasks elsewhere.  It could still be linked to GC, e.g. we
> could start a timer during the GC so its run at the next
> opportunity.

That's a project for another pandemic ;-)  I want a simple enough
solution for now that we could install on the release branch.

Message #44 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rcopley <at> gmail.com, dancol <at> dancol.org, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Fri, 17 Apr 2020 12:00:27 -0400

>> > Obviously, we cannot allow GC to run while regex routines do their
>> > work, because they are passed C pointers to buffer text.  The question
>> > is, where to disable GC?  We could do it inside
>> > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
>> > from many places that evidently have no problems with GC.  So my
>> > suggestion would be to disable GC inside re_match_2_internal instead.
>> > Comments?
>> Looks fine to me.
> So you prefer disabling GC to setting the inhibit_shrinking flag?
> I tend to agree with Daniel here, FWIW.

No, what I meant was that I'm fine with doing it "inside
re_match_2_internal" instead of "inside update_syntax_table_forward".
But I prefer if the "it" is setting `inhibit_shrinking` instead of
inhibiting the GC altogether.

> That's a project for another pandemic ;-)  I want a simple enough
> solution for now that we could install on the release branch.

Oh, for `emacs-27` that's clearly out, indeed.  But for `master` it
should not be that hard.  I think we can reuse `pending_funcalls`
for that.


        Stefan

Message #47 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: rcopley <at> gmail.com, dancol <at> dancol.org, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Sat, 18 Apr 2020 14:57:16 +0300

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: rcopley <at> gmail.com,  dancol <at> dancol.org,  40661 <at> debbugs.gnu.org
> Date: Fri, 17 Apr 2020 12:00:27 -0400
> 
> >> > Obviously, we cannot allow GC to run while regex routines do their
> >> > work, because they are passed C pointers to buffer text.  The question
> >> > is, where to disable GC?  We could do it inside
> >> > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called
> >> > from many places that evidently have no problems with GC.  So my
> >> > suggestion would be to disable GC inside re_match_2_internal instead.
> >> > Comments?
> >> Looks fine to me.
> > So you prefer disabling GC to setting the inhibit_shrinking flag?
> > I tend to agree with Daniel here, FWIW.
> 
> No, what I meant was that I'm fine with doing it "inside
> re_match_2_internal" instead of "inside update_syntax_table_forward".
> But I prefer if the "it" is setting `inhibit_shrinking` instead of
> inhibiting the GC altogether.

OK, thank you and Daniel for the feedback, and thanks to Richard for
finding this bug in the first place.

I installed a fix on the emacs-27 branch, it avoids the crashes in the
recipe posted by Richard.  Please eyeball the fix, in case I made some
stupid mistake (my environment is rather noisy today, so I trust
myself even less than I usually do).

Message #50 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Richard Copley <rcopley <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Daniel Colascione <dancol <at> dancol.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Mon, 20 Apr 2020 19:30:29 +0100

On Sat, 18 Apr 2020 at 12:57, Eli Zaretskii <eliz <at> gnu.org> wrote:
> OK, thank you and Daniel for the feedback, and thanks to Richard for
> finding this bug in the first place.
>
> I installed a fix on the emacs-27 branch, it avoids the crashes in the
> recipe posted by Richard.  Please eyeball the fix, in case I made some
> stupid mistake (my environment is rather noisy today, so I trust
> myself even less than I usually do).

There are a couple of points about the patch that aren't clear to me.
I think you have probably thought them already. I hope you don't mind
me asking for clarification.

I have a little worry about the situation where the procedure of doing
a "decoding" entails a regexp buffer search. (I can't make out whether
or not "decoding" means executing arbitrary Lisp code.) Is it
possible, with your patch, that we might re-enable buffer shrinking
sooner than desirable? (This is reiterating a worry you yourself
mentioned earlier, "I'm not sure we
want to conflate these two purposes".)

In the comment, you mention relocation as well as shrinking. Does it
make sense to combine this new guard with the existing
freeze_buffer_relocation in some way?

Message #53 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Richard Copley <rcopley <at> gmail.com>
Cc: dancol <at> dancol.org, monnier <at> iro.umontreal.ca, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Mon, 20 Apr 2020 22:06:59 +0300

> From: Richard Copley <rcopley <at> gmail.com>
> Date: Mon, 20 Apr 2020 19:30:29 +0100
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, Daniel Colascione <dancol <at> dancol.org>, 40661 <at> debbugs.gnu.org
> 
> I have a little worry about the situation where the procedure of doing
> a "decoding" entails a regexp buffer search. (I can't make out whether
> or not "decoding" means executing arbitrary Lisp code.)

This can happen if the coding-system has a post-read-conversion
attribute, yes.

> Is it possible, with your patch, that we might re-enable buffer
> shrinking sooner than desirable?

I don't think I understand the scenario you have in mind.  If decoding
calls regexp search, then we inhibit shrinking inside the regex
routines, not outside of them.  Which code will re-enable shrinking
before the regex routines return?

> (This is reiterating a worry you yourself mentioned earlier, "I'm
> not sure we want to conflate these two purposes".)

coding.c uses this flag only while it runs purely C code.  Lisp cannot
run at that point.  If a coding-system has a post-read-conversion, it
will run only after the C decoder finishes.

> In the comment, you mention relocation as well as shrinking.

It's the relocation that causes the crash, because following the
relocation we unmap a portion of the process memory, where previously
we had buffer text, from the process's address space.  Shrinking of
the gap is what triggers the relocation (when the original gap was
very large).

> Does it make sense to combine this new guard with the existing
> freeze_buffer_relocation in some way?

I thought about that, but decided against it, for two reasons.  First,
freeze_buffer_relocation should at some point go away: it is not a
no-op only in an Emacs that uses ralloc.c, which currently only the
MS-DOS build does, AFAIK.  And second, it freezes relocation at a
higher level than needed, above the re_match_2_internal function which
is the only one that can call Lisp.

Message #56 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Richard Copley <rcopley <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Daniel Colascione <dancol <at> dancol.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Tue, 21 Apr 2020 00:04:23 +0100

On Mon, 20 Apr 2020 at 20:07, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > From: Richard Copley <rcopley <at> gmail.com>
> > Date: Mon, 20 Apr 2020 19:30:29 +0100
> > Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, Daniel Colascione <dancol <at> dancol.org>, 40661 <at> debbugs.gnu.org
> >
> > I have a little worry about the situation where the procedure of doing
> > a "decoding" entails a regexp buffer search. (I can't make out whether
> > or not "decoding" means executing arbitrary Lisp code.)
>
> This can happen if the coding-system has a post-read-conversion
> attribute, yes.
>
> > Is it possible, with your patch, that we might re-enable buffer
> > shrinking sooner than desirable?
>
> I don't think I understand the scenario you have in mind.  If decoding
> calls regexp search, then we inhibit shrinking inside the regex
> routines, not outside of them.  Which code will re-enable shrinking
> before the regex routines return?
> > (This is reiterating a worry you yourself mentioned earlier, "I'm
> > not sure we want to conflate these two purposes".)
>

What I had in mind was that we do inhibit shrinking outside the regex
search routines, for example (coding.c:8005),

  current_buffer->text->inhibit_shrinking = 1;
  decode_coding (coding);
  current_buffer->text->inhibit_shrinking = 0;

I was worried about the case where a regex buffer search happens 'half
way through' decode_coding. Then something bad might happen during the
second half, when inhibit_shrinking is zero.

But ...

> coding.c uses this flag only while it runs purely C code.  Lisp cannot
> run at that point.  If a coding-system has a post-read-conversion, it
> will run only after the C decoder finishes.

... so that's OK then.

> > In the comment, you mention relocation as well as shrinking.
>
> It's the relocation that causes the crash, because following the
> relocation we unmap a portion of the process memory, where previously
> we had buffer text, from the process's address space.  Shrinking of
> the gap is what triggers the relocation (when the original gap was
> very large).
>
> > Does it make sense to combine this new guard with the existing
> > freeze_buffer_relocation in some way?
>
> I thought about that, but decided against it, for two reasons.  First,
> freeze_buffer_relocation should at some point go away: it is not a
> no-op only in an Emacs that uses ralloc.c, which currently only the
> MS-DOS build does, AFAIK.

OK.

> And second, it freezes relocation at a
> higher level than needed, above the re_match_2_internal function which
> is the only one that can call Lisp.

Right, but freeze_buffer_relocation was surely done like that on
purpose, to hoist it outside the loop in search_buffer_re. Anyway, I'm
sure it's no big deal.

Message #59 received at 40661 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Richard Copley <rcopley <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Daniel Colascione <dancol <at> dancol.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 40661 <at> debbugs.gnu.org
Subject: Re: bug#40661: Crash in regex search during redisplay
Date: Wed, 30 Sep 2020 05:49:17 +0200

Richard Copley <rcopley <at> gmail.com> writes:

>> And second, it freezes relocation at a
>> higher level than needed, above the re_match_2_internal function which
>> is the only one that can call Lisp.
>
> Right, but freeze_buffer_relocation was surely done like that on
> purpose, to hoist it outside the loop in search_buffer_re. Anyway, I'm
> sure it's no big deal.

If I'm skimming this thread correctly, the reported bug was fixed by Eli
at the time, and there was then some followup discussion.   But there
doesn't seem to be anything more to be done here, so I'm closing this
bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.