GNU bug report logs - #15107
24.3; replace-regexp-in-string wrong on \`

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Kevin Ryde <user42@HIDDEN>; Keywords: confirmed; dated Thu, 15 Aug 2013 22:17:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.
bug Marked as found in versions 25.1. Request was from npostavs@HIDDEN to control <at> debbugs.gnu.org. Full text available.
Added tag(s) confirmed. Request was from npostavs@HIDDEN to control <at> debbugs.gnu.org. Full text available.

Message received at 15107 <at> debbugs.gnu.org:


Received: (at 15107) by debbugs.gnu.org; 6 Mar 2016 06:35:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 06 01:35:21 2016
Received: from localhost ([127.0.0.1]:36080 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1acSHg-00016x-JD
	for submit <at> debbugs.gnu.org; Sun, 06 Mar 2016 01:35:21 -0500
Received: from mail-ig0-f178.google.com ([209.85.213.178]:32932)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <wrightmikea@HIDDEN>) id 1acQ9g-0005yh-4B
 for 15107 <at> debbugs.gnu.org; Sat, 05 Mar 2016 23:18:56 -0500
Received: by mail-ig0-f178.google.com with SMTP id ig19so2985514igb.0
 for <15107 <at> debbugs.gnu.org>; Sat, 05 Mar 2016 20:18:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:date:message-id:subject:from:to;
 bh=UFA8re6XnRuDq4Lr4tqpPF1qhWKL6SnhUvKtopeo1h0=;
 b=lbrifFJvYHr5/NZWu2v4v/QfvcsqUicHcUgsl5vPij9kE8FYOWn+QFx7NmshmVQ3wV
 Tqy/txRAYDR6RHaeU0ej3r0Q7byIle0wpTnB9y7/vgxzLj6PZM9cS/DZi4IsFZ2+unmC
 NRAnkg+3N7ksp5FAjFiRMXnOX+/nUE8U0z6cf3JLDie58Lsute3Gey6mewwabNA84pM6
 yuTUQapL/QVt9t1yhBuEnMl95fLrOWwtcfESxY4KJxT5GpH3Z/laL8CM5BiJYbhg6/pE
 7cXhqtCJ8ptgkaHFzUFp5VV04kfEfa/z20rmcVc/g/JbHhSggvq6LXcfXYrAH2e3MKPc
 l0yQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:mime-version:date:message-id:subject:from:to;
 bh=UFA8re6XnRuDq4Lr4tqpPF1qhWKL6SnhUvKtopeo1h0=;
 b=U+YBJ8vt0Lv7RDdUtoyCDJ4H4jao8rWA79mRXXRvBCduMoeH6G6wA//cZCoeLM+f3J
 4z7wXKY3mAubddOf3p+nAnAwTJgsGsWCsWt3Rgj9+OL6FI3nGk/PMfxBxW/H31dIsaur
 Js8EvWlJBN2jdTcDoASXyhbKkEbMuDCwVnzYb0e9QXYH1luwulqMrH9+iGz3e4KsXjXL
 7ssq+achl3mB1UEbqaThF2x/ixAHrDRxGAXG2G8ZgBQbQmk5M1lwoyxCsr3PsD0gX1we
 P5M6vT256eRXaS/+uzQT84wGuHtWLXgDI4Ohwh8Y1Grb3XNIEaPFDybNsCWpZB1Ke0EO
 FCCQ==
X-Gm-Message-State: AD7BkJJbZwwpJyRdvcsPrVBJ6LMJh6ZkaysUL6tBonh3UJxyW5xAvxbTlFh6BaN5gy6GANwyHI0MWK1gqtzOcg==
MIME-Version: 1.0
X-Received: by 10.50.160.9 with SMTP id xg9mr4046625igb.41.1457237930512; Sat,
 05 Mar 2016 20:18:50 -0800 (PST)
Received: by 10.50.30.234 with HTTP; Sat, 5 Mar 2016 20:18:50 -0800 (PST)
Date: Sat, 5 Mar 2016 20:18:50 -0800
Message-ID: <CAGkLq6ejGGE3r9HsNYehfd63=6WPJf3HWYcEOG7pbmp7D9JOAQ@HIDDEN>
Subject: Re: bug#15107: reproduced on emacs-25 branch
From: Michael Wright <wrightmikea@HIDDEN>
To: 15107 <at> debbugs.gnu.org
Content-Type: multipart/alternative; boundary=001a11c31746339b7e052d59a529
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 15107
X-Mailman-Approved-At: Sun, 06 Mar 2016 01:35:18 -0500
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

--001a11c31746339b7e052d59a529
Content-Type: text/plain; charset=UTF-8

Kevin Ryde <user42@HIDDEN> writes:

> replace-regexp-in-string behaves incorrectly if a regexp has \` among
> its matches.
>
>     (replace-regexp-in-string "\\`\\|X" "Z" "--XX--" t t)
>     =>
>     "Z--ZXZX--"
>
> where I expected
>
>     "Z--ZZ--"
>
> This seems to be due to the optimization in replace-regexp-in-string
> which re-matches on the matched substring.  \' can match the substring
> where it did not match in the middle of the full string.  In the example
> above "X" is the match in the full string, but on taking that "X" as a
> substring it can match "\\`".

I built the emacs-25 git branch I recreated the above bug today.

GNU Emacs 25.0.92.1 (x86_64-apple-darwin13.4.0, NS appkit-1265.21 Version
10.9.5 (Build 13F1507))
 of 2016-03-05

--001a11c31746339b7e052d59a529
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Kevin Ryde &lt;<a href=3D"mailto:user42@HIDDEN">u=
ser42@HIDDEN</a>&gt; writes:<br></div><div><br></div><div>&gt; replace-=
regexp-in-string behaves incorrectly if a regexp has \` among</div><div>&gt=
; its matches.</div><div>&gt;</div><div>&gt; =C2=A0 =C2=A0 (replace-regexp-=
in-string &quot;\\`\\|X&quot; &quot;Z&quot; &quot;--XX--&quot; t t)</div><d=
iv>&gt; =C2=A0 =C2=A0 =3D&gt;</div><div>&gt; =C2=A0 =C2=A0 &quot;Z--ZXZX--&=
quot;</div><div>&gt;</div><div>&gt; where I expected</div><div>&gt;</div><d=
iv>&gt; =C2=A0 =C2=A0 &quot;Z--ZZ--&quot;</div><div>&gt;</div><div>&gt; Thi=
s seems to be due to the optimization in replace-regexp-in-string</div><div=
>&gt; which re-matches on the matched substring. =C2=A0\&#39; can match the=
 substring</div><div>&gt; where it did not match in the middle of the full =
string.=C2=A0 In the example</div><div>&gt; above &quot;X&quot; is the matc=
h in the full string, but on taking that &quot;X&quot; as a</div><div>&gt; =
substring it can match &quot;\\`&quot;.</div><div><br></div><div>I built th=
e emacs-25 git branch I recreated the above bug today.=C2=A0</div><div><br>=
</div><div><div>GNU Emacs 25.0.92.1 (x86_64-apple-darwin13.4.0, NS appkit-1=
265.21 Version 10.9.5 (Build 13F1507))</div><div>=C2=A0of 2016-03-05</div><=
/div></div>

--001a11c31746339b7e052d59a529--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#15107; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 15 Aug 2013 22:16:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Aug 15 18:16:57 2013
Received: from localhost ([127.0.0.1]:33081 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1VA5qj-0002hp-Fl
	for submit <at> debbugs.gnu.org; Thu, 15 Aug 2013 18:16:57 -0400
Received: from eggs.gnu.org ([208.118.235.92]:40205)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <gg@HIDDEN>) id 1VA5qh-0002ha-4h
 for submit <at> debbugs.gnu.org; Thu, 15 Aug 2013 18:16:56 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <gg@HIDDEN>) id 1VA5qR-0003Z1-Vb
 for submit <at> debbugs.gnu.org; Thu, 15 Aug 2013 18:16:49 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-100.0 required=5.0 tests=BAYES_20, USER_IN_WHITELIST
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:36596)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <gg@HIDDEN>) id 1VA5qR-0003Yw-Sj
 for submit <at> debbugs.gnu.org; Thu, 15 Aug 2013 18:16:39 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:41801)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <gg@HIDDEN>) id 1VA5qJ-0001Ps-Pe
 for bug-gnu-emacs@HIDDEN; Thu, 15 Aug 2013 18:16:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <gg@HIDDEN>) id 1VA5qB-0003VZ-Nt
 for bug-gnu-emacs@HIDDEN; Thu, 15 Aug 2013 18:16:31 -0400
Received: from mailout2-12.pacific.net.au ([125.255.80.139]:54069
 helo=mailout4-syd3.pacific.net.au)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <gg@HIDDEN>) id 1VA5qB-0003UV-DV
 for bug-gnu-emacs@HIDDEN; Thu, 15 Aug 2013 18:16:23 -0400
Received: from mailproxy3-syd3.pacific.net.au (mailproxy3-syd3.pacific.net.au
 [61.8.2.164])
 by mailout4-syd3.pacific.net.au (Postfix) with ESMTP id 53ECF480F79
 for <bug-gnu-emacs@HIDDEN>; Fri, 16 Aug 2013 08:16:18 +1000 (EST)
Received: from blah.blah (unknown [203.26.175.102])
 by mailproxy3-syd3.pacific.net.au (Postfix) with ESMTP id 7D6AF578071
 for <bug-gnu-emacs@HIDDEN>; Fri, 16 Aug 2013 08:16:17 +1000 (EST)
Received: from gg by blah.blah with local (Exim 4.80)
 (envelope-from <gg@HIDDEN>) id 1VA5ph-00018B-3H
 for bug-gnu-emacs@HIDDEN; Fri, 16 Aug 2013 08:15:53 +1000
From: Kevin Ryde <user42@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 24.3; replace-regexp-in-string wrong on \`
Date: Fri, 16 Aug 2013 08:15:53 +1000
Message-ID: <87eh9uk3c6.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -3.4 (---)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.4 (---)

replace-regexp-in-string behaves incorrectly if a regexp has \` among
its matches.

    (replace-regexp-in-string "\\`\\|X" "Z" "--XX--" t t)
    =>
    "Z--ZXZX--"

where I expected

    "Z--ZZ--"

This seems to be due to the optimization in replace-regexp-in-string
which re-matches on the matched substring.  \' can match the substring
where it did not match in the middle of the full string.  In the example
above "X" is the match in the full string, but on taking that "X" as a
substring it can match "\\`".

Probably similar mismatches on the substring occur for things like \' ^
$ \b \< etc.  Maybe the comment in the code about munging the match data
would be a better way.





In GNU Emacs 24.3.1 (i486-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2013-05-29 on blah.blah, modified by Debian
System Description:	Debian GNU/Linux testing/unstable

Configured using:
 `configure '--build' 'i486-linux-gnu' '--build' 'i486-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/i386-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=lucid' '--with-toolkit-scroll-bars' '--without-gconf'
 'build_alias=i486-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'
 'LDFLAGS=-Wl,-z,relro -Wl,-znocombreloc'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LANG: en_AU
  locale-coding-system: iso-latin-1-unix
  default enable-multibyte-characters: t




Acknowledgement sent to Kevin Ryde <user42@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#15107; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sat, 6 Aug 2016 00:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.