GNU bug report logs - #65536
30.0.50; replace-regexp-in-string documentation does not mention it saves match data

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 65536 in the body.
You can then email your comments to 65536 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; replace-regexp-in-string documentation does not mention it
 saves match data
Date: Fri, 25 Aug 2023 15:16:29 +0200

Not sure about this one, maybe I have overlooked something ...

The Emacs Lisp manual says:

   Notice that all functions are allowed to overwrite the match data
   unless they’re explicitly documented not to do so.

And in particular function `replace-regexp-in-string' could be
assumed to modify the match data, but it uses an explicit call to
`save-match-data' to not do so.

Maybe this should be explicitly documented in its doc string and/or
the Emacs Lisp manual ((elisp) Search and Replace) so that users of
the function can rely on that fact?

Message #8 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Cc: 65536 <at> debbugs.gnu.org
Subject: Re: bug#65536: 30.0.50;
 replace-regexp-in-string documentation does not mention it saves
 match data
Date: Fri, 25 Aug 2023 17:18:36 +0300

> Date: Fri, 25 Aug 2023 15:16:29 +0200
> From:  Jens Schmidt via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> Not sure about this one, maybe I have overlooked something ...
> 
> The Emacs Lisp manual says:
> 
>    Notice that all functions are allowed to overwrite the match data
>    unless they’re explicitly documented not to do so.
> 
> And in particular function `replace-regexp-in-string' could be
> assumed to modify the match data, but it uses an explicit call to
> `save-match-data' to not do so.
> 
> Maybe this should be explicitly documented in its doc string and/or
> the Emacs Lisp manual ((elisp) Search and Replace) so that users of
> the function can rely on that fact?

Is it important to promise never to clobber match-data in this
function?

Message #11 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 65536 <at> debbugs.gnu.org
Subject: Re: bug#65536: 30.0.50; replace-regexp-in-string documentation does
 not mention it saves match data
Date: Fri, 25 Aug 2023 17:26:03 +0200

On 2023-08-25  16:18, Eli Zaretskii wrote:
>> Date: Fri, 25 Aug 2023 15:16:29 +0200
>> From:  Jens Schmidt via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>

>> Maybe this should be explicitly documented in its doc string and/or
>> the Emacs Lisp manual ((elisp) Search and Replace) so that users of
>> the function can rely on that fact?
> 
> Is it important to promise never to clobber match-data in this
> function?

Not sure whether the following is convincing enough ...

- I came across this question in the following scenario:

  (match-fat-regexp with a lot of subres, one of which matches quoted
   chars: "\\(?5:\\(?:[[:alnum:]_]+\\|\\\\.\\)+\\)")
  (let ((a (match-string 1 input))
        ;; unquote quoted chars
        (b (replace-regexp-in-string
            "\\\\\\(.\\)" "\\1"
            (match-string 5 input)
            t nil))
        (c (match-string 7 input)))
    ...)

  So for that and similar scenarios it would be helpful.

- And since `replace-regexp-in-string' contains that `save-match-data'
  for a long time already (at least since Emacs 23) I guess that a lot
  of authors have been relying on that fact, either consciously, after
  having peeked into the function, or unconsciously.

- If it helps, I could provide a patch ...

Message #14 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 65536 <at> debbugs.gnu.org
Subject: Re: bug#65536: 30.0.50; replace-regexp-in-string documentation does
 not mention it saves match data
Date: Fri, 25 Aug 2023 18:56:15 +0200

On 2023-08-25  17:26, Jens Schmidt wrote:
> On 2023-08-25  16:18, Eli Zaretskii wrote:
>>> Date: Fri, 25 Aug 2023 15:16:29 +0200
>>> From:  Jens Schmidt via "Bug reports for GNU Emacs,
>>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
>>> Maybe this should be explicitly documented in its doc string and/or
>>> the Emacs Lisp manual ((elisp) Search and Replace) so that users of
>>> the function can rely on that fact?
>>
>> Is it important to promise never to clobber match-data in this
>> function?
> 
> Not sure whether the following is convincing enough ...

OTOH, I just found the following in the doc string of
`save-match-data', which means that `replace-regexp-in-string'
is actually not "convention-conforming":

  NOTE: The convention in Elisp is that any function, except for a few
  exceptions like car/assoc/+/goto-char, can clobber the match data,
  so ‘save-match-data’ should normally be used to save *your* match data
  rather than your caller’s match data.

The final decision is yours ...

Message #17 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>,
 Stefan Kangas <stefankangas <at> gmail.com>
Cc: 65536 <at> debbugs.gnu.org
Subject: Re: bug#65536: 30.0.50; replace-regexp-in-string documentation does
 not mention it saves match data
Date: Sat, 26 Aug 2023 09:09:11 +0300

> Date: Fri, 25 Aug 2023 18:56:15 +0200
> From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> Cc: 65536 <at> debbugs.gnu.org
> 
> On 2023-08-25  17:26, Jens Schmidt wrote:
> > On 2023-08-25  16:18, Eli Zaretskii wrote:
> >>> Date: Fri, 25 Aug 2023 15:16:29 +0200
> >>> From:  Jens Schmidt via "Bug reports for GNU Emacs,
> >>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> > 
> >>> Maybe this should be explicitly documented in its doc string and/or
> >>> the Emacs Lisp manual ((elisp) Search and Replace) so that users of
> >>> the function can rely on that fact?
> >>
> >> Is it important to promise never to clobber match-data in this
> >> function?
> > 
> > Not sure whether the following is convincing enough ...
> 
> OTOH, I just found the following in the doc string of
> `save-match-data', which means that `replace-regexp-in-string'
> is actually not "convention-conforming":
> 
>   NOTE: The convention in Elisp is that any function, except for a few
>   exceptions like car/assoc/+/goto-char, can clobber the match data,
>   so ‘save-match-data’ should normally be used to save *your* match data
>   rather than your caller’s match data.
> 
> The final decision is yours ...

I think it's okay to leave things as they are.

Stefan, WDYT?

Message #20 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> posteo.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 65536 <at> debbugs.gnu.org, Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65536: 30.0.50; replace-regexp-in-string documentation does
 not mention it saves match data
Date: Sat, 26 Aug 2023 06:24:55 +0000

Eli Zaretskii <eliz <at> gnu.org> writes:

>>   NOTE: The convention in Elisp is that any function, except for a few
>>   exceptions like car/assoc/+/goto-char, can clobber the match data,
>>   so ‘save-match-data’ should normally be used to save *your* match data
>>   rather than your caller’s match data.
>> 
>> The final decision is yours ...
>
> I think it's okay to leave things as they are.

I can share some experience from me dropping some `save-match-data' from
Org parser - I had to fight the consequences, as a lot of Org's own
code implicitly assumed that match data is saved, which caused huge
number of tests to fail. Fixing this was not fun, although I was able to
do it because that particular function had a history modifying match
data in the past and only relatively recent code additions were
affected.

If something like `replace-regexp-in-string' suddenly changes its
behaviour (even undocumented), I expect issues in third-party code.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Message #23 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 65536 <at> debbugs.gnu.org, Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Subject: Re: bug#65536: 30.0.50; replace-regexp-in-string documentation does
 not mention it saves match data
Date: Sat, 26 Aug 2023 10:40:18 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> I think it's okay to leave things as they are.
>
> Stefan, WDYT?

We do not want to change replace-regexp-in-string's behavior.  This
goes without saying IMO, and needs no documentation changes.

I also see nothing highly surprising or unusual about it saving match
data.  Interested users can easily read the code to see if it does,
just like Jens did.

So I also think it's fine not to change anything here, on balance.

Message #26 received at 65536 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 65536 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#65536: 30.0.50; replace-regexp-in-string documentation does
 not mention it saves match data
Date: Sat, 26 Aug 2023 12:09:57 +0300

tags 65536 notabug wontfix
close 65536
thanks

> From: Stefan Kangas <stefankangas <at> gmail.com>
> Date: Sat, 26 Aug 2023 10:40:18 +0200
> Cc: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>, 65536 <at> debbugs.gnu.org
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > I think it's okay to leave things as they are.
> >
> > Stefan, WDYT?
> 
> We do not want to change replace-regexp-in-string's behavior.  This
> goes without saying IMO, and needs no documentation changes.
> 
> I also see nothing highly surprising or unusual about it saving match
> data.  Interested users can easily read the code to see if it does,
> just like Jens did.
> 
> So I also think it's fine not to change anything here, on balance.

Thanks, so I'm closing this bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.