GNU bug report logs - #42296
27.0.91; Correct manual entry for 'concat' w.r.t. allocation [PATCH]

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 42296 in the body.
You can then email your comments to 42296 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.91; Correct manual entry for 'concat' w.r.t. allocation [PATCH]
Date: Thu, 9 Jul 2020 17:54:10 +0200

[Message part 1 (text/plain, inline)]

'concat' hasn't guaranteed the allocation of a new string since 1997 at least (79d137ffe7d). Remove claims to the contrary from the manual.

[0001-Correct-concat-manual-entry.patch (application/octet-stream, attachment)]

Message #8 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91;
 Correct manual entry for 'concat' w.r.t. allocation [PATCH]
Date: Thu, 09 Jul 2020 20:31:38 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Thu, 9 Jul 2020 17:54:10 +0200
> 
> 'concat' hasn't guaranteed the allocation of a new string since 1997 at least (79d137ffe7d). Remove claims to the contrary from the manual.

This loses too much useful information, IMO.  Let's please say that
the result is sometimes ("frequently"?), but not always, a new string,
and that programs should not rely on that aspect.

Thanks.

Message #11 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Thu, 9 Jul 2020 19:56:59 +0200

[Message part 1 (text/plain, inline)]

9 juli 2020 kl. 19.31 skrev Eli Zaretskii <eliz <at> gnu.org>:

> This loses too much useful information, IMO.  Let's please say that
> the result is sometimes ("frequently"?), but not always, a new string,
> and that programs should not rely on that aspect.

Here is a new version that adds this information, and recommends copy-sequence for mutability.

[0001-Correct-concat-manual-entry.patch (application/octet-stream, attachment)]

Message #14 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Thu, 09 Jul 2020 21:51:19 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Thu, 9 Jul 2020 19:56:59 +0200
> Cc: 42296 <at> debbugs.gnu.org
> 
> > This loses too much useful information, IMO.  Let's please say that
> > the result is sometimes ("frequently"?), but not always, a new string,
> > and that programs should not rely on that aspect.
> 
> Here is a new version that adds this information, and recommends copy-sequence for mutability.

That's not really what I asked for.

And how does mutability enter the picture?  We could say something
about it (but then we'd have to be less terse), but that doesn't in
any way replace the need to say that in many cases the value will be a
new string, IMO.

Thanks.

Message #17 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Thu, 9 Jul 2020 21:17:14 +0200

9 juli 2020 kl. 20.51 skrev Eli Zaretskii <eliz <at> gnu.org>:

> That's not really what I asked for.

Then I misunderstood; would you explain what you mean in a different way?
The old text incorrectly stated that the return value always is a new string; the last patch changes this to saying that the value may or may not be a new string, and that the user therefore shouldn't assume that the value be mutable. If that was not what you asked for, then what was it, and why?

> And how does mutability enter the picture?  We could say something
> about it (but then we'd have to be less terse), but that doesn't in
> any way replace the need to say that in many cases the value will be a
> new string, IMO.

Sorry, but I still don't understand. What salient quality is there other than mutability? There is identity (uniqueness), but that is included as well. No user is ever worried about that the returned value may actually be a new string; it's very much the other way around.

Message #20 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: mattiase <at> acm.org
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91;
 Correct manual entry for 'concat' w.r.t. allocation [PATCH]
Date: Thu, 09 Jul 2020 22:20:20 +0300

> Date: Thu, 09 Jul 2020 21:51:19 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 42296 <at> debbugs.gnu.org
> 
> That's not really what I asked for.
> 
> And how does mutability enter the picture?  We could say something
> about it (but then we'd have to be less terse), but that doesn't in
> any way replace the need to say that in many cases the value will be a
> new string, IMO.

Here's what I had in mind:

  This function frequently, but not always, constructs a new string
  that is not @code{eq} to any existing string.  Lisp programs should
  not rely on the result being a new string nor on it being @code{eq}
  to an existing string.

  When this function returns a string @code{eq] to another, changing
  the result will also change that other string; to avoid that, use
  @code{copy-sequence} on the result.

WDYT?

Message #23 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Thu, 09 Jul 2020 22:24:03 +0300

> Feedback-ID:mattiase <at> acm.or
> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Thu, 9 Jul 2020 21:17:14 +0200
> Cc: 42296 <at> debbugs.gnu.org
> 
> 9 juli 2020 kl. 20.51 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> > That's not really what I asked for.
> 
> Then I misunderstood; would you explain what you mean in a different way?

I tried to explain that by showing the text I had in mind; I sent that
a minute ago.

> > And how does mutability enter the picture?  We could say something
> > about it (but then we'd have to be less terse), but that doesn't in
> > any way replace the need to say that in many cases the value will be a
> > new string, IMO.
> 
> Sorry, but I still don't understand. What salient quality is there other than mutability? There is identity (uniqueness), but that is included as well. No user is ever worried about that the returned value may actually be a new string; it's very much the other way around.

To my mind, immutability (or, rather, the adverse effects of modifying
the result of 'concat') is the result of the identity.  So it is
conceptually wrong to talk about mutability _instead_ of the identity.
Identity is the basic reason, immutability is the side effect.  Which
is what I tried to say in the text I suggested.

Thanks.

Message #26 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Fri, 10 Jul 2020 19:04:48 +0200

9 juli 2020 kl. 21.24 skrev Eli Zaretskii <eliz <at> gnu.org>:

>  This function frequently, but not always, constructs a new string
>  that is not @code{eq} to any existing string.  Lisp programs should
>  not rely on the result being a new string nor on it being @code{eq}
>  to an existing string.
> 
>  When this function returns a string @code{eq] to another, changing
>  the result will also change that other string; to avoid that, use
>  @code{copy-sequence} on the result.

Thank you! First a minor detail: the word 'frequently' doesn't convey any useful information since the user isn't supposed to take any chances -- either the returned value is always new and unaliased, or there is no such guarantee. The frequency isn't relevant, and we shouldn't encourage the user to act as if it were by talking about it.

> To my mind, immutability (or, rather, the adverse effects of modifying
> the result of 'concat') is the result of the identity.  So it is
> conceptually wrong to talk about mutability _instead_ of the identity.
> Identity is the basic reason, immutability is the side effect.  Which
> is what I tried to say in the text I suggested.

I understand what you mean, but do not necessarily agree in every respect. A string may be unsuitable for modification for several (overlapping) reasons:

* It is a string literal that occurs in the program.
* It is a string produced by macro-expansion.
* It is a string resulting from other compile-time evaluation (eg, constant folding).
* It is a string residing in read-only storage of some kind (via dumping).
* It aliases another string in the program.
* It is an actual read-only string value (which we don't really have today, but we should not preclude its introduction in the future).

Thus mutability and identity are related concepts but neither clearly dominates the other; stating both is probably useful, for clarity if nothing else. Saying that mutating the return value merely entails changing some other string is understating and misleading: the consequences are not well-defined, and may include mutating programs and the raising of errors. (It will never reach C levels of undefined behaviour if we can help it, however.)

There is also the question whether the function can return one of its arguments. If so, the caller not only needs to abstain from mutating the returned value, but also the argument, even though it may be safely created by the caller. Currently, (concat x) never returns a string argument, unless it's a compile-time constant that cannot be mutated anyway. It's unclear whether we can guarantee this behaviour.

But let's be constructive: Taking your proposed text and performing some light surgery, we might end up with:

 This function does not always allocate a new string.  Callers should
 not rely on the result being a new string nor on it being @code{eq}
 to an existing string.

 In particular, the returned value should not be altered. To obtain
 a string that can be mutated, use @code{copy-sequence} on the result.

Message #29 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Fri, 10 Jul 2020 21:08:44 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Fri, 10 Jul 2020 19:04:48 +0200
> Cc: 42296 <at> debbugs.gnu.org
> 
> 9 juli 2020 kl. 21.24 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> >  This function frequently, but not always, constructs a new string
> >  that is not @code{eq} to any existing string.  Lisp programs should
> >  not rely on the result being a new string nor on it being @code{eq}
> >  to an existing string.
> > 
> >  When this function returns a string @code{eq] to another, changing
> >  the result will also change that other string; to avoid that, use
> >  @code{copy-sequence} on the result.
> 
> Thank you! First a minor detail: the word 'frequently' doesn't convey any useful information since the user isn't supposed to take any chances -- either the returned value is always new and unaliased, or there is no such guarantee. The frequency isn't relevant, and we shouldn't encourage the user to act as if it were by talking about it.

"Frequently" describes what actually happens.  Describing facts is not
"encouraging" users to do anything, especially since the very next
sentence tells them not to draw any far-reaching conclusions.

IOW, we should treat our users as grown-up adults, not as children
from whom we need to hide information.

>  This function does not always allocate a new string.  Callers should
>  not rely on the result being a new string nor on it being @code{eq}
>  to an existing string.
> 
>  In particular, the returned value should not be altered. To obtain
>  a string that can be mutated, use @code{copy-sequence} on the result.

Fine with me, except that "should not be altered": I object to that,
unless we explain why.  My proposed text included such an explanation;
without it, this looks like another dogma that someone sooner or later
will come up and challenge.

Thanks.

Message #32 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 13:15:39 +0200

10 juli 2020 kl. 20.08 skrev Eli Zaretskii <eliz <at> gnu.org>:

> Fine with me, except that "should not be altered": I object to that,
> unless we explain why.  My proposed text included such an explanation;
> without it, this looks like another dogma that someone sooner or later
> will come up and challenge.

Let's try again then, elaborating the second paragraph:

 This function does not always allocate a new string.  Callers should
 not rely on the result being a new string nor on it being @code{eq}
 to an existing string.

 In particular, the returned value should not be altered, because doing
 so may inadvertently change another string, alter a constant string in
 the program, or even raise an error.  To obtain a string that can be
 mutated, use @code{copy-sequence} on the result.

Message #35 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 14:52:25 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sat, 11 Jul 2020 13:15:39 +0200
> Cc: 42296 <at> debbugs.gnu.org
> 
>  In particular, the returned value should not be altered, because doing
>  so may inadvertently change another string, alter a constant string in
>  the program, or even raise an error.

Too definitive to my palate, sorry.

You may wish reading the recent long discussions of what is/should be
and what isn't/shouldn't be immutable.  Turns out some of us are very
sensitive to "should not" being used where "we recommend you not" or
"make sure you know what you are doing" is what actually is the truth.

Once again, we are not talking to kindergarten kids.

Message #38 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 14:51:06 +0200

11 juli 2020 kl. 13.52 skrev Eli Zaretskii <eliz <at> gnu.org>:

> Too definitive to my palate, sorry.

Apologies -- I do my best to read your mind.

> You may wish reading the recent long discussions of what is/should be
> and what isn't/shouldn't be immutable.

Thank you, but reading it once was quite enough.

>  Turns out some of us are very
> sensitive to "should not" being used where "we recommend you not" or
> "make sure you know what you are doing" is what actually is the truth.

Very well, here is a revised proposal:

 This function does not always allocate a new string.  Callers are
 advised not rely on the result being a new string nor on it being @code{eq}
 to an existing string.

 In particular, mutating the returned value may inadvertently change another
 string, alter a constant string in the program, or even raise an error.
 To obtain a string that can be mutated, use @code{copy-sequence} on the result.

Message #41 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 16:07:32 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sat, 11 Jul 2020 14:51:06 +0200
> Cc: 42296 <at> debbugs.gnu.org
> 
> 11 juli 2020 kl. 13.52 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> > Too definitive to my palate, sorry.
> 
> Apologies -- I do my best to read your mind.

I posted my proposed text so that you won't have to.

>  This function does not always allocate a new string.  Callers are
>  advised not rely on the result being a new string nor on it being @code{eq}
>  to an existing string.
> 
>  In particular, mutating the returned value may inadvertently change another
>  string, alter a constant string in the program, or even raise an error.
>  To obtain a string that can be mutated, use @code{copy-sequence} on the result.

Fine with me, with one correction: last sentence will sound better if
reworded like this:

  To obtain a string that you can safely mutate, use
  @code{copy-sequence} on the result.

Message #46 received at 42296-done <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296-done <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 15:16:24 +0200

11 juli 2020 kl. 15.07 skrev Eli Zaretskii <eliz <at> gnu.org>:

> Fine with me, with one correction:

Thank you -- correction applied, and result pushed to master (by mistake) and then to emacs-27.

Message #49 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 16:37:17 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sat, 11 Jul 2020 15:16:24 +0200
> Cc: 42296-done <at> debbugs.gnu.org
> 
> Thank you -- correction applied, and result pushed to master (by mistake) and then to emacs-27.

Thanks.

Next time you have to do something like this, use "git cherry-pick -x"
to bring commits from master to the release branch, since that helps
merging in the opposite direction (I think the way you did it might
cause merge conflicts).

Message #52 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 15:47:05 +0200

11 juli 2020 kl. 15.37 skrev Eli Zaretskii <eliz <at> gnu.org>:

> Next time you have to do something like this, use "git cherry-pick -x"
> to bring commits from master to the release branch, since that helps
> merging in the opposite direction (I think the way you did it might
> cause merge conflicts).

Understood.

Message #55 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Mattias Engdegård
 <mattiase <at> acm.org>
Cc: 42296 <at> debbugs.gnu.org
Subject: RE: bug#42296: 27.0.91; Correct manual entry for 'concat' w.r.t.
 allocation [PATCH]
Date: Sat, 11 Jul 2020 09:17:30 -0700 (PDT)

> >  This function does not always allocate a new string.  Callers are
> >  advised not rely on the result being a new string nor on it being
> @code{eq}
> >  to an existing string.
> >
> >  In particular, mutating the returned value may inadvertently change
> another
> >  string, alter a constant string in the program, or even raise an
> error.
> >  To obtain a string that can be mutated, use @code{copy-sequence} on
> the result.
> 
> Fine with me, with one correction: last sentence will sound better if
> reworded like this:
> 
>   To obtain a string that you can safely mutate, use
>   @code{copy-sequence} on the result.

FTR, I think that the doc of `concat' should really
make clear that it (now, and apparently for a while
now) has this odd (yes) behavior of not guaranteeing
a new string.  Not sure why we made this change, but
I imagine it was in a zeal to optimize.

At a minimum, please make this very clear, and say
explicitly that to ensure that you get a new string
you can use `copy-sequence' (as you've mentioned),
and that you can alternatively use `seq-concatenate':

 (seq-concatenate 'string &rest SEQUENCES)

Message #58 received at 42296 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: mattiase <at> acm.org, 42296 <at> debbugs.gnu.org
Subject: Re: bug#42296: 27.0.91;
 Correct manual entry for 'concat' w.r.t. allocation [PATCH]
Date: Sat, 11 Jul 2020 23:03:31 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >  To obtain a string that can be mutated, use @code{copy-sequence} on the result.

  > Fine with me, with one correction: last sentence will sound better if
  > reworded like this:

  >   To obtain a string that you can safely mutate, use
  >   @code{copy-sequence} on the result.

Yes, it is almost always better to switch to active voice
and state the subject.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.