Received: (at 79678) by debbugs.gnu.org; 26 Oct 2025 14:30:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 26 10:30:39 2025
Received: from localhost ([127.0.0.1]:45883 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vD1lV-0000VZ-T1
for submit <at> debbugs.gnu.org; Sun, 26 Oct 2025 10:30:38 -0400
Received: from mail.muc.de ([193.149.48.3]:43687)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1vD1lS-00008j-T5
for 79678 <at> debbugs.gnu.org; Sun, 26 Oct 2025 10:30:35 -0400
Received: (qmail 52425 invoked by uid 3782); 26 Oct 2025 15:30:27 +0100
Received: from muc.de (p4fe154dd.dip0.t-ipconnect.de [79.225.84.221]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Sun, 26 Oct 2025 15:30:27 +0100
Received: (qmail 29844 invoked by uid 1000); 26 Oct 2025 14:30:26 -0000
Date: Sun, 26 Oct 2025 14:30:26 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#79678: Problems with the symbols with position mechanism
Message-ID: <aP4wgllfIAT-NMy-@MAC.fritz.box>
References: <aPkdPnGEi5sRT7km@HIDDEN>
<jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN>
<aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN>
<aPok-wTfzY_3LpEZ@HIDDEN> <86v7k5u53e.fsf@HIDDEN>
<aPo2whTJUUjpqUmy@HIDDEN> <86placu9e8.fsf@HIDDEN>
<aPuNwYL3_6NwKero@HIDDEN>
<jwvms5g71fs.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvms5g71fs.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 79678
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 79678 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Fri, Oct 24, 2025 at 12:03:47 -0400, Stefan Monnier wrote:
> >> Stefan suggested at least 2, AFAIU.
> > Yes, but they have disadvantages which he pointed out.
> I also suggested `eval-when/and-compile-preserving-sympos`, which AFAIK
> is the simplest and safest solution to the immediate problem.
It's cheap and nasty. Cheap for the reasons you give, and nasty because
an internal implementation detail of the Lisp engine leaks into user
level code.
> > Several at best misleading notions about my patch were circulating:
> > In particular:
> > o - .... presumably O(1) operation `set` can now not only mutate the
> > value it receives but traverse it completely.
> OK, I now see that I missed the part of your patch which defines a new
> `byte-run-strip-symbol-positions-copy`, so indeed it doesn't mutate.
> Instead it builds a new value.
More accurately, it creates a copy with the same value.
> For reference, here's the relevant part of your patch:
> @@ -894,6 +914,8 @@ DEFUN ("fset", Ffset, Sfset, 2, 2, 0,
> signal a `cyclic-function-indirection' error. */)
> (register Lisp_Object symbol, Lisp_Object definition)
> {
> + Lisp_Object def_1 = definition;
> +
> CHECK_SYMBOL (symbol);
> /* Perhaps not quite the right error signal, but seems good enough. */
> if (NILP (symbol) && !NILP (definition))
> @@ -919,7 +941,12 @@ DEFUN ("fset", Ffset, Sfset, 2, 2, 0,
> calln (Qcomp_subr_trampoline_install, symbol);
> #endif
> - set_symbol_function (symbol, definition);
> + if (byte_compile_macroing
> + && !NILP (Vbyte_compile_current_buffer)
> + && Ffboundp (Qbyte_run_strip_symbol_positions_copy))
> + def_1 = calln (Qbyte_run_strip_symbol_positions_copy, def_1);
> +
> + set_symbol_function (symbol, def_1);
> return definition;
> }
> So, clearly, `Fset` can now take O(N) time and allocate O(N).
"Can" is very different from "does". In the normal case, there is no
allocation at all, and the extra time taken is for a machine level
comparison of a global and a conditional jump, and I think here an
additional assignement, if it isn't optimised away by the C compiler.
It is only when macro expansion is in progress during a byte compilation
that the extra resources are used.
> And we now have the weird behavior that
> (progn (setq x y) (eq x y))
> can sometimes return nil.
Inside a macro expansion? That's a fair point, if it can actually
happen.
> > o - The problem is not just CPU time but the change in semantics.
> This described an opinion of mine rather than a fact about your patch,
> so I'm not sure how you could even contest this.
You could have pointed out that this change would be restricted to the
leakage in macro expansions and its fix. You made it look like Lisp
programs would behave differently from before.
> > o - .... this is spreading the SWP special treatment deeper into the core
> > of the ELisp language
> In which sense is this not true? Just look at the patch, seeing how it
> touches `Fset` and all the variable-binding code.
It does not touch the Lisp language; any Lisp which worked before would
work in the same way afterwards, and vice versa. You made it look like
my patch broke Lisp. There was no change in Fset beyond stripping SWPs
when needed.
> > o - .... making the operation take a time proportional to the size of the
> > second arg.
> > o - .... changing `set` from O(1) to O(N)
> These are meant to say the same. I definitely stand by the O(N) claim.
> You can argue that "in practice" you don't see this and I have no reason
> not to believe you, but I'm pretty sure it *will* bite sooner or later.
Unless I'm ignorant about some academic convention where the O notation
always refers to the worst case, set_internal runs in the same time it
does before, modulo that boolean test and conditional jump instruction.
Only during macro expansion can the function take significantly longer,
and that only when the target binding is the default binding of a
dynamic variable. Your point made it look like the increase in time
happened in the typical case. It doesn't.
> > o - Making it mutate its argument is also a change to the language.
> Yes, I was wrong about that. Instead it performs a "functional update",
> which means that it replaces a value with a different value (not `eq`).
> Not sure which is worse.
That again is a fair criticism. But writing the identical value to the
variable is precisely the leakage problem which needed to be solved.
The SWPs in that value are still required by the macro currently
running.
Perhaps there is a solution involving postponing this writing until the
SWPs in it are no longer needed, then writing it later, or perhaps
writing the original value to begin with, and stripping it of SWPs
later.
This could be done, but is it worth the trouble?
Anyhow, it's looking like my patch is being rejected, and currently
there's no replacement for it. Some people don't like the alteration to
the EQ primitive which the current SWP mechanism necessitates.
I came up with another idea yesterday, which I posted on emacs-devel,
which would enable EQ to be restored to its former state.
You have vaguely alluded to other ideas recently and in the past. You
also have at least some familiarity with how other Lisp systems cope
with the problem. Could you perhaps firm up these ideas into a coherent
proposal and post it onto emacs-devel for discussion? The problem we're
facing is outputting correct line/column numbers for warning/error
messages generated by the byte compiler.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
Received: (at 79678) by debbugs.gnu.org; 24 Oct 2025 16:04:05 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 24 12:04:05 2025
Received: from localhost ([127.0.0.1]:36816 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vCKGq-0001I3-TE
for submit <at> debbugs.gnu.org; Fri, 24 Oct 2025 12:04:05 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:23700)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <monnier@HIDDEN>)
id 1vCKGm-0001HW-F2
for 79678 <at> debbugs.gnu.org; Fri, 24 Oct 2025 12:04:03 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id B87C781697;
Fri, 24 Oct 2025 12:03:53 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1761321832;
bh=XdoP59L/WN/+M79yA6bdXVPO1vdVD1ZDgWW0F+f4cy8=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=OEWOmJykCEpe3GyhFdJhMw43ffCONX8kDpWjXyTHE6Jdera/UOX1q/MNnMvvpgyPq
a2zUOA8FRV3jSbo6xqW+1zMmorU7HwLo/gKQAf5PSbjUiNgueH1WP5JZhS3TWUi4Sd
sCrSRReKiJzLOnVE4620KjOqfPNF+bZT7Ye/gZCJaooTD08MSJ1n13zMvfaVk2IWTn
yI6AegF/h2PuCfe68QbgRgd9DJn1zum71q9jB2j4x9CJz1Qg39Nc09C4Ro4tTM2+Yv
MoqYwO7fUnisRtJ1PIGnu5MemaanPcAeqD9t394iR7SDpOQCk9bvThzPFcXvgkLDpN
nF2FJDUlT7LaQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id CA4B08034A;
Fri, 24 Oct 2025 12:03:52 -0400 (EDT)
Received: from asado (unknown [181.28.45.30])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3BADC12022D;
Fri, 24 Oct 2025 12:03:50 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#79678: Problems with the symbols with position mechanism
In-Reply-To: <aPuNwYL3_6NwKero@HIDDEN>
Message-ID: <jwvms5g71fs.fsf-monnier+emacs@HIDDEN>
References: <aPkdPnGEi5sRT7km@HIDDEN>
<jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN>
<aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN>
<aPok-wTfzY_3LpEZ@HIDDEN> <86v7k5u53e.fsf@HIDDEN>
<aPo2whTJUUjpqUmy@HIDDEN> <86placu9e8.fsf@HIDDEN>
<aPuNwYL3_6NwKero@HIDDEN>
Date: Fri, 24 Oct 2025 12:03:47 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.178 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 79678
Cc: Eli Zaretskii <eliz@HIDDEN>, 79678 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> Stefan suggested at least 2, AFAIU.
> Yes, but they have disadvantages which he pointed out.
I also suggested `eval-when/and-compile-preserving-sympos`, which AFAIK
is the simplest and safest solution to the immediate problem.
> Several at best misleading notions about my patch were circulating:
> In particular:
> o - .... presumably O(1) operation `set` can now not only mutate the
> value it receives but traverse it completely.
OK, I now see that I missed the part of your patch which defines a new
`byte-run-strip-symbol-positions-copy`, so indeed it doesn't mutate.
Instead it builds a new value.
For reference, here's the relevant part of your patch:
@@ -894,6 +914,8 @@ DEFUN ("fset", Ffset, Sfset, 2, 2, 0,
signal a `cyclic-function-indirection' error. */)
(register Lisp_Object symbol, Lisp_Object definition)
{
+ Lisp_Object def_1 = definition;
+
CHECK_SYMBOL (symbol);
/* Perhaps not quite the right error signal, but seems good enough. */
if (NILP (symbol) && !NILP (definition))
@@ -919,7 +941,12 @@ DEFUN ("fset", Ffset, Sfset, 2, 2, 0,
calln (Qcomp_subr_trampoline_install, symbol);
#endif
- set_symbol_function (symbol, definition);
+ if (byte_compile_macroing
+ && !NILP (Vbyte_compile_current_buffer)
+ && Ffboundp (Qbyte_run_strip_symbol_positions_copy))
+ def_1 = calln (Qbyte_run_strip_symbol_positions_copy, def_1);
+
+ set_symbol_function (symbol, def_1);
return definition;
}
So, clearly, `Fset` can now take O(N) time and allocate O(N).
And we now have the weird behavior that
(progn (setq x y) (eq x y))
can sometimes return nil.
> o - The problem is not just CPU time but the change in semantics.
This described an opinion of mine rather than a fact about your patch,
so I'm not sure how you could even contest this.
> o - .... this is spreading the SWP special treatment deeper into the core
> of the ELisp language
In which sense is this not true? Just look at the patch, seeing how it
touches `Fset` and all the variable-binding code.
> o - .... making the operation take a time proportional to the size of the
> second arg.
> o - .... changing `set` from O(1) to O(N)
These are meant to say the same. I definitely stand by the O(N) claim.
You can argue that "in practice" you don't see this and I have no reason
not to believe you, but I'm pretty sure it *will* bite sooner or later.
> o - Making it mutate its argument is also a change to the language.
Yes, I was wrong about that. Instead it performs a "functional update",
which means that it replaces a value with a different value (not `eq`).
Not sure which is worse.
Stefan
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
Received: (at 79678) by debbugs.gnu.org; 24 Oct 2025 14:31:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 24 10:31:41 2025
Received: from localhost ([127.0.0.1]:36557 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vCIpQ-0001nP-QU
for submit <at> debbugs.gnu.org; Fri, 24 Oct 2025 10:31:41 -0400
Received: from mail.muc.de ([193.149.48.3]:58447)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1vCIpN-0001mu-N7
for 79678 <at> debbugs.gnu.org; Fri, 24 Oct 2025 10:31:39 -0400
Received: (qmail 92635 invoked by uid 3782); 24 Oct 2025 16:31:30 +0200
Received: from muc.de (p4fe15bd9.dip0.t-ipconnect.de [79.225.91.217]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Fri, 24 Oct 2025 16:31:29 +0200
Received: (qmail 16589 invoked by uid 1000); 24 Oct 2025 14:31:29 -0000
Date: Fri, 24 Oct 2025 14:31:29 +0000
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#79678: Problems with the symbols with position mechanism
Message-ID: <aPuNwYL3_6NwKero@HIDDEN>
References: <aPkdPnGEi5sRT7km@HIDDEN>
<jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN>
<aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN>
<aPok-wTfzY_3LpEZ@HIDDEN> <86v7k5u53e.fsf@HIDDEN>
<aPo2whTJUUjpqUmy@HIDDEN> <86placu9e8.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <86placu9e8.fsf@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 79678
Cc: acm@HIDDEN, 79678 <at> debbugs.gnu.org, monnier@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Eli.
On Fri, Oct 24, 2025 at 09:00:15 +0300, Eli Zaretskii wrote:
> > Date: Thu, 23 Oct 2025 14:08:02 +0000
> > Cc: monnier@HIDDEN, 79678 <at> debbugs.gnu.org, acm@HIDDEN
> > From: Alan Mackenzie <acm@HIDDEN>
> > > > I'm not responsible for the extensive use of BASE_EQ. It is purely an
> > > > optimisation, and it could always be replaced by EQ.
> > > There's no chance in the world that we will be able to prevent people
> > > from optimizing the Emacs code. So this cannot possibly be a
> > > solution.
> > It was meant as an explanation, not a possible solution. Sorry! In the
> > original implementation of SWPs, BASE_EQ was solely used in lisp.h as a
> > part of the expanded EQ.
> I know, but what happened and still happens after that is part of the
> consequence. Given the effect of symbols with position on perfomance,
> we could even predict that this will happen in advance.
I wasn't aware at the time that this optimsation might happen. I've
never been a big fan for squeezing the last 2-3% performance out of a
system. Especially given the power of modern PCs.
> > > > > I'm afraid we are already way too far on this slippery slope.
> > > > I think you'll agree with me that line/column numbers are absolutely
> > > > necessary in byte compiler warning/error messages.
> > > They are good to have, but I think we are paying too high a price for
> > > that nicety.
> > What would you have as an alternative? Compiler warning messages with no
> > position information at all? I don't think that's the way to go.
> I don't know. But I don't think we have explored any alternatives
> seriously at the time. AFAIR, you pressed hard to have this included,
> and both myself and Stefan were skeptical and didn't think the problem
> was serious enough to justify such changes.
See below.
> So one alternative would be to live with the problem, as we did for 40
> years before that. Maybe there are others.
Around ten years ago, there were at least six open bug reports about the
problem, from six different people. One of them, bug#24128 was raised by
a certain Eli Zaretskii. On 2016-08-02 he posted "So it's high time we
had it fixed, IMO.".
Well, when other people were just talking, talking, talking about how
difficult it would be to fix, I sat down and actually fixed it. Only the
second version of the fix (which refrained from positioning Qnil) was
accepted into master.
My fix was such that the basic form of the byte compiler was left
unchanged, and modulo a few bugs which got fixed, simply worked, mostly.
The patch at the centre of this thread is an attempt to get rid of that
annoying "mostly".
My recollection of the time is that there were no practical alternative
implementation ideas. There were a few vague suggestions which dissolved
into nothingness when looked at more closely.
We are still lacking a coherent workable alternative. Maybe we could
build a new byte compiler which rather than reading a Lisp form at a time
with the Lisp reader, would process the input text a line at a time.
Much like how C compilers process their input text.
Or, perhaps the current read-positioning-symbols could continue to be
used, but with the positions stripped from the symbols at an early stage
of the compilation and stored separately.
Or maybe we could use a hash table from conses' opening parentheses to
positions, in which case the reader would return conses with position
rather than symbols with position. Somehow.
Or maybe we could copy the strategy used by some other Lisp system. I'm
not at all familiar with these, but Stefan is. Maybe he could come up
with some idea based on one of these other systems.
Whatever we do, I think it's imperative that we can compile the Lisp
currently in existence without having to change it.
[ .... ]
> > > > If you think a less invasive fix is possible, please tell me the
> > > > details, and I'll work on it.
> > > What kind of discussion is it if a dissenting opinion is dismissed
> > > unless it comes with code?
> > Not a good one. I wasn't doing that. I was suggesting that working code
> > was some considerable advance on just an idea, and should be taken
> > seriously.
> Stefan suggested at least 2, AFAIU.
Yes, but they have disadvantages which he pointed out.
[ From Stefan's post: ]
>>> 1. It requires distinguishing "code" and "data". To improve on the
>>> the current situation, this requires changing macros so as to
>>> indicate which parts of its arguments is code and which part is data,
>>> probably also change our interpreter (aka `eval`) so it can work with
>>> code ....
If this distinguishing of code from data can be done by the compiler,
maybe it would be a workable strategy. But not if this indication has to
be done in advance by the programmer, perhaps via something like a debug
spec.
>>> 2. Don't distinguish code and data. I.e. allow SWPs everywhere all
>>> the time, eliminate `symbols-with-pos-enabled` ....
I'm not sure how scheme 2 is an advance on the current situation. It
would have even more SWPs in circulation, and EQ would likely be no
faster than at the moment.
> > > > Anyhow, why do you judge that the cost of my patch is too high?
> > > > It is a ~2,000 line patch published only yesterday. To make that
> > > > judgment fairly, you would have to have read it in some detail.
> > > > I don't think you've done this; there hasn't been time.
> > > There was enough time for me to get the general idea of the patch,
> > > trust me. More generally, please try responding to the point, instead
> > > of assuming that your opponents don't know what they are talking about
> > > and don't understand your code, because it simply isn't true.
> > I asked Stefan if he'd read the patch, and got no answer.
> That question is already problematic, in my book. It hints that
> people respond to you without knowing what they are talking about. It
> should not be a problem to get the gist of a patch by skimming it, and
> I trust Stefan he did that before replying. The fact is that his
> response was pointing to specific aspects of your patch, which would
> be impossible had he not read the patch.
When I asked that question, I was hoping for an answer something like "I
skimmed it in ~10 minutes", or "I've read the Lisp part of the patch
thoroughly, but not the C part". That would enable me to talk about the
patch in terms which Stefan would be familiar with, in more or less
detail.
> So such questions, in such context, are part of the attitude which we
> should steer away of, if we want to keep the discussion technical and
> efficient.
[ .... ]
> > I'll just repeat the last request from my last post. That is to have
> > my patch fairly assessed by somebody who doesn't immediately dismiss
> > its core idea as "hideous".
> The "hideous" word aside, what is unfair in how your patch was
> assessed?
Several at best misleading notions about my patch were circulating: In
particular:
o - .... presumably O(1) operation `set` can now not only mutate the
value it receives but traverse it completely.
o - .... making the operation take a time proportional to the size of the
second arg.
o - The problem is not just CPU time but the change in semantics.
o - .... this is spreading the SWP special treatment deeper into the core
of the ELisp language
o - .... changing `set` from O(1) to O(N)
o - Making it mutate its argument is also a change to the language.
All of these, in so far as they were not actually false, were misleading
or irrelevant. They were specific enough to raise alarms in anybody
reading them, yet vague enough not to be completely false. For example,
in the last point ("mutat[ing] its argument") the only relevant thing the
patch did was strip SWPs. The value was left unchanged.
I would wish my patch to be reviewed by somebody who either hadn't read
these points or who had sufficient understanding of their irrelevance not
to be influenced by them. I accept your judgment on the patch now, given
that we have discussed these six notions and cleared the air about them.
At the same time, if we are going to be replacing the byte compiler's
position mechanism some time soon anyway, why not accept the patch in the
meantime? It might well make the currently incomplete SWP mechanism
complete.
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.Received: (at 79678) by debbugs.gnu.org; 24 Oct 2025 06:00:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 24 02:00:28 2025 Received: from localhost ([127.0.0.1]:34833 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1vCAqi-0001i7-56 for submit <at> debbugs.gnu.org; Fri, 24 Oct 2025 02:00:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58440) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1vCAqf-0001hf-Gl for 79678 <at> debbugs.gnu.org; Fri, 24 Oct 2025 02:00:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1vCAqY-00087j-Uy; Fri, 24 Oct 2025 02:00:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=QrDdXBiXPBpAR6m+iYTX8bn+FtLs3vGB6N+BZkceNdg=; b=PKG7/CblpRK+ qITbs1MfhowGWRFXDf47U3U6I47Q/RUpM0uu+dIkaUtdEHj9F3iX4VZx/N9DMcGDaNOFDf6H6Cxyi 4JpjYg4e0+j4AVeUJUlFalMrl0zRgbs7zkXgzqSJJKADGdG9CiyFzKixNTxDjTztfcbMiK3rJez8R ZunBUvrnsrKkcViMnEmVrvH4S+xmTcXsmTfWQDoCxmQdRsTg2vBpsJqG8lOmV5fIgGXJ4B8MGS8L+ x2rYuglBl2p1pWfyd3JrTCrcb3lTm1XrYoFCXTI+GCfh8dNqhuNemyD+NlSNZjRE5V7mKXFWVXeiT 8gvf7eQsaPsZKD8goAmIqw==; Date: Fri, 24 Oct 2025 09:00:15 +0300 Message-Id: <86placu9e8.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Alan Mackenzie <acm@HIDDEN> In-Reply-To: <aPo2whTJUUjpqUmy@HIDDEN> (message from Alan Mackenzie on Thu, 23 Oct 2025 14:08:02 +0000) Subject: Re: bug#79678: Problems with the symbols with position mechanism References: <aPkdPnGEi5sRT7km@HIDDEN> <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> <aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN> <aPok-wTfzY_3LpEZ@HIDDEN> <86v7k5u53e.fsf@HIDDEN> <aPo2whTJUUjpqUmy@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 79678 Cc: 79678 <at> debbugs.gnu.org, monnier@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Thu, 23 Oct 2025 14:08:02 +0000 > Cc: monnier@HIDDEN, 79678 <at> debbugs.gnu.org, acm@HIDDEN > From: Alan Mackenzie <acm@HIDDEN> > > > > I'm not responsible for the extensive use of BASE_EQ. It is purely an > > > optimisation, and it could always be replaced by EQ. > > > There's no chance in the world that we will be able to prevent people > > from optimizing the Emacs code. So this cannot possibly be a > > solution. > > It was meant as an explanation, not a possible solution. Sorry! In the > original implementation of SWPs, BASE_EQ was solely used in lisp.h as a > part of the expanded EQ. I know, but what happened and still happens after that is part of the consequence. Given the effect of symbols with position on perfomance, we could even predict that this will happen in advance. > > > > I'm afraid we are already way too far on this slippery slope. > > > > I think you'll agree with me that line/column numbers are absolutely > > > necessary in byte compiler warning/error messages. > > > They are good to have, but I think we are paying too high a price for > > that nicety. > > What would you have as an alternative? Compiler warning messages with no > position information at all? I don't think that's the way to go. I don't know. But I don't think we have explored any alternatives seriously at the time. AFAIR, you pressed hard to have this included, and both myself and Stefan were skeptical and didn't think the problem was serious enough to justify such changes. So one alternative would be to live with the problem, as we did for 40 years before that. Maybe there are others. > > > I feel hard done by that after my work on this necessary Emacs component, > > > I've had little else but hostile criticism of how bad it is, how it is a > > > "slippery slope", and worse. > > > There's nothing wrong in expressing an opinion about the relative > > merits and demerits of a feature and its implementation. I don't > > understand why you perceive that as "hostile", (and am quite > > astonished that you do). "Slippery slope" has no derogatory meaning, > > it describes a situation and a trend, not any kind of judgment against > > you and your work. When you comment on someone else's code, you > > frequently use much harsher epithets. > > > It is impossible to have a rational discussion about a technical issue > > when every dissenting opinion is perceived as hostile. > > We're both projecting extremes, when I think we would both rather tend > toward moderation in such descriptions. Then please refrain from "charged" words like "hostile", "half-truth", etc., okay? > > > If you think a less invasive fix is possible, please tell me the > > > details, and I'll work on it. > > > What kind of discussion is it if a dissenting opinion is dismissed > > unless it comes with code? > > Not a good one. I wasn't doing that. I was suggesting that working code > was some considerable advance on just an idea, and should be taken > seriously. Stefan suggested at least 2, AFAIU. > > > Anyhow, why do you judge that the cost of my patch is too high? It is a > > > ~2,000 line patch published only yesterday. To make that judgment > > > fairly, you would have to have read it in some detail. I don't think > > > you've done this; there hasn't been time. > > > There was enough time for me to get the general idea of the patch, > > trust me. More generally, please try responding to the point, instead > > of assuming that your opponents don't know what they are talking about > > and don't understand your code, because it simply isn't true. > > I asked Stefan if he'd read the patch, and got no answer. That question is already problematic, in my book. It hints that people respond to you without knowing what they are talking about. It should not be a problem to get the gist of a patch by skimming it, and I trust Stefan he did that before replying. The fact is that his response was pointing to specific aspects of your patch, which would be impossible had he not read the patch. So such questions, in such context, are part of the attitude which we should steer away of, if we want to keep the discussion technical and efficient. > > I don't know how we can keep discussing this seriously on these terms. > > It certainly is no way of convincing your opponents. > > I'll just repeat the last request from my last post. That is to have my > patch fairly assessed by somebody who doesn't immediately dismiss its > core idea as "hideous". The "hideous" word aside, what is unfair in how your patch was assessed?
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.Received: (at 79678) by debbugs.gnu.org; 23 Oct 2025 14:08:14 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 23 10:08:14 2025 Received: from localhost ([127.0.0.1]:33092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1vBvzB-00059H-QZ for submit <at> debbugs.gnu.org; Thu, 23 Oct 2025 10:08:14 -0400 Received: from mail.muc.de ([193.149.48.3]:52960) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1vBvz8-000591-BX for 79678 <at> debbugs.gnu.org; Thu, 23 Oct 2025 10:08:11 -0400 Received: (qmail 122 invoked by uid 3782); 23 Oct 2025 16:08:03 +0200 Received: from muc.de (p4fe153d1.dip0.t-ipconnect.de [79.225.83.209]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 23 Oct 2025 16:08:03 +0200 Received: (qmail 10984 invoked by uid 1000); 23 Oct 2025 14:08:02 -0000 Date: Thu, 23 Oct 2025 14:08:02 +0000 To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#79678: Problems with the symbols with position mechanism Message-ID: <aPo2whTJUUjpqUmy@HIDDEN> References: <aPkdPnGEi5sRT7km@HIDDEN> <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> <aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN> <aPok-wTfzY_3LpEZ@HIDDEN> <86v7k5u53e.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86v7k5u53e.fsf@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 79678 Cc: acm@HIDDEN, 79678 <at> debbugs.gnu.org, monnier@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Eli. On Thu, Oct 23, 2025 at 16:20:53 +0300, Eli Zaretskii wrote: > > Date: Thu, 23 Oct 2025 12:52:11 +0000 > > Cc: monnier@HIDDEN, 79678 <at> debbugs.gnu.org > > From: Alan Mackenzie <acm@HIDDEN> > > > I've been watching over the last several years how the BASE_EQ thingy > > > keeps seeping into our sources in more and more places. Compared to > > > the original and well-understood EQ, this makes the code harder to read > > > and (more importantly) harder to write correctly, because there isn't > > > an easy way of knowing where a symbol could have position and where it > > > cannot. > > I'm not responsible for the extensive use of BASE_EQ. It is purely an > > optimisation, and it could always be replaced by EQ. > There's no chance in the world that we will be able to prevent people > from optimizing the Emacs code. So this cannot possibly be a > solution. It was meant as an explanation, not a possible solution. Sorry! In the original implementation of SWPs, BASE_EQ was solely used in lisp.h as a part of the expanded EQ. > > > I'm afraid we are already way too far on this slippery slope. > > I think you'll agree with me that line/column numbers are absolutely > > necessary in byte compiler warning/error messages. > They are good to have, but I think we are paying too high a price for > that nicety. What would you have as an alternative? Compiler warning messages with no position information at all? I don't think that's the way to go. > > I feel hard done by that after my work on this necessary Emacs component, > > I've had little else but hostile criticism of how bad it is, how it is a > > "slippery slope", and worse. > There's nothing wrong in expressing an opinion about the relative > merits and demerits of a feature and its implementation. I don't > understand why you perceive that as "hostile", (and am quite > astonished that you do). "Slippery slope" has no derogatory meaning, > it describes a situation and a trend, not any kind of judgment against > you and your work. When you comment on someone else's code, you > frequently use much harsher epithets. > It is impossible to have a rational discussion about a technical issue > when every dissenting opinion is perceived as hostile. We're both projecting extremes, when I think we would both rather tend toward moderation in such descriptions. > > > He said "not at any cost", and I agree. Speaking generally (and you > > > asked above a general question about fixing bugs), some costs are too > > > high, in which case we should either look for less invasive fixes or > > > perhaps decide not to fix something if the problem is not grave and > > > workarounds exist. > > My patch is working code. These "less invasive fixes" are pure > > non-existent fantasy. > That doesn't mean they cannot be implemented. > > If you think a less invasive fix is possible, please tell me the > > details, and I'll work on it. > What kind of discussion is it if a dissenting opinion is dismissed > unless it comes with code? Not a good one. I wasn't doing that. I was suggesting that working code was some considerable advance on just an idea, and should be taken seriously. > At least I, as a person who has a lot to do every day for Emacs and too > little time to do it, cannot be possibly required to code by myself > every implementation idea I favor more than some submitted code. I didn't mean to suggest that. I was suggesting that you formulate the idea to some moderate degree of detail, and I would implement it. Having recently coded up my patch, I'm not in a frame of mind to think up alternative approaches. > One could favor alternative ideas for fixing a problem without being > required to code it by him/herself, don't you think? Absolutely! See above. > > Anyhow, why do you judge that the cost of my patch is too high? It is a > > ~2,000 line patch published only yesterday. To make that judgment > > fairly, you would have to have read it in some detail. I don't think > > you've done this; there hasn't been time. > There was enough time for me to get the general idea of the patch, > trust me. More generally, please try responding to the point, instead > of assuming that your opponents don't know what they are talking about > and don't understand your code, because it simply isn't true. I asked Stefan if he'd read the patch, and got no answer. You could have said how intensively you'd read it, making it easier for me to formulate my next post to match. > You should trust us that after decades of reviewing patches, we can > grasp the general idea of the design and implementation by just > skimming a patch, even if it is large. > I decided not to respond to the rest of your message because it > mistakenly takes legitimate and politely expressed criticism for > unfairness, cheap politics, half-truths, and hostility. I think you ought to be concerned as to whether what I wrote might be true. And if not, what prompted me to write it, and what caused my mistake, if such it be. > I don't know how we can keep discussing this seriously on these terms. > It certainly is no way of convincing your opponents. I'll just repeat the last request from my last post. That is to have my patch fairly assessed by somebody who doesn't immediately dismiss its core idea as "hideous". -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.Received: (at 79678) by debbugs.gnu.org; 23 Oct 2025 13:21:15 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 23 09:21:15 2025 Received: from localhost ([127.0.0.1]:60502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1vBvFj-00025x-5g for submit <at> debbugs.gnu.org; Thu, 23 Oct 2025 09:21:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40648) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1vBvFc-00025J-5z for 79678 <at> debbugs.gnu.org; Thu, 23 Oct 2025 09:21:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1vBvFU-0000L0-OQ; Thu, 23 Oct 2025 09:21:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=2AyxAuXh5oIg55zyU3WSElSWj8BrsRjtYYL9OjYywUU=; b=Xh3LFXrJ5uMk UR1IG4wfFUqtRRErOLQRutfjDoQo4plwTzoeZbk1golpIqyLHYgtz+lK/qyibv1Xz77Eljv5a5qAL uHVvhgDAiUR+r/w41JvRO1uYEzduEzs+vyrq5/sJw42iJEsNkPDjwFYdjf3ZkwXOSZDnDoe1/52Lh jo8yyChQ5FgKdVVcSht0WGThOzDepjkln51smbCPDcWnsGZ1fTv/tNpw1G3YaTEmuUMqLyj+9qbGG QqGppzLVEi+k4zLVYvt17LvPUsg2NKCC7jkJugFDTBFuBjofphz+iIXWB9+8xnKkPdn3HABu4t9jv RFo9JUrDwVRY7AVPwZpGxA==; Date: Thu, 23 Oct 2025 16:20:53 +0300 Message-Id: <86v7k5u53e.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Alan Mackenzie <acm@HIDDEN> In-Reply-To: <aPok-wTfzY_3LpEZ@HIDDEN> (message from Alan Mackenzie on Thu, 23 Oct 2025 12:52:11 +0000) Subject: Re: bug#79678: Problems with the symbols with position mechanism References: <aPkdPnGEi5sRT7km@HIDDEN> <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> <aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN> <aPok-wTfzY_3LpEZ@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 79678 Cc: 79678 <at> debbugs.gnu.org, monnier@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Thu, 23 Oct 2025 12:52:11 +0000 > Cc: monnier@HIDDEN, 79678 <at> debbugs.gnu.org > From: Alan Mackenzie <acm@HIDDEN> > > > I've been watching over the last several years how the BASE_EQ thingy > > keeps seeping into our sources in more and more places. Compared to > > the original and well-understood EQ, this makes the code harder to read > > and (more importantly) harder to write correctly, because there isn't > > an easy way of knowing where a symbol could have position and where it > > cannot. > > I'm not responsible for the extensive use of BASE_EQ. It is purely an > optimisation, and it could always be replaced by EQ. There's no chance in the world that we will be able to prevent people from optimizing the Emacs code. So this cannot possibly be a solution. > > I'm afraid we are already way too far on this slippery slope. > > I think you'll agree with me that line/column numbers are absolutely > necessary in byte compiler warning/error messages. They are good to have, but I think we are paying too high a price for that nicety. > I feel hard done by that after my work on this necessary Emacs component, > I've had little else but hostile criticism of how bad it is, how it is a > "slippery slope", and worse. There's nothing wrong in expressing an opinion about the relative merits and demerits of a feature and its implementation. I don't understand why you perceive that as "hostile", (and am quite astonished that you do). "Slippery slope" has no derogatory meaning, it describes a situation and a trend, not any kind of judgment against you and your work. When you comment on someone else's code, you frequently use much harsher epithets. It is impossible to have a rational discussion about a technical issue when every dissenting opinion is perceived as hostile. > > He said "not at any cost", and I agree. Speaking generally (and you > > asked above a general question about fixing bugs), some costs are too > > high, in which case we should either look for less invasive fixes or > > perhaps decide not to fix something if the problem is not grave and > > workarounds exist. > > My patch is working code. These "less invasive fixes" are pure > non-existent fantasy. That doesn't mean they cannot be implemented. > If you think a less invasive fix is possible, please tell me the > details, and I'll work on it. What kind of discussion is it if a dissenting opinion is dismissed unless it comes with code? At least I, as a person who has a lot to do every day for Emacs and too little time to do it, cannot be possibly required to code by myself every implementation idea I favor more than some submitted code. One could favor alternative ideas for fixing a problem without being required to code it by him/herself, don't you think? > Anyhow, why do you judge that the cost of my patch is too high? It is a > ~2,000 line patch published only yesterday. To make that judgment > fairly, you would have to have read it in some detail. I don't think > you've done this; there hasn't been time. There was enough time for me to get the general idea of the patch, trust me. More generally, please try responding to the point, instead of assuming that your opponents don't know what they are talking about and don't understand your code, because it simply isn't true. You should trust us that after decades of reviewing patches, we can grasp the general idea of the design and implementation by just skimming a patch, even if it is large. I decided not to respond to the rest of your message because it mistakenly takes legitimate and politely expressed criticism for unfairness, cheap politics, half-truths, and hostility. I don't know how we can keep discussing this seriously on these terms. It certainly is no way of convincing your opponents.
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.Received: (at 79678) by debbugs.gnu.org; 23 Oct 2025 12:52:22 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 23 08:52:22 2025 Received: from localhost ([127.0.0.1]:60389 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1vBunl-0000Pt-V4 for submit <at> debbugs.gnu.org; Thu, 23 Oct 2025 08:52:22 -0400 Received: from mail.muc.de ([193.149.48.3]:34694) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1vBuni-0000P5-JP for 79678 <at> debbugs.gnu.org; Thu, 23 Oct 2025 08:52:19 -0400 Received: (qmail 1343 invoked by uid 3782); 23 Oct 2025 14:52:12 +0200 Received: from muc.de (p4fe153d1.dip0.t-ipconnect.de [79.225.83.209]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 23 Oct 2025 14:52:11 +0200 Received: (qmail 6398 invoked by uid 1000); 23 Oct 2025 12:52:11 -0000 Date: Thu, 23 Oct 2025 12:52:11 +0000 To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#79678: Problems with the symbols with position mechanism Message-ID: <aPok-wTfzY_3LpEZ@HIDDEN> References: <aPkdPnGEi5sRT7km@HIDDEN> <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> <aPlaZha-LePL4CKH@HIDDEN> <86bjlyursb.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86bjlyursb.fsf@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 79678 Cc: 79678 <at> debbugs.gnu.org, monnier@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Eli. On Thu, Oct 23, 2025 at 08:10:44 +0300, Eli Zaretskii wrote: > > Cc: acm@HIDDEN, 79678 <at> debbugs.gnu.org > > Date: Wed, 22 Oct 2025 22:27:50 +0000 > > From: Alan Mackenzie <acm@HIDDEN> > > > IOW, this is spreading the SWP special treatment deeper into the core of > > > the ELisp language (it's currently mostly limited to `eq` and that is > > > already a source of concern). > > Concern to whom? > Concern to me, at the very least. OK. > I've been watching over the last several years how the BASE_EQ thingy > keeps seeping into our sources in more and more places. Compared to > the original and well-understood EQ, this makes the code harder to read > and (more importantly) harder to write correctly, because there isn't > an easy way of knowing where a symbol could have position and where it > cannot. I'm not responsible for the extensive use of BASE_EQ. It is purely an optimisation, and it could always be replaced by EQ. BASE_EQ can be used when it is known that at least one of its arguments cannot be Qnil, and only then. > I'm afraid we are already way too far on this slippery slope. I think you'll agree with me that line/column numbers are absolutely necessary in byte compiler warning/error messages. SWPs are the way this is done, and there is, so far, no alternative. I feel hard done by that after my work on this necessary Emacs component, I've had little else but hostile criticism of how bad it is, how it is a "slippery slope", and worse. None of these critics has formulated a usable alternative. If they did, I would be the first to volunteer to implement it, ripping out the SWP mechanism in the process. After this successful work of a few years ago, I would have expected some thanks at the very least. > > The SWP mechanism _is_ part of the core of the Lisp > > machine and has been for several years. You've recently been > > encouraging its expansion and wider use in semel.el. > semel.el (or any other application-level Lisp program) is not the core > language, because it doesn't define primitives or special forms, and > doesn't modify them. The core is in the code written in C and in a > few low-level places in Lisp (like subr.el). > > > So I'm not interested in fixing those cases "at any cost". > > We have at least one bug here, a nasty bug, and I've fixed it. Why do > > you prefer for bugs not to be fixed? > He said "not at any cost", and I agree. Speaking generally (and you > asked above a general question about fixing bugs), some costs are too > high, in which case we should either look for less invasive fixes or > perhaps decide not to fix something if the problem is not grave and > workarounds exist. My patch is working code. These "less invasive fixes" are pure non-existent fantasy. If you think a less invasive fix is possible, please tell me the details, and I'll work on it. If such a fix can be written, then surely my patch should be installed right away, and later superseded by the superior solution. Anyhow, why do you judge that the cost of my patch is too high? It is a ~2,000 line patch published only yesterday. To make that judgment fairly, you would have to have read it in some detail. I don't think you've done this; there hasn't been time. > Software engineering is not mathematics, so we don't have to be > pedantic in our assessment of bugs and their solutions. We can accept > compromise where it's justified. > This is software engineering 101, shich I'm sure you are well aware > of, so I'm surprised you raise these issues, even rhetorically. To me these bugs are significant, which is why I took the trouble to fix them. > > > Your proposal seems to me to be way beyond what I'm willing to pay to > > > fix those cases. Just look at all the "dont_strip" management to handle > > > the various combinations of let-bound and buffer-local; this is hideous, > > > making already complex code even worse. > > The increase in complexity is moderate, not "hideous", and quite in > > keeping with the essentially complicated things the code has to do, > > given the complicated nature of variables in Emacs Lisp. > With all due respect, please leave it to us, the Emacs maintainers, to > decide whether some increase in complexity is hideous or not. That is unfair. Of the three people involved in this exchange, only one is familiar with the patch, and that's me. To call my patch "hideous", without even looking at it properly, is disparaging. It is an insult to my capabilities and good sense. It certainly doesn't conform to the GNU injunction to be nice to eachother. Stefan is a consummate politician, and he knows how to persuade by telling falsehoods, exaggerations, and half-truths. His replies to me were full of such things, and I regret you may have taken on these rhetorical tricks rather than reaching your own judgment of the patch. I'm reasonably sure Stefan hasn't looked at the patch, at least not much. However, he called it "hideous", he said that it "changed Lisp", that it turned set_internal "from O(1) to O(N)", that set "can mutate the value it receives". All these things are false, or at best misleading, and I fear you've absorbed them and accepted them as facts. The truth is that my patch doesn't slow down Emacs, except perhaps very slightly, it runs exactly the same Lisp as before, and conforms in general with the way Emacs has been coded. > Most, if not all, of the price to pay for this complexity will fall on > our shoulders, not yours. So our opinions on this matter much more, no > offense. I was able to work my way into the existing code (e.g. data.c and eval.c), and anybody coming after me, or the Emacs maintainers, would be able to do the same with my patch applied. It does not increase the complexity of the code significantly. > > It is clear you don't like the SWP mechanism. But I think the only real > > criticism you've made of the patch is of the increase in complexity of > > the code. I disagree with that criticism. > FTR, I also don't like symbols with position. I think it gave us very > little in gains, while the price in complexity and code obfuscation we > pay every day, and will continue to pay for as long as Emacs is alive. See above. You criticize without suggesting an alternative. Maybe I don't like SWPs either, but I do like having correct line/column numbers on byte compiler warning messages. > So, if my opinion on this matters, I'm also not interested in sliding > down this slope any farther. I ask for a fair assessment of my patch. That cannot come from somebody who describes it as "hideous" without even looking at it or trying it out. To say nothing of all the other falsehoods. -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.Received: (at 79678) by debbugs.gnu.org; 23 Oct 2025 05:10:58 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 23 01:10:58 2025 Received: from localhost ([127.0.0.1]:59279 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1vBnbF-00047d-HU for submit <at> debbugs.gnu.org; Thu, 23 Oct 2025 01:10:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41488) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1vBnbB-00046l-MC for 79678 <at> debbugs.gnu.org; Thu, 23 Oct 2025 01:10:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1vBnb5-0005xZ-In; Thu, 23 Oct 2025 01:10:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Npyop1p779t9t6heOnJ4xW4puHOo2R+60MHOvHiJVmA=; b=omXk4yMcZOQl Cpd+qpXEq4N5Epyxho1tgi5meHdYf0FQldKq9ztEGBimsU8f8MRKJBWH0EIQd24Iuxrlbc/Oqguxo RY+iKMTVad9ZlrocG7l6s1yzpbqIjaoP/vlgv+NV4ndWYusI3pvZIvJfe5VMSQ89ZxQgTed5CFesS PElRFDC/ZEkfLwbHemyZpBgmM0XAN7upgCEj1Wq7g8xogEZTDt/A4U2BqZCQWkLzoYL/dTzcXRTMx GqKEzhC4lyGjvkhXzG+tk3NW4E8CK35FWiOFmxMYF08ctmBjb/LU7NgEnC1tR9jobno/lgN6gKrZX wJQbuz2/bsLhGiK0RTryMQ==; Date: Thu, 23 Oct 2025 08:10:44 +0300 Message-Id: <86bjlyursb.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Alan Mackenzie <acm@HIDDEN> In-Reply-To: <aPlaZha-LePL4CKH@HIDDEN> (message from Alan Mackenzie on Wed, 22 Oct 2025 22:27:50 +0000) Subject: Re: bug#79678: Problems with the symbols with position mechanism References: <aPkdPnGEi5sRT7km@HIDDEN> <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> <aPlaZha-LePL4CKH@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 79678 Cc: 79678 <at> debbugs.gnu.org, monnier@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: acm@HIDDEN, 79678 <at> debbugs.gnu.org > Date: Wed, 22 Oct 2025 22:27:50 +0000 > From: Alan Mackenzie <acm@HIDDEN> > > > IOW, this is spreading the SWP special treatment deeper into the core of > > the ELisp language (it's currently mostly limited to `eq` and that is > > already a source of concern). > > Concern to whom? Concern to me, at the very least. I've been watching over the last several years how the BASE_EQ thingy keeps seeping into our sources in more and more places. Compared to the original and well-understood EQ, this makes the code harder to read and (more importantly) harder to write correctly, because there isn't an easy way of knowing where a symbol could have position and where it cannot. I'm afraid we are already way too far on this slippery slope. > The SWP mechanism _is_ part of the core of the Lisp > machine and has been for several years. You've recently been > encouraging its expansion and wider use in semel.el. semel.el (or any other application-level Lisp program) is not the core language, because it doesn't define primitives or special forms, and doesn't modify them. The core is in the code written in C and in a few low-level places in Lisp (like subr.el). > > So I'm not interested in fixing those cases "at any cost". > > We have at least one bug here, a nasty bug, and I've fixed it. Why do > you prefer for bugs not to be fixed? He said "not at any cost", and I agree. Speaking generally (and you asked above a general question about fixing bugs), some costs are too high, in which case we should either look for less invasive fixes or perhaps decide not to fix something if the problem is not grave and workarounds exist. Software engineering is not mathematics, so we don't have to be pedantic in our assessment of bugs and their solutions. We can accept compromise where it's justified. This is software engineering 101, shich I'm sure you are well aware of, so I'm surprised you raise these issues, even rhetorically. > > Your proposal seems to me to be way beyond what I'm willing to pay to > > fix those cases. Just look at all the "dont_strip" management to handle > > the various combinations of let-bound and buffer-local; this is hideous, > > making already complex code even worse. > > The increase in complexity is moderate, not "hideous", and quite in > keeping with the essentially complicated things the code has to do, > given the complicated nature of variables in Emacs Lisp. With all due respect, please leave it to us, the Emacs maintainers, to decide whether some increase in complexity is hideous or not. Most, if not all, of the price to pay for this complexity will fall on our shoulders, not yours. So our opinions on this matter much more, no offense. > It is clear you don't like the SWP mechanism. But I think the only real > criticism you've made of the patch is of the increase in complexity of > the code. I disagree with that criticism. FTR, I also don't like symbols with position. I think it gave us very little in gains, while the price in complexity and code obfuscation we pay every day, and will continue to pay for as long as Emacs is alive. So, if my opinion on this matters, I'm also not interested in sliding down this slope any farther.
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
Received: (at 79678) by debbugs.gnu.org; 23 Oct 2025 00:03:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 22 20:03:01 2025
Received: from localhost ([127.0.0.1]:58445 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vBinE-0001F0-UD
for submit <at> debbugs.gnu.org; Wed, 22 Oct 2025 20:03:01 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:14071)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <monnier@HIDDEN>)
id 1vBin9-0001EV-Ir
for 79678 <at> debbugs.gnu.org; Wed, 22 Oct 2025 20:02:58 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id A79081002F0;
Wed, 22 Oct 2025 20:02:48 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1761177767;
bh=0CXBjGc53MNi4ZCVgT/kXOMpT7nEK+1/2uBmeUgG4+I=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=BrWVSX4YUzdW2LLpQIyBKJWCWfcW/HuHKHI5LSOaZ4u0NRkksYmmufTluMGR/aoXZ
pxgtQ5lVt3WRlBQNPywpTTzOIHKDn7RN6Pwvq9WV33yoyJFNaK38r7fTkc+JMHFgmd
qPP3yOANYbZss0pJevr3KZVUVaEC9Zn/8GBuxp4DDL2liJXOB+v1HIjmHA7dNusB7g
cRz2Ag2CcEE3viwOYjDxMGWBkWXza0bDppOC8kPIbyK0XSlxcdQUwerWZUsNNaFSX0
IVuQQrnwHOwD/IHPJBh9NnssSdvPIqKNo3/+5uXwWkalhk5XJHUtiGxAOeugNzKAi8
DUt2ZIsVAZHvw==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 8A93210013E;
Wed, 22 Oct 2025 20:02:47 -0400 (EDT)
Received: from asado (unknown [181.28.45.30])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 5A24A12028D;
Wed, 22 Oct 2025 20:02:45 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#79678: Problems with the symbols with position mechanism
In-Reply-To: <aPlaZha-LePL4CKH@HIDDEN>
Message-ID: <jwvqzuubkcc.fsf-monnier+emacs@HIDDEN>
References: <aPkdPnGEi5sRT7km@HIDDEN>
<jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN>
<aPlaZha-LePL4CKH@HIDDEN>
Date: Wed, 22 Oct 2025 20:02:41 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.010 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 79678
Cc: 79678 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> > The stripping happens only when macro expansion is in progress.
>> That means all those "assignment primitives" will need to test a runtime
>> boolean indicating whether we're in macroexpansion mode?
> Yes, of course. That won't even take a nanosecond on a modern machine.
The problem is not just CPU time but the change in semantics.
>> IOW, this is spreading the SWP special treatment deeper into the core of
>> the ELisp language (it's currently mostly limited to `eq` and that is
>> already a source of concern).
> Concern to whom?
To the core Emacs contributors working on the ELisp level, like
Matthias, and myself, but also others have expressed their dislike (and
sometimes stronger feelings) at the idea of changing `eq` the way the
SWP currently does).
> The SWP mechanism _is_ part of the core of the Lisp
> machine and has been for several years. You've recently been
> encouraging its expansion and wider use in semel.el.
I'm quite happy with `read-positioning-symbols`, yes.
I'm not happy about `symbols-with-pos-enabled`, although I reluctantly
tolerate it for lack of a better solution for now.
> If you look more closely at the patch you'll see that it has no effect
> on the Lisp language.
We attribute completely different meanings to "effect on the language".
What you're saying is that you believe that existing ELisp code will not
need to be modified because the changes are sufficiently discrete.
Your patch changes `set_internal` and a lot of the variable-binding
code. I.e. it changes details at the very core of the Lisp language.
Some of them are "implementation only" (i.e. don't change the language,
indeed), but changing `set` form O(1) to O(N) is a change to the
language. Making it mutate its argument is also a change to
the language.
> One of my main motivations in doing this work
> (and it was a significant amount of work) was precisely to avoid the
> sort of workaround you suggested earlier; things like
> `eval-when-compile-preserving-sympos' which _would_ have affected the
> Lisp language quite markedly.
You lost me here.
You're suggesting the purpose of this 70kB patch which modifies core
ELisp primitives is to avoid defining a 10-line macro?
And how would defining a new macro "affect the Lisp language quite
markedly"? Which existing code would be affected?
> Have you actually tried out the patch, yet?
I don't need to try out the patch to see the damage it does.
I take your word for it that it does what it says in the tin, since
you've amply demonstrated that you're a competent coder.
>> 1- Try and keep source-code-positions out of the core language, confining
>> them to the compiler. That's what we've tried to do so far, where the
>> `eq` and `symbols-with-pos-enabled` are tolerated as "temporary" wrinkles
>> until we replace it with something better.
> If there is anything better, I'll be quite surprised. Since even before
> I implemented SWPs, nobody has been able to come up with a coherent
> alternative to replace them.
The "something better" I'm referring to is basically to remove the
`symbols-with-pos-enabled` and instead change the affected ELisp code.
IOW, mostly change relevant uses of `eq(l)` to use some other test
instead (I'd be in favor of using `equal` for that).
It's a non-trivial undertaking, admittedly.
> Do you have a better way of fixing this bug?
No. But I prefer to keep it unfixed than installing that patch.
> It is clear you don't like the SWP mechanism.
Reality is a bit more nuanced. As mentioned above, I'm happy with
`read-positioning-symbols`, but not with `symbols-with-pos-enabled`.
We need to think of ways to slowly make it unnecessary.
Stefan
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.Received: (at 79678) by debbugs.gnu.org; 22 Oct 2025 22:28:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 22 18:28:03 2025 Received: from localhost ([127.0.0.1]:58222 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1vBhJK-0003hQ-Fb for submit <at> debbugs.gnu.org; Wed, 22 Oct 2025 18:28:03 -0400 Received: from mail.muc.de ([193.149.48.3]:29497) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1vBhJG-0003h2-WE for 79678 <at> debbugs.gnu.org; Wed, 22 Oct 2025 18:28:00 -0400 Received: (qmail 77975 invoked by uid 3782); 23 Oct 2025 00:27:51 +0200 Received: from muc.de (p4fe1531c.dip0.t-ipconnect.de [79.225.83.28]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 23 Oct 2025 00:27:51 +0200 Received: (qmail 1964 invoked by uid 1000); 22 Oct 2025 22:27:50 -0000 Date: Wed, 22 Oct 2025 22:27:50 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#79678: Problems with the symbols with position mechanism Message-ID: <aPlaZha-LePL4CKH@HIDDEN> References: <aPkdPnGEi5sRT7km@HIDDEN> <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 79678 Cc: acm@HIDDEN, 79678 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Wed, Oct 22, 2025 at 16:33:38 -0400, Stefan Monnier wrote: > > The cause of both of these bugs at an abstract level is the same. > > That is the violation of two principles which were not fully > > understood when SWPs were first implemented: > AFAIK they were very well understood back then already, we discussed > them at length. > > To fix this bug: > > 1. SWPs must be in the forms to eval-when/and-compile. > > 2. SWPs must be stripped, when not wanted, from values for setq. > What's special about `setq`? What about `put`, `puthash`, `setc[ad]r`, > `aset`, etc...? I'm aware of these, but one thing at a time! The patch is already big enough as it is. > > The stripping happens only when macro expansion is in progress. > That means all those "assignment primitives" will need to test a runtime > boolean indicating whether we're in macroexpansion mode? Yes, of course. That won't even take a nanosecond on a modern machine. > IOW, this is spreading the SWP special treatment deeper into the core of > the ELisp language (it's currently mostly limited to `eq` and that is > already a source of concern). Concern to whom? The SWP mechanism _is_ part of the core of the Lisp machine and has been for several years. You've recently been encouraging its expansion and wider use in semel.el. If you look more closely at the patch you'll see that it has no effect on the Lisp language. One of my main motivations in doing this work (and it was a significant amount of work) was precisely to avoid the sort of workaround you suggested earlier; things like `eval-when-compile-preserving-sympos' which _would_ have affected the Lisp language quite markedly. > If you look at your `test-cc-awk.el` examples, you'll see that there > will always be variants of it where we either don't have any position > info or it's imprecise. I disagree. Perhaps you could construct an example.el where some error or warning message gets no position or a wrong position. (As I've already said elsewhere, I don't accept your euphemism "imprecise".) Have you actually tried out the patch, yet? > E.g. if that code gets byte-compiled before it's evaluated, the info > will be lost because the `.elc` file currently doesn't preserve > position information, or the error/warning will be able something that > does not involve a symbol, or ... > So I'm not interested in fixing those cases "at any cost". We have at least one bug here, a nasty bug, and I've fixed it. Why do you prefer for bugs not to be fixed? > Your proposal seems to me to be way beyond what I'm willing to pay to > fix those cases. Just look at all the "dont_strip" management to handle > the various combinations of let-bound and buffer-local; this is hideous, > making already complex code even worse. The increase in complexity is moderate, not "hideous", and quite in keeping with the essentially complicated things the code has to do, given the complicated nature of variables in Emacs Lisp. > And then we have the fact that the presumably O(1) operation `set` can > now not only mutate the value it receives but traverse it completely, > making the operation take a time proportional to the size of the > second arg. Only during the expansion of a macro during byte compilation. In practice, the new code doesn't make Emacs more than unnoticeably slower. Time a bootstrap build in master, and compare it with one using the patch. You might be pleasantly surprised. > That's a non-starter for me. > AFAIC, I can see 2 directions in which we can go: > 1- Try and keep source-code-positions out of the core language, confining > them to the compiler. That's what we've tried to do so far, where the > `eq` and `symbols-with-pos-enabled` are tolerated as "temporary" wrinkles > until we replace it with something better. If there is anything better, I'll be quite surprised. Since even before I implemented SWPs, nobody has been able to come up with a coherent alternative to replace them. > That's also what's used in virtually all other languages/compilers. > It requires distinguishing "code" and "data". To improve on the > current situation, this requires changing macros so as to indicate > which parts of its arguments is code and which part is data, probably > also change our interpreter (aka `eval`) so it can work with code > (it currently expects its code argument to be in the form of data, > i.e. without SWPs). In other words, radically changing Emacs, rendering a vast quantity of existing Lisp code unusable. > 2- Don't distinguish code and data. I.e. allow SWPs everywhere all the > time, eliminate `symbols-with-pos-enabled` and behave as if it were > always non-nil, get rid of `byte-run-strip-symbol-positions`, ... > This will require changes like printing SWPs as bare symbols rather than > as "#<symbol FOO at BAR>", and will slow down `EQ` (at least until > we can change enough code to be able to revert `EQ` to what is now > `BASE_EQ`). > I'm not fond of option (2), but I prefer it to your half-way > proposal. I think option 2 is also a non-starter. Do you have a better way of fixing this bug? If so, I'd be interested, and suggest it supersede my patch once that better way has been implemented. It is clear you don't like the SWP mechanism. But I think the only real criticism you've made of the patch is of the increase in complexity of the code. I disagree with that criticism. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
Received: (at 79678) by debbugs.gnu.org; 22 Oct 2025 20:33:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 22 16:33:55 2025
Received: from localhost ([127.0.0.1]:58112 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vBfWt-0006ez-DC
for submit <at> debbugs.gnu.org; Wed, 22 Oct 2025 16:33:55 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:35483)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <monnier@HIDDEN>)
id 1vBfWq-0006ej-7I
for 79678 <at> debbugs.gnu.org; Wed, 22 Oct 2025 16:33:53 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 9EB58819C0;
Wed, 22 Oct 2025 16:33:45 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1761165224;
bh=7HmrQOUcxHr4qrvsQmg6z8jzMWMpmyow+FK0ugGyG6A=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=J4fozSlowbrdBMNUVuCg+PZahLt9886ZWdYnI8eWiqxbVr/DX0M1ngyB6+r7aiYL8
ohn3dwkT5M9T/DVDbSHkradyvr5Xs2wszzZdIqMtAmv6Zv8nD1mkBHTWERX8omgypw
CGBsVxEBn0yVS9fWL2jmGGUu7vgEQcoJNPkeQimi0HIcExrmaEqaOgYt/Gkm9wdU+d
axJ5sNVSGd+sPYzp3/D6X5PSlBnWq5uD3pTxnfwB/eVOs3W+7QWhxM7ve8KyuOc4r1
/zUu5ovN4xpSWXbOTZ9DeaWOcH8in0FqT0zhTYroeEl5NgXoj5punlfuzFJYefdNDc
jjbajpgtGbxtQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 976B481690;
Wed, 22 Oct 2025 16:33:44 -0400 (EDT)
Received: from asado (unknown [181.28.45.30])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 556E2120426;
Wed, 22 Oct 2025 16:33:43 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#79678: Problems with the symbols with position mechanism
In-Reply-To: <aPkdPnGEi5sRT7km@HIDDEN>
Message-ID: <jwv8qh2d8mf.fsf-monnier+emacs@HIDDEN>
References: <aPkdPnGEi5sRT7km@HIDDEN>
Date: Wed, 22 Oct 2025 16:33:38 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.134 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 79678
Cc: 79678 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> The cause of both of these bugs at an abstract level is the same.
> That is the violation of two principles which were not fully
> understood when SWPs were first implemented:
AFAIK they were very well understood back then already, we discussed
them at length.
> To fix this bug:
> 1. SWPs must be in the forms to eval-when/and-compile.
> 2. SWPs must be stripped, when not wanted, from values for setq.
What's special about `setq`? What about `put`, `puthash`, `setc[ad]r`,
`aset`, etc...?
> The stripping happens only when macro expansion is in progress.
That means all those "assignment primitives" will need to test a runtime
boolean indicating whether we're in macroexpansion mode?
IOW, this is spreading the SWP special treatment deeper into the core of
the ELisp language (it's currently mostly limited to `eq` and that is
already a source of concern).
If you look at your `test-cc-awk.el` examples, you'll see that there
will always be variants of it where we either don't have any position
info or it's imprecise. E.g. if that code gets byte-compiled before
it's evaluated, the info will be lost because the `.elc` file currently
doesn't preserve position information, or the error/warning will be able
something that does not involve a symbol, or ...
So I'm not interested in fixing those cases "at any cost".
Your proposal seems to me to be way beyond what I'm willing to pay to
fix those cases. Just look at all the "dont_strip" management to handle
the various combinations of let-bound and buffer-local; this is hideous,
making already complex code even worse.
And then we have the fact that the presumably O(1) operation `set` can
now not only mutate the value it receives but traverse it completely,
making the operation take a time proportional to the size of the
second arg.
That's a non-starter for me.
AFAIC, I can see 2 directions in which we can go:
1- Try and keep source-code-positions out of the core language, confining
them to the compiler. That's what we've tried to do so far, where the
`eq` and `symbols-with-pos-enabled` are tolerated as "temporary" wrinkles
until we replace it with something better.
That's also what's used in virtually all other languages/compilers.
It requires distinguishing "code" and "data". To improve on the
current situation, this requires changing macros so as to indicate
which parts of its arguments is code and which part is data, probably
also change our interpreter (aka `eval`) so it can work with code
(it currently expects its code argument to be in the form of data,
i.e. without SWPs).
2- Don't distinguish code and data. I.e. allow SWPs everywhere all the
time, eliminate `symbols-with-pos-enabled` and behave as if it were
always non-nil, get rid of `byte-run-strip-symbol-positions`, ...
This will require changes like printing SWPs as bare symbols rather than
as "#<symbol FOO at BAR>", and will slow down `EQ` (at least until
we can change enough code to be able to revert `EQ` to what is now
`BASE_EQ`).
I'm not fond of option (2), but I prefer it to your half-way
proposal.
Stefan
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
Received: (at 79678) by debbugs.gnu.org; 22 Oct 2025 19:07:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 22 15:07:12 2025
Received: from localhost ([127.0.0.1]:57897 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vBeAx-0001KP-3V
for submit <at> debbugs.gnu.org; Wed, 22 Oct 2025 15:07:11 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:44290)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1vBeAr-0001Jh-Aw
for 79678 <at> debbugs.gnu.org; Wed, 22 Oct 2025 15:07:08 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1vBeAl-0006r1-Oa; Wed, 22 Oct 2025 15:06:59 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
Date; bh=zufGBFYmnHri4/6g6xomRdA1lIaSI8dsAcBemXjRagw=; b=rXRJhdOiyhFsS7lSa4o4
Ia/kRR7NUdLQbtuL0rBoJzgyVxXmBwM0PWFRfujY1mRtxONUn1XIQtt3XhVGf7Poyjw/lUvwLLF+4
VbXchBKeSJ+O9l5rLXp5k4SJPJ9tHGD6QLdggL0sYjCEjckzi/7OSFHCoG/TYekeC3sXMQXGhNimB
Ap9UzolmmkwjSAJEdfABnrJgD5FTQoLlvygScpE4DHptBthhVxu17Vx+3SGhXfN3bv/yiNNKrcwXc
L+nWEVIR6wRlrChqRdKcoRDzwzvVfJXr6F4SfwkLvqGyIgXw30kAoXp9GoG1KfT60k45eKp22vkzg
Ji8+Q1N699VukQ==;
Date: Wed, 22 Oct 2025 22:06:55 +0300
Message-Id: <86cy6evjqo.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>,
Stefan Monnier <monnier@HIDDEN>
In-Reply-To: <aPkdPnGEi5sRT7km@HIDDEN> (message from Alan Mackenzie on
Wed, 22 Oct 2025 18:06:54 +0000)
Subject: Re: bug#79678: Problems with the symbols with position mechanism
References: <aPkdPnGEi5sRT7km@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 79678
Cc: 79678 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Cc: acm@HIDDEN
> Date: Wed, 22 Oct 2025 18:06:54 +0000
> From: Alan Mackenzie <acm@HIDDEN>
>
> Hello, Emacs.
>
> As has become apparent with some recent bugs, the implementation of the
> symbols with position mechanism is unfinished.
>
> For example, the source file test-cc-awk.el in bug#78872, on being byte
> compiled, produces these messages:
>
> In toplevel form:
> ~/cc-mackenzie-2025-06-21/test-cc-awk.el:152:33: Warning: reference to free variable ‘c-awk-escaped-nls*’
> ~/cc-mackenzie-2025-06-21/test-cc-awk.el:153:17: Warning: reference to free variable ‘c-awk-escaped-nls*-with-space*’
> ~/cc-mackenzie-2025-06-21/test-cc-awk.el: Warning: ‘font-lock-keyword-face’ is an obsolete variable (as of 31.1); use the quoted symbol instead: 'font-lock-keyword-face
> ~/cc-mackenzie-2025-06-21/test-cc-awk.el:63:4: Error: Symbol’s value as variable is void: c-awk-escaped-nls*
>
> .. The line/column position in the first two messages is correct, that of the
> fourth is wrong (it should be 152:33, not 63:4), and there is no position on
> the third message (it should be 68:14).
>
> The following source file on being compiled:
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> ;; -*- lexical-binding: t -*-
> (defvar baz nil)
>
> (defmacro foo (bar)
> (setq baz bar)
> `(car ,bar))
>
> (defun quux ()
> (let ((bar '(Car . Cdr)))
> (foo `,bar)))
>
> (eval-when-compile
> (let (print-symbols-bare)
> (message "%S" baz)))
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> , ought to output `,bar. Instead it outputs `,#<symbol bar at 156> - the
> symbol with position ("SWP") has "leaked" from the macro foo. This is a Bad
> Thing.
>
> #########################################################################
>
> The cause of both of these bugs at an abstract level is the same. That is the
> violation of two principles which were not fully understood when SWPs were
> first implemented:
>
> 1. Source forms passed to macros must have SWPs so that correct warning/error
> positions can be output.
> 2. Macros setq'ing to dynamic variables must strip the SWPs to prevent
> leakage.
>
> The first bug, with ~/test-cc-mode.el is due to the violation of 1. The
> second, due to the violation of 2.
>
> 1. The forms given to most macros contain SWPs, but eval-when-compile and
> eval-and-compile are exceptions. SWPs were stripped from these macros' forms
> as an expedient back when SWPs were first implemented. It is however wrong.
>
> 2. Emacs fails to strip the SWP from BAZ in (setq baz bar) in macro FOO.
>
> #########################################################################
>
> To fix this bug:
> 1. SWPs must be in the forms to eval-when/and-compile.
> 2. SWPs must be stripped, when not wanted, from values for setq.
>
> The stripping happens only when macro expansion is in progress.
>
> For 2., we need to distinguish cases needing stripping from those that don't.
> It turns out we need never strip for lexical bindings. For dynamic bindings,
> let-bound bindings and function parameters do not need stripping, since they
> are not being written into the permanent value of the variable. Only the
> binding for a default value needs its value's SWPs stripping.
>
> This distinction is made by adding a boolean flag, dont_strip, to each dynamic
> variable binding. It is bound to true in let forms, etc. On binding such a
> variable, the old value of dont_strip is saved with the rest of the binding on
> specpdl. (Exceptionally, some variables used in byte compilation must retain
> their SWPs in their default value. This is catered for.)
>
> #########################################################################
>
> With the following patch, the bugs above are fully fixed. It should be
> virtually invisible to the rest of Emacs. make check runs without any
> unexpected results.
>
> As for execution speed, a full build with native compilation was around half a
> second slower (in a 2 minute build). This is likely due to having slightly
> more code to compile rather than the changes slowing down compilation.
Thanks, I've added Stefan to the discussion.
bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 22 Oct 2025 18:07:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 22 14:07:28 2025
Received: from localhost ([127.0.0.1]:57745 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1vBdF6-0005lZ-Ot
for submit <at> debbugs.gnu.org; Wed, 22 Oct 2025 14:07:28 -0400
Received: from lists.gnu.org ([2001:470:142::17]:48418)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1vBdF0-0005l1-Fa
for submit <at> debbugs.gnu.org; Wed, 22 Oct 2025 14:07:22 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <acm@HIDDEN>) id 1vBdEu-0005HK-BL
for bug-gnu-emacs@HIDDEN; Wed, 22 Oct 2025 14:07:12 -0400
Received: from mail.muc.de ([193.149.48.3])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <acm@HIDDEN>) id 1vBdEo-0006kx-Mj
for bug-gnu-emacs@HIDDEN; Wed, 22 Oct 2025 14:07:12 -0400
Received: (qmail 46195 invoked by uid 3782); 22 Oct 2025 20:06:55 +0200
Received: from muc.de (p4fe1531c.dip0.t-ipconnect.de [79.225.83.28]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Wed, 22 Oct 2025 20:06:55 +0200
Received: (qmail 4584 invoked by uid 1000); 22 Oct 2025 18:06:54 -0000
Date: Wed, 22 Oct 2025 18:06:54 +0000
To: bug-gnu-emacs@HIDDEN
Subject: Problems with the symbols with position mechanism
Message-ID: <aPkdPnGEi5sRT7km@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
Received-SPF: pass client-ip=193.149.48.3; envelope-from=acm@HIDDEN;
helo=mail.muc.de
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.9 (/)
X-Debbugs-Envelope-To: submit
Cc: acm@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.1 (/)
Hello, Emacs.
As has become apparent with some recent bugs, the implementation of the
symbols with position mechanism is unfinished.
For example, the source file test-cc-awk.el in bug#78872, on being byte
compiled, produces these messages:
In toplevel form:
~/cc-mackenzie-2025-06-21/test-cc-awk.el:152:33: Warning: reference to free variable ‘c-awk-escaped-nls*’
~/cc-mackenzie-2025-06-21/test-cc-awk.el:153:17: Warning: reference to free variable ‘c-awk-escaped-nls*-with-space*’
~/cc-mackenzie-2025-06-21/test-cc-awk.el: Warning: ‘font-lock-keyword-face’ is an obsolete variable (as of 31.1); use the quoted symbol instead: 'font-lock-keyword-face
~/cc-mackenzie-2025-06-21/test-cc-awk.el:63:4: Error: Symbol’s value as variable is void: c-awk-escaped-nls*
.. The line/column position in the first two messages is correct, that of the
fourth is wrong (it should be 152:33, not 63:4), and there is no position on
the third message (it should be 68:14).
The following source file on being compiled:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; -*- lexical-binding: t -*-
(defvar baz nil)
(defmacro foo (bar)
(setq baz bar)
`(car ,bar))
(defun quux ()
(let ((bar '(Car . Cdr)))
(foo `,bar)))
(eval-when-compile
(let (print-symbols-bare)
(message "%S" baz)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
, ought to output `,bar. Instead it outputs `,#<symbol bar at 156> - the
symbol with position ("SWP") has "leaked" from the macro foo. This is a Bad
Thing.
#########################################################################
The cause of both of these bugs at an abstract level is the same. That is the
violation of two principles which were not fully understood when SWPs were
first implemented:
1. Source forms passed to macros must have SWPs so that correct warning/error
positions can be output.
2. Macros setq'ing to dynamic variables must strip the SWPs to prevent
leakage.
The first bug, with ~/test-cc-mode.el is due to the violation of 1. The
second, due to the violation of 2.
1. The forms given to most macros contain SWPs, but eval-when-compile and
eval-and-compile are exceptions. SWPs were stripped from these macros' forms
as an expedient back when SWPs were first implemented. It is however wrong.
2. Emacs fails to strip the SWP from BAZ in (setq baz bar) in macro FOO.
#########################################################################
To fix this bug:
1. SWPs must be in the forms to eval-when/and-compile.
2. SWPs must be stripped, when not wanted, from values for setq.
The stripping happens only when macro expansion is in progress.
For 2., we need to distinguish cases needing stripping from those that don't.
It turns out we need never strip for lexical bindings. For dynamic bindings,
let-bound bindings and function parameters do not need stripping, since they
are not being written into the permanent value of the variable. Only the
binding for a default value needs its value's SWPs stripping.
This distinction is made by adding a boolean flag, dont_strip, to each dynamic
variable binding. It is bound to true in let forms, etc. On binding such a
variable, the old value of dont_strip is saved with the rest of the binding on
specpdl. (Exceptionally, some variables used in byte compilation must retain
their SWPs in their default value. This is catered for.)
#########################################################################
With the following patch, the bugs above are fully fixed. It should be
virtually invisible to the rest of Emacs. make check runs without any
unexpected results.
As for execution speed, a full build with native compilation was around half a
second slower (in a 2 minute build). This is likely due to having slightly
more code to compile rather than the changes slowing down compilation.
diff --git a/lisp/emacs-lisp/byte-run.el b/lisp/emacs-lisp/byte-run.el
index 05ee0615fec..be4c95205aa 100644
--- a/lisp/emacs-lisp/byte-run.el
+++ b/lisp/emacs-lisp/byte-run.el
@@ -32,9 +32,12 @@
(defvar byte-run--ssp-seen nil
"Which conses/vectors/records have been processed in strip-symbol-positions?
-The value is a hash table, the keys being the elements and the values being t.
+The value is a hash table, the keys being the elements and the values
+being t. Used in byte-run-strip-symbol-positions and
+byte-run-strip-symbol-poisitions-copy, where the value is the key or a
+replacement value.
-The purpose of this is to detect circular structures.")
+The purpose of this is to detect circular structures and repeated conses.")
(defalias 'byte-run--strip-list
#'(lambda (arg)
@@ -90,15 +93,203 @@ 'byte-run-strip-symbol-positions
ARG is any Lisp object, but is usually a list or a vector or a
record, containing symbols with position."
- (setq byte-run--ssp-seen (make-hash-table :test 'eq))
+ (let (byte-compile-macroing)
+ (setq byte-run--ssp-seen (make-hash-table :test 'eq))
+ (cond
+ ((symbol-with-pos-p arg)
+ (bare-symbol arg))
+ ((consp arg)
+ (byte-run--strip-list arg))
+ ((or (vectorp arg) (recordp arg))
+ (byte-run--strip-vector/record arg))
+ (t arg)))))
+
+
+(defalias 'byte-run--strip-elt-copy
+ #'(lambda (arg)
+ "Strip any positions from the Lisp element ARG.
+Return the changed element, or nil if there was no change."
(cond
((symbol-with-pos-p arg)
(bare-symbol arg))
((consp arg)
- (byte-run--strip-list arg))
+ (byte-run--strip-list-copy arg))
((or (vectorp arg) (recordp arg))
- (byte-run--strip-vector/record arg))
- (t arg))))
+ (byte-run--strip-vector/record-copy arg))
+ ((interpreted-function-p arg)
+ (unwind-protect
+ (progn
+ (closure-to-record arg)
+ (let ((new (byte-run--strip-vector/record-copy arg)))
+ (and new (if (recordp new) ; It might already be a closure,
+ ; if it was found in the hash table.
+ (record-to-closure new)
+ new))))
+ (if (recordp arg)
+ (record-to-closure arg))))
+ (t nil))))
+
+;; The source for the following macro. Use `macroexpand-all' to regenerate.
+;; (defmacro byte-run--changed-hash (arg)
+;; "Return `(gethash ARG byte-run--ssp-seen)' when different from ARG.
+;; This indicates an ARG with a change in it. ARG should be a variable."
+;; (declare (debug t))
+;; `(let ((hash (gethash ,arg byte-run--ssp-seen)))
+;; (and (not (eq hash ,arg)) hash)))
+(defalias 'byte-run--changed-hash
+ (cons 'macro
+ #'(lambda (arg)
+ "Return `(gethash ARG byte-run--ssp-seen)' when different from ARG.
+This indicates an ARG with a change in it. ARG should be a variable."
+ (list 'let (list (list 'hash
+ (cons 'gethash
+ (cons arg '(byte-run--ssp-seen)))))
+ (cons 'and (cons (list 'not (list 'eq 'hash arg)) '(hash)))))))
+(put 'byte-run--changed-hash 'edebug-form-spec 't)
+
+(defalias 'byte-run--strip-list-copy
+ #'(lambda (arg)
+ "Strip the positions from symbols with position in the list ARG.
+If no changes are made, return nil, otherwise return a modified copy of
+ARG, leaving the original ARG unchanged. Note that a non-nil result
+shares as many components as possible with ARG."
+ (let ((a arg)
+ hash res changed)
+ ;; Scan A, assuming we won't need to create a changed copy of
+ ;; ARG, until we find a changed element.
+ (while (and a
+ (not (setq hash (byte-run--changed-hash a))) ; circular list
+ (not (setq changed (byte-run--strip-elt-copy
+ (if (consp a) (car a) a)))))
+ (puthash a a byte-run--ssp-seen)
+ (setq a (and (consp a) (cdr a))))
+
+ ;; If anything in the list has changed, we need to make a copy
+ ;; of ARG, incorporating that change.
+ (if a
+ (let ((z arg))
+ ;; Copy the leading unchanged elements (including the
+ ;; unchanged car of a dotted list).
+ (while (and (consp z)
+ (not (eq z a)))
+ (setq res (cons (car z) res))
+ (puthash z res byte-run--ssp-seen)
+ (setq z (cdr z)))
+
+ (if (not hash)
+ (progn
+ (if (consp z)
+ ;; Copy the first changed element, for which the
+ ;; replacement will already be in HASH or
+ ;; CHANGED. We postpone this in the case of
+ ;; HASH or a cdr of a dotted list.
+ (progn
+ (setq res (cons changed res))
+ (puthash z res byte-run--ssp-seen)
+ (setq z (cdr z))))
+
+ ;; Copy the possibly changed rest of the elements.
+ (while (and (consp z)
+ (not (setq hash (byte-run--changed-hash z))))
+ (setq changed (byte-run--strip-elt-copy (car z)))
+ (setq res (cons (or changed (car z)) res))
+ (puthash z res byte-run--ssp-seen)
+ (setq z (cdr z)))
+
+ ;; The cdr of a dotted pair not at A.
+ (if (and z (atom z)
+ (not (and a (atom a))))
+ (or
+ (setq hash (byte-run--changed-hash z))
+ (setq changed (byte-run--strip-elt-copy z))))
+
+ (setq res (nreverse res))
+ (cond
+ ;; A dotted pair cdr.
+ ((and z (atom z))
+ (setcdr (last res) (or hash changed z))
+ res)
+ (hash (setq res (nconc res hash)))
+ (t res)))
+ ;; Deal with a HASH found earlier on.
+ (setq res (nreverse res))
+ (setq res (nconc res hash)))
+ res)
+ nil))))
+
+(defalias 'byte-run--strip-vector/record-copy
+ #'(lambda (arg)
+ "Strip the positions from symbols with position in the vector/record ARG.
+If no changes are made, return nil, otherwise return a modified copy of
+ARG, leaving the original ARG unchanged. As many components of ARG as
+possible are shared by the result and ARG."
+ (let* ((len (length arg))
+ (i 0)
+ elt hash changed)
+ (if (setq hash (byte-run--changed-hash arg);; (gethash arg byte-run--ssp-seen)
+ )
+ ;; (and (not (eq hash arg))
+ hash;; )
+ ;; Be careful of circular structures.
+ (puthash arg arg byte-run--ssp-seen)
+ ;; Scan the vector/record for elements needing changing.
+ (while (and (< i len)
+ (progn
+ (setq elt (aref arg i))
+ (not (setq changed
+ (byte-run--strip-elt-copy elt)))))
+ (setq i (1+ i)))
+
+ (if (< i len)
+ (let ((res (if (vectorp arg)
+ (make-vector len nil)
+ (make-record nil (1- len) nil)))
+ (j 0))
+ (puthash arg res byte-run--ssp-seen)
+ ;; Copy the unchanged initial elements of the vector/record.
+ (while (< j i)
+ (aset res j (aref arg j))
+ (setq j (1+ j)))
+ ;; Copy the rest of the vector/record, possibly changed.
+ (while (progn (aset res j (or changed (aref arg j)))
+ (setq j (1+ j))
+ (< j len))
+ (setq changed (byte-run--strip-elt-copy (aref arg j))))
+ res)
+ nil)))))
+
+(defalias 'byte-run-strip-symbol-positions-copy
+ #'(lambda (arg)
+ "Strip all positions from symbols in ARG.
+The original ARG does not get modified. The result will share
+components with ARG where possible.
+
+ARG is any Lisp object, but is usually a list or a vector or a
+record, containing symbols with position."
+ (let (byte-compile-macroing) ; Prevent recursive call with
+ ; the following `setq's
+ (cond
+ ((symbol-with-pos-p arg)
+ (bare-symbol arg))
+ ((consp arg)
+ (setq byte-run--ssp-seen (make-hash-table :test 'eq))
+ (prog1 (or (byte-run--strip-list-copy arg) arg)
+ (clrhash byte-run--ssp-seen)))
+ ((or (vectorp arg) (recordp arg))
+ (setq byte-run--ssp-seen (make-hash-table :test 'eq))
+ (prog1 (or (byte-run--strip-vector/record-copy arg) arg)
+ (clrhash byte-run--ssp-seen)))
+ ((interpreted-function-p arg)
+ (setq byte-run--ssp-seen (make-hash-table :test 'eq))
+ (unwind-protect
+ (progn
+ (closure-to-record arg)
+ (record-to-closure (or (byte-run--strip-vector/record-copy arg)
+ arg)))
+ (if (recordp arg)
+ (record-to-closure arg))
+ (clrhash byte-run--ssp-seen)))
+ (t arg)))))
(defalias 'function-put
;; We don't want people to just use `put' because we can't conveniently
diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 7cb944e3b08..69f60fc2915 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -466,6 +466,7 @@ byte-compile-call-tree
where CALLERS is a list of functions that call FUNCTION, and CALLS
is a list of functions for which calls were generated while compiling
FUNCTION.")
+(byte-compile-never-strip 'byte-compile-call-tree)
(defcustom byte-compile-call-tree-sort 'name
"If non-nil, sort the call tree.
@@ -480,6 +481,8 @@ byte-compile-jump-tables
"List of all jump tables used during compilation of this form.")
(defvar byte-compile-constants nil
"List of all constants encountered during compilation of this form.")
+(byte-compile-never-strip 'byte-compile-constants)
+
(defvar byte-compile-variables nil
"List of all variables encountered during compilation of this form.")
(defvar byte-compile-bound-variables nil
@@ -513,40 +516,47 @@ byte-compile-recurse-toplevel
(funcall non-toplevel-case form)))
-(defvar bytecomp--copy-tree-seen)
-
-(defun bytecomp--copy-tree-1 (tree)
- ;; TREE must be a cons.
- (or (gethash tree bytecomp--copy-tree-seen)
- (let* ((next (cdr tree))
- (result (cons nil next))
- (copy result))
- (while (progn
- (puthash tree copy bytecomp--copy-tree-seen)
- (let ((a (car tree)))
- (setcar copy (if (consp a)
- (bytecomp--copy-tree-1 a)
- a)))
- (and (consp next)
- (let ((tail (gethash next bytecomp--copy-tree-seen)))
- (if tail
- (progn (setcdr copy tail)
- nil)
- (setq tree next)
- (setq next (cdr next))
- (let ((prev copy))
- (setq copy (cons nil next))
- (setcdr prev copy)
- t))))))
- result)))
-
-(defun bytecomp--copy-tree (tree)
- "Make a copy of TREE, preserving any circular structure therein.
-Only conses are traversed and duplicated, not arrays or any other structure."
- (if (consp tree)
- (let ((bytecomp--copy-tree-seen (make-hash-table :test #'eq)))
- (bytecomp--copy-tree-1 tree))
- tree))
+;; (defvar bytecomp--copy-tree-seen)
+
+;; (defun bytecomp--copy-tree-1 (tree)
+;; ;; TREE must be a cons.
+;; (or (gethash tree bytecomp--copy-tree-seen)
+;; (let* ((next (cdr tree))
+;; (result (cons nil next))
+;; (copy result))
+;; (while (progn
+;; (puthash tree copy bytecomp--copy-tree-seen)
+;; (let ((a (car tree)))
+;; (setcar copy (if (consp a)
+;; (bytecomp--copy-tree-1 a)
+;; a)))
+;; (and (consp next)
+;; (let ((tail (gethash next bytecomp--copy-tree-seen)))
+;; (if tail
+;; (progn (setcdr copy tail)
+;; nil)
+;; (setq tree next)
+;; (setq next (cdr next))
+;; (let ((prev copy))
+;; (setq copy (cons nil next))
+;; (setcdr prev copy)
+;; t))))))
+;; result)))
+
+;; (defun bytecomp--copy-tree (tree)
+;; "Make a copy of TREE, preserving any circular structure therein.
+;; Only conses are traversed and duplicated, not arrays or any other structure."
+;; (if (consp tree)
+;; (let ((bytecomp--copy-tree-seen (make-hash-table :test #'eq)))
+;; (bytecomp--copy-tree-1 tree))
+;; tree))
+
+(defvar byte-compile-evaluating nil
+ "Non-nil when `eval-when-compile' or `eval-and-compile' is active.
+It is used to inhibit the generation of invocations of
+`internal--with-suppressed-warnings' in contexts where that \"macro\" is
+not available to the compiler.")
+(defvar byte-compile--for-effect)
(defconst byte-compile-initial-macro-environment
`(
@@ -554,7 +564,8 @@ byte-compile-initial-macro-environment
;; (apply 'byte-compiler-options-handler forms)))
(declare-function . byte-compile-macroexpand-declare-function)
(eval-when-compile . ,(lambda (&rest body)
- (let ((result nil))
+ (let ((result nil)
+ (byte-compile-evaluating t))
(byte-compile-recurse-toplevel
(macroexp-progn body)
(lambda (form)
@@ -567,32 +578,38 @@ byte-compile-initial-macro-environment
(let ((byte-compile-unresolved-functions
byte-compile-unresolved-functions)
(byte-compile-new-defuns
- byte-compile-new-defuns))
+ byte-compile-new-defuns)
+ (byte-compile-macroing t))
(setf result
(byte-compile-eval
- (byte-run-strip-symbol-positions
- (byte-compile-top-level
- (byte-compile-preprocess form))))))))
+ (byte-compile-close-variables-subset
+ nil byte-compile-reserved-constants
+ ;; Half-compile for error messages.
+ (byte-compile-no-output
+ (byte-compile-preprocess form))
+ (macroexpand-all
+ form macroexpand-all-environment)))))))
(list 'quote result))))
(eval-and-compile . ,(lambda (&rest body)
- (byte-compile-recurse-toplevel
- (macroexp-progn body)
- (lambda (form)
- ;; Don't compile here, since we don't know
- ;; whether to compile as byte-compile-form
- ;; or byte-compile-file-form.
- (let ((expanded
- (macroexpand--all-toplevel
- form
- macroexpand-all-environment)))
- (eval (byte-run-strip-symbol-positions
- (bytecomp--copy-tree expanded))
- (when lexical-binding
- (or (append
- macroexp--dynvars
- byte-compile-bound-variables)
- t)))
- expanded)))))
+ (let ((byte-compile-evaluating t))
+ (byte-compile-recurse-toplevel
+ (macroexp-progn body)
+ (lambda (form)
+ ;; Don't compile here, since we don't know
+ ;; whether to compile as byte-compile-form
+ ;; or byte-compile-file-form.
+ (let* ((byte-compile-macroing t)
+ (expanded
+ (macroexpand--all-toplevel
+ form
+ macroexpand-all-environment)))
+ (eval expanded
+ (when lexical-binding
+ (or (append
+ macroexp--dynvars
+ byte-compile-bound-variables)
+ t)))
+ expanded))))))
(with-suppressed-warnings
. ,(lambda (warnings &rest body)
;; We let-bind `byte-compile--suppressed-warnings' here in order
@@ -606,10 +623,12 @@ byte-compile-initial-macro-environment
;; This function doesn't exist, but is just a placeholder
;; symbol to hook up with the
;; `byte-hunk-handler'/`byte-defop-compiler-1' machinery.
- `(internal--with-suppressed-warnings
- ',warnings
- ,(macroexpand-all `(progn ,@body)
- macroexpand-all-environment)))
+ (if (not byte-compile-evaluating)
+ `(internal--with-suppressed-warnings
+ ',warnings
+ ,(macroexpand-all `(progn ,@body)
+ macroexpand-all-environment))
+ `(progn ,@body)))
(macroexp-warn-and-return
(format-message "`with-suppressed-warnings' with empty body")
nil '(empty-body with-suppressed-warnings) t warnings)))))
@@ -654,6 +673,7 @@ byte-compile-new-defuns
;; Variables for lexical binding
(defvar byte-compile--lexical-environment nil
"The current lexical environment.")
+(byte-compile-never-strip 'byte-compile--lexical-environment)
(defvar byte-compile-tag-number 0)
(defvar byte-compile-output nil
@@ -1100,7 +1120,7 @@ byte-compile-lapcode
;; Since this may change the hash table key, we remove
;; the entry from the table and reinsert it outside the
;; scope of the `maphash'.
- (setq value (byte-run-strip-symbol-positions value))
+ (setq value (byte-run-strip-symbol-positions-copy value))
(push (cons value pc) alist)
(remhash value hash-table))
hash-table)
@@ -1204,10 +1224,12 @@ emacs-lisp-compilation-recompile
(byte-compile-file emacs-lisp-compilation--current-file))
(defvar byte-compile-current-form nil)
+(byte-compile-never-strip 'byte-compile-current-form)
+
(defvar byte-compile-dest-file nil)
(defvar byte-compile-current-file nil)
(defvar byte-compile-current-group nil)
-(defvar byte-compile-current-buffer nil)
+(defvar byte-compile-current-buffer) ;; Definition moved to data.c, 2025-09-24
;; Log something that isn't a warning.
(defmacro byte-compile-log (format-string &rest args)
@@ -1247,26 +1269,48 @@ byte-compile-abbreviate-file
(f2 (file-relative-name file dir)))
(if (< (length f2) (length f1)) f2 f1)))
-(defun byte-compile--first-symbol-with-pos (form)
+(defvar byte-compile--ssp-seen nil
+ "Which conses/vectors/records have been processed, seeking a SWP.
+The value is a hash table, the keys being the elements, the values being t.
+
+The purpose of this is to detect circular structures and repeated conses
+in `byte-compile--first-symbol-with-pos'.")
+
+(defun byte-compile--first-symbol-with-pos-1 (form)
"Return the first symbol with position in form, or nil if none.
Order is by depth-first search."
- (named-let loop ((form form)
- (depth 10)) ;Arbitrary limit.
- (cond
- ((<= depth 0) nil) ;Avoid cycles (bug#58601).
- ((symbol-with-pos-p form) form)
- ((consp form)
- (or (loop (car form) (1- depth))
- (loop (cdr form) (1- depth))))
- ((or (vectorp form) (recordp form))
+ (cond
+ ((symbol-with-pos-p form) form)
+ ((consp form)
+ (unless (gethash form byte-compile--ssp-seen)
+ (puthash form t byte-compile--ssp-seen)
+ (or (byte-compile--first-symbol-with-pos-1 (car form))
+ (let ((sym nil))
+ (setq form (cdr form))
+ (while (and (consp form)
+ (not (setq sym (byte-compile--first-symbol-with-pos-1
+ (car form)))))
+ (setq form (cdr form)))
+ (or sym
+ (and form (byte-compile--first-symbol-with-pos-1 form)))))))
+ ((or (vectorp form) (recordp form))
+ (unless (gethash form byte-compile--ssp-seen)
+ (puthash form t byte-compile--ssp-seen)
(let ((len (length form))
(i 0)
(sym nil))
(while (and (< i len)
- (not (setq sym (loop (aref form i) (1- depth)))))
+ (not (setq sym (byte-compile--first-symbol-with-pos-1
+ (aref form i)))))
(setq i (1+ i)))
sym)))))
+(defun byte-compile--first-symbol-with-pos (form)
+ "Return the first symbol with position in form, or nil if none.
+Order is by depth-first search."
+ (setq byte-compile--ssp-seen (make-hash-table :test 'eq))
+ (byte-compile--first-symbol-with-pos-1 form))
+
(defun byte-compile--warning-source-offset ()
"Return a source offset from `byte-compile-form-stack' or nil if none."
(let ((sym (byte-compile--first-symbol-with-pos byte-compile-form-stack)))
@@ -1942,6 +1986,26 @@ byte-compile-close-variables
(setq byte-to-native-plist-environment
overriding-plist-environment)))))
+(defmacro byte-compile-close-variables-subset (for-effect reserved-csts &rest body)
+ (declare (debug t))
+ `(let (;;
+ ;; Close over a subset of the variables in
+ ;; `byte-compil-close-variables', sufficient to evaluate
+ ;; `byte-compile-top-level' without breaking an enclosing
+ ;; compilation. The first two arguments, if non-nil, are the
+ ;; initial values bound for those variables.
+ ;;
+ (byte-compile--for-effect ,for-effect)
+ (byte-compile-constants nil)
+ (byte-compile-variables nil)
+ (byte-compile-tag-number 0)
+ (byte-compile-depth 0)
+ (byte-compile-maxdepth 0)
+ (byte-compile-reserved-constants (or ,reserved-csts 0))
+ (byte-compile-output nil)
+ (byte-compile-jump-tables nil))
+ ,@body))
+
(defmacro displaying-byte-compile-warnings (&rest body) ;FIXME: namespace!
(declare (debug (def-body)))
`(bytecomp--displaying-warnings (lambda () ,@body)))
@@ -1953,12 +2017,13 @@ bytecomp--displaying-warnings
(funcall body-fn)
;; Use a `handler-bind' to remember the `byte-compile-form-stack'
;; active at the time the error is signaled, so as to
- ;; get more precise error locations.
+ ;; get correct error locations.
(let ((form-stack nil))
(condition-case error-info
(handler-bind
- ((error (lambda (_err)
- (setq form-stack byte-compile-form-stack))))
+ ((error (lambda (err)
+ (setq form-stack
+ (cons err byte-compile-form-stack)))))
(funcall body-fn))
(error (let ((byte-compile-form-stack form-stack))
(byte-compile-report-error error-info))))))))
@@ -2562,8 +2627,6 @@ byte-compile-output-file-form
(prin1 form byte-compile--outbuffer)
nil))
-(defvar byte-compile--for-effect)
-
(defun byte-compile-keep-pending (form &optional handler)
(if (memq byte-optimize '(t source))
(setq form (byte-optimize-one-form form t)))
@@ -3098,8 +3161,9 @@ byte-compile-lambda
(error "Not a lambda list: %S" fun))
(byte-compile-check-lambda-list (nth 1 fun))
(let* ((arglist (nth 1 fun))
- (bare-arglist (byte-run-strip-symbol-positions arglist)) ; for compile-defun.
- (arglistvars (byte-run-strip-symbol-positions
+ (bare-arglist (byte-run-strip-symbol-positions-copy
+ arglist)) ; for compile-defun.
+ (arglistvars (byte-run-strip-symbol-positions-copy
(byte-compile-arglist-vars arglist)))
(byte-compile-bound-variables
(append (if (not lexical-binding) arglistvars)
@@ -3136,22 +3200,28 @@ byte-compile-lambda
(setq command-modes (cdr (cdr int)))
;; If the interactive spec is a call to `list', don't
;; compile it, because `call-interactively' looks at the
- ;; args of `list'. Actually, compile it to get warnings,
- ;; but don't use the result.
+ ;; args of `list'. Actually, compile it partially to get
+ ;; warnings, and use the result for lexical bindings.
(let* ((form (nth 1 int))
- (newform (byte-compile-top-level form)))
- (while (memq (car-safe form) '(let let* progn save-excursion))
- (while (consp (cdr form))
- (setq form (cdr form)))
- (setq form (car form)))
- (if (or (not (eq (car-safe form) 'list))
- ;; For code using lexical-binding, form is not
- ;; valid lisp, but rather an intermediate form
- ;; which may include "calls" to
- ;; internal-make-closure (Bug#29988).
- lexical-binding)
- (setq int `(,(car int) ,newform))
- (setq int (byte-run-strip-symbol-positions int))))) ; for compile-defun.
+ newform)
+ (byte-compile-close-variables-subset
+ nil nil
+ (byte-compile-no-output form)
+ (while (memq (car-safe form) '(let let* progn save-excursion))
+ (while (consp (cdr form))
+ (setq form (cdr form)))
+ (setq form (car form)))
+ (if (or (not (eq (car-safe form) 'list))
+ ;; For code using lexical-binding, form is not
+ ;; valid lisp, but rather an intermediate form
+ ;; which may include "calls" to
+ ;; internal-make-closure (Bug#29988).
+ lexical-binding)
+ (progn
+ (setq newform (byte-compile-out-toplevel nil nil))
+ (setq int `(,(car int) ,newform)))
+ (setq int (byte-run-strip-symbol-positions-copy
+ int))))))
((cdr int) ; Invalid (interactive . something).
(byte-compile-warn-x int "malformed interactive spec: %s"
int))))
@@ -3207,7 +3277,9 @@ byte-compile-constants-vector
;; The rest of the constants and variables need 3-byte byte-codes.
(let* ((i (1- byte-compile-reserved-constants))
(rest (nreverse byte-compile-variables)) ; nreverse because the first
- (other (nreverse byte-compile-constants)) ; vars often are used most.
+ (other (nreverse
+ (byte-run-strip-symbol-positions
+ byte-compile-constants))) ; vars often are used most.
ret tmp
(limits '(5 ; Use the 1-byte varref codes,
63 ; 1-constlim ; 1-byte byte-constant codes,
@@ -3232,24 +3304,21 @@ byte-compile-constants-vector
(setq other rest))))
(apply 'vector (nreverse (mapcar 'car ret)))))
-;; Given an expression FORM, compile it and return an equivalent byte-code
-;; expression (a call to the function byte-code).
-(defun byte-compile-top-level (form &optional for-effect output-type
- lexenv reserved-csts)
- ;; OUTPUT-TYPE advises about how form is expected to be used:
- ;; 'eval or nil -> a single form,
- ;; 'lambda -> body of a lambda,
- ;; 'file -> used at file-level.
- (let ((byte-compile--for-effect for-effect)
- (byte-compile-constants nil)
- (byte-compile-variables nil)
- (byte-compile-tag-number 0)
- (byte-compile-depth 0)
- (byte-compile-maxdepth 0)
- (byte-compile--lexical-environment lexenv)
- (byte-compile-reserved-constants (or reserved-csts 0))
- (byte-compile-output nil)
- (byte-compile-jump-tables nil))
+(defun byte-compile-no-output (form &optional output-type lexenv)
+ "Do the first stage of compilation of FORM.
+FORM is already macro expanded.
+
+The results are left in the variables bound in
+`byte-compile-close-variables-subset'.
+
+OUTPUT-TYPE advises about how form is expected to be used:
+ \\='eval or nil -> a single form,
+ \\='lambda -> body of a lambda,
+ \\='file -> used at file-level.
+
+This function does not change FORM. In particular, it doesn't strip
+positions from the symbols with position in FORM."
+ (let ((byte-compile--lexical-environment lexenv))
(if (memq byte-optimize '(t source))
(setq form (byte-optimize-one-form form byte-compile--for-effect)))
(while (and (eq (car-safe form) 'progn) (null (cdr (cdr form))))
@@ -3265,7 +3334,18 @@ byte-compile-top-level
(byte-compile-out-tag (byte-compile-make-tag))))
;; Now compile FORM
(byte-compile-form form byte-compile--for-effect)
- (byte-compile-out-toplevel byte-compile--for-effect output-type)))
+ byte-compile--for-effect))
+
+;; Given an expression FORM, compile it and return an equivalent byte-code
+;; expression (a call to the function byte-code).
+(defun byte-compile-top-level (form &optional for-effect output-type
+ lexenv reserved-csts)
+ "Compile (macro expanded) expression FORM to a byte-code expression.
+OUTPUT-TYPE is as described in `byte-compile-no-output'."
+
+ (byte-compile-close-variables-subset for-effect reserved-csts
+ (setq for-effect (byte-compile-no-output form output-type lexenv))
+ (byte-compile-out-toplevel for-effect output-type)))
(defun byte-compile-out-toplevel (&optional for-effect output-type)
;; OUTPUT-TYPE can be like that of `byte-compile-top-level'.
@@ -3380,7 +3460,7 @@ byte-compile-macroexpand-declare-function
(setq byte-compile-noruntime-functions
(delq fn byte-compile-noruntime-functions))
;; Delegate the rest to the normal macro definition.
- (let ((print-symbols-bare t)) ; Possibly redundant binding.
+ (let ((print-symbols-bare t)) ; Possibly redundant binding.
(macroexpand `(declare-function ,fn ,file ,@args))))
@@ -4202,7 +4282,7 @@ byte-compile-make-closure
(aref fun 3) ; Stack depth of function
(if docstring-exp
(cons
- (eval (byte-run-strip-symbol-positions
+ (eval (byte-run-strip-symbol-positions-copy
docstring-exp)
t)
(cdr opt-args)) ; The interactive spec will
@@ -4217,7 +4297,7 @@ byte-compile-make-closure
,(aref fun 1) ; The byte-code.
(vconcat (vector . ,env) ,(aref fun 2)) ; constant vector
,(aref fun 3) ; max stack depth
- ,(byte-run-strip-symbol-positions docstring-exp)
+ ,(byte-run-strip-symbol-positions-copy docstring-exp)
;; optional interactive spec and anything else, all quoted
,@(mapcar (lambda (x) `',x) (drop 5 (append fun nil)))))))))
@@ -5569,8 +5649,10 @@ byte-compile-out
"Push the operation onto `byte-compile-output'.
OP is an opcode, a symbol. OPERAND is either nil or a number or
a one-element list of a Lisp form."
- (when (and (consp operand) (null (cdr operand)))
- (setq operand (byte-run-strip-symbol-positions operand)))
+ (when (and (consp operand) (null (cdr operand))
+ ;; We've got a constant contained in `byte-compile-constants'.
+ (symbolp (car operand)))
+ (setcar operand (bare-symbol (car operand))))
(push (cons op operand) byte-compile-output)
(if (eq op 'byte-return)
;; This is actually an unnecessary case, because there should be no
@@ -5585,9 +5667,10 @@ byte-compile-out
;;; call tree stuff
(defun byte-compile-annotate-call-tree (form)
- (let ((current-form (byte-run-strip-symbol-positions
+ (let ((current-form (byte-run-strip-symbol-positions-copy
byte-compile-current-form))
- (bare-car-form (byte-run-strip-symbol-positions (car form)))
+ (bare-car-form (byte-run-strip-symbol-positions-copy
+ (car form)))
entry)
;; annotate the current call
(if (setq entry (assq bare-car-form byte-compile-call-tree))
diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index 63d85623fbe..ba762ca9619 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -2040,6 +2040,7 @@ cl-progv
(defconst cl--labels-magic (make-symbol "cl--labels-magic"))
(defvar cl--labels-convert-cache nil)
+(byte-compile-never-strip 'cl--labels-convert-cache)
(defun cl--labels-convert (f)
"Special macro-expander to rename (function F) references in `cl-labels'."
diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el
index 2bf9e0eb451..d3f77b29682 100644
--- a/lisp/emacs-lisp/macroexp.el
+++ b/lisp/emacs-lisp/macroexp.el
@@ -224,7 +224,8 @@ macroexpand-1
(env-expander (assq head environment)))
(if env-expander
(if (cdr env-expander)
- (apply (cdr env-expander) (cdr form))
+ (let ((byte-compile-macroing t))
+ (apply (cdr env-expander) (cdr form)))
form)
(if (not (and (symbolp head) (fboundp head)))
form
@@ -236,7 +237,8 @@ macroexpand-1
((not (consp def)) form)
(t
(if (eq 'macro (car def))
- (apply (cdr def) (cdr form))
+ (let ((byte-compile-macroing t))
+ (apply (cdr def) (cdr form)))
form))))))))
(t form)))
@@ -319,7 +321,8 @@ macroexp-preserve-posification
(defun macroexp-macroexpand (form env)
"Like `macroexpand' but checking obsolescence."
- (let* ((macroexpand-all-environment env)
+ (let* ((byte-compile-macroing t)
+ (macroexpand-all-environment env)
new-form)
(macroexp-preserve-posification
form
@@ -606,7 +609,8 @@ macroexpand-all
If no macros are expanded, FORM is returned unchanged.
The second optional arg ENVIRONMENT specifies an environment of macro
definitions to shadow the loaded ones for use in file byte-compilation."
- (let ((macroexpand-all-environment environment)
+ (let ((byte-compile-macroing t)
+ (macroexpand-all-environment environment)
(macroexp--dynvars macroexp--dynvars))
(macroexp--expand-all form)))
@@ -614,7 +618,8 @@ macroexpand-all
;; forms. It does not dynbind `macroexp--dynvars' because we want
;; top-level `defvar' declarations to be recorded in that variable.
(defun macroexpand--all-toplevel (form &optional environment)
- (let ((macroexpand-all-environment environment))
+ (let ((byte-compile-macroing t)
+ (macroexpand-all-environment environment))
(macroexp--expand-all form)))
;;; Handy functions to use in macros.
@@ -920,7 +925,7 @@ internal-macroexpand-for-load
;; but in case it does happen, let's catch the error and give the
;; code a chance to macro-expand later.
(error "Eager macro-expansion failure: %S" err)
- form)))))
+ form)))))
;; ¡¡¡ Big Ugly Hack !!!
;; src/bootstrap-emacs is mostly used to compile .el files, so it needs
diff --git a/lisp/obsolete/cl.el b/lisp/obsolete/cl.el
index 8a7e54a981c..38e85b28e34 100644
--- a/lisp/obsolete/cl.el
+++ b/lisp/obsolete/cl.el
@@ -302,7 +302,10 @@ cl--pass-args-to-cl-declare
;; it's not 100% compatible.
(defvar cl-closure-vars nil)
+(byte-compile-never-strip 'cl-closure-vars)
(defvar cl--function-convert-cache nil)
+(byte-compile-never-strip 'cl--function-convert-cache)
+
(defun cl--function-convert (f)
"Special macro-expander for special cases of (function F).
diff --git a/src/alloc.c b/src/alloc.c
index 4ed76d9d368..9696cde3f2a 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -3670,6 +3670,7 @@ init_symbol (Lisp_Object val, Lisp_Object name)
p->u.s.interned = SYMBOL_UNINTERNED;
p->u.s.trapped_write = SYMBOL_UNTRAPPED_WRITE;
p->u.s.declared_special = false;
+ p->u.s.dont_strip = false;
}
DEFUN ("make-symbol", Fmake_symbol, Smake_symbol, 1, 1, 0,
diff --git a/src/buffer.c b/src/buffer.c
index e44b6daf587..fa53c15f906 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -1157,6 +1157,9 @@ reset_buffer_local_variables (struct buffer *b, int permanent_too)
bset_case_eqv_table (b, XCHAR_TABLE (Vascii_downcase_table)->extras[2]);
bset_invisibility_spec (b, Qt);
+ /* None of the local variables will be bound. */
+ b->local_dont_strip_set_list = Qnil;
+
/* Reset all (or most) per-buffer variables to their defaults. */
if (permanent_too == 1)
bset_local_var_alist (b, Qnil);
diff --git a/src/buffer.h b/src/buffer.h
index 302daac3e04..d1d82099b9e 100644
--- a/src/buffer.h
+++ b/src/buffer.h
@@ -361,6 +361,10 @@ #define BVAR(buf, field) ((buf)->field ## _)
symbols, just the symbol appears as the element. */
Lisp_Object local_var_alist_;
+ /* List of symbols which have a buffer local binding, and that
+ binding has a DONT_STRIP value of 1. */
+ Lisp_Object local_dont_strip_set_list;
+
/* Symbol naming major mode (e.g., lisp-mode). */
Lisp_Object major_mode_;
@@ -1168,6 +1172,59 @@ BUFFER_CHECK_INDIRECTION (struct buffer *b)
}
}
+/* Routines to get the value of DONT_STRIP for the buffer local binding
+ of a symbol, to set this value to true and to clear it to false. BUF
+ can be Qnil, meaning use the buffer local variable's default
+ binding's value. This is invariably at SYM->u.s.dont_strip. */
+INLINE bool
+BUF_GET_DONT_STRIP (Lisp_Object buf, Lisp_Object symbol)
+{
+ if (NILP (buf))
+ return XSYMBOL (symbol)->u.s.dont_strip;
+ /* There may be no point to the next check. If we have a non-buffer,
+ Emacs cannot continue without inconsistencies. Letting it
+ terminate with a seg fault might be the better option. ACM,
+ 2025-10-18. */
+ CHECK_BUFFER (buf);
+ return !NILP (memq_no_quit (symbol,
+ XBUFFER (buf)->local_dont_strip_set_list));
+}
+
+INLINE void
+BUF__SET_DONT_STRIP (struct buffer *b, Lisp_Object symbol)
+{
+ eassert (BARE_SYMBOL_P (symbol));
+
+ if (NILP (memq_no_quit (symbol, b->local_dont_strip_set_list)))
+ b->local_dont_strip_set_list =
+ Fcons (symbol, b->local_dont_strip_set_list);
+}
+
+INLINE void
+BUF__CLEAR_DONT_STRIP (struct buffer *b, Lisp_Object symbol)
+{
+ b->local_dont_strip_set_list
+ = delq_no_quit (symbol, b->local_dont_strip_set_list);
+}
+
+INLINE void
+BUF_PUT_DONT_STRIP (Lisp_Object buf, Lisp_Object symbol, bool dont_strip)
+{
+ struct buffer *b;
+
+ if (NILP (buf))
+ {
+ XSYMBOL (symbol)->u.s.dont_strip = dont_strip;
+ return;
+ }
+ CHECK_BUFFER (buf);
+ b = XBUFFER (buf);
+ if (dont_strip)
+ BUF__SET_DONT_STRIP (b, symbol);
+ else
+ BUF__CLEAR_DONT_STRIP (b, symbol);
+}
+
#ifdef HAVE_TREE_SITTER
INLINE struct ts_linecol
diff --git a/src/bytecode.c b/src/bytecode.c
index e3e36d9b315..50c91e60208 100644
--- a/src/bytecode.c
+++ b/src/bytecode.c
@@ -711,7 +711,10 @@ #define DEFINE(name, value) [name] = &&insn_ ## name,
Lisp_Object sym = vectorp[arg];
Lisp_Object val = POP;
if (XBARE_SYMBOL (sym)->u.s.redirect == SYMBOL_PLAINVAL
- && !XBARE_SYMBOL (sym)->u.s.trapped_write)
+ && !XBARE_SYMBOL (sym)->u.s.trapped_write
+ && (!byte_compile_macroing
+ || NILP (Vbyte_compile_current_buffer)
+ || XBARE_SYMBOL (sym)->u.s.dont_strip))
SET_SYMBOL_VAL (XBARE_SYMBOL (sym), val);
else
set_internal (sym, val, Qnil, SET_INTERNAL_SET);
diff --git a/src/data.c b/src/data.c
index 333d908aedc..f8e29a8f1f1 100644
--- a/src/data.c
+++ b/src/data.c
@@ -831,6 +831,26 @@ DEFUN ("symbol-name", Fsymbol_name, Ssymbol_name, 1, 1, 0,
return name;
}
+DEFUN ("symbol-innards", Fsymbol_innards, Ssymbol_innards, 1, 1, 0,
+ doc: /* Return the invisible components of SYMBOL, a symbol.
+These are, in order: .redirect, .trapped_write, .interned (each of these
+being converted to a small integer), .declared_special, .dont_strip (these
+being converted to nil or t). */)
+ (Lisp_Object symbol)
+{
+ struct Lisp_Symbol *sym;
+
+ CHECK_SYMBOL (symbol);
+ sym = XSYMBOL (symbol);
+ return Flist (5,
+ (Lisp_Object [])
+ {make_fixnum (sym->u.s.redirect),
+ make_fixnum (sym->u.s.trapped_write),
+ make_fixnum (sym->u.s.interned),
+ sym->u.s.declared_special ? Qt : Qnil,
+ find_dont_strip (sym) ? Qt : Qnil});
+}
+
DEFUN ("bare-symbol", Fbare_symbol, Sbare_symbol, 1, 1, 0,
doc: /* Extract, if need be, the bare symbol from SYM.
SYM must be either a symbol or a symbol with position, otherwise
@@ -894,6 +914,8 @@ DEFUN ("fset", Ffset, Sfset, 2, 2, 0,
signal a `cyclic-function-indirection' error. */)
(register Lisp_Object symbol, Lisp_Object definition)
{
+ Lisp_Object def_1 = definition;
+
CHECK_SYMBOL (symbol);
/* Perhaps not quite the right error signal, but seems good enough. */
if (NILP (symbol) && !NILP (definition))
@@ -919,7 +941,12 @@ DEFUN ("fset", Ffset, Sfset, 2, 2, 0,
calln (Qcomp_subr_trampoline_install, symbol);
#endif
- set_symbol_function (symbol, definition);
+ if (byte_compile_macroing
+ && !NILP (Vbyte_compile_current_buffer)
+ && Ffboundp (Qbyte_run_strip_symbol_positions_copy))
+ def_1 = calln (Qbyte_run_strip_symbol_positions_copy, def_1);
+
+ set_symbol_function (symbol, def_1);
return definition;
}
@@ -1549,18 +1576,16 @@ swap_in_symval_forwarding (struct Lisp_Symbol *symbol, struct Lisp_Buffer_Local_
if (NILP (tem1)
|| current_buffer != XBUFFER (tem1))
{
+ Lisp_Object var;
+ XSETSYMBOL (var, symbol);
/* Unload the previously loaded binding. */
- tem1 = blv->valcell;
if (blv->fwd.fwdptr)
- set_blv_value (blv, do_symval_forwarding (blv->fwd));
+ set_blv_value (blv, do_symval_forwarding (blv->fwd));
+
/* Choose the new binding. */
- {
- Lisp_Object var;
- XSETSYMBOL (var, symbol);
- tem1 = assq_no_quit (var, BVAR (current_buffer, local_var_alist));
- set_blv_where (blv, Fcurrent_buffer ());
- }
+ tem1 = assq_no_quit (var, BVAR (current_buffer, local_var_alist));
+ set_blv_where (blv, Fcurrent_buffer ());
if (!(blv->found = !NILP (tem1)))
tem1 = blv->defcell;
@@ -1571,6 +1596,101 @@ swap_in_symval_forwarding (struct Lisp_Symbol *symbol, struct Lisp_Buffer_Local_
}
}
+/* Find the value of the flag symbol_dont_strip for SYMBOL's
+ current binding. */
+bool
+find_dont_strip (struct Lisp_Symbol *sym)
+{
+ Lisp_Object symbol;
+ struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
+
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+ XSETSYMBOL (symbol, sym);
+ switch (sym->u.s.redirect)
+ {
+ case SYMBOL_PLAINVAL:
+ return sym->u.s.dont_strip;
+ case SYMBOL_LOCALIZED:
+ return BUF_GET_DONT_STRIP (blv->where, symbol);
+ case SYMBOL_FORWARDED:
+ {
+ lispfwd valcontents = sym->u.s.val.fwd;
+ if (XFWDTYPE (valcontents) == Lisp_Fwd_Buffer_Obj)
+ return BUF_GET_DONT_STRIP (Fcurrent_buffer (), symbol);
+ else if (XFWDTYPE (valcontents) == Lisp_Fwd_Kboard_Obj)
+ {
+ KBOARD *kbd = kboard_for_bindings ();
+ return !NILP (memq_no_quit (symbol,
+ kbd->local_dont_strip_set_list));
+ }
+ else
+ return sym->u.s.dont_strip;
+ }
+ default: emacs_abort ();
+ }
+}
+
+DEFUN ("find-dont-strip", Ffind_dont_strip, Sfind_dont_strip, 1, 1, 0,
+ doc: /* Call the C function find_dont_strip with SYMBOL, returning nil or t. */)
+ (Lisp_Object symbol)
+{
+ struct Lisp_Symbol *sym;
+
+ CHECK_SYMBOL (symbol);
+ sym = XSYMBOL (symbol);
+ return find_dont_strip (sym) ? Qt : Qnil;
+}
+
+/* Set the value of the appropriate field `dont_strip' to FLAG for
+ SYMBOL. */
+void
+set_dont_strip (struct Lisp_Symbol *sym, bool flag)
+{
+ Lisp_Object symbol;
+
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+ XSETSYMBOL (symbol, sym);
+
+ switch (sym->u.s.redirect)
+ {
+ case SYMBOL_PLAINVAL:
+ sym->u.s.dont_strip = flag;
+ break;
+ case SYMBOL_LOCALIZED:
+ {
+ struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
+ BUF_PUT_DONT_STRIP (blv->where, symbol, flag);
+ break;
+ }
+ case SYMBOL_FORWARDED:
+ {
+ lispfwd valcontents = sym->u.s.val.fwd;
+ if (XFWDTYPE (valcontents) == Lisp_Fwd_Buffer_Obj)
+ BUF_PUT_DONT_STRIP (Fcurrent_buffer (), symbol, flag);
+ else if (XFWDTYPE (valcontents) == Lisp_Fwd_Kboard_Obj)
+ {
+ KBOARD *kbd = kboard_for_bindings ();
+ if (flag)
+ {
+ if (NILP (memq_no_quit (symbol,
+ kbd->local_dont_strip_set_list)))
+ kbd->local_dont_strip_set_list =
+ Fcons (symbol, kbd->local_dont_strip_set_list);
+ }
+ else
+ kbd->local_dont_strip_set_list =
+ delq_no_quit (symbol, kbd->local_dont_strip_set_list);
+ }
+ else
+ sym->u.s.dont_strip = flag;
+ break;
+ }
+ default: emacs_abort ();
+ }
+}
+
/* Find the value of a symbol, returning Qunbound if it's not bound.
This is helpful for code which just wants to get a variable's value
if it has one, without signaling an error.
@@ -1632,6 +1752,53 @@ DEFUN ("set", Fset, Sset, 2, 2, 0,
return newval;
}
+DEFUN ("byte-compile-never-strip", Fbyte_compile_never_strip,
+ Sbyte_compile_never_strip, 1, 1, 0,
+ doc: /* Mark SYMBOL never to have values stripped before assignment.
+
+Normally, before setting a symbol which hasn't been let-bound to a value,
+when expanding a macro during byte or native compilation, any symbols with
+position in the value are first stripped of their positions.
+
+This prevents the proper working of some variables used in byte compilation.
+After calling this function on these variables' symbols, this stripping is not
+done. It should be invoked immediately after a variable's `defvar'. It
+won't work reliably on a variable which has already been let-bound or is being
+used as a function's argument.
+
+The result of this function is SYMBOL. It should be used only for variables
+used in byte compilation. */)
+ (Lisp_Object symbol)
+{
+ CHECK_SYMBOL (symbol);
+ struct Lisp_Symbol *sym = XSYMBOL (symbol);
+
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+ sym->u.s.dont_strip = true;
+ return symbol;
+}
+
+/* If we are evaluating a macro in a byte/native compilation, SYM's
+ current binding is its default, and BINDFLAG is SET_INTERNAL_SET
+ (i.e. for a setq etc., rather than a let binding), strip the
+ positions from symbols with position in NEWVAL, making a copy of it.
+ Return the possibly changed NEWVAL. */
+
+static Lisp_Object
+maybe_strip_symbol_positions (struct Lisp_Symbol *sym,
+ Lisp_Object newval,
+ enum Set_Internal_Bind bindflag)
+{
+ if (byte_compile_macroing
+ && bindflag == SET_INTERNAL_SET
+ && !NILP (Vbyte_compile_current_buffer)
+ && Ffboundp (Qbyte_run_strip_symbol_positions_copy)
+ && !find_dont_strip (sym))
+ newval = calln (Qbyte_run_strip_symbol_positions_copy, newval);
+ return newval;
+}
+
/* Store the value NEWVAL into SYMBOL.
If buffer-locality is an issue, WHERE specifies which context to use.
(nil stands for the current buffer/frame).
@@ -1645,12 +1812,19 @@ DEFUN ("set", Fset, Sset, 2, 2, 0,
set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
enum Set_Internal_Bind bindflag)
{
+ struct Lisp_Symbol *sym;
bool unbinding_p = BASE_EQ (newval, Qunbound);
/* If restoring in a dead buffer, do nothing. */
CHECK_SYMBOL (symbol);
- struct Lisp_Symbol *sym = XSYMBOL (symbol);
+ sym = XSYMBOL (symbol);
+
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+
+ newval = maybe_strip_symbol_positions (sym, newval, bindflag);
+
switch (sym->u.s.trapped_write)
{
case SYMBOL_NOWRITE:
@@ -1680,10 +1854,8 @@ set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
default: emacs_abort ();
}
- start:
switch (sym->u.s.redirect)
{
- case SYMBOL_VARALIAS: sym = SYMBOL_ALIAS (sym); goto start;
case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
case SYMBOL_LOCALIZED:
{
@@ -1711,8 +1883,13 @@ set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
if (blv->fwd.fwdptr)
set_blv_value (blv, do_symval_forwarding (blv->fwd));
- /* Find the new binding. */
XSETSYMBOL (symbol, sym); /* May have changed via aliasing. */
+
+ /* The new value of DONT_STRIP is that of the variable's
+ default, not that of the old binding. */
+ BUF_PUT_DONT_STRIP (blv->where, symbol, sym->u.s.dont_strip);
+
+ /* Find the new binding. */
Lisp_Object tem1
= assq_no_quit (symbol,
BVAR (XBUFFER (where), local_var_alist));
@@ -2025,15 +2202,17 @@ set_default_internal (Lisp_Object symbol, Lisp_Object value,
default: emacs_abort ();
}
- start:
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
switch (sym->u.s.redirect)
{
- case SYMBOL_VARALIAS: sym = SYMBOL_ALIAS (sym); goto start;
case SYMBOL_PLAINVAL: set_internal (symbol, value, Qnil, bindflag); return;
case SYMBOL_LOCALIZED:
{
struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
+ value = maybe_strip_symbol_positions (sym, value, bindflag);
+
/* Store new value into the DEFAULT-VALUE slot. */
XSETCDR (blv->defcell, value);
@@ -2054,6 +2233,8 @@ set_default_internal (Lisp_Object symbol, Lisp_Object value,
int offset = XBUFFER_OBJFWD (valcontents)->offset;
int idx = PER_BUFFER_IDX (offset);
+ value = maybe_strip_symbol_positions (sym, value, bindflag);
+
set_per_buffer_default (offset, value);
/* If this variable is not always local in all buffers,
@@ -2062,6 +2243,8 @@ set_default_internal (Lisp_Object symbol, Lisp_Object value,
{
Lisp_Object buf, tail;
+ value = maybe_strip_symbol_positions (sym, value, bindflag);
+
/* Do this only in live buffers, so that if there are
a lot of buffers which are dead, that doesn't slow
down let-binding of variables that are
@@ -2082,6 +2265,7 @@ set_default_internal (Lisp_Object symbol, Lisp_Object value,
char *base = (char *) (where ? where
: kboard_for_bindings ());
char *p = base + XKBOARD_OBJFWD (valcontents)->offset;
+ maybe_strip_symbol_positions (sym, value, bindflag);
*(Lisp_Object *) p = value;
}
else
@@ -2166,16 +2350,21 @@ DEFUN ("make-variable-buffer-local", Fmake_variable_buffer_local,
struct Lisp_Buffer_Local_Value *blv = NULL;
union Lisp_Val_Fwd valcontents UNINIT;
bool forwarded UNINIT;
+ Lisp_Object variable_a;
CHECK_SYMBOL (variable);
sym = XSYMBOL (variable);
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+ XSETSYMBOL (variable_a, sym);
- start:
switch (sym->u.s.redirect)
{
- case SYMBOL_VARALIAS: sym = SYMBOL_ALIAS (sym); goto start;
case SYMBOL_PLAINVAL:
forwarded = 0; valcontents.value = SYMBOL_VAL (sym);
+ /* NEW STOUGH, 2025-10-04. Correct, 2025-10-16, OLD STOUGH, 2025-10-16 */
+ /* dont_strip = sym->u.s.dont_strip; */
+ /* END OF NEW STOUGH */
if (BASE_EQ (valcontents.value, Qunbound))
valcontents.value = Qnil;
break;
@@ -2193,7 +2382,7 @@ DEFUN ("make-variable-buffer-local", Fmake_variable_buffer_local,
default: emacs_abort ();
}
- if (SYMBOL_CONSTANT_P (variable))
+ if (SYMBOL_CONSTANT_P (variable_a))
xsignal1 (Qsetting_constant, variable);
if (!blv)
@@ -2201,6 +2390,7 @@ DEFUN ("make-variable-buffer-local", Fmake_variable_buffer_local,
blv = make_blv (sym, forwarded, valcontents);
sym->u.s.redirect = SYMBOL_LOCALIZED;
SET_SYMBOL_BLV (sym, blv);
+ BUF_PUT_DONT_STRIP (blv->where, variable_a, sym->u.s.dont_strip);
}
blv->local_if_set = 1;
@@ -2230,6 +2420,7 @@ DEFUN ("make-local-variable", Fmake_local_variable, Smake_local_variable,
(Lisp_Object variable)
{
Lisp_Object tem;
+ Lisp_Object variable_a;
bool forwarded UNINIT;
union Lisp_Val_Fwd valcontents UNINIT;
struct Lisp_Symbol *sym;
@@ -2237,13 +2428,14 @@ DEFUN ("make-local-variable", Fmake_local_variable, Smake_local_variable,
CHECK_SYMBOL (variable);
sym = XSYMBOL (variable);
-
- start:
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+
switch (sym->u.s.redirect)
{
- case SYMBOL_VARALIAS: sym = SYMBOL_ALIAS (sym); goto start;
case SYMBOL_PLAINVAL:
- forwarded = 0; valcontents.value = SYMBOL_VAL (sym); break;
+ forwarded = 0; valcontents.value = SYMBOL_VAL (sym);
+ break;
case SYMBOL_LOCALIZED:
blv = SYMBOL_BLV (sym);
break;
@@ -2259,6 +2451,8 @@ DEFUN ("make-local-variable", Fmake_local_variable, Smake_local_variable,
if (sym->u.s.trapped_write == SYMBOL_NOWRITE)
xsignal1 (Qsetting_constant, variable);
+ XSETSYMBOL (variable_a, sym); /* Update in case of aliasing. */
+ /* Make sure this buffer has its own value of symbol. */
if (!blv)
{
if (forwarded && BUFFER_OBJFWDP (valcontents.fwd))
@@ -2273,12 +2467,12 @@ DEFUN ("make-local-variable", Fmake_local_variable, Smake_local_variable,
}
blv = make_blv (sym, forwarded, valcontents);
sym->u.s.redirect = SYMBOL_LOCALIZED;
+ BUF_PUT_DONT_STRIP (Fcurrent_buffer (), variable_a,
+ sym->u.s.dont_strip);
SET_SYMBOL_BLV (sym, blv);
}
- /* Make sure this buffer has its own value of symbol. */
- XSETSYMBOL (variable, sym); /* Update in case of aliasing. */
- tem = assq_no_quit (variable, BVAR (current_buffer, local_var_alist));
+ tem = assq_no_quit (variable_a, BVAR (current_buffer, local_var_alist));
if (NILP (tem))
{
if (let_shadows_buffer_binding_p (sym))
@@ -2295,8 +2489,10 @@ DEFUN ("make-local-variable", Fmake_local_variable, Smake_local_variable,
bset_local_var_alist
(current_buffer,
- Fcons (Fcons (variable, XCDR (blv->defcell)),
+ Fcons (Fcons (variable_a, XCDR (blv->defcell)),
BVAR (current_buffer, local_var_alist)));
+ BUF_PUT_DONT_STRIP (Fcurrent_buffer (), variable_a,
+ sym->u.s.dont_strip);
/* If the symbol forwards into a C variable, then load the binding
for this buffer now, to preserve the invariant that forwarded
@@ -2319,11 +2515,16 @@ DEFUN ("kill-local-variable", Fkill_local_variable, Skill_local_variable,
(register Lisp_Object variable)
{
register Lisp_Object tem;
+ Lisp_Object variable_a;
struct Lisp_Buffer_Local_Value *blv;
struct Lisp_Symbol *sym;
CHECK_SYMBOL (variable);
sym = XSYMBOL (variable);
+ while (sym->u.s.redirect == SYMBOL_VARALIAS)
+ sym = SYMBOL_ALIAS (sym);
+ XSETSYMBOL (variable_a, sym); /* For an aliased variable. */
+
start:
switch (sym->u.s.redirect)
@@ -2343,6 +2544,8 @@ DEFUN ("kill-local-variable", Fkill_local_variable, Skill_local_variable,
SET_PER_BUFFER_VALUE_P (current_buffer, idx, 0);
set_per_buffer_value (current_buffer, offset,
per_buffer_default (offset));
+ BUF_PUT_DONT_STRIP (Fcurrent_buffer (), variable_a,
+ sym->u.s.dont_strip);
}
}
return variable;
@@ -2354,15 +2557,19 @@ DEFUN ("kill-local-variable", Fkill_local_variable, Skill_local_variable,
}
if (sym->u.s.trapped_write == SYMBOL_TRAPPED_WRITE)
- notify_variable_watchers (variable, Qnil, Qmakunbound, Fcurrent_buffer ());
+ notify_variable_watchers (variable_a, Qnil, Qmakunbound,
+ Fcurrent_buffer ());
/* Get rid of this buffer's alist element, if any. */
- XSETSYMBOL (variable, sym); /* Propagate variable indirection. */
- tem = assq_no_quit (variable, BVAR (current_buffer, local_var_alist));
+ tem = assq_no_quit (variable_a, BVAR (current_buffer, local_var_alist));
if (!NILP (tem))
- bset_local_var_alist
- (current_buffer,
- Fdelq (tem, BVAR (current_buffer, local_var_alist)));
+ {
+ bset_local_var_alist
+ (current_buffer,
+ Fdelq (tem, BVAR (current_buffer, local_var_alist)));
+ BUF_PUT_DONT_STRIP (Fcurrent_buffer (), variable_a,
+ sym->u.s.dont_strip);
+ }
/* If the symbol is set up with the current buffer's binding
loaded, recompute its value. We have to do it now, or else
@@ -2646,6 +2853,31 @@ DEFUN ("aset", Faset, Saset, 3, 3, 0,
}
return newelt;
}
+
+DEFUN ("closure-to-record", Fclosure_to_record, Sclosure_to_record,
+ 1, 1, 0, doc: /* Change the closure field in the vector ARG to record.
+Return changed ARG.
+For internal use only. */)
+ (Lisp_Object arg)
+{
+ CHECK_TYPE (CLOSUREP (arg), Qclosurep, arg);
+ XVECTOR (arg)->header.size &= ~PVEC_TYPE_MASK;
+ XSETPVECTYPE (XVECTOR (arg), PVEC_RECORD);
+ return arg;
+}
+
+DEFUN ("record-to-closure", Frecord_to_closure, Srecord_to_closure,
+ 1, 1, 0, doc: /* Change the record field in the vector ARG to closure.
+Return changed ARG.
+For internal use only. */)
+ (Lisp_Object arg)
+{
+ CHECK_TYPE (RECORDP (arg), Qrecordp, arg);
+ XVECTOR (arg)->header.size &= ~PVEC_TYPE_MASK;
+ XSETPVECTYPE (XVECTOR (arg), PVEC_CLOSURE);
+ return arg;
+}
+
/* Arithmetic functions */
@@ -4041,6 +4273,7 @@ syms_of_data (void)
DEFSYM (Qbufferp, "bufferp");
DEFSYM (Qvectorp, "vectorp");
DEFSYM (Qrecordp, "recordp");
+ DEFSYM (Qclosurep, "closurep");
DEFSYM (Qbool_vector_p, "bool-vector-p");
DEFSYM (Qchar_or_string_p, "char-or-string-p");
DEFSYM (Qmarkerp, "markerp");
@@ -4057,6 +4290,7 @@ syms_of_data (void)
DEFSYM (Qvector_or_char_table_p, "vector-or-char-table-p");
DEFSYM (Qfixnum_or_symbol_with_pos_p, "fixnum-or-symbol-with-pos-p");
DEFSYM (Qoclosure_interactive_form, "oclosure-interactive-form");
+ DEFSYM (Qbyte_run_strip_symbol_positions_copy, "byte-run-strip-symbol-positions-copy");
DEFSYM (Qsubrp, "subrp");
DEFSYM (Qunevalled, "unevalled");
@@ -4212,6 +4446,7 @@ #define PUT_ERROR(sym, tail, msg) \
DEFSYM (Qbyte_code_function_p, "byte-code-function-p");
defsubr (&Sindirect_variable);
+ defsubr (&Sfind_dont_strip);
defsubr (&Sinteractive_form);
defsubr (&Scommand_modes);
defsubr (&Seq);
@@ -4262,6 +4497,7 @@ #define PUT_ERROR(sym, tail, msg) \
defsubr (&Sindirect_function);
defsubr (&Ssymbol_plist);
defsubr (&Ssymbol_name);
+ defsubr (&Ssymbol_innards);
defsubr (&Sbare_symbol);
defsubr (&Ssymbol_with_pos_pos);
defsubr (&Sremove_pos_from_symbol);
@@ -4275,6 +4511,7 @@ #define PUT_ERROR(sym, tail, msg) \
defsubr (&Ssetplist);
defsubr (&Ssymbol_value);
defsubr (&Sset);
+ defsubr (&Sbyte_compile_never_strip);
defsubr (&Sdefault_boundp);
defsubr (&Sdefault_value);
defsubr (&Sset_default);
@@ -4286,6 +4523,8 @@ #define PUT_ERROR(sym, tail, msg) \
defsubr (&Svariable_binding_locus);
defsubr (&Saref);
defsubr (&Saset);
+ defsubr (&Sclosure_to_record);
+ defsubr (&Srecord_to_closure);
defsubr (&Snumber_to_string);
defsubr (&Sstring_to_number);
defsubr (&Seqlsign);
@@ -4352,6 +4591,17 @@ #define PUT_ERROR(sym, tail, msg) \
Bind this to non-nil in applications such as the byte compiler. */);
symbols_with_pos_enabled = false;
+ DEFSYM (Qbyte_compile_macroing, "byte-compile-macroing");
+ DEFVAR_BOOL ("byte-compile-macroing", byte_compile_macroing,
+ doc: /* If non-nil, the byte compiler is expanding a macro.
+This is used as a flag to strip positions from symbols with positions in
+certain circumstances before storing to certain types of binding. */);
+ byte_compile_macroing = false;
+
+ DEFVAR_LISP ("byte-compile-current-buffer", Vbyte_compile_current_buffer,
+ doc: /* The buffer of the source currently being compiled, or nil. */);
+ Vbyte_compile_current_buffer = Qnil;
+
DEFSYM (Qwatchers, "watchers");
DEFSYM (Qmakunbound, "makunbound");
DEFSYM (Qunlet, "unlet");
diff --git a/src/eval.c b/src/eval.c
index 0c29de9f3ad..89b8909393e 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -85,6 +85,13 @@ set_specpdl_old_value (union specbinding *pdl, Lisp_Object val)
pdl->let.old_value = val;
}
+static bool
+specpdl_old_dont_strip (union specbinding *pdl)
+{
+ eassert (pdl->kind >= SPECPDL_LET);
+ return pdl->let.dont_strip;
+}
+
static Lisp_Object
specpdl_where (union specbinding *pdl)
{
@@ -903,6 +910,8 @@ DEFUN ("internal--define-uninitialized-variable",
symbol);
XSYMBOL (symbol)->u.s.declared_special = true;
+ if (byte_compile_macroing)
+ XSYMBOL (symbol)->u.s.dont_strip = true;
if (!NILP (doc))
{
Fput (symbol, Qvariable_documentation, doc);
@@ -1270,6 +1279,9 @@ DEFUN ("macroexpand", Fmacroexpand, Smacroexpand, 1, 2, 0,
{
/* With cleanups from Hallvard Furuseth. */
register Lisp_Object expander, sym, def, tem;
+ specpdl_ref count = SPECPDL_INDEX ();
+
+ specbind (Qbyte_compile_macroing, Qt);
while (1)
{
@@ -1323,7 +1335,7 @@ DEFUN ("macroexpand", Fmacroexpand, Smacroexpand, 1, 2, 0,
form = newform;
}
}
- return form;
+ return unbind_to (count, form);
}
DEFUN ("catch", Fcatch, Scatch, 1, UNEVALLED, 0,
@@ -3401,7 +3413,8 @@ funcall_lambda (Lisp_Object fun, ptrdiff_t nargs, Lisp_Object *arg_vector)
/* Lexically bind NEXT by adding it to the lexenv alist. */
lexenv = Fcons (Fcons (next, arg), lexenv);
else
- /* Dynamically bind NEXT. */
+ /* Dynamically bind NEXT. Note: the argument's dont_strip
+ flag is set to t in the following specbind. */
specbind (next, arg);
previous_rest = false;
}
@@ -3577,6 +3590,7 @@ do_specbind (struct Lisp_Symbol *sym, union specbinding *bind,
switch (sym->u.s.redirect)
{
case SYMBOL_PLAINVAL:
+ sym->u.s.dont_strip = true;
if (!sym->u.s.trapped_write)
SET_SYMBOL_VAL (sym, value);
else
@@ -3587,12 +3601,14 @@ do_specbind (struct Lisp_Symbol *sym, union specbinding *bind,
if (BUFFER_OBJFWDP (SYMBOL_FWD (sym))
&& specpdl_kind (bind) == SPECPDL_LET_DEFAULT)
{
+ set_dont_strip (sym, true);
set_default_internal (specpdl_symbol (bind), value, bindflag,
NULL);
return;
}
FALLTHROUGH;
case SYMBOL_LOCALIZED:
+ set_dont_strip (sym, true);
set_internal (specpdl_symbol (bind), value, Qnil, bindflag);
break;
@@ -3631,6 +3647,7 @@ specbind (Lisp_Object symbol, Lisp_Object value)
specpdl_ptr->let.symbol = symbol;
specpdl_ptr->let.old_value = SYMBOL_VAL (sym);
specpdl_ptr->let.where.kbd = NULL;
+ specpdl_ptr->let.dont_strip = sym->u.s.dont_strip;
break;
case SYMBOL_LOCALIZED:
case SYMBOL_FORWARDED:
@@ -3640,6 +3657,7 @@ specbind (Lisp_Object symbol, Lisp_Object value)
specpdl_ptr->let.symbol = symbol;
specpdl_ptr->let.old_value = ovalue;
specpdl_ptr->let.where.buf = Fcurrent_buffer ();
+ specpdl_ptr->let.dont_strip = find_dont_strip (sym);
eassert (sym->u.s.redirect != SYMBOL_LOCALIZED
|| (BASE_EQ (SYMBOL_BLV (sym)->where, Fcurrent_buffer ())));
@@ -3812,6 +3830,8 @@ do_one_unbind (union specbinding *this_binding, bool unwinding,
Lisp_Object sym = specpdl_symbol (this_binding);
if (SYMBOLP (sym) && XSYMBOL (sym)->u.s.redirect == SYMBOL_PLAINVAL)
{
+ XSYMBOL (sym)->u.s.dont_strip
+ = specpdl_old_dont_strip (this_binding);
if (XSYMBOL (sym)->u.s.trapped_write == SYMBOL_UNTRAPPED_WRITE)
SET_SYMBOL_VAL (XSYMBOL (sym), specpdl_old_value (this_binding));
else
@@ -3825,21 +3845,27 @@ do_one_unbind (union specbinding *this_binding, bool unwinding,
kbdwhere = specpdl_kboard (this_binding);
FALLTHROUGH;
case SPECPDL_LET_DEFAULT:
- set_default_internal (specpdl_symbol (this_binding),
- specpdl_old_value (this_binding),
- bindflag, kbdwhere);
- break;
+ set_dont_strip (XSYMBOL (specpdl_symbol (this_binding)),
+ specpdl_old_dont_strip (this_binding));
+ set_default_internal (specpdl_symbol (this_binding),
+ specpdl_old_value (this_binding),
+ bindflag, kbdwhere);
+ break;
case SPECPDL_LET_LOCAL:
{
Lisp_Object symbol = specpdl_symbol (this_binding);
Lisp_Object where = specpdl_where (this_binding);
Lisp_Object old_value = specpdl_old_value (this_binding);
+ struct Lisp_Symbol *end = XSYMBOL (specpdl_symbol (this_binding));
eassert (BUFFERP (where));
/* If this was a local binding, reset the value in the appropriate
buffer, but only if that buffer's binding still exists. */
if (!NILP (Flocal_variable_p (symbol, where)))
- set_internal (symbol, old_value, where, bindflag);
+ {
+ set_dont_strip (end, specpdl_old_dont_strip (this_binding));
+ set_internal (symbol, old_value, where, bindflag);
+ }
}
break;
}
@@ -4144,8 +4170,11 @@ specpdl_unrewind (union specbinding *pdl, int distance, bool vars_only)
&& XSYMBOL (sym)->u.s.redirect == SYMBOL_PLAINVAL)
{
Lisp_Object old_value = specpdl_old_value (tmp);
+ bool old_dont_strip = specpdl_old_dont_strip (tmp);
set_specpdl_old_value (tmp, SYMBOL_VAL (XSYMBOL (sym)));
SET_SYMBOL_VAL (XSYMBOL (sym), old_value);
+ tmp->let.dont_strip = (XSYMBOL (sym))->u.s.dont_strip;
+ (XSYMBOL (sym))->u.s.dont_strip = old_dont_strip;
break;
}
}
@@ -4157,9 +4186,12 @@ specpdl_unrewind (union specbinding *pdl, int distance, bool vars_only)
{
Lisp_Object sym = specpdl_symbol (tmp);
Lisp_Object old_value = specpdl_old_value (tmp);
+ bool old_dont_strip = specpdl_old_dont_strip (tmp);
set_specpdl_old_value (tmp, default_value (sym));
set_default_internal (sym, old_value, SET_INTERNAL_THREAD_SWITCH,
kbdwhere);
+ tmp->let.dont_strip = find_dont_strip (XSYMBOL (sym));
+ set_dont_strip (XSYMBOL (sym), old_dont_strip);
}
break;
case SPECPDL_LET_LOCAL:
@@ -4167,6 +4199,7 @@ specpdl_unrewind (union specbinding *pdl, int distance, bool vars_only)
Lisp_Object symbol = specpdl_symbol (tmp);
Lisp_Object where = specpdl_where (tmp);
Lisp_Object old_value = specpdl_old_value (tmp);
+ bool old_dont_strip = specpdl_old_dont_strip (tmp);
eassert (BUFFERP (where));
/* If this was a local binding, reset the value in the appropriate
@@ -4177,6 +4210,8 @@ specpdl_unrewind (union specbinding *pdl, int distance, bool vars_only)
(tmp, buffer_local_value (symbol, where));
set_internal (symbol, old_value, where,
SET_INTERNAL_THREAD_SWITCH);
+ tmp->let.dont_strip = BUF_GET_DONT_STRIP (where, symbol);
+ BUF_PUT_DONT_STRIP (where, symbol, old_dont_strip);
}
else
/* If the var is not local any more, it can't be undone nor
@@ -4553,6 +4588,8 @@ syms_of_eval (void)
A value of `(t)' indicates an empty environment, otherwise it is an
alist of active lexical bindings. */);
Vinternal_interpreter_environment = Qnil;
+ /* We never want to strip symbols with position in this variable, so ... */
+ Fbyte_compile_never_strip (Qinternal_interpreter_environment);
/* Don't export this variable to Elisp, so no one can mess with it
(Just imagine if someone makes it buffer-local). */
Funintern (Qinternal_interpreter_environment, Qnil);
diff --git a/src/fns.c b/src/fns.c
index 5334c9f94a8..1f044718c1b 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -2105,6 +2105,27 @@ DEFUN ("delq", Fdelq, Sdelq, 2, 2, 0,
return list;
}
+Lisp_Object
+delq_no_quit (Lisp_Object elt, Lisp_Object list)
+{
+ Lisp_Object prev = Qnil, tail;
+
+ for (tail = list; CONSP (tail); tail = XCDR (tail))
+ {
+ Lisp_Object tem = XCAR (tail);
+ if (EQ (elt, tem))
+ {
+ if (NILP (prev))
+ list = XCDR (tail);
+ else
+ Fsetcdr (prev, XCDR (tail));
+ }
+ else
+ prev = tail;
+ }
+ return list;
+}
+
DEFUN ("delete", Fdelete, Sdelete, 2, 2, 0,
doc: /* Delete members of SEQ which are `equal' to ELT, and return the result.
SEQ must be a sequence (i.e. a list, a vector, or a string).
diff --git a/src/keyboard.c b/src/keyboard.c
index a9cbd107dde..fda177ab986 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -13054,6 +13054,7 @@ init_kboard (KBOARD *kb, Lisp_Object type)
kset_last_prefix_arg (kb, Qnil);
kset_kbd_queue (kb, Qnil);
kb->kbd_queue_has_data = false;
+ kb->local_dont_strip_set_list = Qnil;
kb->immediate_echo = false;
kset_echo_string (kb, Qnil);
kset_echo_prompt (kb, Qnil);
@@ -14521,6 +14522,7 @@ mark_kboards (void)
mark_object (KVAR (kb, Vlast_prefix_arg));
mark_object (KVAR (kb, kbd_queue));
mark_object (KVAR (kb, defining_kbd_macro));
+ mark_object (kb->local_dont_strip_set_list);
mark_object (KVAR (kb, Vlast_kbd_macro));
mark_object (KVAR (kb, Vsystem_key_alist));
mark_object (KVAR (kb, system_key_syms));
diff --git a/src/keyboard.h b/src/keyboard.h
index 5e04b54eb74..d5249261dac 100644
--- a/src/keyboard.h
+++ b/src/keyboard.h
@@ -120,6 +120,10 @@ #define KVAR(kboard, field) ((kboard)->field ## _)
/* Where to store the next keystroke of the macro. */
Lisp_Object *kbd_macro_ptr;
+ /* List of these variables whose bindings have aa `dont_strip'
+ value of true. */
+ Lisp_Object local_dont_strip_set_list;
+
/* The finalized section of the macro starts at kbd_macro_buffer and
ends before this. This is not the same as kbd_macro_ptr, because
we advance this to kbd_macro_ptr when a key's command is complete.
diff --git a/src/lisp.h b/src/lisp.h
index e7b15069f00..a436ae781f5 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -814,6 +814,12 @@ #define XUNTAG(a, type, ctype) \
special (with `defvar' etc), and shouldn't be lexically bound. */
bool_bf declared_special : 1;
+ /* True means that values will be written into the current
+ SYMBOL_PLAINVAL or SYMBOL_FORWARDED binding without stripping
+ positions from symbols with position in the value. Only
+ significant when `byte-compile-macroing' is non-nil. */
+ bool_bf dont_strip :1;
+
/* The symbol's name, as a Lisp string. */
Lisp_Object name;
@@ -3637,6 +3643,7 @@ #define DEFVAR_KBOARD(lname, vname, doc) \
ENUM_BF (specbind_tag) kind : CHAR_BIT;
/* `where' is not used in the case of SPECPDL_LET,
unless the symbol is forwarded to a KBOARD. */
+ bool_bf dont_strip : 1;
Lisp_Object symbol, old_value;
union {
/* KBOARD object to which SYMBOL forwards, in the case of
@@ -4187,6 +4194,8 @@ modiff_to_integer (modiff_count a)
extern void notify_variable_watchers (Lisp_Object, Lisp_Object,
Lisp_Object, Lisp_Object);
extern Lisp_Object indirect_function (Lisp_Object);
+extern bool find_dont_strip (struct Lisp_Symbol *);
+extern void set_dont_strip (struct Lisp_Symbol *, bool);
extern Lisp_Object find_symbol_value (Lisp_Object);
enum {
@@ -4306,6 +4315,7 @@ #define CONS_TO_INTEGER(cons, type, var) \
extern void syms_of_fns (void);
extern void mark_fns (void);
Lisp_Object memq_no_quit (Lisp_Object elt, Lisp_Object list);
+Lisp_Object delq_no_quit (Lisp_Object elt, Lisp_Object list);
/* Defined in sort.c */
extern void tim_sort (Lisp_Object, Lisp_Object, Lisp_Object *, const ptrdiff_t,
diff --git a/src/lread.c b/src/lread.c
index 1a667ce163a..ddc1eeb0158 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -1411,6 +1411,7 @@ DEFUN ("load", Fload, Sload, 1, 5, 0,
lread_close (fd);
clear_unwind_protect (fd_index);
}
+ specbind (Qbyte_compile_macroing, Qnil);
val = calln (Vload_source_file_function, found, hist_file_name,
NILP (noerror) ? Qnil : Qt,
(NILP (nomessage) || force_load_messages) ? Qnil : Qt);
@@ -1493,6 +1494,7 @@ DEFUN ("load", Fload, Sload, 1, 5, 0,
specbind (Qload_true_file_name, found);
specbind (Qinhibit_file_name_operation, Qnil);
specbind (Qload_in_progress, Qt);
+ specbind (Qbyte_compile_macroing, Qnil);
if (is_module)
{
@@ -3859,7 +3861,6 @@ read0 (source_t *source, bool locate_syms)
.u.vector.elems = Qnil,
.u.vector.old_locate_syms = locate_syms,
});
- locate_syms = false;
goto read_obj;
case '^':
diff --git a/src/pdumper.c b/src/pdumper.c
index 4fa9c917124..f188f027d50 100644
--- a/src/pdumper.c
+++ b/src/pdumper.c
@@ -2499,6 +2499,7 @@ dump_symbol (struct dump_context *ctx, Lisp_Object object,
DUMP_FIELD_COPY (&out, symbol, u.s.trapped_write);
DUMP_FIELD_COPY (&out, symbol, u.s.interned);
DUMP_FIELD_COPY (&out, symbol, u.s.declared_special);
+ DUMP_FIELD_COPY (&out, symbol, u.s.dont_strip);
dump_field_lv (ctx, &out, symbol, &symbol->u.s.name, WEIGHT_STRONG);
switch (symbol->u.s.redirect)
{
@@ -2819,6 +2820,7 @@ dump_buffer (struct dump_context *ctx, const struct buffer *in_buffer)
buffer->window_count = 0;
else
eassert (buffer->window_count == -1);
+ buffer->local_dont_strip_set_list = Qnil;
buffer->local_minor_modes_ = Qnil;
buffer->last_name_ = Qnil;
buffer->last_selected_window_ = Qnil;
diff --git a/test/lisp/emacs-lisp/bytecomp-tests.el b/test/lisp/emacs-lisp/bytecomp-tests.el
index 0245ad3c977..20a0625653c 100644
--- a/test/lisp/emacs-lisp/bytecomp-tests.el
+++ b/test/lisp/emacs-lisp/bytecomp-tests.el
@@ -970,7 +970,7 @@ bytecomp-tests--warnings
(defun my--test2 (arg) (+ arg 1)))))
(with-current-buffer (get-buffer-create "*Compile-Log*")
(goto-char (point-min))
- ;; Should warn that mt--test1[12] are first used as functions.
+ ;; Should warn that my--test1[12] are first used as functions.
;; The second alternative is for when the file name is so long
;; that pretty-printing starts the message on the next line.
(should (or (re-search-forward "my--test11:\n.*macro" nil t)
@@ -2195,34 +2195,36 @@ byte-compile-file/no-byte-compile
(should (eq (byte-compile-file src-file) 'no-byte-compile))
(should-not (file-exists-p dest-file))))
-(ert-deftest bytecomp--copy-tree ()
- (should (null (bytecomp--copy-tree nil)))
- (let ((print-circle t))
- (let* ((x '(1 2 (3 4)))
- (y (bytecomp--copy-tree x)))
- (should (equal (prin1-to-string (list x y))
- "((1 2 (3 4)) (1 2 (3 4)))")))
- (let* ((x '#1=(a #1#))
- (y (bytecomp--copy-tree x)))
- (should (equal (prin1-to-string (list x y))
- "(#1=(a #1#) #2=(a #2#))")))
- (let* ((x '#1=(#1# a))
- (y (bytecomp--copy-tree x)))
- (should (equal (prin1-to-string (list x y))
- "(#1=(#1# a) #2=(#2# a))")))
- (let* ((x '((a . #1=(b)) #1#))
- (y (bytecomp--copy-tree x)))
- (should (equal (prin1-to-string (list x y))
- "(((a . #1=(b)) #1#) ((a . #2=(b)) #2#))")))
- (let* ((x '#1=(a #2=(#1# b . #3=(#2# c . #1#)) (#3# d)))
- (y (bytecomp--copy-tree x)))
- (should (equal (prin1-to-string (list x y))
- (concat
- "("
- "#1=(a #2=(#1# b . #3=(#2# c . #1#)) (#3# d))"
- " "
- "#4=(a #5=(#4# b . #6=(#5# c . #4#)) (#6# d))"
- ")"))))))
+;; Commented out by ACM 2025-10-18 due to the removal of the function
+;; from bytecomp.el.
+;; (ert-deftest bytecomp--copy-tree ()
+;; (should (null (bytecomp--copy-tree nil)))
+;; (let ((print-circle t))
+;; (let* ((x '(1 2 (3 4)))
+;; (y (bytecomp--copy-tree x)))
+;; (should (equal (prin1-to-string (list x y))
+;; "((1 2 (3 4)) (1 2 (3 4)))")))
+;; (let* ((x '#1=(a #1#))
+;; (y (bytecomp--copy-tree x)))
+;; (should (equal (prin1-to-string (list x y))
+;; "(#1=(a #1#) #2=(a #2#))")))
+;; (let* ((x '#1=(#1# a))
+;; (y (bytecomp--copy-tree x)))
+;; (should (equal (prin1-to-string (list x y))
+;; "(#1=(#1# a) #2=(#2# a))")))
+;; (let* ((x '((a . #1=(b)) #1#))
+;; (y (bytecomp--copy-tree x)))
+;; (should (equal (prin1-to-string (list x y))
+;; "(((a . #1=(b)) #1#) ((a . #2=(b)) #2#))")))
+;; (let* ((x '#1=(a #2=(#1# b . #3=(#2# c . #1#)) (#3# d)))
+;; (y (bytecomp--copy-tree x)))
+;; (should (equal (prin1-to-string (list x y))
+;; (concat
+;; "("
+;; "#1=(a #2=(#1# b . #3=(#2# c . #1#)) (#3# d))"
+;; " "
+;; "#4=(a #5=(#4# b . #6=(#5# c . #4#)) (#6# d))"
+;; ")"))))))
(require 'backtrace)
--
Alan Mackenzie (Nuremberg, Germany).
Alan Mackenzie <acm@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#79678; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.