Received: (at 67455) by debbugs.gnu.org; 8 Apr 2024 12:00:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Apr 08 08:00:54 2024
Received: from localhost ([127.0.0.1]:45470 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rtngE-0008Cg-Gv
for submit <at> debbugs.gnu.org; Mon, 08 Apr 2024 08:00:54 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:49022)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rtng9-0008Bs-IK
for 67455 <at> debbugs.gnu.org; Mon, 08 Apr 2024 08:00:53 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 87ED280DB3;
Mon, 8 Apr 2024 08:00:36 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1712577635;
bh=ZqlJHyvdnF94obZeZkmQWkXHnPFV4UgS2xwNtCo9kHY=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=NXQ6V05t96TToRvci5lIIyg7K1c0SFVCaML9Xds0gYNS175EN0r2LJX2ESGrJv5DB
J9MAxjNDMuRBQNLQaxiW04HtKmPSrEDtXLISM7RyqRrcVyXzpAzo3IcNE8LfObZd/Y
zxyIg0EKQFbZsYKNLIJZWj5I+aaDL4PUaq58ZDcUhOUk1IdXUAQZ1I1XCVhzjmUEo5
lI307Mrg2V8kevp+33xHwy8vnCNpst4IDRCAv8fn9S6bPzKF+7a68uwQSLohxkCTSg
5eJNmZ4P949DfxE2pw9Jy6Bai/iq/1im/JSAC4CNtwHxVk7w+7cWbRQqiQtNr6LeZD
CBjdIRFFHM2Dg==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 9A66D80BDD;
Mon, 8 Apr 2024 08:00:35 -0400 (EDT)
Received: from pastel (unknown [45.72.201.215])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 711A812020E;
Mon, 8 Apr 2024 08:00:35 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZhOruaGJftPClb4h@ACM> (Alan Mackenzie's message of "Mon, 8 Apr
2024 08:32:57 +0000")
Message-ID: <jwvil0sdqnl.fsf-monnier+emacs@HIDDEN>
References: <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
<ZgfxinngV469zNSY@ACM> <jwvplvbta02.fsf-monnier+emacs@HIDDEN>
<ZhJ8Ltr3DuuAyFOD@ACM> <jwvsezwegsq.fsf-monnier+emacs@HIDDEN>
<ZhOruaGJftPClb4h@ACM>
Date: Mon, 08 Apr 2024 08:00:34 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.096 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> How? `macroexp--expand-all` will not be passed this `lambda` because
>> it's not an *expression*.
> Well, when I commented out that pcase arm, the lambda no longer got
> stripped. I'm not sure what you mean by expression.
In Edebug specs we call it `form`.
The ((x 3) (y 4)) in a `let` is not an expression, for example.
(4 . 5) could appear in a piece of code in the position where we expect
an expression but it is not a valid expression (it will result in a syntax
error at compile- or run-time).
The grammar of ELisp is made of various elements, one of which would
traditionally be called "expressions". If we could specify it in EBNF
it could look something like the following:
<formalargs> ::= "(" <id>* [ "&optional" <id>* [ "&rest" <id> ]] ")"
<decls> ::= "(" <decl>* ")"
<decl> ::= <id> | "(" <id> <exp> ")"
<func> ::= <id> | "(" "lambda" <formalargs> <exp>* ")"
...
<exp> ::= <id>
| "(" <func> <exp>* ")"
| "(" "quote" <sexp> ")"
| "(" "let" <decls> <exp>* ")"
| "(" "function" <func> ")"
| ...
Only the <exp> parts go through `macroexp--expand-all`.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 8 Apr 2024 08:33:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Apr 08 04:33:16 2024
Received: from localhost ([127.0.0.1]:45246 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rtkRI-0003me-D3
for submit <at> debbugs.gnu.org; Mon, 08 Apr 2024 04:33:16 -0400
Received: from mail.muc.de ([193.149.48.3]:26581)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rtkRF-0003mP-So
for 67455 <at> debbugs.gnu.org; Mon, 08 Apr 2024 04:33:14 -0400
Received: (qmail 24139 invoked by uid 3782); 8 Apr 2024 10:33:00 +0200
Received: from muc.de (p4fe15203.dip0.t-ipconnect.de [79.225.82.3]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Mon, 08 Apr 2024 10:32:59 +0200
Received: (qmail 6741 invoked by uid 1000); 8 Apr 2024 08:32:57 -0000
Date: Mon, 8 Apr 2024 08:32:57 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZhOruaGJftPClb4h@ACM>
References: <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
<ZgfxinngV469zNSY@ACM> <jwvplvbta02.fsf-monnier+emacs@HIDDEN>
<ZhJ8Ltr3DuuAyFOD@ACM> <jwvsezwegsq.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <jwvsezwegsq.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Sun, Apr 07, 2024 at 23:16:13 -0400, Stefan Monnier wrote:
> > Pretending the problem doesn't exist won't solve it. In the ;POS...
> > structures for a lambda, there are two pointers - one to the definition
> > of the lambda, the other to the point of use.
> Fancy. Could you give me an example where I see this in play?
> [ To help me understand also what you mean by "definition of the
> lambda" and "point of use"? ]
> I looked around but all I could see where position info like
> [foo foo.el 41 nil]
> which point to "the definition" of the function.
Apologies. I was thinking of my latest not yet committed version, where
I've added a fifth element into the position information, the buffer
containing the lambda. This should enable buttons to be set on the
interactive backtrace, pointing back at the two source code positions.
> >> BTW, AFAIK the above is conceptually what the byte-compiler does (exce=
pt
> >> it performs a few more transformations between `macroexp--expand-all`
> >> and `strip-all-symbol-positions`).
> > It is a bad idea to conflate these two radically different uses of SWPs.
> In what way are they radically different uses of SWPs?
You're confused in precisely the way I feared. "conceptually what the
byte-compiler does" is what it does to strip the SWPs used as WARNING
POSITIONS. When the SWPs are used for function position structures
(whether in interpreted or compiled code) the handling is radically
different - the P in the SWP is stripped as soon as possible after its
use. In the warning pos use, the positions are preserved for as long as
possible.
> >> Is it the case that `cl-defmethod` generates a function whose source
> >> position (partly) points to `generic.el:403` if `cl-generic.el` was
> >> interpreted but not if it compiled?
> > No, the intention is that the source positions are independent of wheth=
er
> > the code is compiled.
> Good. So why do the interpreted and compiled cases need to be
> "radically different uses of SWPs"?
They're not. It is the use for warning positions that differs from the
use for putting pos info into the doc string.
> >> > (defmacro foo (lambda bar)
> >> > `(cons ,lambda ,bar))
> >> > expands to
> >> > (macro closure (t) (lambda bar) ";POS^^^A^A^A [foo *scratch* 158 nil]
> >> > " (list 'cons lambda bar))
> >> IIUC your reader will make the `lambda` formal argument into an SWP.
> >> Where is that SWP stripped?
> > In macroexp--expand-all in the "guard arm" near the end.
> How? `macroexp--expand-all` will not be passed this `lambda` because
> it's not an *expression*.
Well, when I commented out that pcase arm, the lambda no longer got
stripped. I'm not sure what you mean by expression.
> `eval-buffer` of a buffer containing the above defmacro does:
> 1 -> (macroexp--expand-all (defalias 'foo (cons 'macro #'{foo} (lambda (#=
<symbol lambda at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" `(cons ,lambd=
a ,bar)))))
> | 2 -> (macroexp--expand-all 'foo)
> | 2 <- macroexp--expand-all: 'foo
> | 2 -> (macroexp--expand-all (cons 'macro #'{foo} (lambda (#<symbol lambd=
a at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" `(cons ,lambda ,bar))))
> | | 3 -> (macroexp--expand-all 'macro)
> | | 3 <- macroexp--expand-all: 'macro
> | | 3 -> (macroexp--expand-all #'{foo} (lambda (#<symbol lambda at 46> ba=
r) ";POS=01=01 [foo foo.el 41 nil]\n" `(cons ,lambda ,bar)))
> | | | 4 -> (macroexp--expand-all ";POS=01=01 [foo foo.el 41 nil]\n")
> | | | 4 <- macroexp--expand-all: ";POS=01=01 [foo foo.el 41 nil]\n"
> | | | 4 -> (macroexp--expand-all `(cons ,lambda ,bar))
> | | | | 5 -> (macroexp--expand-all 'cons)
> | | | | 5 <- macroexp--expand-all: 'cons
> | | | | 5 -> (macroexp--expand-all lambda)
> | | | | 5 <- macroexp--expand-all: lambda
> | | | | 5 -> (macroexp--expand-all bar)
> | | | | 5 <- macroexp--expand-all: bar
> | | | 4 <- macroexp--expand-all: (list 'cons lambda bar)
> | | 3 <- macroexp--expand-all: #'{foo} (lambda (#<symbol lambda at 46> ba=
r) ";POS=01=01 [foo foo.el 41 nil]\n" (list 'cons lambda bar))
> | 2 <- macroexp--expand-all: (cons 'macro #'{foo} (lambda (#<symbol lambd=
a at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" (list 'cons lambda bar)))
> 1 <- macroexp--expand-all: (defalias 'foo (cons 'macro #'{foo} (lambda (#=
<symbol lambda at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" (list 'cons l=
ambda bar))))
> So we see that indeed it returns code where the formal argument `lambda`
> is (incorrectly) a SYMPOS. Yet somehow the sympos is stripped after
> macroexpansion somewhere since `(symbol-function 'foo)` shows the
> resulting function doesn't have any symposes in it.
OK, this needs clearing up.
> >> > so it is clear this case is getting handled OK. I'm afraid I can't
> >> > point out the exact place in the code at the moment where this is
> >> > getting done.
> >> I think it would be good to know, so as to be able to decide whether
> >> it'll indeed always work right, or we just got lucky this time.
> > See above.
> Yes, please, see above =F0=9F=99=82
> >> Could you explain what you think makes it intrinsically complex?
> > The mass of detail that needs dealing with that Emacs has collected over
> > the decades. As a counter question, why do you think the exercise ought
> > to be simple (assuming you do think this)?
> Because you solved the hard part when you added the symposes for the comp=
iler.
OK, but that's not the way things turned out.
> Stefan
--=20
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 8 Apr 2024 03:16:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 07 23:16:31 2024
Received: from localhost ([127.0.0.1]:45081 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rtfUl-0005QF-3j
for submit <at> debbugs.gnu.org; Sun, 07 Apr 2024 23:16:31 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:21179)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rtfUj-0005Q3-Ml
for 67455 <at> debbugs.gnu.org; Sun, 07 Apr 2024 23:16:30 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 66FAB80BC1;
Sun, 7 Apr 2024 23:16:16 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1712546175;
bh=n9P42IekFT9nDOQ2MOiwDGjl4XWcWsBcpQCVEDk8kPA=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=nCVWbyh6YgDBumfexN0UzGCmpeAz6zt2JCGNVH/Jcm2lcHnpdULpyV0mqyNAhFQyZ
bE9n/S4JvmNTl51iL5br50FyFnnfPI8cGgUkHrsqd/KvtywtG5M9aV50D2ufbQz33g
DnOcOziIa7e4sbJr81lngXeDWF67Vu6H4xPbwOPTdZ7BJtwvinfAM4L0/cSeK5j0Zk
YDP5xO+YvfWkWI8wB0sVAexCOFbnrv0mSesfHCdefdQr0mxnzSkU7Hv8PdzZDkrb4X
/G+edehZrn+M1kYPouiUu6kUytIMyK2V7WDBF19dEoSZgHicu+BAHpzCRtTrRS9nXR
7rFr7BZHf3BRA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 36314803B1;
Sun, 7 Apr 2024 23:16:15 -0400 (EDT)
Received: from pastel (unknown [45.72.201.215])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 06A05120250;
Sun, 7 Apr 2024 23:16:14 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZhJ8Ltr3DuuAyFOD@ACM> (Alan Mackenzie's message of "Sun, 7 Apr
2024 10:57:50 +0000")
Message-ID: <jwvsezwegsq.fsf-monnier+emacs@HIDDEN>
References: <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
<ZgfxinngV469zNSY@ACM> <jwvplvbta02.fsf-monnier+emacs@HIDDEN>
<ZhJ8Ltr3DuuAyFOD@ACM>
Date: Sun, 07 Apr 2024 23:16:13 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.097 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Pretending the problem doesn't exist won't solve it. In the ;POS...
> structures for a lambda, there are two pointers - one to the definition
> of the lambda, the other to the point of use.
Fancy. Could you give me an example where I see this in play?
[ To help me understand also what you mean by "definition of the
lambda" and "point of use"? ]
I looked around but all I could see where position info like
[foo foo.el 41 nil]
which point to "the definition" of the function.
>> BTW, AFAIK the above is conceptually what the byte-compiler does (except
>> it performs a few more transformations between `macroexp--expand-all`
>> and `strip-all-symbol-positions`).
> It is a bad idea to conflate these two radically different uses of SWPs.
In what way are they radically different uses of SWPs?
>> Is it the case that `cl-defmethod` generates a function whose source
>> position (partly) points to `generic.el:403` if `cl-generic.el` was
>> interpreted but not if it compiled?
> No, the intention is that the source positions are independent of whether
> the code is compiled.
Good. So why do the interpreted and compiled cases need to be
"radically different uses of SWPs"?
>> > (defmacro foo (lambda bar)
>> > `(cons ,lambda ,bar))
>
>> > expands to
>
>> > (macro closure (t) (lambda bar) ";POS^^^A^A^A [foo *scratch* 158 nil]
>> > " (list 'cons lambda bar))
>
>> IIUC your reader will make the `lambda` formal argument into an SWP.
>> Where is that SWP stripped?
>
> In macroexp--expand-all in the "guard arm" near the end.
How? `macroexp--expand-all` will not be passed this `lambda` because
it's not an *expression*.
`eval-buffer` of a buffer containing the above defmacro does:
1 -> (macroexp--expand-all (defalias 'foo (cons 'macro #'{foo} (lambda (#<s=
ymbol lambda at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" `(cons ,lambda =
,bar)))))
| 2 -> (macroexp--expand-all 'foo)
| 2 <- macroexp--expand-all: 'foo
| 2 -> (macroexp--expand-all (cons 'macro #'{foo} (lambda (#<symbol lambda =
at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" `(cons ,lambda ,bar))))
| | 3 -> (macroexp--expand-all 'macro)
| | 3 <- macroexp--expand-all: 'macro
| | 3 -> (macroexp--expand-all #'{foo} (lambda (#<symbol lambda at 46> bar)=
";POS=01=01 [foo foo.el 41 nil]\n" `(cons ,lambda ,bar)))
| | | 4 -> (macroexp--expand-all ";POS=01=01 [foo foo.el 41 nil]\n")
| | | 4 <- macroexp--expand-all: ";POS=01=01 [foo foo.el 41 nil]\n"
| | | 4 -> (macroexp--expand-all `(cons ,lambda ,bar))
| | | | 5 -> (macroexp--expand-all 'cons)
| | | | 5 <- macroexp--expand-all: 'cons
| | | | 5 -> (macroexp--expand-all lambda)
| | | | 5 <- macroexp--expand-all: lambda
| | | | 5 -> (macroexp--expand-all bar)
| | | | 5 <- macroexp--expand-all: bar
| | | 4 <- macroexp--expand-all: (list 'cons lambda bar)
| | 3 <- macroexp--expand-all: #'{foo} (lambda (#<symbol lambda at 46> bar)=
";POS=01=01 [foo foo.el 41 nil]\n" (list 'cons lambda bar))
| 2 <- macroexp--expand-all: (cons 'macro #'{foo} (lambda (#<symbol lambda =
at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" (list 'cons lambda bar)))
1 <- macroexp--expand-all: (defalias 'foo (cons 'macro #'{foo} (lambda (#<s=
ymbol lambda at 46> bar) ";POS=01=01 [foo foo.el 41 nil]\n" (list 'cons lam=
bda bar))))
So we see that indeed it returns code where the formal argument `lambda`
is (incorrectly) a SYMPOS. Yet somehow the sympos is stripped after
macroexpansion somewhere since `(symbol-function 'foo)` shows the
resulting function doesn't have any symposes in it.
>> > so it is clear this case is getting handled OK. I'm afraid I can't
>> > point out the exact place in the code at the moment where this is
>> > getting done.
>> I think it would be good to know, so as to be able to decide whether
>> it'll indeed always work right, or we just got lucky this time.
> See above.
Yes, please, see above =F0=9F=99=82
>> Could you explain what you think makes it intrinsically complex?
> The mass of detail that needs dealing with that Emacs has collected over
> the decades. As a counter question, why do you think the exercise ought
> to be simple (assuming you do think this)?
Because you solved the hard part when you added the symposes for the compil=
er.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 8 Apr 2024 02:56:38 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 07 22:56:38 2024 Received: from localhost ([127.0.0.1]:45067 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rtfBV-0003xE-UW for submit <at> debbugs.gnu.org; Sun, 07 Apr 2024 22:56:38 -0400 Received: from mail.muc.de ([193.149.48.3]:27119) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rtfBT-0003wH-FU for 67455 <at> debbugs.gnu.org; Sun, 07 Apr 2024 22:56:36 -0400 Received: (qmail 43870 invoked by uid 3782); 8 Apr 2024 04:56:21 +0200 Received: from muc.de (p4fe15203.dip0.t-ipconnect.de [79.225.82.3]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 08 Apr 2024 04:56:21 +0200 Received: (qmail 5627 invoked by uid 1000); 8 Apr 2024 02:56:20 -0000 Date: Mon, 8 Apr 2024 02:56:20 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZhNc1FNq8TO-nJFn@ACM> References: <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> <ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN> <ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> <ZgfhIfpwjSiUr_2U@ACM> <jwvv853taqa.fsf-monnier+emacs@HIDDEN> <ZhKE92pGS3SG4pNy@ACM> <jwvy19oeheq.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <jwvy19oeheq.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Sun, Apr 07, 2024 at 22:19:28 -0400, Stefan Monnier wrote: > > The definition starts when the reader reads (defun foo ...). It ends > > when that Fdefalias has been evaluated. Between those two events, > > defining-symbol is bound to foo. > When the code is compiled, years can pass between those two events. Really? Then defining-symbol would stay bound over these years, too. Where's the problem? > > The critical thing here is the variable defining-symbol. I think you're > > suggesting that its value could accidentally find its way into other > > symbols' position structures. When are those other symbols getting > > defined? Surely if a secondary defun, defmacro, defvar, ... happens > > during the main defun, its defining symbol is that of the main defun? > That secondary symbol might be defined by the macro for its own use, > rather than for the use of the returned code. > > Well, m-a-e is defined too late. > According to your self-imposed rule 🙂 > > Even if it weren't, how would it solve any of these problems you see? > > It's just swapping one dynamically bound variable for another. > Indeed, m-a-e has to fight the same issues, in theory, but in practice, > those issues have already been addressed over the years. It's rebound > to nil "all the time" to try and make sure its effect doesn't leak. defining-symbol isn't bound like this, in order for its value to be available when needed. > BTW, it would be nice to separate your patches into some that add > position info to lambda's docstring, and then others that add > "defining symbol" to the available position info. All this could be moot. Byte compiling is appreciably slower on this branch than on master. make bootstrap is taking me nearly 7 minutes compared with master's 4min 40sec. Running make -j17 check is taking 1min 50sec rather than 50sec. Once compiled, the code runs at the same speed, though. I think this issue needs sorting out first. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 8 Apr 2024 02:19:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 07 22:19:46 2024
Received: from localhost ([127.0.0.1]:45057 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rtebq-00010M-7Q
for submit <at> debbugs.gnu.org; Sun, 07 Apr 2024 22:19:46 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:2386)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rtebo-000106-4K
for 67455 <at> debbugs.gnu.org; Sun, 07 Apr 2024 22:19:44 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7D7A880BC1;
Sun, 7 Apr 2024 22:19:30 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1712542769;
bh=D5oY0mUikOGk7A35Rwr/yGvQT2tEUYG4KC3aZtL0cuU=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=gwy0r+t/nZkV1DxlwunYPt7EOzLS8zco1q1U6PQiDEdD3u5cjYo/JF432gfniOZ6S
NQPnynyO9o4kbZo8AAWjmRr9dp6W1aCr7YpeOaia4GM10hskcYCs0bGO7QToQFDujh
G7HUghR+3vy5OHe6es+fB98dBwWQBsWQDIamowMJwZt8rmcNq/O+o4264nSdpRnjE1
MuxjKkA2wrx5j1qpyXnS19lerFkFnUagNtN+OcSuxZrvbghfjD1reRA3iNeCVYSO2Q
1WrR89S2AtNNAbg4oizN8+GoUpK60G8SgZHqs55JTGcUI0JtWWtkeHnJAvtlgL6BYq
xmpe2oQfzY/fA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 6647F8036C;
Sun, 7 Apr 2024 22:19:29 -0400 (EDT)
Received: from pastel (unknown [45.72.201.215])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3BAD012082A;
Sun, 7 Apr 2024 22:19:29 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZhKE92pGS3SG4pNy@ACM> (Alan Mackenzie's message of "Sun, 7 Apr
2024 11:35:19 +0000")
Message-ID: <jwvy19oeheq.fsf-monnier+emacs@HIDDEN>
References: <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
<ZgfhIfpwjSiUr_2U@ACM> <jwvv853taqa.fsf-monnier+emacs@HIDDEN>
<ZhKE92pGS3SG4pNy@ACM>
Date: Sun, 07 Apr 2024 22:19:28 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.098 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> The definition starts when the reader reads (defun foo ...). It ends
> when that Fdefalias has been evaluated. Between those two events,
> defining-symbol is bound to foo.
When the code is compiled, years can pass between those two events.
> The critical thing here is the variable defining-symbol. I think you're
> suggesting that its value could accidentally find its way into other
> symbols' position structures. When are those other symbols getting
> defined? Surely if a secondary defun, defmacro, defvar, ... happens
> during the main defun, its defining symbol is that of the main defun?
That secondary symbol might be defined by the macro for its own use,
rather than for the use of the returned code.
> Well, m-a-e is defined too late.
According to your self-imposed rule =F0=9F=99=82
> Even if it weren't, how would it solve any of these problems you see?
> It's just swapping one dynamically bound variable for another.
Indeed, m-a-e has to fight the same issues, in theory, but in practice,
those issues have already been addressed over the years. It's rebound
to nil "all the time" to try and make sure its effect doesn't leak.
BTW, it would be nice to separate your patches into some that add
position info to lambda's docstring, and then others that add
"defining symbol" to the available position info.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 7 Apr 2024 11:35:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 07 07:35:37 2024 Received: from localhost ([127.0.0.1]:41709 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rtQoD-0000Jl-2m for submit <at> debbugs.gnu.org; Sun, 07 Apr 2024 07:35:37 -0400 Received: from mail.muc.de ([193.149.48.3]:59965) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rtQoB-0000JY-OR for 67455 <at> debbugs.gnu.org; Sun, 07 Apr 2024 07:35:36 -0400 Received: (qmail 96078 invoked by uid 3782); 7 Apr 2024 13:35:22 +0200 Received: from muc.de (pd953a949.dip0.t-ipconnect.de [217.83.169.73]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 07 Apr 2024 13:35:22 +0200 Received: (qmail 9924 invoked by uid 1000); 7 Apr 2024 11:35:19 -0000 Date: Sun, 7 Apr 2024 11:35:19 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZhKE92pGS3SG4pNy@ACM> References: <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> <ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN> <ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> <ZgfhIfpwjSiUr_2U@ACM> <jwvv853taqa.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvv853taqa.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Sat, Mar 30, 2024 at 22:22:52 -0400, Stefan Monnier wrote: > >> We're still miscommunicating. You're talking about how your code is > >> implemented, apparently, whereas I'm asking about what is the > >> intended behavior. > > I am still mystified by your failure to understand "currently being > > defined", a phrase that to me could hardly be clearer. > AFAIK, the definition itself happens inside `Fdefalias`. It's a very > short amount of time during which none of your code is executed, so that > can't be it. The definition starts when the reader reads (defun foo ...). It ends when that Fdefalias has been evaluated. Between those two events, defining-symbol is bound to foo. > The rest is the actual construction of the value/function object which > will make up the definition. This construction is done piecemeal in > various phases at potentially various different times, not all of them > necessarily on the same machine. Really? :-) > Some of the code executed during the course of the construction of this > object have nothing at all to do with that object, they just happen to > be used by some code which participates in the construction of that > object. That's a bit too abstract. I can't see the potential problems that you see, possibly because I've already solved them. > So "currently" (i.e. referring to *time*) is very problematic because > it's only loosely correlated to what you're interested in. Again, I don't see these problems. > >> It's like I'm asking what the C spec says and you're answering me by > >> telling me how GCC works. > > OK, let's try again. defining-symbol records the symbol currently being > > defined. It's used to set the defining symbol and buffer offset fields > > in the position structure in that symbol's doc string, and also in the > > doc strings of contained lambda forms. > I can try it again also if you want: to do it right, I think you want to > store that information in `macroexpand-all-environment`. That variable doesn't exist, yet. > > Is my previous paragraph sufficiently clear? If so, can you envisage a > > scenario where a symbol being defined would fail to get the two fields > > correctly set in its doc string or a lambda form's doc string? > Yes, for example I can imagine a lazy macroexpansion done to run some > code during the eager macroexpansion could mistakenly think that it is > part of the eagerly macroexpanded function (whereas it's only part of > the code used during the construction of that function). The critical thing here is the variable defining-symbol. I think you're suggesting that its value could accidentally find its way into other symbols' position structures. When are those other symbols getting defined? Surely if a secondary defun, defmacro, defvar, ... happens during the main defun, its defining symbol is that of the main defun? > I'm sure there are other cases, by thinking of it makes my head hurt. > Which is why I recommend you put that info into > `macroexpand-all-environment` which is designed specifically for such > purpose of "currently within the scope of something". Well, m-a-e is defined too late. Even if it weren't, how would it solve any of these problems you see? It's just swapping one dynamically bound variable for another. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 7 Apr 2024 10:58:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 07 06:58:11 2024 Received: from localhost ([127.0.0.1]:41697 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rtQDy-0002mU-M6 for submit <at> debbugs.gnu.org; Sun, 07 Apr 2024 06:58:11 -0400 Received: from mail.muc.de ([193.149.48.3]:42604) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rtQDv-0002l2-Am for 67455 <at> debbugs.gnu.org; Sun, 07 Apr 2024 06:58:09 -0400 Received: (qmail 52805 invoked by uid 3782); 7 Apr 2024 12:57:54 +0200 Received: from muc.de (pd953a949.dip0.t-ipconnect.de [217.83.169.73]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 07 Apr 2024 12:57:54 +0200 Received: (qmail 9814 invoked by uid 1000); 7 Apr 2024 10:57:51 -0000 Date: Sun, 7 Apr 2024 10:57:50 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZhJ8Ltr3DuuAyFOD@ACM> References: <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> <ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN> <ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> <ZgfxinngV469zNSY@ACM> <jwvplvbta02.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvplvbta02.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. Sorry about the delay - I lost my email server after an obsolete SSL library got deleted from my system, and one or two other things, too. On Sat, Mar 30, 2024 at 22:54:18 -0400, Stefan Monnier wrote: > > Some symbols must not be stripped. For example, in cl-generic.el L403 > > we have: > > (fun `(cl-function (lambda ,plain-args ,@body))) > > .. There the position on the lambda must be preserved until ME2 time > > when it becomes clear what the shape of the lambda is. In particular, > > whether there is already a doc string in ,@body to amend, or we need to > > insert a new one. > I could see some reasons you *may* want to keep some info here, but it's > definitely not a "must" because the source position of those functions > should generally not point to `cl-generic.el:403` but to where > `cl-defmethod` was used. Pretending the problem doesn't exist won't solve it. In the ;POS... structures for a lambda, there are two pointers - one to the definition of the lambda, the other to the point of use. > Also, if you do want to preserve some info there (presumably with the > intent to combine it with the more important info that will be available > at ME2) it will need cooperation from `cl-generic.el` because, as far as > the semantic of Emacs Lisp is concerned, the above constructs a list > with a `lambda` symbol inside of it, with no guarantee that it will be > used as a function, .... Mostly there is the symbol `function' in that position. Here we've got `cl-function' which expands to function. Surely with (function (lambda ....)) we know we're dealing with a function. > .... and even if ever used as a function there's no guarantee that this > list will pass through the few places where we strip SWPs, so keeping > SWPs in there without some explicit request from `cl-generic.el` would > be a bug. I don't think this is right. The code will pass through macroexp--expand-all, which is where the SWP wii be stripped. > IOW, I don't think it's a good reason to rule out > (strip-all-symbol-positions > (macroexp--expand-all > (read-positioning-symbols))) As I've said, we'd need code to preserve the SWPs on "complicated" lambdas. I haven't even begun to think about how this could work. > BTW, AFAIK the above is conceptually what the byte-compiler does (except > it performs a few more transformations between `macroexp--expand-all` > and `strip-all-symbol-positions`). It is a bad idea to conflate these two radically different uses of SWPs. That can only lead to confusion and bugs. > Is it the case that `cl-defmethod` generates a function whose source > position (partly) points to `generic.el:403` if `cl-generic.el` was > interpreted but not if it compiled? No, the intention is that the source positions are independent of whether the code is compiled. > >> >> >> Also, IIUC you don't have a separate phase to strip the SWPs when > >> >> >> loading from source, but instead you strip them as you "consume" their > >> >> >> info during macroexpansion. If so, how/when/where do you strip the > >> >> >> false positives that may occur inside quoted data or in code like: > >> >> >> (defmacro foo (lambda bar) ...) > > (defmacro foo (lambda bar) > > `(cons ,lambda ,bar)) > > expands to > > (macro closure (t) (lambda bar) ";POS^^^A^A^A [foo *scratch* 158 nil] > > " (list 'cons lambda bar)) > IIUC your reader will make the `lambda` formal argument into an SWP. > Where is that SWP stripped? In macroexp--expand-all in the "guard arm" near the end. > > so it is clear this case is getting handled OK. I'm afraid I can't > > point out the exact place in the code at the moment where this is > > getting done. > I think it would be good to know, so as to be able to decide whether > it'll indeed always work right, or we just got lucky this time. See above. > >> I don't actually know whether it will be better. It just seems it could > >> lead to simpler code, with no change at all to the reader, for example. > > The exercise is intrinsically complicated. > Could you explain what you think makes it intrinsically complex? The mass of detail that needs dealing with that Emacs has collected over the decades. As a counter question, why do you think the exercise ought to be simple (assuming you do think this)? > > What you're suggesting is that the code to decide which SWPs to strip > > is going to be simpler than the enhancements to the reader. > As seen above, I suggest to leave the reader unchanged and to strip all > SWPs. I'm pretty sure it would give comparable info to what you have > and it would be simpler (also, it would make it much less likely to > have discrepancies between the compiled case and the interpreted case). "Comparable" isn't good enough - we need the position info on "complicated" lambdas to endure, somehow. There are no discrepancies between compiled and interpreted forms because they both use the same mechanism in macro expansion. > My main worry with it would be performance. Yhat, too. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 31 Mar 2024 02:54:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 30 22:54:35 2024
Received: from localhost ([127.0.0.1]:46391 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rqlL8-0007X4-1Q
for submit <at> debbugs.gnu.org; Sat, 30 Mar 2024 22:54:35 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:8481)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rqlL2-0007WM-Jp
for 67455 <at> debbugs.gnu.org; Sat, 30 Mar 2024 22:54:32 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 5B59780D32;
Sat, 30 Mar 2024 22:54:20 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711853659;
bh=Kmy0kHc7a0ROClM4leLEGhaH10nEt9IbyIeyerymD3M=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=gMYmmuMAskngBgFueaIk8sakczYYAeoYxeBmzvWOyB23W+g7/zDrVbPPtaB2GfJOR
AEZIuWpI8VZAzA8hHe7buh21Yppuvn8sPa+XHh/9hTDdGr/lBeH1PMc9UMqWcZeXef
BTxbCHSNJyG9BZdMx6HZmRFL0OCSyEVTJy8ARlQaX6GZJtRwGJxApg6V1TmCBWrmYJ
+T5Oc2AZpdbJELqISSnoiYlWix1Fuy6P6DWl9aguPIiLO+DhUhKglyTb8Wb700IVse
W4sCRCV/4Oc9EIGxRIIdnjw35DrOL3lZEG1XlQvOdLsH1PKYLLAbBuLXm7XqupeD+1
YIlu1eXvK4ekA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 36E808036C;
Sat, 30 Mar 2024 22:54:19 -0400 (EDT)
Received: from pastel (unknown [45.72.201.215])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 0D608120426;
Sat, 30 Mar 2024 22:54:19 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgfxinngV469zNSY@ACM> (Alan Mackenzie's message of "Sat, 30 Mar
2024 11:03:38 +0000")
Message-ID: <jwvplvbta02.fsf-monnier+emacs@HIDDEN>
References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
<ZgfxinngV469zNSY@ACM>
Date: Sat, 30 Mar 2024 22:54:18 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.135 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Some symbols must not be stripped. For example, in cl-generic.el L403
> we have:
>
> (fun `(cl-function (lambda ,plain-args ,@body)))
>
> .. There the position on the lambda must be preserved until ME2 time
> when it becomes clear what the shape of the lambda is. In particular,
> whether there is already a doc string in ,@body to amend, or we need to
> insert a new one.
I could see some reasons you *may* want to keep some info here, but it's
definitely not a "must" because the source position of those functions
should generally not point to `cl-generic.el:403` but to where
`cl-defmethod` was used.
Also, if you do want to preserve some info there (presumably with the
intent to combine it with the more important info that will be available
at ME2) it will need cooperation from `cl-generic.el` because, as far as
the semantic of Emacs Lisp is concerned, the above constructs a list
with a `lambda` symbol inside of it, with no guarantee that it will be
used as a function, and even if ever used as a function there's no
guarantee that this list will pass through the few places where we strip
SWPs, so keeping SWPs in there without some explicit request from
`cl-generic.el` would be a bug.
IOW, I don't think it's a good reason to rule out
(strip-all-symbol-positions
(macroexp--expand-all
(read-positioning-symbols)))
BTW, AFAIK the above is conceptually what the byte-compiler does (except
it performs a few more transformations between `macroexp--expand-all`
and `strip-all-symbol-positions`). Is it the case that `cl-defmethod`
generates a function whose source position (partly) points to
`generic.el:403` if `cl-generic.el` was interpreted but not if
it compiled?
>> >> >> Also, IIUC you don't have a separate phase to strip the SWPs when
>> >> >> loading from source, but instead you strip them as you "consume" their
>> >> >> info during macroexpansion. If so, how/when/where do you strip the
>> >> >> false positives that may occur inside quoted data or in code like:
>
>> >> >> (defmacro foo (lambda bar) ...)
>
> (defmacro foo (lambda bar)
> `(cons ,lambda ,bar))
>
> exoands to
>
> (macro closure (t) (lambda bar) ";POS^^^A^A^A [foo *scratch* 158 nil]
> " (list 'cons lambda bar))
IIUC your reader will make the `lambda` formal argument into an SWP.
Where is that SWP stripped?
> so it is clear this case is getting handled OK. I'm afraid I can't
> point out the exact place in the code at the moment where this is
> getting done.
I think it would be good to know, so as to be able to decide whether
it'll indeed always work right, or we just got lucky this time.
>> I don't actually know whether it will be better. It just seems it could
>> lead to simpler code, with no change at all to the reader, for example.
> The exercise is intrinsically complicated.
Could you explain what you think makes it intrinsically complex?
> What you're suggesting is that the code to decide which SWPs to strip
> is going to be simpler than the enhancements to the reader.
As seen above, I suggest to leave the reader unchanged and to strip all
SWPs. I'm pretty sure it would give comparable info to what you have
and it would be simpler (also, it would make it much less likely to
have discrepancies between the compiled case and the interpreted case).
My main worry with it would be performance.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 31 Mar 2024 02:23:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 30 22:23:23 2024
Received: from localhost ([127.0.0.1]:46385 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rqkqw-0005wA-Ve
for submit <at> debbugs.gnu.org; Sat, 30 Mar 2024 22:23:23 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:62519)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rqkqq-0005vM-TC
for 67455 <at> debbugs.gnu.org; Sat, 30 Mar 2024 22:23:20 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 5E11C44230A;
Sat, 30 Mar 2024 22:23:08 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711851786;
bh=9d/8XKl4jDoCKn4v+BxJDJhRSnK3C0ld9YotuqTn4aY=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=cx7k2sHshcJbI2zPZkHcXCs6rEKe6uBgLB3AMQcUMpi9YwTsaN9R16VUhfsXvflxc
2zT+5+b6YrpaIFQPP1wl4+SKE7a2mqyepHY40b+hmthdY7zNmI7+A8lHlRehU+P8ab
v1VtE/GwgUHp/YNqduW0LsxXfvVOysl/58Zw5KBgxuG0BOENNJjulRsTbc0/z9xD8T
/GPccvR7LcTlOe3KUmXh1ld5EWvDL+GDD6wpAhrWhy2mtisrkFMOwMoRYpYm9ELnD1
XV/yEIvUF8iT26NKd4rD6YINb1C/bNBDaEFm2WQZbOWCtof/T7owGQOaqj6XNukeWf
1L2ImHsCbAedQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id D4E8F4422CD;
Sat, 30 Mar 2024 22:23:06 -0400 (EDT)
Received: from pastel (unknown [45.72.201.215])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A73781206FE;
Sat, 30 Mar 2024 22:23:06 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgfhIfpwjSiUr_2U@ACM> (Alan Mackenzie's message of "Sat, 30 Mar
2024 09:53:37 +0000")
Message-ID: <jwvv853taqa.fsf-monnier+emacs@HIDDEN>
References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
<ZgfhIfpwjSiUr_2U@ACM>
Date: Sat, 30 Mar 2024 22:22:52 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.133 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> We're still miscommunicating. You're talking about how your code is
>> implemented, apparently, whereas I'm asking about what is the
>> intended behavior.
> I am still mystified by your failure to understand "currently being
> defined", a phrase that to me could hardly be clearer.
AFAIK, the definition itself happens inside `Fdefalias`. It's a very
short amount of time during which none of your code is executed, so that
can't be it. The rest is the actual construction of the value/function
object which will make up the definition. This construction is done
piecemeal in various phases at potentially various different times, not
all of them necessarily on the same machine. Some of the code executed
during the course of the construction of this object have nothing at all
to do with that object, they just happen to be used by some code which
participates in the construction of that object.
So "currently" (i.e. referring to *time*) is very problematic because
it's only loosely correlated to what you're interested in.
>> It's like I'm asking what the C spec says and you're answering me by
>> telling me how GCC works.
>
> OK, let's try again. defining-symbol records the symbol currently being
> defined. It's used to set the defining symbol and buffer offset fields
> in the position structure in that symbol's doc string, and also in the
> doc strings of contained lambda forms.
I can try it again also if you want: to do it right, I think you want to
store that information in `macroexpand-all-environment`.
> Is my previous paragraph sufficiently clear? If so, can you envisage a
> scenario where a symbol being defined would fail to get the two fields
> correctly set in its doc string or a lambda form's doc string?
Yes, for example I can imagine a lazy macroexpansion done to run some
code during the eager macroexpansion could mistakenly think that it is
part of the eagerly macroexpanded function (whereas it's only part of
the code used during the construction of that function).
I'm sure there are other cases, by thinking of it makes my head hurt.
Which is why I recommend you put that info into
`macroexpand-all-environment` which is designed specifically for such
purpose of "currently within the scope of something".
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 30 Mar 2024 11:03:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 30 07:03:49 2024
Received: from localhost ([127.0.0.1]:44062 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rqWV2-0006Bp-Vl
for submit <at> debbugs.gnu.org; Sat, 30 Mar 2024 07:03:49 -0400
Received: from mail.muc.de ([193.149.48.3]:36502)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rqWV1-0006Bc-3P
for 67455 <at> debbugs.gnu.org; Sat, 30 Mar 2024 07:03:47 -0400
Received: (qmail 61056 invoked by uid 3782); 30 Mar 2024 12:03:39 +0100
Received: from acm.muc.de (p4fe15358.dip0.t-ipconnect.de [79.225.83.88])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Sat, 30 Mar 2024 12:03:38 +0100
Received: (qmail 6296 invoked by uid 1000); 30 Mar 2024 11:03:38 -0000
Date: Sat, 30 Mar 2024 11:03:38 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgfxinngV469zNSY@ACM>
References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Thu, Mar 28, 2024 at 12:25:11 -0400, Stefan Monnier wrote:
[ .... ]
> >> >> Do you have rough numbers comparing the cost of `read`,
> >> >> `read-positioning-symbols`, and `read-positioning-DEFINED-symbols`?
> >> > No, but they will be very close to eachother (and very cheap)
> >> Then I think we should use `read-positioning-symbols`, which
> >> requires fewer code changes.
> > It won't. Required would be Lisp code to determine whether a particular
> > SWP needs to be stripped or not. This is not going to be simple. It is
> > likely to be about as complicated as the existing enhancements to read0.
> They'd all need to be stripped, AFAICT, so we'd do:
> (strip-all-symbol-positions
> (macroexp--expand-all
> (read-positioning-symbols)))
> What would be hard about it?
Some symbols must not be stripped. For example, in cl-generic.el L403
we have:
(fun `(cl-function (lambda ,plain-args ,@body)))
.. There the position on the lambda must be preserved until ME2 time
when it becomes clear what the shape of the lambda is. In particular,
whether there is already a doc string in ,@body to amend, or we need to
insert a new one.
> >> >> Also, IIUC you don't have a separate phase to strip the SWPs when
> >> >> loading from source, but instead you strip them as you "consume" their
> >> >> info during macroexpansion. If so, how/when/where do you strip the
> >> >> false positives that may occur inside quoted data or in code like:
> >> >> (defmacro foo (lambda bar) ...)
(defmacro foo (lambda bar)
`(cons ,lambda ,bar))
exoands to
(macro closure (t) (lambda bar) ";POS^^^A^A^A [foo *scratch* 158 nil]
" (list 'cons lambda bar))
..
> >> >> (defmacro foo (defun bar) ...)
> >> >> (let* ((lambda foo)
> >> >> (defun bar))
> >> >> ...)
Similarly, we get
(defun baz ()
(let ((lambda 'foo)
(defun 'bar))
(cons lambda defun)))
(symbol-function 'baz)
(closure (t) nil ";POS^^^A^A^A [baz *scratch* 323 nil]
" (let ((lambda 'foo) (defun 'bar)) (cons lambda defun)))
, so it is clear this case is getting handled OK. I'm afraid I can't
point out the exact place in the code at the moment where this is
getting done.
> >> > There's a pcase arm right at the end of macroexp--expand-all which strips
> >> > SWPs of their positions. Recursing through macroexp--all-forms will
> >> > eventually hit this pcase arm for these lambdas.
> Actually, now that I look at the code I only see:
> ((guard (and (not byte-compile-in-progress)
> (symbol-with-pos-p form)))
> (bare-symbol form))
> is that the "arm" you're talking about?
Yes.
> AFAICT this will handle only those symbols which appear as Lisp
> expressions (IOW symbols which are variable references), so it will
> strip the `bar` in the second example but not the `bar` in my first
> exmple, nor the two `lambda`s, nor those in
> '(lambda (defun bar))
See above.
[ .... ]
> > Why do you think this design change will be better than the existing
> > design?
> I don't actually know whether it will be better. It just seems it could
> lead to simpler code, with no change at all to the reader, for example.
The exercise is intrinsically complicated. read0 is largely self
contained, meaning any compexity introduced there won't spill over into
other functions. What you're suggesting is that the code to decide
which SWPs to strip is going to be simpler than the enhancements to the
reader.
> I'm here asking what lead you to the current design, under the
> assumption that the complexity you introduced was the result of
> other experiments.
The complexity is essential to the task being done. I'm convinced it
cannot be avoided, though it could be moved around the code base, to
some extent. I don't have a record of the precise reasons for the
design. To a large extent it was a sequence of solutions to the often
awkward problems the code presented.
> Am I to understand that the current code is basically your first attempt
> at adding such functionality?
By no means. This is the second implementation, the first having been
rejected by Eli and you last autumn. I'm very much looking forward to
not needing a third implementation.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 30 Mar 2024 09:53:48 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 30 05:53:48 2024 Received: from localhost ([127.0.0.1]:44030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rqVPI-0002ok-6j for submit <at> debbugs.gnu.org; Sat, 30 Mar 2024 05:53:48 -0400 Received: from mail.muc.de ([193.149.48.3]:47448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rqVPG-0002o1-C4 for 67455 <at> debbugs.gnu.org; Sat, 30 Mar 2024 05:53:46 -0400 Received: (qmail 76732 invoked by uid 3782); 30 Mar 2024 10:53:38 +0100 Received: from acm.muc.de (p4fe15358.dip0.t-ipconnect.de [79.225.83.88]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 30 Mar 2024 10:53:38 +0100 Received: (qmail 6134 invoked by uid 1000); 30 Mar 2024 09:53:37 -0000 Date: Sat, 30 Mar 2024 09:53:37 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZgfhIfpwjSiUr_2U@ACM> References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN> <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> <ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN> <ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Thu, Mar 28, 2024 at 12:25:11 -0400, Stefan Monnier wrote: [ .... ] > >> >> My crystal ball suggests that "currently" may be the wrong way to think > >> >> about it: maybe instead of thinking of "when" (as in "during the > >> >> definition of function FOO") what you're looking for might be "where" > >> >> (as in "within the body of FOO"). > >> >> [ That's the same difference as the difference between dynamic and > >> >> static scoping. ] > >> > I'm having trouble understanding what you're saying, here. > >> Is it because you don't understand the difference between dynamic > >> scoping and static scoping, or because you don't see the relationship > >> with that and your notion of "currently being defined"? > > The latter, I think. defining-symbol is entirely dynamically scoped. > We're still miscommunicating. You're talking about how your code is > implemented, apparently, whereas I'm asking about what is the > intended behavior. I am still mystified by your failure to understand "currently being defined", a phrase that to me could hardly be clearer. > It's like I'm asking what the C spec says and you're answering me by > telling me how GCC works. OK, let's try again. defining-symbol records the symbol currently being defined. It's used to set the defining symbol and buffer offset fields in the position structure in that symbol's doc string, and also in the doc strings of contained lambda forms. > > I'm convinced it does. Can you suggest a scenario where the > > defining-symbol mechanism (outlined above) might fail? > Without knowing what it is intended to do, the only thing we can say is > that it does what it does, so no indeed it won't fail to do what it > does, since that's what it does. 🙂 Is my previous paragraph sufficiently clear? If so, can you envisage a scenario where a symbol being defined would fail to get the two fields correctly set in its doc string or a lambda form's doc string? [ .... ] > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 30 Mar 2024 09:10:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 30 05:10:25 2024 Received: from localhost ([127.0.0.1]:43942 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rqUjJ-0000iN-Gl for submit <at> debbugs.gnu.org; Sat, 30 Mar 2024 05:10:25 -0400 Received: from mail.muc.de ([193.149.48.3]:28692) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rqUjG-0000i6-Oy for 67455 <at> debbugs.gnu.org; Sat, 30 Mar 2024 05:10:23 -0400 Received: (qmail 28356 invoked by uid 3782); 30 Mar 2024 10:10:14 +0100 Received: from acm.muc.de (p4fe15358.dip0.t-ipconnect.de [79.225.83.88]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 30 Mar 2024 10:10:14 +0100 Received: (qmail 6125 invoked by uid 1000); 30 Mar 2024 09:10:14 -0000 Date: Sat, 30 Mar 2024 09:10:14 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZgfW9siFQXw5RbdU@ACM> References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN> <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> <ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN> <ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Thu, Mar 28, 2024 at 12:25:11 -0400, Stefan Monnier wrote: [ .... ] > >> The earlier the better, in theory, but not at any cost. > > No, the earlier the better, full stop. > Please "full stop" being absolutist. We're talking about opinions and > preferences here. What you're proposing is only handling some fuctions because you think we're collectively not clever enough to maintain byte-run--posify-defining-form. This would leave Emacs inconsistent, some functions failing to be handled not for any functional reason, but because of an alleged lack of our capability. > When hitting an error, I spend more time reading the code (and modifying > it) than looking at debug output, so to me the clarity of the code is > more important than whether a few lambdas get some addition positional > info, especially since I usually know full well where those lambdas > come from. My prime method of debugging is reading code, too. But you're conflating the clarity of b-r--p-defining-f with the clarity of the code you're debugging. They're different things. The former is less important than the latter. The whole point in these changes is to give info precisely in those anonymous lambda entries in backtraces which currently contain no information. > I understand it affects us differently, but the tradeoff is real. > >> Having to write all that code within the very restrictive sublanguage > >> available before subr.el and backquote.el is a cost I don't think > >> justifies it. This is done in other functions, too. > > The cost has already been paid, by me. > Code is not "fire and forget". [ .... ] > >> But your current code in byte-run.el is a Bad Thing as well. > > What, precisely, do you find bad about it? It may be possible to improve > > it without wholesale redesign. > A lot of it is hard to read because it is constrained to a restrictive > subset of ELisp. byte-run--posify-defining-form uses the same techniques as other declare clause handlers, such as byte-run--set-interactive-only and many others. Why is b-r--p-defining-f objectionable, but not b-r--s-i-only? It was tedious rather than difficult to write, and it is tedious rather than difficult to read. > >> I'm not suggesting to drop support for lambdas loaded from source. > >> I'm saying we don't need to support it for the first N files loaded into > >> `src/emacs-bootstrap`. > > You're suggesting dropping support for many source files, where that > > support is most needed. > "Most needed" according to which criteria? The difficulty of debugging in early bootstrap compared with when further debugging tools have already been loaded. [ .... ] > Code costs by merely existing. Inconsistencies and sloppy implementation cost too. [ .... ] > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 28 Mar 2024 16:48:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 28 12:48:15 2024
Received: from localhost ([127.0.0.1]:41043 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpsvG-0007BR-Ql
for submit <at> debbugs.gnu.org; Thu, 28 Mar 2024 12:48:15 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:31864)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpsvD-0007AP-F6
for 67455 <at> debbugs.gnu.org; Thu, 28 Mar 2024 12:48:14 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 0E5D280672;
Thu, 28 Mar 2024 12:48:05 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711644484;
bh=xGb0DG6g773ZT65sX3vTLIKwEZuHdXk3BnnxQzGLGl8=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=YM5HytUqIsROdnKywoTwlTqDpdmqn+9i8G/XoJaxEfFGRQWyGTqb8YhHLdTSNXy5n
YYRxau6FJtKCppS8R6cNDyluMyUh8O4G/SiI/+MhKoTEykOzaDxKuHs2LQ9F6j5McM
idaaWEOVuYREStFOnpHJQerLyqZk1GCN3YEvy2JzPPlAaq+sP9ZjSTXiRhKB7wVpfd
ygbVnBqmGj/xcfiiFeHHGBfVNcr9gz8lvyezxnFrvLqKnkfUvzVkO1hAGeL5JFjL2z
bUCtbE5lwSJ5R1dsbN3YzD3iJQpFguUAWqXwS5VvCGGXYIJ45bxtnZdLPzl6RwLSa5
AMoinrVt+TrxA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 03B4C80469;
Thu, 28 Mar 2024 12:48:04 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id D199412026E;
Thu, 28 Mar 2024 12:48:03 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN> (Stefan Monnier's message
of "Thu, 28 Mar 2024 12:25:11 -0400")
Message-ID: <jwvplvexpzg.fsf-monnier+emacs@HIDDEN>
References: <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM> <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
Date: Thu, 28 Mar 2024 12:48:03 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.137 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> The "callee" I'm talking about is `load-source-file-function` (which is
^^^
not
> instead another "caller", beside the byte compiler), it is the
> code that tests `byte-compile-in-progress`.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 28 Mar 2024 16:25:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 28 12:25:33 2024
Received: from localhost ([127.0.0.1]:41012 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpsZI-0006D3-9D
for submit <at> debbugs.gnu.org; Thu, 28 Mar 2024 12:25:32 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:48304)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpsZE-0006CL-Ci
for 67455 <at> debbugs.gnu.org; Thu, 28 Mar 2024 12:25:30 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 71ECA4429C0;
Thu, 28 Mar 2024 12:25:21 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711643119;
bh=BpvhuISHjL0jGFojQFdesEy+N+/FupKVVq9t5IXvgzI=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=XK7Oscplr07xbbdJ2/1jmDpKqXw4nsK5DgrpYpY4/bQk9pvTROUQ0ZhicOkHqU2wu
XV0XicUbfoozmcAHM6KXyOwBvYT0ZTy2lZaQqOfb1p03OXltA0jOmgaV2GOtRVMVgU
6AE7me7xDv1j3AFbq/x+8+/dW273VvELUpMga6227pvqVyxNvk5dmQEaMy+Voz7GUQ
FDlC8OA2l3oT43+vy2PCGD9EHfCosylIWwJURjOOniNUBzxIdgisgfsdjoFeb69ZLj
DJj8XYX0RiH/VPobs+bbNz83s26ZhzEoYmlISTECWe4vATvv8qROSZEhh+EE2JA7CB
tjf8MbLYzsqWA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 222D44429C4;
Thu, 28 Mar 2024 12:25:19 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id F16071207EA;
Thu, 28 Mar 2024 12:25:18 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgSTAR9IpCVQ_xCC@ACM> (Alan Mackenzie's message of "Wed, 27 Mar
2024 21:43:29 +0000")
Message-ID: <jwvfrwaz8so.fsf-monnier+emacs@HIDDEN>
References: <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
<ZgSTAR9IpCVQ_xCC@ACM>
User-Agent: Gnus/5.13 (Gnus v5.13)
Date: Thu, 28 Mar 2024 12:25:11 -0400
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.288 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> I think it is better to regard the byte compilation as the special case.
> Only in byte compilation do we want to preserve the SWPs on forms
> getting posified.
[...]
> Byte compilation is NOT calling loading from source. We don't have a
> caller/callee relationship here. We are doing posification either from
The "callee" I'm talking about is `load-source-file-function` (which is
instead another "caller", beside the byte compiler), it is the
code that tests `byte-compile-in-progress`.
Based on the above two elements, I suggest the name
`macroexp--inhibit-strip-sympos` (and set/pass/bind it as needed in the
compiler).
[ FWIW, the reason I associate this variable with
`load-source-file-function` rather than with the compiler is because
the macroexp code which tests this variable only strips the SWPs at
a very few different spots, more specifically those very spots that
are expected to get SWPs in the `load-source-file-function`.
Different minds work differently, I guess. =F0=9F=99=82 ]
>> Also, I think as a general rule it's better for the caller to set
>> a callee variable that controls how the callee behaves, rather than for =
the
>> callee to check a caller variable to decide how to behave, because
>> it's normal for the caller to "know about" its callee (after all, it's
>> the caller which decides to call the callee), whereas it's not normal
>> for the callee to know about specific callers (it creates undesirable
>> dependencies).
> byte compilation or from somewhere else. It is analogous to testing
> lexical binding. Here, the variable is called lexical-binding; it is
> not named after a particular activity to be carried out differently for
> l-b and not l-b.
Indeed testing `lexical-binding` in macros (like we do at a few places)
sucks; it's an ugly hack. We use it because that was the least bad
option we could come up with given the need to preserve
backward compatibility.
Here there's no such problem.
>> The earlier the better, in theory, but not at any cost.
> No, the earlier the better, full stop.
Please "full stop" being absolutist. We're talking about opinions and
preferences here.
When hitting an error, I spend more time reading the code (and modifying
it) than looking at debug output, so to me the clarity of the code is
more important than whether a few lambdas get some addition positional
info, especially since I usually know full well where those lambdas
come from.
I understand it affects us differently, but the tradeoff is real.
>> Having to write all that code within the very restrictive sublanguage
>> available before subr.el and backquote.el is a cost I don't think
>> justifies it.
> The cost has already been paid, by me.
Code is not "fire and forget".
>> >> My crystal ball suggests that "currently" may be the wrong way to thi=
nk
>> >> about it: maybe instead of thinking of "when" (as in "during the
>> >> definition of function FOO") what you're looking for might be "where"
>> >> (as in "within the body of FOO").
>> >> [ That's the same difference as the difference between dynamic and
>> >> static scoping. ]
>> > I'm having trouble understanding what you're saying, here.
>> Is it because you don't understand the difference between dynamic
>> scoping and static scoping, or because you don't see the relationship
>> with that and your notion of "currently being defined"?
> The latter, I think. defining-symbol is entirely dynamically scoped.
We're still miscommunicating. You're talking about how your code is
implemented, apparently, whereas I'm asking about what is the
intended behavior.
It's like I'm asking what the C spec says and you're answering me by
telling me how GCC works.
> I'm convinced it does. Can you suggest a scenario where the
> defining-symbol mechanism (outlined above) might fail?
Without knowing what it is intended to do, the only thing we can say is
that it does what it does, so no indeed it won't fail to do what it
does, since that's what it does. =F0=9F=99=82
>> >> Do you have rough numbers comparing the cost of `read`,
>> >> `read-positioning-symbols`, and `read-positioning-DEFINED-symbols`?
>> > No, but they will be very close to eachother (and very cheap)
>> Then I think we should use `read-positioning-symbols`, which
>> requires fewer code changes.
> It won't. Required would be Lisp code to determine whether a particular
> SWP needs to be stripped or not. This is not going to be simple. It is
> likely to be about as complicated as the existing enhancements to read0.
They'd all need to be stripped, AFAICT, so we'd do:
(strip-all-symbol-positions
(macroexp--expand-all
(read-positioning-symbols)))
What would be hard about it?
>> >> Also, IIUC you don't have a separate phase to strip the SWPs when
>> >> loading from source, but instead you strip them as you "consume" their
>> >> info during macroexpansion. If so, how/when/where do you strip the
>> >> false positives that may occur inside quoted data or in code like:
>
>> >> (defmacro foo (lambda bar) ...)
>> >> (defmacro foo (defun bar) ...)
>
>> >> (let* ((lambda foo)
>> >> (defun bar))
>> >> ...)
>
>> > There's a pcase arm right at the end of macroexp--expand-all which str=
ips
>> > SWPs of their positions. Recursing through macroexp--all-forms will
>> > eventually hit this pcase arm for these lambdas.
Actually, now that I look at the code I only see:
((guard (and (not byte-compile-in-progress)
(symbol-with-pos-p form)))
(bare-symbol form))
is that the "arm" you're talking about? AFAICT this will handle only
those symbols which appear as Lisp expressions (IOW symbols which are
variable references), so it will strip the `bar` in the second example
but not the `bar` in my first exmple, nor the two `lambda`s, nor
those in
'(lambda (defun bar))
>> But your current code in byte-run.el is a Bad Thing as well.
> What, precisely, do you find bad about it? It may be possible to improve
> it without wholesale redesign.
A lot of it is hard to read because it is constrained to a restrictive
subset of ELisp.
>> I'm not suggesting to drop support for lambdas loaded from source.
>> I'm saying we don't need to support it for the first N files loaded into
>> `src/emacs-bootstrap`.
> You're suggesting dropping support for many source files, where that
> support is most needed.
"Most needed" according to which criteria?
> I'm not playing on words. My point is that
> read-positioning-defined-symbols exists and works. It is not a
> speculative "would be nice to have". The work has already been done.
Code costs by merely existing.
> Why do you think this design change will be better than the existing
> design?
I don't actually know whether it will be better. It just seems it could
lead to simpler code, with no change at all to the reader, for example.
I'm here asking what lead you to the current design, under the
assumption that the complexity you introduced was the result of
other experiments.
Am I to understand that the current code is basically your first attempt
at adding such functionality?
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 22:00:22 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 27 18:00:22 2024 Received: from localhost ([127.0.0.1]:38482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rpbJl-0000Ul-1a for submit <at> debbugs.gnu.org; Wed, 27 Mar 2024 18:00:22 -0400 Received: from mail.muc.de ([193.149.48.3]:54072) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rpbJf-0000SJ-Rs for 67455 <at> debbugs.gnu.org; Wed, 27 Mar 2024 18:00:19 -0400 Received: (qmail 62400 invoked by uid 3782); 27 Mar 2024 23:00:09 +0100 Received: from acm.muc.de (pd953a0c3.dip0.t-ipconnect.de [217.83.160.195]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 27 Mar 2024 23:00:09 +0100 Received: (qmail 9444 invoked by uid 1000); 27 Mar 2024 22:00:09 -0000 Date: Wed, 27 Mar 2024 22:00:09 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZgSW6bEMgiGdzOiz@ACM> References: <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN> <ZgL99Pp3BdrA6ycb@ACM> <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN> <ZgMuXkD07TRSe__n@ACM> <jwvcyrg3ewv.fsf-monnier+emacs@HIDDEN> <ZgOUDdqHsnd2goA4@ACM> <jwvjzln29bz.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvjzln29bz.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Wed, Mar 27, 2024 at 08:23:52 -0400, Stefan Monnier wrote: > >> > r-p-defined-s positions only lambdas and NAMEs defined by defun, > >> > defmacro, defvar, .... (around 50 defining symbols). r-p-s positions > >> > every symbol apart from nil. They have different purposes. r-p-d-s > >> > gets info for the doc strings, which requires SWPs only for some > >> > symbols. r-p-s is needed to get warning message locations. Were r-p-s > >> > used for the doc string position information, most of the symbols would > >> > need to be stripped of their positions before the form could be used. > >> > It is simpler and faster not to position them at all. > >> In terms of code, I can't see why it'd be simpler: we already have the > >> r-p-s function, .... > > We also already have r-p-d-s. > You're playing on words here: we don't "already have" `r-p-d-s` on master. I'm not playing on words. My point is that read-positioning-defined-symbols exists and works. It is not a speculative "would be nice to have". The work has already been done. > > Both functions (together with plain read) have read0 as their core > > engine. The enhancement to read0 to support r-p-d-s was only moderate > > in size and not complicated to anybody who understands finite > > state machines. > Just because it's not a complex change doesn't mean it's "simpler" than > no change at all. No change isn't an option, here. Something has to determine which symbols are to be positioned / stripped of their positions. Currently, this is done economically in r-p-d-s. > >> .... and we already have a function to strip that info when we don't > >> need it any more, so it would be less new code to write if we just > >> used r-p-s, I think. > > I think you're envisaging an extensive redesign where SWPs would not be > > tightly and individually controlled as they are at the moment, but > > instead would be created en masse and stripped en masse a bit later. > Yes, that's the starting design I had in mind (and that I described > a few emails back). it's also what we do in the byte-compilation case, ... It's not. In each case, the optimum number of symbols gets positioned. Positioning "everything" is sub-optimal for everything bar byte compilation. > so it's code we already have and use. See above. > The "en masse" doesn't make it complex. It changes the design extensively, thus introducing new complications. Why do you think this design change will be better than the existing design? > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 21:43:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 27 17:43:44 2024
Received: from localhost ([127.0.0.1]:38461 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpb3f-0007N4-G0
for submit <at> debbugs.gnu.org; Wed, 27 Mar 2024 17:43:44 -0400
Received: from mail.muc.de ([193.149.48.3]:52188)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rpb3Z-0007MG-IK
for 67455 <at> debbugs.gnu.org; Wed, 27 Mar 2024 17:43:38 -0400
Received: (qmail 43918 invoked by uid 3782); 27 Mar 2024 22:43:30 +0100
Received: from acm.muc.de (pd953a0c3.dip0.t-ipconnect.de [217.83.160.195])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Wed, 27 Mar 2024 22:43:30 +0100
Received: (qmail 9387 invoked by uid 1000); 27 Mar 2024 21:43:29 -0000
Date: Wed, 27 Mar 2024 21:43:29 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgSTAR9IpCVQ_xCC@ACM>
References: <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM> <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Wed, Mar 27, 2024 at 08:22:27 -0400, Stefan Monnier wrote:
> > More precisely, to @dfn{posify} their containing forms by writing the
> > position information into their doc strings. We do this in the byte
> > compilation case, too. The difference is that in the "load from source"
> > case we want to strip the SWP, in byte compilation, we don't.
> [ Nitpick: in byte-compilation, we also do. We want to keep the SWPs
> longer, but in the end we also want to strip them away necause we don't
> want them in the resulting compiled code. ]
In byte compilation we want to keeo the SWPs for AS LONG AS POSSIBLE.
In other cases, we want to keep them for AS SHORT A TIME AS POSSIBLE.
> > I don't think that is a good name. The byte compiler has no business
> > setting "internal" variables for the posification processing. Instead it
> > should announce it's running and expect the posification to respect that.
> > I think byte-compile-in-progress is a good name for this.
> AFAIK we want those SWPs stripped if and only if we're in a "load from
> source" case. The compilation case is one of those where we don't want
> to strip them, but it's not the only one, so the compiler should not do
> anything w.r.t to that. Instead it's the code that does the "load from
> source" which should set some indicator that stripping is requested.
I think it is better to regard the byte compilation as the special case.
Only in byte compilation do we want to preserve the SWPs on forms
getting posified.
> Also, I think as a general rule it's better for the caller to set
> a callee variable that controls how the callee behaves, rather than for the
> callee to check a caller variable to decide how to behave, because
> it's normal for the caller to "know about" its callee (after all, it's
> the caller which decides to call the callee), whereas it's not normal
> for the callee to know about specific callers (it creates undesirable
> dependencies).
Byte compilation is NOT calling loading from source. We don't have a
caller/callee relationship here. We are doing posification either from
byte compilation or from somewhere else. It is analogous to testing
lexical binding. Here, the variable is called lexical-binding; it is
not named after a particular activity to be carried out differently for
l-b and not l-b.
> >> Better yet: to avoid the problem of dynamic scope extending "too far"
> >> (i.e. accidentally applying to nested loads/evals/compile/...), you
> >> could put the relevant info into `macroexpand-all-environment`.
> >> [ That var is also dynamically bound, but we're already careful to
> >> rebind it when needed so it doesn't apply to nested uses
> >> of macroexpansion. ]
> > That variable is only loaded in the 17th loaded Lisp file. The new
> > facility should be working at the earliest stages of loading Lisp, as it
> > does at the moment.
> The earlier the better, in theory, but not at any cost.
No, the earlier the better, full stop. The earlier an error happens in
bootstrapping, the more important it is to provide debugging support.
The new facilities were exceptionally helpful whilst debugging the
partially finished code.
> Having to write all that code within the very restrictive sublanguage
> available before subr.el and backquote.el is a cost I don't think
> justifies it.
The cost has already been paid, by me. The cost of maintaining that
code will be small by comparison; byte-run--posify-defining-form is
_not_ all that difficult to understand and amend.
> If we *really* want that, then we should explore other avenues, such as
> keeping pre-macroexpanded versions of the files (for bootstrapping
> purposes) but generating those files from files where a more normal
> coding style can be used.
> [ Something similar to the ldefs-boot.el. ]
Possibly - but that will also introduce complications.
> > Besides, macroexpand-all-environment is not
> > documented anywhere, what it is, what it's for, etc.
> Feel free to disregard my advice if you don't like it.
> I'm just pointing out that it's probably the tool which gives you the
> semantics you want.
OK.
> >> My crystal ball suggests that "currently" may be the wrong way to think
> >> about it: maybe instead of thinking of "when" (as in "during the
> >> definition of function FOO") what you're looking for might be "where"
> >> (as in "within the body of FOO").
> >> [ That's the same difference as the difference between dynamic and
> >> static scoping. ]
> > I'm having trouble understanding what you're saying, here.
> Is it because you don't understand the difference between dynamic
> scoping and static scoping, or because you don't see the relationship
> with that and your notion of "currently being defined"?
The latter, I think. defining-symbol is entirely dynamically scoped.
> The above citation is in the context of my question about what you mean
> by "currently" in:
> doc: /* The symbol currently being defined by a defining form.
> I personally don't really understand it, and AFAICT, you don't really
> understand it either because you haven't been able to describe it.
I understand it fully. I'm puzzled by your failure to understand what
I've written.
The flow goes as follows:
(i) defining-symbol gets bound to nil in readevalloop_eager_eval_loop.
(ii) d-s gets setq'd to NAME in defun, defvar, cl-defgeneric, .....
Usually, the setq has been generated by
byte-run--posify-defining-form.
(iii) d-s gets used in byte-run-posify-doc-string, which writes its
details to a new or existing doc string.
By "currently", I mean that d-s holds NAME between (ii) and (iii) and
until the process of defining the new form is complete.
> >> > Ideally, I would like to have bound defining-symbol inside defun.
> >> (defmacro my-defun (name args &rest body)
> >> `(cl-macrolet ((defining-symbol () '',name))
> >> (defun ,name ,args ,@body)))
> >> (my-defun my-foo (x) (list x (defining-symbol)))
> >> (symbol-function 'my-foo)
> >> ==> #f(lambda (x) [t] (list x 'my-foo))
> >> `cl-macrolet` uses `macroexpand-all-environment` for that.
> > cl-macs gets loaded far too late for such an approach to be useful.
> That's not really relevant since we're just trying to understand what
> you mean by "currently". What is relevant is whether it gives
> the intended semantics.
I'm convinced it does. Can you suggest a scenario where the
defining-symbol mechanism (outlined above) might fail?
> But if you insist, here's the equivalent version without `cl-macs`
> (using the same underlying technique as used by `cl-macrolet`):
> (defmacro my-defun (name args &rest body)
> (macroexpand-all
> `(defun ,name ,args ,@body)
> (cons
> (cons 'defining-symbol (lambda () `',name))
> macroexpand-all-environment)))
Thanks.
> >> Do you have rough numbers comparing the cost of `read`,
> >> `read-positioning-symbols`, and `read-positioning-DEFINED-symbols`?
> > No, but they will be very close to eachother (and very cheap)
> Then I think we should use `read-positioning-symbols`, which
> requires fewer code changes.
It won't. Required would be Lisp code to determine whether a particular
SWP needs to be stripped or not. This is not going to be simple. It is
likely to be about as complicated as the existing enhancements to read0.
> >> Also, IIUC you don't have a separate phase to strip the SWPs when
> >> loading from source, but instead you strip them as you "consume" their
> >> info during macroexpansion. If so, how/when/where do you strip the
> >> false positives that may occur inside quoted data or in code like:
> >> (defmacro foo (lambda bar) ...)
> >> (defmacro foo (defun bar) ...)
> >> (let* ((lambda foo)
> >> (defun bar))
> >> ...)
> > There's a pcase arm right at the end of macroexp--expand-all which strips
> > SWPs of their positions. Recursing through macroexp--all-forms will
> > eventually hit this pcase arm for these lambdas.
> Ah, so it's like a "strip phase" but "fused" into the macroexpansion phase.
I suppose so.
> >> Not at all. Those will remain without position, but only in
> >> `src/bootstrap-emacs`.
> > This would be a Bad Thing.
> But your current code in byte-run.el is a Bad Thing as well.
What, precisely, do you find bad about it? It may be possible to improve
it without wholesale redesign.
> It's all a question of trade-offs :-(
> >> In the real `src/emacs` they will get the position because they'll come
> >> from the `.el[cn]` file and by the time we get compile those files
> >> `macro-declarations-alist` will be fully populated.
> > The understanding we reached in November was that loading from source
> > files would be handled, too.
> I'm not suggesting to drop support for lambdas loaded from source.
> I'm saying we don't need to support it for the first N files loaded into
> `src/emacs-bootstrap`.
You're suggesting dropping support for many source files, where that
support is most needed.
You're suggesting introducing awkward special cases where the code won't
work. As currently implemented, the code DOES work.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 12:24:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 27 08:24:02 2024
Received: from localhost ([127.0.0.1]:35928 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpSK2-0001eq-7L
for submit <at> debbugs.gnu.org; Wed, 27 Mar 2024 08:24:02 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:62196)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpSK0-0001eM-DD
for 67455 <at> debbugs.gnu.org; Wed, 27 Mar 2024 08:24:00 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id E8CF344123C;
Wed, 27 Mar 2024 08:23:54 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711542233;
bh=2VIMf2lKriAiylS5JyXC14jrzjhkq+nvfQfX5Kd46uk=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=nGtDcIc2QBIobor2/A/2auj3a9BOItavNvT4qq6swnwtOZFhbny1IUXv0TNVS+R+d
GXnYO/FV0dWJ1G2ALg/b3mqgzcPyZ982QoHNJ7LiaaFl9i8WuaXHWHtgnpzbJ19Jys
hDy6rX+xzAYxdyioYIvZfvBOWcNSy6eSmQc52XgrP0XzxFlrbQt+uv1gApfRjT323p
K+FKr0EPPpE4wMkqwjYU+zZLcvIQEVF++ureYrbwloNc0aS5EHoUF0nREOW2NWNb2q
GRRpeoa0yv5/Dk3TadDXjumvYu5Ujx/Gc/ZSZR3ZKR0OL87chmgcFfG7Emriv7ZHu9
Hb49UdQ31do2Q==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 84B36440175;
Wed, 27 Mar 2024 08:23:53 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 618991201BE;
Wed, 27 Mar 2024 08:23:53 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgOUDdqHsnd2goA4@ACM> (Alan Mackenzie's message of "Wed, 27 Mar
2024 03:35:41 +0000")
Message-ID: <jwvjzln29bz.fsf-monnier+emacs@HIDDEN>
References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
<ZgL99Pp3BdrA6ycb@ACM> <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN>
<ZgMuXkD07TRSe__n@ACM> <jwvcyrg3ewv.fsf-monnier+emacs@HIDDEN>
<ZgOUDdqHsnd2goA4@ACM>
Date: Wed, 27 Mar 2024 08:23:52 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.431 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> > r-p-defined-s positions only lambdas and NAMEs defined by defun,
>> > defmacro, defvar, .... (around 50 defining symbols). r-p-s positions
>> > every symbol apart from nil. They have different purposes. r-p-d-s
>> > gets info for the doc strings, which requires SWPs only for some
>> > symbols. r-p-s is needed to get warning message locations. Were r-p-s
>> > used for the doc string position information, most of the symbols would
>> > need to be stripped of their positions before the form could be used.
>> > It is simpler and faster not to position them at all.
>> In terms of code, I can't see why it'd be simpler: we already have the
>> r-p-s function, ....
> We also already have r-p-d-s.
You're playing on words here: we don't "already have" `r-p-d-s` on master.
> Both functions (together with plain read) have read0 as their core
> engine. The enhancement to read0 to support r-p-d-s was only moderate
> in size and not complicated to anybody who understands finite
> state machines.
Just because it's not a complex change doesn't mean it's "simpler" than
no change at all.
>> .... and we already have a function to strip that info when we don't
>> need it any more, so it would be less new code to write if we just
>> used r-p-s, I think.
> I think you're envisaging an extensive redesign where SWPs would not be
> tightly and individually controlled as they are at the moment, but
> instead would be created en masse and stripped en masse a bit later.
Yes, that's the starting design I had in mind (and that I described
a few emails back). it's also what we do in the byte-compilation case,
so it's code we already have and use.
The "en masse" doesn't make it complex.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 12:22:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 27 08:22:43 2024
Received: from localhost ([127.0.0.1]:35919 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpSIk-0001am-GW
for submit <at> debbugs.gnu.org; Wed, 27 Mar 2024 08:22:43 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:1122)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpSIg-0001aT-Sp
for 67455 <at> debbugs.gnu.org; Wed, 27 Mar 2024 08:22:40 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 5DBC744122C;
Wed, 27 Mar 2024 08:22:32 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711542150;
bh=MtRMs6iilx23Cs1GZOHrslJRqmsvKsGhH4pbPr92Q9o=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=CFmlj53hlHbYoWhEraJ9VukQhG7BRNUqeRt+v7guvO7L8zyys3wORrhuM1MqV4BE1
LNQ1Ue7QaCQm/PmnbfXld2ehbF7AdMc493SDfJmozQdjOjeGjZUhC6GX3p07tVg/YB
bBXiI2Bg4nzTCOTUe2dG6zf9pibdo1fLilgy6yj8enaXeFgRRYwfwjTLxEGNvjE4Ka
j1O2WThuGBoIjBMN46A2V5JnapehFCwxRSpbH1ZRVQHoJT3ZVQ/uQv63YKcDwEkeg1
jPx3Eh4FNbOoG1jYPOVmNeZggoXmgJsuazNkZkfDzp/90xHVrxPhoPAKD9cEjp0Tkm
eSaJkumYTGjCw==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id F25EB44121A;
Wed, 27 Mar 2024 08:22:29 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BDAAF1203BC;
Wed, 27 Mar 2024 08:22:29 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgPvE96-OisH-g8S@ACM> (Alan Mackenzie's message of "Wed, 27 Mar
2024 10:04:03 +0000")
Message-ID: <jwvfrwb293o.fsf-monnier+emacs@HIDDEN>
References: <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
<ZgPvE96-OisH-g8S@ACM>
Date: Wed, 27 Mar 2024 08:22:27 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.575 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> More precisely, to @dfn{posify} their containing forms by writing the
> position information into their doc strings. We do this in the byte
> compilation case, too. The difference is that in the "load from source"
> case we want to strip the SWP, in byte compilation, we don't.
[ Nitpick: in byte-compilation, we also do. We want to keep the SWPs
longer, but in the end we also want to strip them away necause we don't
want them in the resulting compiled code. ]
> I don't think that is a good name. The byte compiler has no business
> setting "internal" variables for the posification processing. Instead it
> should announce it's running and expect the posification to respect that.
> I think byte-compile-in-progress is a good name for this.
AFAIK we want those SWPs stripped if and only if we're in a "load from
source" case. The compilation case is one of those where we don't want
to strip them, but it's not the only one, so the compiler should not do
anything w.r.t to that. Instead it's the code that does the "load from
source" which should set some indicator that stripping is requested.
Also, I think as a general rule it's better for the caller to set
a callee variable that controls how the callee behaves, rather than for the
callee to check a caller variable to decide how to behave, because
it's normal for the caller to "know about" its callee (after all, it's
the caller which decides to call the callee), whereas it's not normal
for the callee to know about specific callers (it creates undesirable
dependencies).
>> Better yet: to avoid the problem of dynamic scope extending "too far"
>> (i.e. accidentally applying to nested loads/evals/compile/...), you
>> could put the relevant info into `macroexpand-all-environment`.
>> [ That var is also dynamically bound, but we're already careful to
>> rebind it when needed so it doesn't apply to nested uses
>> of macroexpansion. ]
>
> That variable is only loaded in the 17th loaded Lisp file. The new
> facility should be working at the earliest stages of loading Lisp, as it
> does at the moment.
The earlier the better, in theory, but not at any cost. Having to write
all that code within the very restrictive sublanguage available before
subr.el and backquote.el is a cost I don't think justifies it.
If we *really* want that, then we should explore other avenues, such as
keeping pre-macroexpanded versions of the files (for bootstrapping
purposes) but generating those files from files where a more normal
coding style can be used.
[ Something similar to the ldefs-boot.el. ]
> Besides, macroexpand-all-environment is not
> documented anywhere, what it is, what it's for, etc.
Feel free to disregard my advice if you don't like it.
I'm just pointing out that it's probably the tool which gives you the
semantics you want.
>> My crystal ball suggests that "currently" may be the wrong way to think
>> about it: maybe instead of thinking of "when" (as in "during the
>> definition of function FOO") what you're looking for might be "where"
>> (as in "within the body of FOO").
>> [ That's the same difference as the difference between dynamic and
>> static scoping. ]
> I'm having trouble understanding what you're saying, here.
Is it because you don't understand the difference between dynamic
scoping and static scoping, or because you don't see the relationship
with that and your notion of "currently being defined"?
The above citation is in the context of my question about what you mean
by "currently" in:
doc: /* The symbol currently being defined by a defining form.
I personally don't really understand it, and AFAICT, you don't really
understand it either because you haven't been able to describe it.
>> > Ideally, I would like to have bound defining-symbol inside defun.
>
>> (defmacro my-defun (name args &rest body)
>> `(cl-macrolet ((defining-symbol () '',name))
>> (defun ,name ,args ,@body)))
>
>> (my-defun my-foo (x) (list x (defining-symbol)))
>
>> (symbol-function 'my-foo)
>> ==> #f(lambda (x) [t] (list x 'my-foo))
>
>> `cl-macrolet` uses `macroexpand-all-environment` for that.
>
> cl-macs gets loaded far too late for such an approach to be useful.
That's not really relevant since we're just trying to understand what
you mean by "currently". What is relevant is whether it gives
the intended semantics.
But if you insist, here's the equivalent version without `cl-macs`
(using the same underlying technique as used by `cl-macrolet`):
(defmacro my-defun (name args &rest body)
(macroexpand-all
`(defun ,name ,args ,@body)
(cons
(cons 'defining-symbol (lambda () `',name))
macroexpand-all-environment)))
>> Do you have rough numbers comparing the cost of `read`,
>> `read-positioning-symbols`, and `read-positioning-DEFINED-symbols`?
> No, but they will be very close to eachother (and very cheap)
Then I think we should use `read-positioning-symbols`, which
requires fewer code changes.
>> Also, IIUC you don't have a separate phase to strip the SWPs when
>> loading from source, but instead you strip them as you "consume" their
>> info during macroexpansion. If so, how/when/where do you strip the
>> false positives that may occur inside quoted data or in code like:
>
>> (defmacro foo (lambda bar) ...)
>> (defmacro foo (defun bar) ...)
>
>> (let* ((lambda foo)
>> (defun bar))
>> ...)
>
> There's a pcase arm right at the end of macroexp--expand-all which strips
> SWPs of their positions. Recursing through macroexp--all-forms will
> eventually hit this pcase arm for these lambdas.
Ah, so it's like a "strip phase" but "fused" into the macroexpansion phase.
>> Not at all. Those will remain without position, but only in
>> `src/bootstrap-emacs`.
> This would be a Bad Thing.
But your current code in byte-run.el is a Bad Thing as well.
It's all a question of trade-offs :-(
>> In the real `src/emacs` they will get the position because they'll come
>> from the `.el[cn]` file and by the time we get compile those files
>> `macro-declarations-alist` will be fully populated.
> The understanding we reached in November was that loading from source
> files would be handled, too.
I'm not suggesting to drop support for lambdas loaded from source.
I'm saying we don't need to support it for the first N files loaded into
`src/emacs-bootstrap`.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 10:04:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 27 06:04:19 2024
Received: from localhost ([127.0.0.1]:35828 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpQ8o-0000Ye-JE
for submit <at> debbugs.gnu.org; Wed, 27 Mar 2024 06:04:19 -0400
Received: from mail.muc.de ([193.149.48.3]:34923)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rpQ8i-0000Xq-HD
for 67455 <at> debbugs.gnu.org; Wed, 27 Mar 2024 06:04:17 -0400
Received: (qmail 52467 invoked by uid 3782); 27 Mar 2024 11:04:05 +0100
Received: from acm.muc.de (pd953a0c3.dip0.t-ipconnect.de [217.83.160.195])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Wed, 27 Mar 2024 11:04:05 +0100
Received: (qmail 20666 invoked by uid 1000); 27 Mar 2024 10:04:03 -0000
Date: Wed, 27 Mar 2024 10:04:03 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgPvE96-OisH-g8S@ACM>
References: <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Tue, Mar 26, 2024 at 16:30:06 -0400, Stefan Monnier wrote:
> > We now have two distinct uses of SWPs: providing warning source locations
> > to the compiler (where we want to keep the position as long as possible)
> > and providing position information for the doc string (where we want to
> > strip the position from the symbol ASAP, to avoid trying to use the SWP
> > when we need a plain symbol). If both of these occur together, we want
> > to keep the SWP.
> I think I'm beginning to understand. So in the "load from source case",
> some of your symbols are SWPs and you want to turn them into bare
> symbols "on the fly" during macro-expansion rather than via a separate
> "strip" phase, ....
More precisely, to @dfn{posify} their containing forms by writing the
position information into their doc strings. We do this in the byte
compilation case, too. The difference is that in the "load from source"
case we want to strip the SWP, in byte compilation, we don't.
> .... so you want the macro expansion to know whether it's done
> for "load from source" or for some other purpose and you use
> `byte-compile-in-progress` as a proxy for that information.
> Is that it?
More or less. But byte-compile-in-progress isn't a proxy, it's the prime
criterion for deciding.
> If so, (and if the stripping happens within macros), then indeed passing
> it as a separate argument through all the recursive calls to
> `macroexp--expand-all` would be cumbersome. But I suggest you use
> another name for that(e.g. `macroexp-strip-position`) so the
> intention is made more clear.
I don't think that is a good name. The byte compiler has no business
setting "internal" variables for the posification processing. Instead it
should announce it's running and expect the posification to respect that.
I think byte-compile-in-progress is a good name for this.
> Better yet: to avoid the problem of dynamic scope extending "too far"
> (i.e. accidentally applying to nested loads/evals/compile/...), you
> could put the relevant info into `macroexpand-all-environment`.
> [ That var is also dynamically bound, but we're already careful to
> rebind it when needed so it doesn't apply to nested uses
> of macroexpansion. ]
That variable is only loaded in the 17th loaded Lisp file. The new
facility should be working at the earliest stages of loading Lisp, as it
does at the moment. Besides, macroexpand-all-environment is not
documented anywhere, what it is, what it's for, etc.
> >> > By "currently", I mean that a defining form such as defun or defvar has
> >> > commenced, but not yet terminated; its functions currently occupy stack
> >> > frames.
> >> So you mean we're inside `Fdefalias` or `Fdefvar_1`?
> > Yes, or inside a macro (defun, defmacro, ...) which expands to a
> > defalias.
> These are *very* different times: `Fdefalias` and `Fdefvar_1` are
> executed long after macroexpansion. And they're small C functions which
> run almost no external code at all, so they "occupy stack frames" only
> for a very short time.
I think we were talking about the handling of defining-symbol. It has a
valid binding outside of macros such as defun, and this binding is used
to posify the containing form.
> My crystal ball suggests that "currently" may be the wrong way to think
> about it: maybe instead of thinking of "when" (as in "during the
> definition of function FOO") what you're looking for might be "where"
> (as in "within the body of FOO").
> [ That's the same difference as the difference between dynamic and
> static scoping. ]
I'm having trouble understanding what you're saying, here.
> If my crystal ball is right, then the better place to put that info is
> probably `macroexpand-all-environment`.
See above.
> > Ideally, I would like to have bound defining-symbol inside defun.
> (defmacro my-defun (name args &rest body)
> `(cl-macrolet ((defining-symbol () '',name))
> (defun ,name ,args ,@body)))
> (my-defun my-foo (x) (list x (defining-symbol)))
> (symbol-function 'my-foo)
> ==> #f(lambda (x) [t] (list x 'my-foo))
> `cl-macrolet` uses `macroexpand-all-environment` for that.
cl-macs gets loaded far too late for such an approach to be useful.
> > Fload uses read-positioning-DEFINED-symbols, as contrasted with the
> > compiler, which uses read-positioning-symbols. r-p-d-s positions only
> > lambdas and NAMEs. r-p-s positions all symbols except nil.
> I think I'm beginning to understand (I guess I was struggling with your
> use of "position" as a verb for some reason which made me think that
> symbols were being moved rather than gaining position information).
Sorry about that.
> So, IIUC you use `read-positioning-DEFINED-symbols` instead of
> `read-positioning-symbols` because it's cheaper?
No, because it does the Right Thing.
> Do you have rough numbers comparing the cost of `read`,
> `read-positioning-symbols`, and `read-positioning-DEFINED-symbols`?
No, but they will be very close to eachother (and very cheap) since they
use the same engine, read0 (in lread.c). Each of them will be one or two
orders of magnitude faster than emulating them in Lisp.
> Also, IIUC you don't have a separate phase to strip the SWPs when
> loading from source, but instead you strip them as you "consume" their
> info during macroexpansion. If so, how/when/where do you strip the
> false positives that may occur inside quoted data or in code like:
> (defmacro foo (lambda bar) ...)
> (defmacro foo (defun bar) ...)
> (let* ((lambda foo)
> (defun bar))
> ...)
There's a pcase arm right at the end of macroexp--expand-all which strips
SWPs of their positions. Recursing through macroexp--all-forms will
eventually hit this pcase arm for these lambdas.
> > Ah, right. I hadn't considered this before. The changes are by their
> > very nature essentially complicated and difficult to understand.
> [ Hmm... maybe not the best salespitch. ]
;-) It's the truth, though.
> >> I think you can simply wait to add the entry to
> >> `macro-declarations-alist` until a later time, so the `defining-symbol`
> >> thingies will be ignored during the early bootstrap and once we have
> >> more infrastructure in place we can then register the handler on
> >> `macro-declarations-alist`.
> > This will not be simpler. It would involve re-evaluating defun, then
> > compensating for all the functions up to now whose NAMEs had been read
> > without positions.
> Not at all. Those will remain without position, but only in
> `src/bootstrap-emacs`.
This would be a Bad Thing. The current code is active right after
loading byte-run.
> In the real `src/emacs` they will get the position because they'll come
> from the `.el[cn]` file and by the time we get compile those files
> `macro-declarations-alist` will be fully populated.
The understanding we reached in November was that loading from source
files would be handled, too.
> > There is unavoidable conplexity, here.
> I'm definitely not convinced. I suspect you've been asking yourself
> "can it be made simpler" and you may indeed then convince yourself that
> the answer is no, because of assumptions you don't reconsider.
> Try instead to think about "what would it take to remove that complexity?".
> > I see things somewhat differently. We shouldn't increase the debugging
> > burden even on "expert users".
> Yet it's imposing more complexity (and hence more debugging burden) on
> those same expert users. 🙁
That's a fairly difficult philosophical question. Do we provide full
functionality at the cost of more (difficult) source code, or do we
restrict the functioality to keep the source simpler? I think with Emacs
we usually go with the first alternative.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 03:35:54 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 23:35:54 2024 Received: from localhost ([127.0.0.1]:35492 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rpK4v-0003lu-Q2 for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 23:35:54 -0400 Received: from mail.muc.de ([193.149.48.3]:46607) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rpK4q-0003lF-C0 for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 23:35:52 -0400 Received: (qmail 13490 invoked by uid 3782); 27 Mar 2024 04:35:42 +0100 Received: from acm.muc.de (pd953a0c3.dip0.t-ipconnect.de [217.83.160.195]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 27 Mar 2024 04:35:42 +0100 Received: (qmail 7587 invoked by uid 1000); 27 Mar 2024 03:35:41 -0000 Date: Wed, 27 Mar 2024 03:35:41 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZgOUDdqHsnd2goA4@ACM> References: <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN> <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN> <ZgL99Pp3BdrA6ycb@ACM> <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN> <ZgMuXkD07TRSe__n@ACM> <jwvcyrg3ewv.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvcyrg3ewv.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Tue, Mar 26, 2024 at 16:42:28 -0400, Stefan Monnier wrote: > > r-p-defined-s positions only lambdas and NAMEs defined by defun, > > defmacro, defvar, .... (around 50 defining symbols). r-p-s positions > > every symbol apart from nil. They have different purposes. r-p-d-s > > gets info for the doc strings, which requires SWPs only for some > > symbols. r-p-s is needed to get warning message locations. Were r-p-s > > used for the doc string position information, most of the symbols would > > need to be stripped of their positions before the form could be used. > > It is simpler and faster not to position them at all. > In terms of code, I can't see why it'd be simpler: we already have the > r-p-s function, .... We also already have r-p-d-s. Both functions (together with plain read) have read0 as their core engine. The enhancement to read0 to support r-p-d-s was only moderate in size and not complicated to anybody who understands finite state machines. > .... and we already have a function to strip that info when we don't > need it any more, so it would be less new code to write if we just > used r-p-s, I think. I think you're envisaging an extensive redesign where SWPs would not be tightly and individually controlled as they are at the moment, but instead would be created en masse and stripped en masse a bit later. There seems to me to be little justification for this. It would need more code in byte-run.el, and would be slower, too. It would take a lot of work to implement and debug. At the moment, some SWPs (to do with "complicated" backquoted lambda forms) survive long term, meaning we would need complicated Lisp code to protect these individually. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 21:14:05 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 17:14:05 2024 Received: from localhost ([127.0.0.1]:35275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rpE7Q-0008HU-PS for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 17:14:05 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:34316) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <drew.adams@HIDDEN>) id 1rpE7N-0008Gx-JV for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 17:14:02 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 42QKUkpY028582; Tue, 26 Mar 2024 21:14:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2023-11-20; bh=1MhGNHCuhnngXrF4z1hUHyg06w6TRLzzNFyogIvY8Jk=; b=Dhwrsn8teGQnvwICU0HgduSvu5CVBxSCYEKidjmsLHGWOZJl95aZNN6QtJB+9G9ifSv0 ajSKt7Szg5dNJqe6XRCKjT6x+qKvfXL/Xdd0RnYzfMMPXUjxoSMktQiO7wDsFGy+nf6y m5gGYvUoDPmdLlfgxaVZvIZ8CK81R631wJw+ZD0QKYtgzeO/GqzVqCUkkvbnBlMwdtBk c2deFUIpB3YiRYyFJtWuDoMWjRBJppTIBBPowjCY6jTJI50eclwpUHwFGREfRBU9u84z mi2Jw9ao1bhV5GJq5gKSiGN9ceN/HEyXwlNc4MVGk8qMzHDr1eKFX1YO5SwaPJoLfsJh fA== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3x1pybp20q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 Mar 2024 21:14:00 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 42QLBHfF011614; Tue, 26 Mar 2024 21:13:59 GMT Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2168.outbound.protection.outlook.com [104.47.59.168]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3x1nh7mkth-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 Mar 2024 21:13:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DdI2Go9Ix6EtoClewxpdt3c873eJEHxTxY4Ik1+/s97oR+MkLJ1gYWXarCYGA6HmVqktTuzJqa9wS4vwVvvMg5ZlrF5z8k+C+8dQAawF1AacpFZ8+Hep+9GHms1erw+GzHpzSIFKhY0/gtTvT0md3qPlrQBRINjk50VLSYo+wgrNpXt0iqCcoJkfRDiRWwGdkFlcVbsRKH2rYVnjB6A0U65gHVOWgIQ6isCeLmcC4HbZfQXXimO+iSjXXbfjoPcuKCmPiTbkpavLv70FseCmmDXEhDdSIvjYSROWoisckLjkAwBTzZ6Vb3FMyWW6P4t93wy9HYTEiGrNoajgiom40w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1MhGNHCuhnngXrF4z1hUHyg06w6TRLzzNFyogIvY8Jk=; b=cVZNGqmCvQ4loeFi6AOp3oXOemld43RDBoVV+T2N4An4A3NFhfTWxk3NpLzhT6/nb6Wbd7l5K6zF8kOvIyL1magINe20vbUyAqgIkwAo2Zeb++u24XBafTHt1BbKDVJuOLZacuzxZ8t0sb14xr2Nxv2CPC4QUhl0s+Ti89MXiP4WpN97sQA/0cf0qk4tm/IqznD6fTPoDgsnrdnANKxWgmRPCYv+EsIDdI7Ut6bU+e/8OuxtxluVJZw7sQP3iP/2qO++N6FC53tz/0Iggrou3LIeB+O7fIbnWtKFclU8Jxz0f1c3sy6Vg26vpHjmsyeM3iOI3OVe2cj9njZkhjYuwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1MhGNHCuhnngXrF4z1hUHyg06w6TRLzzNFyogIvY8Jk=; b=EgfY41wSG72iO3MjiUx83XjzFuW6mvqngCX6tzmjmldi0Oz7t08oi41NtMBS2SiEEQ9tRFTTWCETUHlWpiAXocrPdfyy4547dElET4Zdk1SnIyeQo9TanEzUEfwhlX2aQ9VYgQa1BqnZ97oisgMllyPhjQZ0Ih+PRj8wVzOTv2s= Received: from SJ0PR10MB5488.namprd10.prod.outlook.com (2603:10b6:a03:37e::19) by PH7PR10MB5747.namprd10.prod.outlook.com (2603:10b6:510:127::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.31; Tue, 26 Mar 2024 21:13:57 +0000 Received: from SJ0PR10MB5488.namprd10.prod.outlook.com ([fe80::d9bc:c5bb:7fc4:cf9f]) by SJ0PR10MB5488.namprd10.prod.outlook.com ([fe80::d9bc:c5bb:7fc4:cf9f%6]) with mapi id 15.20.7409.031; Tue, 26 Mar 2024 21:13:57 +0000 From: Drew Adams <drew.adams@HIDDEN> To: Stefan Monnier <monnier@HIDDEN>, Alan Mackenzie <acm@HIDDEN> Subject: RE: [External] : bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Thread-Topic: [External] : bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Thread-Index: AQHaf7yg8TMbs6I9p0uWaHeqsMOAkLFKhNqA Date: Tue, 26 Mar 2024 21:13:57 +0000 Message-ID: <SJ0PR10MB548804AC9AB1F4071FA57D3EF3352@HIDDEN> References: <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN> <ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN> <ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN> <ZgKaAUMWzLXv6Alr@ACM> <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> In-Reply-To: <jwvttks3hpj.fsf-monnier+emacs@HIDDEN> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR10MB5488:EE_|PH7PR10MB5747:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6JCQ0PUPG4uFY5ODS3xRsvbv10wOcUVaGEpoglv3e2E5gAxypxUcG9j7gRdm6qXmDu8OvXCpWblAGs/sT09U0ZUaeHUZnGGEw7O2CwbEUaK1tCWcmPBuQ3Y/efeqxGRf7fuXoGaUNIrcuA1Zyc9ucXlmqSHbj2cLmBO+PU72Qwm99SgTt1OKcpj5AD8pV6pF2olLS/gio7IerltHkB+RuUJ/OxxY/w4wwg2HX8FZ9snzxlImulc09xuqmHZxWmXK73zvcp9SUw325t3Ejy0RXSWaneZ/4CyuQH8LtWe+Y0wEUMA2tIxtQKqVAiU5xV8VX4d7zi2azX5b1T0Wx39I+1bSd2uefhYioL+3by5oBgPUzmBz+XGdLI/lCujQT854DFLZ4sGE9r7YzThMsrV4mx3nuwG6NBxNGAo9WwayPVUMj/qKSXmB8NPR7IaaO6vuGQ7g/yBDeCY2S+ZAPdWkLWfh1MpPqjIf3Dz9Z0sY258RF9Mdzt40xDlnR6ePlB8GE9ERhxx6czGMdOiD1MK5OeQ/cobEsoQzA3N5pKti8iQZrDkr3PRXzJICWYpAZ1mWPhVMvrXmhW3dITRn/GDhqPirzFCeWUOP+v+oK7Eg6ps12WfWsNjpcLsWmBmg6MJAZSiW3iicBORcewnrkIKR5wztZlt8G6dJM+kxlrO1wcA= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR10MB5488.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(376005)(1800799015); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?b3Fta2lsbUxpRXNoTzVNcTlEV3dHSGxEQUpRenQvNEswVzZTdVBuQTM1bmhH?= =?utf-8?B?ZkhxZ0tJb0dsRml6U0h1N0hlclVNVmE3cXRVa2ErZnVzUEw2ekUyMjVJQm5q?= =?utf-8?B?NVJyM2pIRVF6a09vL2k4U0t6VDV2cTNBOVlBSXlzT0hnZ2NpUEc3eXlmcWxV?= =?utf-8?B?cy8xQkdxWE5Za0pLK0ttQ05lTjRFQVNVTUFkRFR1UGIwSHZxMDZIQTRzQ1BI?= =?utf-8?B?UkZkZ240Y0x3TkJhVUdqNmlJWmxPQXIvOHN2U3krQnBUM2lIU09yTnExRVFr?= =?utf-8?B?WWpLbHFMUmx6YVpOa29XSlZXTTAzc2U0aU9Wei8wT3lBbjJWZEFpWjBCcFVP?= =?utf-8?B?QWpvZmNsdzVLbDlqWnQyVW83aWtPK05sbjNVOEJUNkM2SW1DaHcxSzdPcjVN?= =?utf-8?B?TGt5SHROYWU4eHc2cVlaSEFtcG1Rc1IrWWErRGZqZDBKU2dIZlg2RFRhWXJ5?= =?utf-8?B?d0lodE9CaUMxUmxSblE5RU5YakdVMytWb0ROZzhGWHlWK1FpK3ppbVk0aFFN?= =?utf-8?B?MmNlVnhBY3NLZ2QzeCs3blVOWGhnTlNnN3NETkxDUDhaNVh5YldabGw1bTFD?= =?utf-8?B?MWMyV1BEckVsUVBBVFd0MktMTW1RQzNDNTNVMG5XL3d0bXNwWHc4dEF0b2tK?= =?utf-8?B?Z0JtUWZNQjRrdW5mb3RVRThybU1LQTEwZS9vY21UNnl6TGlNajRmRGZWMGhn?= =?utf-8?B?ejVESVhxeGZvSnJyRjZvaVh3UXgxOE41b2hjVmtiSmE5cjF6MVN5N1NmM1FB?= =?utf-8?B?SXVuSGhRZ21yUENobzVNRkJHL0pqR0xHVExHT1NOZVFMNmNoZnB3eTBEMUJm?= =?utf-8?B?MGE0VHZ5a3IxaUR6REpIYmREQmF4TkZhY1czUEFsQVh1bUE3RGlaenV3Mjgy?= =?utf-8?B?RW1SeTRwVm5sbFNmS2VRTEUrWWZXOUo5UWpFTTZJbklFV3l1UHNwczFHOG1m?= =?utf-8?B?YXNwUlkyT1NVRkgxSGpRR2tpVCtFY1owUExhSDkySzJqSGZ0cU94UjdaRnZm?= =?utf-8?B?NWhSazJBVHdHNlF6QjIzWWlpNjdDZnhjUmt0dXh1enBEQS9oeVU3dTc0bVUx?= =?utf-8?B?RjlMK3RYdVpwYS9ycXhYOXN6TzhUZWV0YTduTGlHUUY4bkYxa2xmNDQ3VGs5?= =?utf-8?B?ZThqaHVkdnJhMno5c1NmaHhjemZkaWlPem9jVWwyL3BrU1pIdzgvNkZXTE93?= =?utf-8?B?eTJDczFBZXdxOXdFOFpKSkdrS2lHanNRYWsvUnVPcmxoeXFaNmhQOGw4QnQw?= =?utf-8?B?dEpMSkZETHg3VmZqNXJGblhkV0MyS3RCU1JXOUVBRm1UWW12UnQ0a3B4MlN6?= =?utf-8?B?K0J6Z1hYUzhGcTZWVzdnVlVQemlKN01MVjBTVGhpWjlFQXpWeWw1WDlkNUNB?= =?utf-8?B?d2t3N1NaM0hoSmZDNFhHeVBKcllWNzdBNE80T2F2eWw4Zi92VXpjZ0g3VTBq?= =?utf-8?B?cjBMOHprYzAwWDlwRnU0Sm8zV3dVV0hINlR2UFcxZDljdUZYK1FTbXBLSFcx?= =?utf-8?B?aVBnVEZTOWRaSFgzT2h4cWQ4anExRWphVHRXQmdBRENIQWRuMlRFM1V5d3B5?= =?utf-8?B?czZiUVgvTStnK0FzWm0yWnFEWGQ0RHJWVXVCZVJXUlVHa0NkM2xuaFh5TDNp?= =?utf-8?B?MnVBVU96MStMYUtJWEpZdkZueWg0UWd2SHBNNkVidkNkanFtTE8rMjd1Vzlj?= =?utf-8?B?ZVlXd05icG5UYVloNTNseGVLdC90UjZIaGhoNURwWXlVdTltK3NkeGJVR1BY?= =?utf-8?B?SmlaZ0FyM0pKOTJ3K2RCSGVKTVRnMDFYdWRZK2FjbldoYjJVM28veUtDTGdY?= =?utf-8?B?YjRCb1Q1VE1BUDIvVkpibWZpZDlpdUFZMW9wVnBYc2N1Nmo4OCtxM2JHamhO?= =?utf-8?B?TmlQc2RualgyOFZ6ajZySWFDbmZIZDhzcGxkUWQ2aDkwTkEvTFQ0K1cvamdh?= =?utf-8?B?dDhiRWJpV0dEbmNRVE5KNnUvQXNLWE9zcnVVMzAxYzlSZmpPVkdaYkQ2R0c1?= =?utf-8?B?REMweWdxM29LQlFjZzlTZk5mRzl6T2JTeUlKOVpMeWNZcytvbklSQ2pjd1Ny?= =?utf-8?B?ZndXdmo0Rm00SGk5d21MU1dqbFBBaGFpVFM1SEEvanVveXR5dzlNMkhFenJl?= =?utf-8?Q?Zu27VdwyCrNkIdLiqmGIN10pE?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: NmXAcx5sLlVgL93/Vf/Z/aKDiXWb5jR3sYH4FzIIL/Oo0zve0byTanpO/wF/HvdDCiAO+I58fX3Fa/0qCwFimSd9p3eVe+VdAmn6KidrD47a+IKC+v/1S/IKIPaHWd4lVLNzaOquCGyDEqNOeDtuRqf2qBLIP6Q8drCpd7MrPRMFDzIdroQSIFdf3y3wm6kO1YS6Hg8eCq9jBs36QjpwyZ/97hFuCJmRa0SjTScee95KkO+9XBHsomvxvV8vcQbBkjEUWO6cmqKPVvIOGFskOvXir5oU8t0Javuin62f2wMtTlOK+wWNHat9bpTf4SshsAPaCVrbozYCLre/7BstizoQUTEeFqYRHO6KKaBvcywxknjM47pZw7rTDsKbfiQY+aSPF4ZN+uw02ir6IEdBe3lBp6zpde0BYOHTFRmTjVIy3WtZPZLFVEEv+mGpaoV9mjP5puwCMHlGBDuGOpDPhgzEDj6ucH+YEpT7Z8wJVjSdgylaXcGiLUOYGpd/TWWebScwBSb1NIECxJwsoojcmO63bWKpbvkghCdOUTk7VH8av5WfBc17z/jFO61kPVqwHnWLz5MuSbRfkOt1eWmbS19PiGdUs0K0GnDv4H4ftYY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB5488.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 37a07bb5-d164-4830-7888-08dc4dd9a623 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Mar 2024 21:13:57.0190 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: A6X7cvZK/HShxrekGY6aqiSTQHF3u3W1wz1Kp91R6jpALKdJ8G3TZWnTgPyl5KHARWSQEvhNSsvtpdPbMCHE4Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB5747 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-26_08,2024-03-21_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 phishscore=0 mlxscore=0 mlxlogscore=891 spamscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2403210000 definitions=main-2403260153 X-Proofpoint-GUID: NTCqBm0WAP6hN-KqdaY_Jgl6eVY2qmgQ X-Proofpoint-ORIG-GUID: NTCqBm0WAP6hN-KqdaY_Jgl6eVY2qmgQ X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: Eli Zaretskii <eliz@HIDDEN>, "67455 <at> debbugs.gnu.org" <67455 <at> debbugs.gnu.org> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) PiBJIHRoaW5rIEknbSBiZWdpbm5pbmcgdG8gdW5kZXJzdGFuZCAoSSBndWVzcyBJIHdhcyBzdHJ1 Z2dsaW5nIHdpdGggeW91cg0KPiB1c2Ugb2YgInBvc2l0aW9uIiBhcyBhIHZlcmIgZm9yIHNvbWUg cmVhc29uIHdoaWNoIG1hZGUgbWUgdGhpbmsgdGhhdA0KPiBzeW1ib2xzIHdlcmUgYmVpbmcgbW92 ZWQgcmF0aGVyIHRoYW4gZ2FpbmluZyBwb3NpdGlvbiBpbmZvcm1hdGlvbikuDQoNCk5vdCBmb2xs b3dpbmcgdGhpcyB0aHJlYWQgKGF0IGFsbCkuDQoNCkkgc3VnZ2VzdCB0aGF0IGZvciB0aGUgbmV3 IG1lYW5pbmcgKCJnYWluaW5nIHBvc2l0aW9uIGluZm8iKSB5b3UgdXNlIHRoZSAoanVzdCBtaW50 ZWQpDQp2ZXJiICJwb3NpdGlvbml6ZSIgaW5zdGVhZCBvZiAicG9zaXRpb24iLg0KDQpPciBzb21l IG90aGVyIGNoYW5nZSBpbiB2b2NhYnVsYXJ5Lg0K
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 20:42:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 16:42:38 2024
Received: from localhost ([127.0.0.1]:35184 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpDcz-0006Uk-SM
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 16:42:38 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:43645)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpDcx-0006UW-GS
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 16:42:36 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 2152144301B;
Tue, 26 Mar 2024 16:42:30 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711485748;
bh=2DvHqzO5mrIDfpcmkImvtambSDrx2k+GQhqCMSTBWwg=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=UatEy8q+jCR6s3H6bmzAsoX/b/ATVsy8KxHynMPgP1apW5AavFvYyMV44bU1wEmJb
qOtIePDtCpYM9wjz6aBYwNfbRI/HM4SF6q79tgQkqSKzOTY/RiSOJuvjCzfwBu5PqW
8Qb1Yo6OsaV8RnpVnyE1zOLqQ3tZCAS1Z1DWu12slshBc1pWVq7gTidV0SPiA/2C8g
P1WorSGE8NlaLbTnkNhCqi9na9kS/jd1AN9lcNoPqA8Q3ImZPajW3YkilfqZIsE8cv
ivzspvPoYZHx3fsUn3HPkJgFlq8Hb7ItXNtDCS69r38uC0gyiXAj9GH2z1BA9dvEWu
CwF6SLr3JpHrA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C62D5443016;
Tue, 26 Mar 2024 16:42:28 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9D1A31205FB;
Tue, 26 Mar 2024 16:42:28 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgMuXkD07TRSe__n@ACM> (Alan Mackenzie's message of "Tue, 26 Mar
2024 20:21:50 +0000")
Message-ID: <jwvcyrg3ewv.fsf-monnier+emacs@HIDDEN>
References: <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
<ZgL99Pp3BdrA6ycb@ACM> <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN>
<ZgMuXkD07TRSe__n@ACM>
Date: Tue, 26 Mar 2024 16:42:28 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.863 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> r-p-defined-s positions only lambdas and NAMEs defined by defun,
> defmacro, defvar, .... (around 50 defining symbols). r-p-s positions
> every symbol apart from nil. They have different purposes. r-p-d-s
> gets info for the doc strings, which requires SWPs only for some
> symbols. r-p-s is needed to get warning message locations. Were r-p-s
> used for the doc string position information, most of the symbols would
> need to be stripped of their positions before the form could be used.
> It is simpler and faster not to position them at all.
In terms of code, I can't see why it'd be simpler: we already have the
r-p-s function, and we already have a function to strip that info when we
don't need it any more, so it would be less new code to write if we just
used r-p-s, I think.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 20:30:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 16:30:17 2024
Received: from localhost ([127.0.0.1]:35161 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpDR2-0005mp-LJ
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 16:30:17 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:6410)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpDR0-0005ma-SG
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 16:30:16 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 2184D10004C;
Tue, 26 Mar 2024 16:30:09 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711485007;
bh=K1ihFcANv1mBIel9GM7rNXXNyFrL/wuW7SEKc4RekIU=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=XB+xLfjcp173bteBSip9SVeFY+Mb9Iva6kH+PvPsMhSnCC8yJU1nOU6eD862h9i7R
zLJ89s4OiLIegxfsC9PYr9kbUn/RaAq1dNqSbPpGoB4x7gGQcWaxR8QFVerzE2RAMi
kmA6Epm7NxfwJeLjLYcxReit5WKNNlDx9g+m1m8IsLxv3uffdACsEgDw2xcXndflok
KO+E13retXgvoIznt5eLUlc2SA86bmDZttUOOVgoQavqnomvnBzvESzfUth3s0Hjfl
+hmyK6phWswUgbDjBjI9TdAa0IXKUR/rzvdUOKy0tx1I1bsuZvQy+Tkm6/UKhj9vKA
gOpm+dNWHP2SA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 900AF100046;
Tue, 26 Mar 2024 16:30:07 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 65F23120131;
Tue, 26 Mar 2024 16:30:07 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgKaAUMWzLXv6Alr@ACM> (Alan Mackenzie's message of "Tue, 26 Mar
2024 09:48:49 +0000")
Message-ID: <jwvttks3hpj.fsf-monnier+emacs@HIDDEN>
References: <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM>
Date: Tue, 26 Mar 2024 16:30:06 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> We now have two distinct uses of SWPs: providing warning source locations
> to the compiler (where we want to keep the position as long as possible)
> and providing position information for the doc string (where we want to
> strip the position from the symbol ASAP, to avoid trying to use the SWP
> when we need a plain symbol). If both of these occur together, we want
> to keep the SWP.
I think I'm beginning to understand. So in the "load from source case",
some of your symbols are SWPs and you want to turn them into bare
symbols "on the fly" during macro-expansion rather than via a separate
"strip" phase, so you want the macro expansion to know whether it's done
for "load from source" or for some other purpose and you use
`byte-compile-in-progress` as a proxy for that information.
Is that it?
If so, (and if the stripping happens within macros), then indeed passing
it as a separate argument through all the recursive calls to
`macroexp--expand-all` would be cumbersome. But I suggest you use
another name for that(e.g. `macroexp-strip-position`) so the
intention is made more clear.
Better yet: to avoid the problem of dynamic scope extending "too far"
(i.e. accidentally applying to nested loads/evals/compile/...), you
could put the relevant info into `macroexpand-all-environment`.
[ That var is also dynamically bound, but we're already careful to
rebind it when needed so it doesn't apply to nested uses
of macroexpansion. ]
>> > By "currently", I mean that a defining form such as defun or defvar has
>> > commenced, but not yet terminated; its functions currently occupy stack
>> > frames.
>> So you mean we're inside `Fdefalias` or `Fdefvar_1`?
> Yes, or inside a macro (defun, defmacro, ...) which expands to a
> defalias.
These are *very* different times: `Fdefalias` and `Fdefvar_1` are
executed long after macroexpansion. And they're small C functions which
run almost no external code at all, so they "occupy stack frames" only
for a very short time.
My crystal ball suggests that "currently" may be the wrong way to think
about it: maybe instead of thinking of "when" (as in "during the
definition of function FOO") what you're looking for might be "where"
(as in "within the body of FOO").
[ That's the same difference as the difference between dynamic and
static scoping. ]
If my crystal ball is right, then the better place to put that info is
probably `macroexpand-all-environment`.
> Ideally, I would like to have bound defining-symbol inside defun.
(defmacro my-defun (name args &rest body)
`(cl-macrolet ((defining-symbol () '',name))
(defun ,name ,args ,@body)))
(my-defun my-foo (x) (list x (defining-symbol)))
(symbol-function 'my-foo)
=3D=3D> #f(lambda (x) [t] (list x 'my-foo))
`cl-macrolet` uses `macroexpand-all-environment` for that.
> Fload uses read-positioning-DEFINED-symbols, as contrasted with the
> compiler, which uses read-positioning-symbols. r-p-d-s positions only
> lambdas and NAMEs. r-p-s positions all symbols except nil.
I think I'm beginning to understand (I guess I was struggling with your
use of "position" as a verb for some reason which made me think that
symbols were being moved rather than gaining position information).
So, IIUC you use `read-positioning-DEFINED-symbols` instead of
`read-positioning-symbols` because it's cheaper? Do you have rough
numbers comparing the cost of `read`, `read-positioning-symbols`, and
`read-positioning-DEFINED-symbols`?
Also, IIUC you don't have a separate phase to strip the SWPs when
loading from source, but instead you strip them as you "consume" their
info during macroexpansion. If so, how/when/where do you strip the
false positives that may occur inside quoted data or in code like:
(defmacro foo (lambda bar) ...)
(defmacro foo (defun bar) ...)
(let* ((lambda foo)
(defun bar))
...)
> Ah, right. I hadn't considered this before. The changes are by their
> very nature essentially complicated and difficult to understand.
[ Hmm... maybe not the best salespitch. ]
>> I think you can simply wait to add the entry to
>> `macro-declarations-alist` until a later time, so the `defining-symbol`
>> thingies will be ignored during the early bootstrap and once we have
>> more infrastructure in place we can then register the handler on
>> `macro-declarations-alist`.
>
> This will not be simpler. It would involve re-evaluating defun, then
> compensating for all the functions up to now whose NAMEs had been read
> without positions.
Not at all. Those will remain without position, but only in
`src/bootstrap-emacs`. In the real `src/emacs` they will get the
position because they'll come from the `.el[cn]` file and by the time we
get compile those files `macro-declarations-alist` will be
fully populated.
> There is unavoidable conplexity, here.
I'm definitely not convinced. I suspect you've been asking yourself
"can it be made simpler" and you may indeed then convince yourself that
the answer is no, because of assumptions you don't reconsider.
Try instead to think about "what would it take to remove that complexity?".
> I see things somewhat differently. We shouldn't increase the debugging
> burden even on "expert users".
Yet it's imposing more complexity (and hence more debugging burden) on
those same expert users. =F0=9F=99=81
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 20:22:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 16:22:02 2024
Received: from localhost ([127.0.0.1]:35153 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpDJ3-0005IQ-HC
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 16:22:01 -0400
Received: from mail.muc.de ([193.149.48.3]:10280)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rpDIz-0005Ho-M2
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 16:22:00 -0400
Received: (qmail 15113 invoked by uid 3782); 26 Mar 2024 21:21:51 +0100
Received: from acm.muc.de (p4fe15213.dip0.t-ipconnect.de [79.225.82.19])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Tue, 26 Mar 2024 21:21:51 +0100
Received: (qmail 16027 invoked by uid 1000); 26 Mar 2024 20:21:50 -0000
Date: Tue, 26 Mar 2024 20:21:50 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgMuXkD07TRSe__n@ACM>
References: <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
<ZgL99Pp3BdrA6ycb@ACM> <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Tue, Mar 26, 2024 at 15:40:05 -0400, Stefan Monnier wrote:
> >> >> > Sorry about that. A quick summary: defined symbols (and lambda) get
> >> >> > positioned by the new reader function read-positioning-defined symbols.
> >> >> > The new declare clause defining-symbol marks a macro such as defun or
> >> >> > cl-defgeneric as a macro which defines such symbols.
> >> Since I still don't understand the general picture, let me tell you how
> >> I would plan to do it, so you can tell me where it matches your
> >> approach and where it doesn't:
> >> - Change `load-source-file-function` so it uses
> >> `read-positioning-symbols` instead of plain `read`.
> >> [ This means that macro-expansion will now almost always have sympos,
> >> rather than only during compilation, ]
> > load-source-file-function is set to read-positioning-defined-symbols.
> [ I see it's `load-read-function`. ]
Yes!
> How does this differ from `read-positioning-symbols` and why do we need
> it to be different?
r-p-defined-s positions only lambdas and NAMEs defined by defun,
defmacro, defvar, .... (around 50 defining symbols). r-p-s positions
every symbol apart from nil. They have different purposes. r-p-d-s
gets info for the doc strings, which requires SWPs only for some
symbols. r-p-s is needed to get warning message locations. Were r-p-s
used for the doc string position information, most of the symbols would
need to be stripped of their positions before the form could be used.
It is simpler and faster not to position them at all.
> > (In a change to be committed, it gets bound to this function in Fload).
> > In reading
> > (defun foo () "foo doc" (lambda (bar) "lambda doc" (car bar)))
> > , foo gets positioned (because it follows defun), and so does lambda
> > (because it is a lambda following "(").
> IIUC "gets positioned" means that it is a symbol-with-pos rather than
> a bare symbol?
Yes.
> > I'm not entirely sure, but I think in non-eager macro expansion the
> > position information in SWPs is typically available.
> In lazy macro-expansion, SWPs are not available, no.
> But that's OK, it's rare and there's very little we can do about it (the
> reason it's lazy is indeed because we only discover very late that those
> sexps were meant to represent code. It's typically when a sexp is
> passed to `eval`).
OK.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 19:40:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 15:40:19 2024
Received: from localhost ([127.0.0.1]:35098 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpCeg-0002wd-OV
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 15:40:19 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:1398)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rpCeb-0002wN-9q
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 15:40:17 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id DF16110004C;
Tue, 26 Mar 2024 15:40:07 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711482006;
bh=pYJpCt17/vhcX0aXUShCNIosVkff3c3GbfbX/OX4Q1M=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=fnPLLyRAxoOIfbNF23Uw4nAeJftdEfRRPR16lGMjH3DgJuwB9Mkp4DUv7IRf9wSdB
laFZDcdBNuQU2o5ieg581X7dURDoLTRd8OGG5mXkO7icDEhDADG7+dCN5q4TKuMVU8
XQN18wvdDK5HSbJRm3fQ7DFm+KhIaCeVZZpyWlReSgUY3/xVNZvlwVnFk021BXY+hm
avAGiOKc3Tnc0nNxwVFScEQLX7s8j3GVMwpqL+esJ2ZGSj8J5l0ywUIIV/3yXCrRjH
Ku2FdWNaUdBmakWDwJCLFDgoXJlh1chehWNzs5cE1MjzrEIhHs4G9OuLYduV6rbbY6
ts5dVpJK3Xi+w==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id E2451100046;
Tue, 26 Mar 2024 15:40:06 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BB78112082A;
Tue, 26 Mar 2024 15:40:06 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgL99Pp3BdrA6ycb@ACM> (Alan Mackenzie's message of "Tue, 26 Mar
2024 16:55:16 +0000")
Message-ID: <jwvzfuk3inj.fsf-monnier+emacs@HIDDEN>
References: <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
<ZgL99Pp3BdrA6ycb@ACM>
Date: Tue, 26 Mar 2024 15:40:05 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> >> > Sorry about that. A quick summary: defined symbols (and lambda) get
>> >> > positioned by the new reader function read-positioning-defined symbols.
>> >> > The new declare clause defining-symbol marks a macro such as defun or
>> >> > cl-defgeneric as a macro which defines such symbols.
>
>> Since I still don't understand the general picture, let me tell you how
>> I would plan to do it, so you can tell me where it matches your
>> approach and where it doesn't:
>
>> - Change `load-source-file-function` so it uses
>> `read-positioning-symbols` instead of plain `read`.
>> [ This means that macro-expansion will now almost always have sympos,
>> rather than only during compilation, ]
>
> load-source-file-function is set to read-positioning-defined-symbols.
[ I see it's `load-read-function`. ]
How does this differ from `read-positioning-symbols` and why do we need
it to be different?
> (In a change to be committed, it gets bound to this function in Fload).
> In reading
>
> (defun foo () "foo doc" (lambda (bar) "lambda doc" (car bar)))
>
> , foo gets positioned (because it follows defun), and so does lambda
> (because it is a lambda following "(").
IIUC "gets positioned" means that it is a symbol-with-pos rather than
a bare symbol?
> I'm not entirely sure, but I think in non-eager macro expansion the
> position information in SWPs is typically available.
In lazy macro-expansion, SWPs are not available, no.
But that's OK, it's rare and there's very little we can do about it (the
reason it's lazy is indeed because we only discover very late that those
sexps were meant to represent code. It's typically when a sexp is
passed to `eval`).
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 16:55:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 12:55:28 2024
Received: from localhost ([127.0.0.1]:34811 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rpA59-0005mB-Oc
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 12:55:28 -0400
Received: from mail.muc.de ([193.149.48.3]:22272)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rpA56-0005lM-6y
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 12:55:25 -0400
Received: (qmail 80073 invoked by uid 3782); 26 Mar 2024 17:55:18 +0100
Received: from acm.muc.de (p4fe15213.dip0.t-ipconnect.de [79.225.82.19])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Tue, 26 Mar 2024 17:55:17 +0100
Received: (qmail 14083 invoked by uid 1000); 26 Mar 2024 16:55:16 -0000
Date: Tue, 26 Mar 2024 16:55:16 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgL99Pp3BdrA6ycb@ACM>
References: <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM> <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Tue, Mar 26, 2024 at 09:40:19 -0400, Stefan Monnier wrote:
> >> > Sorry about that. A quick summary: defined symbols (and lambda) get
> >> > positioned by the new reader function read-positioning-defined symbols.
> >> > The new declare clause defining-symbol marks a macro such as defun or
> >> > cl-defgeneric as a macro which defines such symbols.
> Since I still don't understand the general picture, let me tell you how
> I would plan to do it, so you can tell me where it matches your
> approach and where it doesn't:
> - Change `load-source-file-function` so it uses
> `read-positioning-symbols` instead of plain `read`.
> [ This means that macro-expansion will now almost always have sympos,
> rather than only during compilation, ]
load-source-file-function is set to read-positioning-defined-symbols.
(In a change to be committed, it gets bound to this function in Fload).
In reading
(defun foo () "foo doc" (lambda (bar) "lambda doc" (car bar)))
, foo gets positioned (because it follows defun), and so does lambda
(because it is a lambda following "(").
> - This in turn requires a strip-sympos pass after the
> eager-macroexpansion phase of `load-source-file-function`.
No such pass is needed, due to the state machine in
read-positioning-defined-symbols.
> - Change macros like `lambda` so as to use the extract position info
> from themselves/theirargs/thecontext (when available, since there will
> still be corner cases where it's not available, such as during
> non-eager macro expansion) ....
The macro lambda has become obsolete; it had no access to the SWP
lambda for which it was invoked. It has been replaced by code in
macroexpand-1 and Fmacroexpand which wraps the lambda form in (function
....) and preserves the SWP lambda.
I'm not entirely sure, but I think in non-eager macro expansion the
position information in SWPs is typically available.
> .... and stash it in their docstring.
> This might be as simple as adding a line
> (setq docstring (add-pos-to-docstring docstring ARG)).
This posification is done (mainly) in a call to
byte-run-posify-lambda-form from the (function (lambda ...)) pcase arm
in macroexp--expand-all.
> - Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 13:49:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 09:49:01 2024
Received: from localhost ([127.0.0.1]:33136 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rp7Aj-0000vY-51
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 09:49:01 -0400
Received: from mail.muc.de ([193.149.48.3]:56714)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rp7Ag-0000v7-9V
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 09:49:00 -0400
Received: (qmail 94037 invoked by uid 3782); 26 Mar 2024 10:48:50 +0100
Received: from acm.muc.de (p4fe15213.dip0.t-ipconnect.de [79.225.82.19])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Tue, 26 Mar 2024 10:48:50 +0100
Received: (qmail 18562 invoked by uid 1000); 26 Mar 2024 09:48:49 -0000
Date: Tue, 26 Mar 2024 09:48:49 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgKaAUMWzLXv6Alr@ACM>
References: <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Mon, Mar 25, 2024 at 18:10:11 -0400, Stefan Monnier wrote:
> >> That sounds rather ugly. I'd rather first try and define precisely
> >> what is we mean by "compilation in progress".
> > That the byte compiler is active, and all symbols (bar nil) get
> > positioned.
> That is much too vague to be usable. Which symbols are we
> talking about? What do we mean by "active"?
The symbols read from the source code being compiled by the byte
compiler. By "active" I mean that a byte compilation has been started,
is not yet complete, and hasn't been temporarily suspended (e.g. for
loading another file with (require 'foo)).
> I think we need to describe it in terms that link the symbol with code
> in bytecomp.el (not based on time, but based on function bytecomp-FOO
> calling BAR calling BAZ ... touching that symbol).
> Note that I also have no idea why we need to care about whether we're in
> the compiled case or the non-compiled case, so maybe my questions are
> "XY" style problems.
We now have two distinct uses of SWPs: providing warning source locations
to the compiler (where we want to keep the position as long as possible)
and providing position information for the doc string (where we want to
strip the position from the symbol ASAP, to avoid trying to use the SWP
when we need a plain symbol). If both of these occur together, we want
to keep the SWP.
This is the purpose of byte-compile-in-progress.
> > An alternative might be to pass an &optional boolean argument meaning
> > "preserve positions on symbols".
> >From where to where should we pass that argument?
> [ And yes, explicit arguments might be preferable unless the
> call-trace is too long for it to be practical or unless some of the
> functions along that call trace can't be changed easily to take such an
> extra argument. ]
To macroexp--expand-all from wherever. Here (L423 of macroexp.el) is
where the position gets stripped from the SWP in the non compilation
case.
> BTW, do you really mean "preserve" (meaning that the symbols were sympos
> to start with)?
Indeed, yes.
> >> I see the same problem with:
> >> DEFVAR_LISP ("defining-symbol", Vdefining_symbol,
> >> doc: /* The symbol currently being defined by a defining form.
> >> This variable is bound in the read-eval-print loop and certain
> >> high-level functions in the byte compiler. It is set to a value by
> >> functions and macros such as `defun', `defmacro', and `defvar'. */);
> >> Lots and lots of things can happen "during the definition" of a form,
> >> including definition of lots of other forms. So I think we'd need to
> >> define much more precisely what you meant by "currently".
> >> In addition, a definition is "intemporal" (it's declarative), so
> >> "currently being defined" is almost like an oxymoron.
> > By "currently", I mean that a defining form such as defun or defvar has
> > commenced, but not yet terminated; its functions currently occupy stack
> > frames.
> So you mean we're inside `Fdefalias` or `Fdefvar_1`?
Yes, or inside a macro (defun, defmacro, ...) which expands to a
defalias.
Ideally, I would like to have bound defining-symbol inside defun. But
this would have lost the binding at the end of defun, before evaluating
the defalias. It was a tricky problem which I think has been solved.
> >> I'm trying to understand your code, but I clearly lack a high-level
> >> overview of the approach you decided to takes, so I don't understand
> >> what's going on there.
> > Sorry about that. A quick summary: defined symbols (and lambda) get
> > positioned by the new reader function read-positioning-defined symbols.
> > The new declare clause defining-symbol marks a macro such as defun or
> > cl-defgeneric as a macro which defines such symbols.
> > The conversion of these SWPs into position structures in doc strings
> > happens at macro expansion time, through byte-run-posify-lambda-form.
> So, IIUC
> (defmacro defun (name args docstring &rest body)
> (declare (defining-symbol 1))
> ...)
> is akin to:
> (defmacro defun (name args docstring &rest body)
> (setq docstring (add-pos-to-docstring
> (symbol-with-pos-pos name) docstring))
> ...)
> ?
Pretty much, yes. (declare (defining-symbol name docstring)) also
informs the reader that NAME is to be positioned when in (defun NAME
....).
> >> If both, could you split it into two, then?
> > I'm not sure that would be possible or sensible - both use a common
> > approach.
> So, IIUC a first part of the change was to make `load` use
> `read-positioning-symbols` just like the compiler?
Fload uses read-positioning-DEFINED-symbols, as contrasted with the
compiler, which uses read-positioning-symbols. r-p-d-s positions only
lambdas and NAMEs. r-p-s positions all symbols except nil.
> >> AFAICT doing it only for compiled functions should be significantly
> >> simpler than for interpreted functions, so it would be a good
> >> stepping stone.
> > The work has already been done, and there is working code. Just as a
> > matter of interest, the branch runs the test suite without errors (not
> > counting "expensive" tests ).
> I'm not talking about a separate branch. I'm talking about splitting
> your changes into understandable commits.
Ah, right. I hadn't considered this before. The changes are by their
very nature essentially complicated and difficult to understand.
> >> On the cosmetic side, you have way too much code in `byte-run.el`.
> >> I think most of this code can be moved elsewhere, e.g. somewhere where
> >> backquote can be used
> > Yes, I noticed this, too. A lot of the bulk is for diagnostic functions
> > for SWPs, and these can eventually be deleted. Or possibly moved into a
> > new file with-pos.el to be loaded before byte-run.el.
> I don't like the idea of adding another file before byte-run.el.
Neither do I, but it's a possibility.
> > byte-run--posify-defining-symbol, the function with the extreme hand
> > expansion of backquotes is used as a declare clause handler, and is
> > needed by defun. Hence it couldn't really be moved to after the loading
> > of backquote.el.
> I think you can simply wait to add the entry to
> `macro-declarations-alist` until a later time, so the `defining-symbol`
> thingies will be ignored during the early bootstrap and once we have
> more infrastructure in place we can then register the handler on
> `macro-declarations-alist`.
This will not be simpler. It would involve re-evaluating defun, then
compensating for all the functions up to now whose NAMEs had been read
without positions. There is unavoidable conplexity, here. We need defun
to build backquote, byte-run--posify-defining-symbol to build defun, and
so we need to write b-r--p-d-s without backquote. All that could be done
is to shift the complexity to a different arm of that dependency
triangle.
> > There are some additional functions which batch-posify functions and
> > variables defined before the posification mechanism is in place. This
> > must be done ASAP, for the benefit of backtraces in early bootstrap.
> That complexifies the early bootstrap code. I'd rather keep that code
> simpler. During early bootstrap, all those functions are interpreted
> and I can't remember ever having difficulty tracking the origin of those
> interpreted lambdas during early bootstrap, so I'm rather opposed to
> such complexity just for the sake of maybe occasionally hypothetically
> giving a bit of help to the 3 of us who have to deal with
> those problems.
Well all I can say here is that the lack of special cases here has been
most helpful in debugging the current code. It's possible (?likely) that
somebody will need to look at it again, sometime.
> I see functions-with-pos as something mostly targeted at "end users" who
> aren't expert enough to figure out the origin of anonymous functions by
> looking at the disassembly of the bytecode and taking an educated guess.
I see things somewhat differently. We shouldn't increase the debugging
burden even on "expert users". My view is that debugging Lisp in Emacs
is too difficult and tedious, and can be improved. debug-early.el and
getting backtraces from redisplay errors are two already implemented such
improvements.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 26 Mar 2024 13:40:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 26 09:40:43 2024
Received: from localhost ([127.0.0.1]:33070 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rp72h-0000Xx-Co
for submit <at> debbugs.gnu.org; Tue, 26 Mar 2024 09:40:43 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:57248)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rp72c-0000Xd-OZ
for 67455 <at> debbugs.gnu.org; Tue, 26 Mar 2024 09:40:42 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 28E8A10004C;
Tue, 26 Mar 2024 09:40:28 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711460426;
bh=vCKSWtCbM7aXHw35dHqb66qvL43JOucDHF0C6cKSc8A=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=VlCAdVEABIPtugJsbd8yypxJXBygiDQ6rg8T5OKhNDgZVW/5eMUlBMIs/AqVjrsjN
kHjampIILAFKC8KXraYkK4hJ3aIBw5KytNWnz4LpKiLz9d8l1OSjOWSuv5/EGmvRUi
F/+a+t9wv+d/3uVESpiy/SJy7Au4H/7cslotY8jxC71atSXgktu73ueBr8nVqVRm3T
anFuPLqdR83lVPON8ndp8dTyv49pjnnGILl3EYC5m6Cmw4pZ2KfZd9KX+iiKX/UZhh
NijAgbBLRu0ujxOOF4hA8mwFxXiHH55ohiJEgZVREAfOoDeLgIMOlSbX3zghbqgyAx
dWMWrqUlH3R6A==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id C410B100046;
Tue, 26 Mar 2024 09:40:26 -0400 (EDT)
Received: from pastel (unknown [45.72.199.112])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9B33C120828;
Tue, 26 Mar 2024 09:40:26 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgKaAUMWzLXv6Alr@ACM> (Alan Mackenzie's message of "Tue, 26 Mar
2024 09:48:49 +0000")
Message-ID: <jwvedbx5dfz.fsf-monnier+emacs@HIDDEN>
References: <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM> <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
<ZgKaAUMWzLXv6Alr@ACM>
Date: Tue, 26 Mar 2024 09:40:19 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> > Sorry about that. A quick summary: defined symbols (and lambda) get
>> > positioned by the new reader function read-positioning-defined symbols.
>> > The new declare clause defining-symbol marks a macro such as defun or
>> > cl-defgeneric as a macro which defines such symbols.
Since I still don't understand the general picture, let me tell you how
I would plan to do it, so you can tell me where it matches your
approach and where it doesn't:
- Change `load-source-file-function` so it uses
`read-positioning-symbols` instead of plain `read`.
[ This means that macro-expansion will now almost always have sympos,
rather than only during compilation, ]
- This in turn requires a strip-sympos pass after the
eager-macroexpansion phase of `load-source-file-function`.
- Change macros like `lambda` so as to use the extract position info
from themselves/theirargs/thecontext (when available, since there will
still be corner cases where it's not available, such as during
non-eager macro expansion) and stash it in their docstring.
This might be as simple as adding a line
(setq docstring (add-pos-to-docstring docstring ARG)).
- Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 25 Mar 2024 22:12:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 25 18:12:45 2024
Received: from localhost ([127.0.0.1]:36459 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rosYd-00037N-Jy
for submit <at> debbugs.gnu.org; Mon, 25 Mar 2024 18:12:45 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:29651)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rosYY-000377-Bb
for 67455 <at> debbugs.gnu.org; Mon, 25 Mar 2024 18:12:42 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 24195442957;
Mon, 25 Mar 2024 18:12:33 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711404751;
bh=XGWAsB2eDcmCpt2UnSunPdg2kBpey/KRKEblCztcei8=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=ei1voFaqlvQQ7o4PcqzHr+XC1UzLE4WkgsAqgkqUYHCA6rVPyVwBFUt/QCe/8Tklm
AR8OIJD9QT7PeusLNKJzAFX46LY3TakUu7CkRXyX410WBpMRdZ/nwBRIJYQymwUYii
1g/NOriCYCN+VRW5WRXtknvJZhrnkccvvVAbc8hCNTrW4ukBKFHh4//DvHtKgiQZfW
KS5R5+cN7zuIqdtci9gVXoTMqKHJWClOzxkLCkgRvAQrPC4MLhlHfSDebrrjbuOXT0
DRiS9/Fqy0afoVC0uDgR9q9T7Uq6sykqPbot6SILYhBo3OEm4RXHYkfavTpRV10bmr
AluRysse6RGrw==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 0F4DA442953;
Mon, 25 Mar 2024 18:12:31 -0400 (EDT)
Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id F15B1120280;
Mon, 25 Mar 2024 18:12:30 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgHmmlpxP5S9set_@ACM> (Alan Mackenzie's message of "Mon, 25 Mar
2024 21:03:22 +0000")
Message-ID: <jwv7chqdm51.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
<ZgHmmlpxP5S9set_@ACM>
Date: Mon, 25 Mar 2024 18:10:11 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.176 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> That sounds rather ugly. I'd rather first try and define precisely
>> what is we mean by "compilation in progress".
> That the byte compiler is active, and all symbols (bar nil) get
> positioned.
That is much too vague to be usable. Which symbols are we
talking about? What do we mean by "active"?
I think we need to describe it in terms that link the symbol with code
in bytecomp.el (not based on time, but based on function bytecomp-FOO
calling BAR calling BAZ ... touching that symbol).
Note that I also have no idea why we need to care about whether we're in
the compiled case or the non-compiled case, so maybe my questions are
"XY" style problems.
> An alternative might be to pass an &optional boolean argument meaning
> "preserve positions on symbols".
From where to where should we pass that argument?
[ And yes, explicit arguments might be preferable unless the
call-trace is too long for it to be practical or unless some of the
functions along that call trace can't be changed easily to take such an
extra argument. ]
BTW, do you really mean "preserve" (meaning that the symbols were sympos
to start with)?
>> I see the same problem with:
>
>> DEFVAR_LISP ("defining-symbol", Vdefining_symbol,
>> doc: /* The symbol currently being defined by a defining form.
>> This variable is bound in the read-eval-print loop and certain
>> high-level functions in the byte compiler. It is set to a value by
>> functions and macros such as `defun', `defmacro', and `defvar'. */);
>
>> Lots and lots of things can happen "during the definition" of a form,
>> including definition of lots of other forms. So I think we'd need to
>> define much more precisely what you meant by "currently".
>> In addition, a definition is "intemporal" (it's declarative), so
>> "currently being defined" is almost like an oxymoron.
>
> By "currently", I mean that a defining form such as defun or defvar has
> commenced, but not yet terminated; its functions currently occupy stack
> frames.
So you mean we're inside `Fdefalias` or `Fdefvar_1`?
>> I'm trying to understand your code, but I clearly lack a high-level
>> overview of the approach you decided to takes, so I don't understand
>> what's going on there.
>
> Sorry about that. A quick summary: defined symbols (and lambda) get
> positioned by the new reader function read-positioning-defined symbols.
> The new declare clause defining-symbol marks a macro such as defun or
> cl-defgeneric as a macro which defines such symbols.
>
> The conversion of these SWPs into position structures in doc strings
> happens at macro expansion time, through byte-run-posify-lambda-form.
So, IIUC
(defmacro defun (name args docstring &rest body)
(declare (defining-symbol 1))
...)
is akin to:
(defmacro defun (name args docstring &rest body)
(setq docstring (add-pos-to-docstring
(symbol-with-pos-pos name) docstring))
...)
?
>> If both, could you split it into two, then?
> I'm not sure that would be possible or sensible - both use a common
> approach.
So, IIUC a first part of the change was to make `load` use
`read-positioning-symbols` just like the compiler?
>> AFAICT doing it only for compiled functions should be significantly
>> simpler than for interpreted functions, so it would be a good
>> stepping stone.
> The work has already been done, and there is working code. Just as a
> matter of interest, the branch runs the test suite without errors (not
> counting "expensive" tests ).
I'm not talking about a separate branch. I'm talking about splitting
your changes into understandable commits.
>> On the cosmetic side, you have way too much code in `byte-run.el`.
>> I think most of this code can be moved elsewhere, e.g. somewhere where
>> backquote can be used
>
> Yes, I noticed this, too. A lot of the bulk is for diagnostic functions
> for SWPs, and these can eventually be deleted. Or possibly moved into a
> new file with-pos.el to be loaded before byte-run.el.
I don't like the idea of adding another file before byte-run.el.
> byte-run--posify-defining-symbol, the function with the extreme hand
> expansion of backquotes is used as a declare clause handler, and is
> needed by defun. Hence it couldn't really be moved to after the loading
> of backquote.el.
I think you can simply wait to add the entry to
`macro-declarations-alist` until a later time, so the `defining-symbol`
thingies will be ignored during the early bootstrap and once we have
more infrastructure in place we can then register the handler on
`macro-declarations-alist`.
> There are some additional functions which batch-posify functions and
> variables defined before the posification mechanism is in place. This
> must be done ASAP, for the benefit of backtraces in early bootstrap.
That complexifies the early bootstrap code. I'd rather keep that code
simpler. During early bootstrap, all those functions are interpreted
and I can't remember ever having difficulty tracking the origin of those
interpreted lambdas during early bootstrap, so I'm rather opposed to
such complexity just for the sake of maybe occasionally hypothetically
giving a bit of help to the 3 of us who have to deal with
those problems.
I see functions-with-pos as something mostly targeted at "end users" who
aren't expert enough to figure out the origin of anonymous functions by
looking at the disassembly of the bytecode and taking an educated guess.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 25 Mar 2024 21:03:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 25 17:03:31 2024
Received: from localhost ([127.0.0.1]:36393 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rorTe-0001Cr-VM
for submit <at> debbugs.gnu.org; Mon, 25 Mar 2024 17:03:31 -0400
Received: from mail.muc.de ([193.149.48.3]:47391)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rorTc-0001CT-QC
for 67455 <at> debbugs.gnu.org; Mon, 25 Mar 2024 17:03:29 -0400
Received: (qmail 20222 invoked by uid 3782); 25 Mar 2024 22:03:23 +0100
Received: from acm.muc.de (pd953a51e.dip0.t-ipconnect.de [217.83.165.30])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Mon, 25 Mar 2024 22:03:23 +0100
Received: (qmail 481 invoked by uid 1000); 25 Mar 2024 21:03:22 -0000
Date: Mon, 25 Mar 2024 21:03:22 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <ZgHmmlpxP5S9set_@ACM>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM> <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
Thanks for taking the time and trouble to review my branch.
On Mon, Mar 25, 2024 at 14:23:50 -0400, Stefan Monnier wrote:
> >> The point is that `byte-compile-in-progress` will be non-nil during
> >> those loads, so you can't use this variable to get the information you need.
> > Yes. How about binding it to nil around `load' and recursive edits, and
> > possibly one or two other things?
> You mean trying to enumerate all the places we can think of where we
> know that compilation is not taking place?
Yes.
> That sounds rather ugly. I'd rather first try and define precisely
> what is we mean by "compilation in progress".
That the byte compiler is active, and all symbols (bar nil) get
positioned. Contrast this with, say, loading a .el file, where only some
symbols get positioned.
An alternative might be to pass an &optional boolean argument meaning
"preserve positions on symbols".
> I see the same problem with:
> DEFVAR_LISP ("defining-symbol", Vdefining_symbol,
> doc: /* The symbol currently being defined by a defining form.
> This variable is bound in the read-eval-print loop and certain
> high-level functions in the byte compiler. It is set to a value by
> functions and macros such as `defun', `defmacro', and `defvar'. */);
> Lots and lots of things can happen "during the definition" of a form,
> including definition of lots of other forms. So I think we'd need to
> define much more precisely what you meant by "currently".
> In addition, a definition is "intemporal" (it's declarative), so
> "currently being defined" is almost like an oxymoron.
By "currently", I mean that a defining form such as defun or defvar has
commenced, but not yet terminated; its functions currently occupy stack
frames.
> I'm trying to understand your code, but I clearly lack a high-level
> overview of the approach you decided to takes, so I don't understand
> what's going on there.
Sorry about that. A quick summary: defined symbols (and lambda) get
positioned by the new reader function read-positioning-defined symbols.
The new declare clause defining-symbol marks a macro such as defun or
cl-defgeneric as a macro which defines such symbols.
The conversion of these SWPs into position structures in doc strings
happens at macro expansion time, through byte-run-posify-lambda-form.
> Is that branch trying to provide function-position for compiled
> functions only, for interpreted functions only, or both?
It not only tries, but succeeds (modulo remaining bugs) in providing
posification for both interpreted and compiled functions.
> If both, could you split it into two, then?
I'm not sure that would be possible or sensible - both use a common
approach.
> AFAICT doing it only for compiled functions should be significantly
> simpler than for interpreted functions, so it would be a good
> stepping stone.
The work has already been done, and there is working code. Just as a
matter of interest, the branch runs the test suite without errors (not
counting "expensive" tests ).
> On the cosmetic side, you have way too much code in `byte-run.el`.
> I think most of this code can be moved elsewhere, e.g. somewhere where
> backquote can be used
Yes, I noticed this, too. A lot of the bulk is for diagnostic functions
for SWPs, and these can eventually be deleted. Or possibly moved into a
new file with-pos.el to be loaded before byte-run.el.
byte-run--posify-defining-symbol, the function with the extreme hand
expansion of backquotes is used as a declare clause handler, and is
needed by defun. Hence it couldn't really be moved to after the loading
of backquote.el.
There are some additional functions which batch-posify functions and
variables defined before the posification mechanism is in place. This
must be done ASAP, for the benefit of backtraces in early bootstrap.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 25 Mar 2024 18:33:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 25 14:33:48 2024
Received: from localhost ([127.0.0.1]:36249 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rop8m-0002kx-FH
for submit <at> debbugs.gnu.org; Mon, 25 Mar 2024 14:33:48 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:12853)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rop8k-0002kk-Tr
for 67455 <at> debbugs.gnu.org; Mon, 25 Mar 2024 14:33:47 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 9C0E4100170;
Mon, 25 Mar 2024 14:26:15 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1711391170;
bh=yQf7z4Uw/azJE8n8l7WDkGITa7SsgZ/mBFXtIyiFTJc=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=C2+vENPss/gyTNdU4h5D052lhMU55294GnJWwmYLFD5btQEZOJ2c2EXgODEsKHJyu
30AynptV2bM4/At+vexkgKAJg9ewshGYMDzcQIExt6Czlq6AykUO+BganMisCp7f/7
9SphxSBl+Q/N+qHGyhTFCdyUp+3k3ThBNo6/d3WnjKUTu6Tr1mydRNoAvtLuVauEcR
A6++pxLxh6M2wWHM8pRizdu7sMPxrR5D3wUa5VP8qq/lnw+CD4iGhIb/ZNa4wcoEB5
3v3EE+1K+6ze8k+7vi2O96Qfjk7/v8WGcU8jktq75FejuvwvOPhzr4y+z0JRqhYhNO
RfNSxNErgq7Ow==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 70CE210004A;
Mon, 25 Mar 2024 14:26:10 -0400 (EDT)
Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6345E12046C;
Mon, 25 Mar 2024 14:26:10 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZgAIpy1Znle1IZiU@ACM> (Alan Mackenzie's message of "Sun, 24 Mar
2024 11:04:07 +0000")
Message-ID: <jwvfrwefae8.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
<ZgAIpy1Znle1IZiU@ACM>
Date: Mon, 25 Mar 2024 14:23:50 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.080 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> The point is that `byte-compile-in-progress` will be non-nil during
>> those loads, so you can't use this variable to get the information you need.
> Yes. How about binding it to nil around `load' and recursive edits, and
> possibly one or two other things?
You mean trying to enumerate all the places we can think of where we
know that compilation is not taking place? That sounds rather ugly.
I'd rather first try and define precisely what is we mean by
"compilation in progress".
I see the same problem with:
DEFVAR_LISP ("defining-symbol", Vdefining_symbol,
doc: /* The symbol currently being defined by a defining form.
This variable is bound in the read-eval-print loop and certain
high-level functions in the byte compiler. It is set to a value by
functions and macros such as `defun', `defmacro', and `defvar'. */);
Lots and lots of things can happen "during the definition" of a form,
including definition of lots of other forms. So I think we'd need to
define much more precisely what you meant by "currently".
In addition, a definition is "intemporal" (it's declarative), so
"currently being defined" is almost like an oxymoron.
I'm trying to understand your code, but I clearly lack a high-level
overview of the approach you decided to takes, so I don't understand
what's going on there.
Is that branch trying to provide function-position for compiled
functions only, for interpreted functions only, or both?
If both, could you split it into two, then?
AFAICT doing it only for compiled functions should be significantly
simpler than for interpreted functions, so it would be a good
stepping stone.
On the cosmetic side, you have way too much code in `byte-run.el`.
I think most of this code can be moved elsewhere, e.g. somewhere where
backquote can be used
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 24 Mar 2024 11:22:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 24 07:22:01 2024 Received: from localhost ([127.0.0.1]:50245 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1roLvN-0002FO-5L for submit <at> debbugs.gnu.org; Sun, 24 Mar 2024 07:22:01 -0400 Received: from mail.muc.de ([193.149.48.3]:36021) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1roLvL-0002Et-6y for 67455 <at> debbugs.gnu.org; Sun, 24 Mar 2024 07:21:59 -0400 Received: (qmail 26008 invoked by uid 3782); 24 Mar 2024 12:21:11 +0100 Received: from acm.muc.de (p4fe15d23.dip0.t-ipconnect.de [79.225.93.35]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 24 Mar 2024 12:21:11 +0100 Received: (qmail 11661 invoked by uid 1000); 24 Mar 2024 11:21:10 -0000 Date: Sun, 24 Mar 2024 11:21:10 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZgAMpg8GvMnfomyM@ACM> References: <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> <Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN> <ZfGF_iqETDDB5M58@ACM> <jwva5n2nydd.fsf-monnier+emacs@HIDDEN> <Zfm649RngE5CxhtK@ACM> <jwvbk7ac5lv.fsf-monnier+emacs@HIDDEN> <ZfoGPKeJcvg_Qb-E@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <ZfoGPKeJcvg_Qb-E@ACM> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Tue, Mar 19, 2024 at 21:40:12 +0000, Alan Mackenzie wrote: > On Tue, Mar 19, 2024 at 16:47:46 -0400, Stefan Monnier wrote: > > > This might work. What do you think? > > I don't know what is the problem you're trying to fix, so it's hard for > > me to have an opinion. > > Could you clarify what is the problem when backquote is not changed > > at all (after all, this is the problem that will also occur if the > > programmer happened to use `cons/list` instead of backquote)? > When Lisp gets read for interpretation, defined symbols (e.g. folliowing > defun or cl-defgneric) get positioned, as do lambdas. When there are ,s > or ,@s on the arg list or the doc string of the lambda, the lambda > currently gets posified by the new code in backquote-process. Without > the new code, the "complicated" lambdas retain their positions, which > cause errors in pdump, which doesn't (and shouldn't) handle SWPs. I was possibly too dogmatic about this. I've actually amended pdumper.c to handle SWPs, and this appears to be working well. The SWPs from "complicated" lambdas get recorded in the dumped image, and later used in ME2. I've reverted my changes to backquote.el. :-) > You're right about my sketched approach not working if the programmer > uses cons/list instead of `, ,, and ,@. (Thanks!) Maybe I can somehow > wait until (cons 'lambda (cons args body)) has been evaluated in ME2, > before posifying the lambda. And also take the change out of > backquote-process. Now coded up. > With this idea, most of the new code would go into the (`(function ,(and > f `(lambda ,_ . ,_))) ...) pcase arm of macrexp--expand-all, with > possibly a new arm to catch and "neutralise" the remaining lambdas, which > aren't functions. It turns out there was little (?nothing) to change in macroexp--expand-all. The code was there already. > > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 24 Mar 2024 11:04:59 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 24 07:04:59 2024 Received: from localhost ([127.0.0.1]:49224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1roLes-0001PH-JL for submit <at> debbugs.gnu.org; Sun, 24 Mar 2024 07:04:58 -0400 Received: from mail.muc.de ([193.149.48.3]:49039) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1roLeq-0001Oq-2J for 67455 <at> debbugs.gnu.org; Sun, 24 Mar 2024 07:04:56 -0400 Received: (qmail 7042 invoked by uid 3782); 24 Mar 2024 12:04:08 +0100 Received: from acm.muc.de (p4fe15d23.dip0.t-ipconnect.de [79.225.93.35]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 24 Mar 2024 12:04:08 +0100 Received: (qmail 11567 invoked by uid 1000); 24 Mar 2024 11:04:07 -0000 Date: Sun, 24 Mar 2024 11:04:07 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZgAIpy1Znle1IZiU@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <handler.67455.B.170100905232659.ack <at> debbugs.gnu.org> <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> <Ze4IXIrh-WbZaTSu@ACM> <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Sun, Mar 10, 2024 at 17:03:28 -0400, Stefan Monnier wrote: > >> >> - Testing `byte-compile-in-progress` can't be right. Do you to test > >> >> whether the result of this backquote will be byte-compiled or do you > >> >> really mean to test whether this backquote happens to be executed > >> >> during compilation (which could be because the compiler ends up > >> >> loading code while executing `eval-when-compile` or `require`)? > >> > Quite simply, during compilation, all symbols (except nil) get read with > >> > position, so to strip their positions here would be wrong. > >> This isn't quite right: during compilation, some code is read with > >> positions (the code that we will compile), but some code is read in the > >> normal way (the code we load for the purpose of running). > >> The distinction is important. > > OK, I wasn't really counting code that we load as "during compilation", > > but I take the point. > The point is that `byte-compile-in-progress` will be non-nil during > those loads, so you can't use this variable to get the information you need. Yes. How about binding it to nil around `load' and recursive edits, and possibly one or two other things? > >> More generally: what goes wrong in the above example if you just treat > >> that as a list of symbol (stripping them all of their position info). > >> AFAICT when *that* macro is expanded (i.e. ME2) you'll presumably get > >> code like > >> (mapatoms #'(lambda (FOO/p) (DO/p SOME/p (THING/p)))) > >> right? [ where "/p" means that the symbol has a sympos. ] > >> Isn't that sufficient info to add a docstring with position? > > It's the lambda which has a position rather than the expanded bits from > > ME2. > Hmm... then I misunderstand something. How can the `lambda` have > a position if you don't include any special treatment of backquote? In read-positioning-defined-symbols, lambdas get positioned, along with symbols being defined by defun, defmacro, cl-defmethod, .... This is not to do with backquote handling. > AFAICT the `lambda` in the result of ME1 should not include position > information because at that time we don't know that it will be used to > build code rather than be some element of a normal list. read-positioning-defined-symbols cannot know how (lambda ...) is going to be used. It is up to other code (here, macroexp--expand-all) to strip the position from the lambda when it would be obtrusive. > And how come the rest doesn't have position information? read-positioning-defined-symbols is specifically coded (with a state machine) to position only the lambdas and the defined symbols. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 19 Mar 2024 22:33:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 19 18:33:45 2024
Received: from localhost ([127.0.0.1]:37539 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rmi1g-0007gp-VC
for submit <at> debbugs.gnu.org; Tue, 19 Mar 2024 18:33:45 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:18402)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rmi1f-0007gU-C3
for 67455 <at> debbugs.gnu.org; Tue, 19 Mar 2024 18:33:43 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 24D4980D32;
Tue, 19 Mar 2024 18:32:58 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710887576;
bh=JmHFr4I9ATc1wNrvtH6Nc5vz3QL3TEof+ZN2dZA4u4U=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=GgpHUFOT6SW7V7mRkIOjuGIApIKvC+YRKf2W4FNpGoLv2CCwxw1XRB5SHVDioW0eI
o8Ujgjy8HvN+YM1CBJQ0k0lNzyhT0Nq1jwKGVjnZFunDDxNFi7YKjWGmUVyVBW1AOl
YoXbxbnohjPONKEej1S6l3F+NPrs7lj2lXgCCfzwozXfjDYcu9iilQ3oppS/Q2HjSI
iu0m21FcUPKsJ07mmqyMyEthsvAXnZ5eZUQQBk75Kor67/6S1fnC194+NLmIFOhajx
1h/AdIcPxODoN7qWKompxYvdJW6uZvIBTfpFBzkvg+jYYozipDtLb9d+fTNQeRItSr
AXhiBcmvFelAw==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id E037A803C1;
Tue, 19 Mar 2024 18:32:56 -0400 (EDT)
Received: from pastel (unknown [104.247.238.200])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B952612072D;
Tue, 19 Mar 2024 18:32:56 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZfoGPKeJcvg_Qb-E@ACM> (Alan Mackenzie's message of "Tue, 19 Mar
2024 21:40:12 +0000")
Message-ID: <jwvttl1c1bx.fsf-monnier+emacs@HIDDEN>
References: <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN>
<ZfGF_iqETDDB5M58@ACM> <jwva5n2nydd.fsf-monnier+emacs@HIDDEN>
<Zfm649RngE5CxhtK@ACM> <jwvbk7ac5lv.fsf-monnier+emacs@HIDDEN>
<ZfoGPKeJcvg_Qb-E@ACM>
Date: Tue, 19 Mar 2024 18:32:54 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.243 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> When Lisp gets read for interpretation, defined symbols (e.g. folliowing
> defun or cl-defgneric) get positioned, as do lambdas.
I don't know what that means.
> When there are ,s or ,@s on the arg list or the doc string of the
> lambda, the lambda currently gets posified by the new code in
> backquote-process.
Are you talking about "lambda" as in "the symbol" or as in "a (lambda
...) expression". If it's "the symbol", then I can't see where a , or ,@
can appear. If the other, then if there's a , or ,@ in there it
presumably means we don't yet know whether it *will* be
a lambda-expression or just a list with a lambda symbol: at that point,
it's just data and we don't know if it will be used to build code.
> Without the new code, the "complicated" lambdas retain their
> positions, which cause errors in pdump, which doesn't (and shouldn't)
> handle SWPs.
So, IIUC, your "get positioned" above means you preserve/add (rather than
strip) the position info on some symbols, most notably those `lambda`s
which "you" predict will be used for code, and if your prediction is
wrong then those sympos end up escaping into the wild.
> You're right about my sketched approach not working if the programmer
> uses cons/list instead of `, ,, and ,@. (Thanks!) Maybe I can somehow
> wait until (cons 'lambda (cons args body)) has been evaluated in ME2,
> before posifying the lambda. And also take the change out of
> backquote-process.
Sounds about right. You'll lose information about the place where the
`lambda` symbol was found in the code, but it's hard to do much better
with what we have,
We could introduce a new `backquote-lisp-form` which works just like
backquote but which additionally asserts that what it builds will be
used as a Lisp form rather than as data.
BTW, we already have such a thing under the name "edebug-\`".
> With this idea, most of the new code would go into the (`(function ,(and
> f `(lambda ,_ . ,_))) ...) pcase arm of macrexp--expand-all, with
> possibly a new arm to catch and "neutralise" the remaining lambdas, which
> aren't functions.
Not sure what other lambdas you're thinking of.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 19 Mar 2024 21:41:00 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 19 17:41:00 2024 Received: from localhost ([127.0.0.1]:35926 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rmhCd-00026t-FU for submit <at> debbugs.gnu.org; Tue, 19 Mar 2024 17:41:00 -0400 Received: from mail.muc.de ([193.149.48.3]:57827) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rmhCb-00026S-T4 for 67455 <at> debbugs.gnu.org; Tue, 19 Mar 2024 17:40:58 -0400 Received: (qmail 11718 invoked by uid 3782); 19 Mar 2024 22:40:13 +0100 Received: from acm.muc.de (pd953ae04.dip0.t-ipconnect.de [217.83.174.4]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 19 Mar 2024 22:40:12 +0100 Received: (qmail 29593 invoked by uid 1000); 19 Mar 2024 21:40:12 -0000 Date: Tue, 19 Mar 2024 21:40:12 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZfoGPKeJcvg_Qb-E@ACM> References: <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> <Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN> <ZfGF_iqETDDB5M58@ACM> <jwva5n2nydd.fsf-monnier+emacs@HIDDEN> <Zfm649RngE5CxhtK@ACM> <jwvbk7ac5lv.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvbk7ac5lv.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Tue, Mar 19, 2024 at 16:47:46 -0400, Stefan Monnier wrote: > > This might work. What do you think? > I don't know what is the problem you're trying to fix, so it's hard for > me to have an opinion. > Could you clarify what is the problem when backquote is not changed > at all (after all, this is the problem that will also occur if the > programmer happened to use `cons/list` instead of backquote)? When Lisp gets read for interpretation, defined symbols (e.g. folliowing defun or cl-defgneric) get positioned, as do lambdas. When there are ,s or ,@s on the arg list or the doc string of the lambda, the lambda currently gets posified by the new code in backquote-process. Without the new code, the "complicated" lambdas retain their positions, which cause errors in pdump, which doesn't (and shouldn't) handle SWPs. You're right about my sketched approach not working if the programmer uses cons/list instead of `, ,, and ,@. (Thanks!) Maybe I can somehow wait until (cons 'lambda (cons args body)) has been evaluated in ME2, before posifying the lambda. And also take the change out of backquote-process. With this idea, most of the new code would go into the (`(function ,(and f `(lambda ,_ . ,_))) ...) pcase arm of macrexp--expand-all, with possibly a new arm to catch and "neutralise" the remaining lambdas, which aren't functions. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 19 Mar 2024 21:10:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 19 17:10:19 2024
Received: from localhost ([127.0.0.1]:33740 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rmgix-0000dq-0o
for submit <at> debbugs.gnu.org; Tue, 19 Mar 2024 17:10:19 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:13276)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rmgNt-00082e-CA
for 67455 <at> debbugs.gnu.org; Tue, 19 Mar 2024 16:48:33 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4E3D6442E96;
Tue, 19 Mar 2024 16:47:48 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710881267;
bh=cAySLi74ux4Dutr4ykQOLj27FRo40zFaYjo666Utd7Y=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=FiUu7W2Q3AXb2pC9Il8OtGvFcR6V1d/Ij80fJ8EUWVGaIPp19WdJFu1UXnL4xODKi
a2SoeYgqOcQmN1+0KGmL7WDEOzGjS0BX+jqrOKEH/2i9eP0TBNCIjfrQxQROpiT5N4
7WA09KnMZBWM2po5iAh7/FfW0PtdJAEVpYdu0c5DbzoKPgcpXFxSOK2f42WP7etRRd
fB46RZubxvYesVW6w/b7wPtknMCyAbhthsKgMHcZiTKEOvwxU8rSp6DWCcoaxZ3L/3
pIMzL5N/6us5/u7l2ARMJ5koxUISYmIffZdxTvLCBLrDNafolidMTzsXw+hJ0PXCB6
B7qcrhVYLyuZQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 2321A442E8D;
Tue, 19 Mar 2024 16:47:47 -0400 (EDT)
Received: from pastel (unknown [104.247.238.200])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id EE9021201E7;
Tue, 19 Mar 2024 16:47:46 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <Zfm649RngE5CxhtK@ACM> (Alan Mackenzie's message of "Tue, 19 Mar
2024 16:18:43 +0000")
Message-ID: <jwvbk7ac5lv.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN>
<ZfGF_iqETDDB5M58@ACM> <jwva5n2nydd.fsf-monnier+emacs@HIDDEN>
<Zfm649RngE5CxhtK@ACM>
Date: Tue, 19 Mar 2024 16:47:46 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.420 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> This might work. What do you think?
I don't know what is the problem you're trying to fix, so it's hard for
me to have an opinion.
Could you clarify what is the problem when backquote is not changed
at all (after all, this is the problem that will also occur if the
programmer happened to use `cons/list` instead of backquote)?
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 19 Mar 2024 16:19:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 19 12:19:32 2024 Received: from localhost ([127.0.0.1]:51231 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rmcBX-0005Zp-Kh for submit <at> debbugs.gnu.org; Tue, 19 Mar 2024 12:19:31 -0400 Received: from mail.muc.de ([193.149.48.3]:18581) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rmcBV-0005Za-6o for 67455 <at> debbugs.gnu.org; Tue, 19 Mar 2024 12:19:29 -0400 Received: (qmail 40253 invoked by uid 3782); 19 Mar 2024 17:18:44 +0100 Received: from acm.muc.de (pd953ae04.dip0.t-ipconnect.de [217.83.174.4]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 19 Mar 2024 17:18:44 +0100 Received: (qmail 27244 invoked by uid 1000); 19 Mar 2024 16:18:43 -0000 Date: Tue, 19 Mar 2024 16:18:43 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <Zfm649RngE5CxhtK@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <handler.67455.B.170100905232659.ack <at> debbugs.gnu.org> <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> <Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN> <ZfGF_iqETDDB5M58@ACM> <jwva5n2nydd.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwva5n2nydd.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello Stefan. On Wed, Mar 13, 2024 at 07:52:50 -0400, Stefan Monnier wrote: > > OK, so it seems like I'll need a new pcase arm in macroexp--expand-all, > > and this new code will need to handle (function (cons 'lambda ...)), and > > the like. > If macroexp--expand-all receives code of the form > (function (cons 'lambda ...)) > it means it received broken code. > IOW, such an arm will never do anything useful (the best it can do is > emit a warning). > I suspect what you're looking for is yet different. How about the following (as yet vague) idea? (i) Amend backquote slightly so that the integer part of the result of backquote-process gets transmitted to the caller, somehow. This would let macroexp--expand-all know it's dealing with smething awkward like `#'(lambda ,@args . ,body). (ii) On encountering such, m--e-all would insert a call to (new macro) macroexp--maybe-posify-lamda-form at ME1 time. (iii) At ME2 time, m--e-all should be able to tell whether or not the lambda form is "real". In this case the macro would be replaced by a call to (existing) byte-run--posify-lambda-position. Whether or not this is done, the SWP lambda would be stripped of its position. This might work. What do you think? > Stefan -- Alan Mackenzie (Nuremberg, Germany)
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 13 Mar 2024 11:53:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 13 07:53:36 2024
Received: from localhost ([127.0.0.1]:44861 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rkNAu-00011w-BR
for submit <at> debbugs.gnu.org; Wed, 13 Mar 2024 07:53:36 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:42939)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rkNAs-00011i-Qr
for 67455 <at> debbugs.gnu.org; Wed, 13 Mar 2024 07:53:35 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id E323B8027D;
Wed, 13 Mar 2024 07:52:53 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710330772;
bh=0K/6M3B+hPH4G0aM8NgqEh5dnQodLdk5Iyx60Jzdc3A=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=gQhQtAp8Zyf5PlJDLu3VbVUiFTZ3qq/u7MsvkSW2VRRndDohYoa1tr7FxsMntl4/J
60Fw7eUQbXfmFq8yO8NeXZ5c+o4fA4pxWxw0GSr302qsyPpE7y5iqP4CLFasDzkkus
XsFOn3+p7KXNKmBPzLMMrJHk6WGDc53SzgnuScU1fsrzNl4vuECgD8M3o3lSbCuMjJ
PZ0v1ED0i+gpLUpee4DguG9Y/iWZcUxT1/SiiWzLvRqu3uQsdFTymu85bt3yxpqRAz
98jQTeqP+fBW2NEhFALlR2HbJtZBVWTxdQ2CGF12mqgg3zRLWjQW9dpCjPqTmzjtze
UTFFUq6ybBeZQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id CC262807A5;
Wed, 13 Mar 2024 07:52:52 -0400 (EDT)
Received: from pastel (unknown [216.154.23.71])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9C7551206A7;
Wed, 13 Mar 2024 07:52:52 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZfGF_iqETDDB5M58@ACM> (Alan Mackenzie's message of "Wed, 13 Mar
2024 10:54:54 +0000")
Message-ID: <jwva5n2nydd.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN>
<ZfGF_iqETDDB5M58@ACM>
Date: Wed, 13 Mar 2024 07:52:50 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.103 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> OK, so it seems like I'll need a new pcase arm in macroexp--expand-all,
> and this new code will need to handle (function (cons 'lambda ...)), and
> the like.
If macroexp--expand-all receives code of the form
(function (cons 'lambda ...))
it means it received broken code.
IOW, such an arm will never do anything useful (the best it can do is
emit a warning).
I suspect what you're looking for is yet different.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 13 Mar 2024 10:55:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 13 06:55:43 2024 Received: from localhost ([127.0.0.1]:44766 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rkMGt-0004zM-0u for submit <at> debbugs.gnu.org; Wed, 13 Mar 2024 06:55:43 -0400 Received: from mail.muc.de ([193.149.48.3]:52450) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rkMGn-0004yr-1y for 67455 <at> debbugs.gnu.org; Wed, 13 Mar 2024 06:55:40 -0400 Received: (qmail 36486 invoked by uid 3782); 13 Mar 2024 11:54:55 +0100 Received: from acm.muc.de (p4fe15744.dip0.t-ipconnect.de [79.225.87.68]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 13 Mar 2024 11:54:54 +0100 Received: (qmail 9203 invoked by uid 1000); 13 Mar 2024 10:54:54 -0000 Date: Wed, 13 Mar 2024 10:54:54 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZfGF_iqETDDB5M58@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <handler.67455.B.170100905232659.ack <at> debbugs.gnu.org> <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> <Ze4z0eZxmW7ya7Gk@ACM> <jwvedchionz.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvedchionz.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Sun, Mar 10, 2024 at 20:50:05 -0400, Stefan Monnier wrote: > >> No matter how many extra tests you add to reduce the frequency, you're > >> fundamentally adding a bug :-( > > Well, macroexp--expand-all has treated (function (lambda ...)) as a > > function long before I started on this project in November. > Of course it does, and that's correct, because by definition the > argument to `macroexp--expand-all` must be source code expression. > > Would trusting the same thing in backquote-process be any > > more dangerous? > Yup, because backquote has no guarantee that the code it must produce is > one that will build a source code expression. > > How about adding (an) extra arm(s) to the large pcase in > > macroexp--expand-all which would recognise backquote's output for > > "evaluated" lambdas. > > What we get back from backquote looks like: > > (cons 'lambda (cons plain-args body)) or > > (list 'lambda args)) or maybe one or two other forms. > Same problem: > (cons 'lambda (cons plain-args body)) > constructs something that may look like a function but that may not be > intended to be used as a function. All we know is that it should build > a list with a `lambda` symbol in it. > It's only when the result of the execution of this code is passed to > `macroexp--expand-all` that we discover that it was meant to build > a list that represents the source code of a function; and only at *that* > point are we allowed to modify that list by macro-expanding expressions > in its body, modifying its docstring, byte-compiling, etc... OK, so it seems like I'll need a new pcase arm in macroexp--expand-all, and this new code will need to handle (function (cons 'lambda ...)), and the like. I'll not actually be able to do too much in the next few days, though. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 11 Mar 2024 00:50:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 10 20:50:56 2024
Received: from localhost ([127.0.0.1]:38174 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rjTsW-0002J5-Ky
for submit <at> debbugs.gnu.org; Sun, 10 Mar 2024 20:50:56 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:61671)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rjTsU-0002Ir-9J
for 67455 <at> debbugs.gnu.org; Sun, 10 Mar 2024 20:50:54 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id B864F80DB3;
Sun, 10 Mar 2024 20:50:12 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710118209;
bh=xwtbAD3fOXPEvUvxvr5fMw3/0Myj9OEpYiwK+QeJh+s=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=LB289A7NV8ARORqJn/eJNbdpKCR1thKUJqKymLPKEvMYL1m7woKyUcOTTPgGbxn0v
3eVoLyYuH32j1MyssxmgzNMF4iLNmQIqjyf7EzuzFEv1i7yD2lUr7ia2/mYXAgc2s0
yu8iF93gVusU+1MkFmd6zicp0u1IsTHgkAPoDZ55vdaIzyUK3yRG/gTTUvfWbUnnqH
lfAdUuHTXAi5tdfc8CkSZXmCcw4itZnhO2f2zPWiaY2+XJEHRcy1FmIT3Eqxa10sOt
Q11XqXU8PSpz/7MPdlQKNc75c3FioDrtLMfBjyv67A4O4PD0Wm0JO3uZv1/lgohvv5
Vz16x4fFeEebg==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 5A77C8093D;
Sun, 10 Mar 2024 20:50:09 -0400 (EDT)
Received: from alfajor (69-165-147-56.dsl.teksavvy.com [69.165.147.56])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 2D967120403;
Sun, 10 Mar 2024 20:50:09 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <Ze4z0eZxmW7ya7Gk@ACM> (Alan Mackenzie's message of "Sun, 10 Mar
2024 22:27:29 +0000")
Message-ID: <jwvedchionz.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4z0eZxmW7ya7Gk@ACM>
Date: Sun, 10 Mar 2024 20:50:05 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> No matter how many extra tests you add to reduce the frequency, you're
>> fundamentally adding a bug :-(
>
> Well, macroexp--expand-all has treated (function (lambda ...)) as a
> function long before I started on this project in November.
Of course it does, and that's correct, because by definition the
argument to `macroexp--expand-all` must be source code expression.
> Would trusting the same thing in backquote-process be any
> more dangerous?
Yup, because backquote has no guarantee that the code it must produce is
one that will build a source code expression.
> How about adding (an) extra arm(s) to the large pcase in
> macroexp--expand-all which would recognise backquote's output for
> "evaluated" lambdas.
>
> What we get back from backquote looks like:
>
> (cons 'lambda (cons plain-args body)) or
> (list 'lambda args)) or maybe one or two other forms.
Same problem:
(cons 'lambda (cons plain-args body))
constructs something that may look like a function but that may not be
intended to be used as a function. All we know is that it should build
a list with a `lambda` symbol in it.
It's only when the result of the execution of this code is passed to
`macroexp--expand-all` that we discover that it was meant to build
a list that represents the source code of a function; and only at *that*
point are we allowed to modify that list by macro-expanding expressions
in its body, modifying its docstring, byte-compiling, etc...
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 10 Mar 2024 22:28:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 10 18:28:12 2024 Received: from localhost ([127.0.0.1]:38139 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rjReO-00074x-2b for submit <at> debbugs.gnu.org; Sun, 10 Mar 2024 18:28:12 -0400 Received: from mail.muc.de ([193.149.48.3]:19990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rjReL-00074j-DY for 67455 <at> debbugs.gnu.org; Sun, 10 Mar 2024 18:28:09 -0400 Received: (qmail 94022 invoked by uid 3782); 10 Mar 2024 23:27:30 +0100 Received: from acm.muc.de (pd953a236.dip0.t-ipconnect.de [217.83.162.54]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 10 Mar 2024 23:27:29 +0100 Received: (qmail 22125 invoked by uid 1000); 10 Mar 2024 22:27:29 -0000 Date: Sun, 10 Mar 2024 22:27:29 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <Ze4z0eZxmW7ya7Gk@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <handler.67455.B.170100905232659.ack <at> debbugs.gnu.org> <ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN> <Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello again, Stefan. On Sun, Mar 10, 2024 at 13:19:03 -0400, Stefan Monnier wrote: [ .... ] > >> - My gut tells me that changing backquote can't be right. > > I tend to agree. I put the code into backquote-process when having > > problems with things like: > > (mapatoms #'(lambda (,(car spec)) ,@body) [ .... ] > >> (lambda (f) ..) *can* appear within a backquote without it being an > >> actual lambda expression. > >> What alternatives have you considered? > > Not a lot of them, as yet. Maybe testing for (function (lambda ...)) > > would be safer. > No matter how many extra tests you add to reduce the frequency, you're > fundamentally adding a bug :-( Well, macroexp--expand-all has treated (function (lambda ...)) as a function long before I started on this project in November. Would trusting the same thing in backquote-process be any more dangerous? Anyway, I've started looking at getting that extra code out of backquote.el. Simply commenting it out produces build time errors, which shows that it's not totally redundant. How about adding (an) extra arm(s) to the large pcase in macroexp--expand-all which would recognise backquote's output for "evaluated" lambdas. What we get back from backquote looks like: (cons 'lambda (cons plain-args body)) or (list 'lambda args)) or maybe one or two other forms. So I could extend such code fragments to add posification code quite easily. I'm not sure it'd be more elegant than what's currently in backquote.el, though. I think I'll give it a try. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 10 Mar 2024 21:04:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 10 17:04:19 2024
Received: from localhost ([127.0.0.1]:38078 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rjQLC-00053g-P1
for submit <at> debbugs.gnu.org; Sun, 10 Mar 2024 17:04:19 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:18994)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rjQLA-00053T-P2
for 67455 <at> debbugs.gnu.org; Sun, 10 Mar 2024 17:04:17 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 324A980B20;
Sun, 10 Mar 2024 17:03:37 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710104612;
bh=qc9l4EalkGXztJtpgRfzLRvo9iUJu8oMmLkQJ7tXHJg=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=o7tUZA0GfjLv6AJ5du4yn+UglWgOufHPLtCQT05yD6WFuIGBp0bzqz62POlpS32m0
uKevdFpNOYoHS53LVtt6ThUibW0WPcTxPeY70Odnz51JnSskP4fHHbfJ/eNPsSDm41
Vu5f+jgAkpvuVkOYjA6JvobePlf1YTgPlSjaTj3bscD72atQSP3XE63WQ+byOtEB2Q
MFkt6gAG/Opv6o+H1BCxNfjitGZ14UYIljc5dL/mSAydqBbEXdY/yG3xB5ueAlLCV9
D0ppzfBhMChL2/chVjVj8gB6yMliGCvY41ZqSGViekz3ooI+OodkqhdM2LGcNK/BgV
btA7BYSy5ButQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id F26E98093D;
Sun, 10 Mar 2024 17:03:31 -0400 (EDT)
Received: from alfajor (69-165-147-56.dsl.teksavvy.com [69.165.147.56])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id C3D49120821;
Sun, 10 Mar 2024 17:03:31 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <Ze4IXIrh-WbZaTSu@ACM> (Alan Mackenzie's message of "Sun, 10 Mar
2024 19:22:04 +0000")
Message-ID: <jwvzfv5iz4j.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
<Ze4IXIrh-WbZaTSu@ACM>
Date: Sun, 10 Mar 2024 17:03:28 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> >> - Testing `byte-compile-in-progress` can't be right. Do you to test
>> >> whether the result of this backquote will be byte-compiled or do you
>> >> really mean to test whether this backquote happens to be executed
>> >> during compilation (which could be because the compiler ends up
>> >> loading code while executing `eval-when-compile` or `require`)?
>
>> > Quite simply, during compilation, all symbols (except nil) get read with
>> > position, so to strip their positions here would be wrong.
>
>> This isn't quite right: during compilation, some code is read with
>> positions (the code that we will compile), but some code is read in the
>> normal way (the code we load for the purpose of running).
>> The distinction is important.
>
> OK, I wasn't really counting code that we load as "during compilation",
> but I take the point.
The point is that `byte-compile-in-progress` will be non-nil during
those loads, so you can't use this variable to get the information you need.
>> More generally: what goes wrong in the above example if you just treat
>> that as a list of symbol (stripping them all of their position info).
>> AFAICT when *that* macro is expanded (i.e. ME2) you'll presumably get
>> code like
>
>> (mapatoms #'(lambda (FOO/p) (DO/p SOME/p (THING/p))))
>
>> right? [ where "/p" means that the symbol has a sympos. ]
>> Isn't that sufficient info to add a docstring with position?
>
> It's the lambda which has a position rather than the expanded bits from
> ME2.
Hmm... then I misunderstand something. How can the `lambda` have
a position if you don't include any special treatment of backquote?
AFAICT the `lambda` in the result of ME1 should not include position
information because at that time we don't know that it will be used to
build code rather than be some element of a normal list.
And how come the rest doesn't have position information?
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 10 Mar 2024 19:22:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 10 15:22:47 2024
Received: from localhost ([127.0.0.1]:37893 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rjOkw-0002Mi-Nh
for submit <at> debbugs.gnu.org; Sun, 10 Mar 2024 15:22:47 -0400
Received: from mail.muc.de ([193.149.48.3]:49142)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rjOku-0002MO-K7
for 67455 <at> debbugs.gnu.org; Sun, 10 Mar 2024 15:22:45 -0400
Received: (qmail 85040 invoked by uid 3782); 10 Mar 2024 20:22:05 +0100
Received: from acm.muc.de (pd953a236.dip0.t-ipconnect.de [217.83.162.54])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Sun, 10 Mar 2024 20:22:04 +0100
Received: (qmail 20945 invoked by uid 1000); 10 Mar 2024 19:22:04 -0000
Date: Sun, 10 Mar 2024 19:22:04 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <Ze4IXIrh-WbZaTSu@ACM>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM> <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Sun, Mar 10, 2024 at 13:19:03 -0400, Stefan Monnier wrote:
> > I've got a version almost ready which actually does something, namely
> > prefixes "anonymous" lines of a backtrace with the name of the defining
> > symbol, like {foo} . It'll soon be time to start seriously thinking
> > about what information ought to go there for the live version.
> Cool!
I've finally got something to show. I've just committed a merge and a
fix for it to branch feature/positioned-lambdas at savannah.
With this Emacs running, type the following into *scratch*
(defun foo () "foo doc" (lambda (bar) "lambda doc" (car bar)))
.. Either evaluate this or byte compile it with compile-defun. Then do
M-: (funcall (foo) 'baz)
.. This will produce a backtrace like:
Debugger entered--Lisp error: (wrong-type-argument listp baz)
car(baz)
{foo} #f(compiled-function (bar) "lambda doc" #<bytecode -0x14ae78a46439bbc>)(baz)
funcall({foo} #f(compiled-function (bar) "lambda doc" #<bytecode -0x14ae78a46439bbc>) baz)
(progn (funcall (foo) 'baz))
eval((progn (funcall (foo) 'baz)) t)
elisp--eval-last-sexp(nil)
{eval-last-sexp} #f(compiled-function () #<bytecode -0x1e8241efdb3d2890>)()
eval-last-sexp(nil)
funcall-interactively(eval-last-sexp nil)
command-execute(eval-last-sexp)
.. Note the {eval-last-sexp} and {foo} on the anonymous functions.
:-)
> >> - Testing `byte-compile-in-progress` can't be right. Do you to test
> >> whether the result of this backquote will be byte-compiled or do you
> >> really mean to test whether this backquote happens to be executed
> >> during compilation (which could be because the compiler ends up
> >> loading code while executing `eval-when-compile` or `require`)?
> > Quite simply, during compilation, all symbols (except nil) get read with
> > position, so to strip their positions here would be wrong.
> This isn't quite right: during compilation, some code is read with
> positions (the code that we will compile), but some code is read in the
> normal way (the code we load for the purpose of running).
> The distinction is important.
OK, I wasn't really counting code that we load as "during compilation",
but I take the point.
> >> - My gut tells me that changing backquote can't be right.
> > I tend to agree. I put the code into backquote-process when having
> > problems with things like:
> > (mapatoms #'(lambda (,(car spec)) ,@body)
> > in cl-macs.el, where it's impossible to know where the doc string (if
> > any) is until after the expansion of the backquotes, or even at run time
> > (as here). In the latter case, rather than "posifying" the doc string
> > at macro expansion time, we have to generate code to do it at run time.
> Hmm... here what you call "run time" is really some later
> macro-expansion, right? The `lambda` symbol comes from the first
> macro-expansion (ME1), but the docstring comes from the second (ME2).
Yes. I often get confused between lots of different macro expansion
times, compile time and run time. It's a lot easier in C. ;-)
> IIUC the problem you face is that you want to get the function's
> position info from the `lambda` symbol, which here would be wrong (even if we
> try to preserve it long enough), is that it?
Something like that. The lambda's position currently gets preserved in
the generated code so that ME2 can use it.
> [ Tho, in more complex cases it becomes debatable whether the function's
> position should point to the position corresponding to ME1 or to that
> of ME2. ]
The code currently preserves both positions. :-) But only one buffer
name.
My latest thoughts on that are perhaps two file names (relative to the
Emacs top directory) would be better than one buffer name. Then I could
put buttons on the backtrace display which on being clicked would open
either of the source files at the right position.
> More generally: what goes wrong in the above example if you just treat
> that as a list of symbol (stripping them all of their position info).
> AFAICT when *that* macro is expanded (i.e. ME2) you'll presumably get
> code like
> (mapatoms #'(lambda (FOO/p) (DO/p SOME/p (THING/p))))
> right? [ where "/p" means that the symbol has a sympos. ]
> Isn't that sufficient info to add a docstring with position?
It's the lambda which has a position rather than the expanded bits from
ME2.
> >> (lambda (f) ..) *can* appear within a backquote without it being an
> >> actual lambda expression.
> >> What alternatives have you considered?
> > Not a lot of them, as yet. Maybe testing for (function (lambda ...))
> > would be safer.
> No matter how many extra tests you add to reduce the frequency, you're
> fundamentally adding a bug :-(
Yes. I'll see what I can do to remove that extra code from
backquote-process.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 10 Mar 2024 17:19:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 10 13:19:46 2024
Received: from localhost ([127.0.0.1]:37794 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rjMpt-0007jj-VJ
for submit <at> debbugs.gnu.org; Sun, 10 Mar 2024 13:19:46 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:60120)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rjMps-0007jW-45
for 67455 <at> debbugs.gnu.org; Sun, 10 Mar 2024 13:19:44 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 39295805EE;
Sun, 10 Mar 2024 13:19:05 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710091144;
bh=d+lgK36D+WN9f1IyJduzV+Qdd135l9IM023nhPOEqeI=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=azvDZBZ+mbhHzxNOKdT3PkYchOZjKxfnR/iS4i95+HlANt0ppm9cdMlPoo51UbNFj
34tarXfVwQEiQEPRlIrQoJhEhHUYuoCxey+RrokV4LH0iEZ4d48PBa0RgFu/6xaUPb
2trbhMfZLdZ5tmTZADNk3/Dz9W89w4BzIeBYNl6WZMG7/qVG879fXsfaMutX1nvzX3
mVke1epaSAQWPC2VxpFuFfWPZHdfLXbdhYNnm2YK1BixbEedsgbVKMdYMU4eA/vNxc
qjfpUGz7jMFa9owNyz+x0pq864uqpCNOVm5Z4apUB45Gn8++dCKopwDkTeFXrPI90C
OKutqTyUxycbQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 1979480240;
Sun, 10 Mar 2024 13:19:04 -0400 (EDT)
Received: from alfajor (69-165-147-56.dsl.teksavvy.com [69.165.147.56])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E1980120454;
Sun, 10 Mar 2024 13:19:03 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <Ze3Zl_hqtKBVrpbD@ACM> (Alan Mackenzie's message of "Sun, 10 Mar
2024 16:02:31 +0000")
Message-ID: <jwvcys2j9wp.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
<Ze3Zl_hqtKBVrpbD@ACM>
Date: Sun, 10 Mar 2024 13:19:03 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> I've got a version almost ready which actually does something, namely
> prefixes "anonymous" lines of a backtrace with the name of the defining
> symbol, like {foo} . It'll soon be time to start seriously thinking
> about what information ought to go there for the live version.
Cool!
>> - Testing `byte-compile-in-progress` can't be right. Do you to test
>> whether the result of this backquote will be byte-compiled or do you
>> really mean to test whether this backquote happens to be executed
>> during compilation (which could be because the compiler ends up
>> loading code while executing `eval-when-compile` or `require`)?
>
> Quite simply, during compilation, all symbols (except nil) get read with
> position, so to strip their positions here would be wrong.
This isn't quite right: during compilation, some code is read with
positions (the code that we will compile), but some code is read in the
normal way (the code we load for the purpose of running).
The distinction is important.
>> - My gut tells me that changing backquote can't be right.
>
> I tend to agree. I put the code into backquote-process when having
> problems with things like:
>
> (mapatoms #'(lambda (,(car spec)) ,@body)
>
> in cl-macs.el, where it's impossible to know where the doc string (if
> any) is until after the expansion of the backquotes, or even at run time
> (as here). In the latter case, rather than "posifying" the doc string
> at macro expansion time, we have to generate code to do it at run time.
Hmm... here what you call "run time" is really some later
macro-expansion, right? The `lambda` symbol comes from the first
macro-expansion (ME1), but the docstring comes from the second (ME2).
IIUC the problem you face is that you want to get the function's
position info from the `lambda` symbol, which here would be wrong (even if we
try to preserve it long enough), is that it?
[ Tho, in more complex cases it becomes debatable whether the function's
position should point to the position corresponding to ME1 or to that
of ME2. ]
More generally: what goes wrong in the above example if you just treat
that as a list of symbol (stripping them all of their position info).
AFAICT when *that* macro is expanded (i.e. ME2) you'll presumably get
code like
(mapatoms #'(lambda (FOO/p) (DO/p SOME/p (THING/p))))
right? [ where "/p" means that the symbol has a sympos. ]
Isn't that sufficient info to add a docstring with position?
>> (lambda (f) ..) *can* appear within a backquote without it being an
>> actual lambda expression.
>> What alternatives have you considered?
> Not a lot of them, as yet. Maybe testing for (function (lambda ...))
> would be safer.
No matter how many extra tests you add to reduce the frequency, you're
fundamentally adding a bug :-(
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 10 Mar 2024 16:03:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 10 12:03:17 2024
Received: from localhost ([127.0.0.1]:37750 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rjLdr-0005TN-7c
for submit <at> debbugs.gnu.org; Sun, 10 Mar 2024 12:03:17 -0400
Received: from mail.muc.de ([193.149.48.3]:61997)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rjLdn-0005T1-TL
for 67455 <at> debbugs.gnu.org; Sun, 10 Mar 2024 12:03:13 -0400
Received: (qmail 59807 invoked by uid 3782); 10 Mar 2024 17:02:32 +0100
Received: from acm.muc.de (pd953a236.dip0.t-ipconnect.de [217.83.162.54])
(using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Sun, 10 Mar 2024 17:02:32 +0100
Received: (qmail 31889 invoked by uid 1000); 10 Mar 2024 16:02:31 -0000
Date: Sun, 10 Mar 2024 16:02:31 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
Message-ID: <Ze3Zl_hqtKBVrpbD@ACM>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM> <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
Thanks for looking at my patch!
I've got a version almost ready which actually does something, namely
prefixes "anonymous" lines of a backtrace with the name of the defining
symbol, like {foo} . It'll soon be time to start seriously thinking
about what information ought to go there for the live version.
Unfortunately, I've still got two blocking issues to resolve - one is a
change in functionality to Fbare_symbol, which I've reported as a bug;
the other is some new code merged in from master, a with-eval-after-load
clause in pcase.el which causes havoc in my build.
Still, progress is being made.
On Sat, Mar 09, 2024 at 16:36:57 -0500, Stefan Monnier wrote:
> > I've just pushed a large commit to feature/positioned-lambdas, a work
> > in progress commit for bug#67455, putting source position information at
> > the start of doc strings. master was merged into it just before the
> > commit.
> I barely started to look at the code, but regarding the following hunk:
> diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
> index 6917128d70a..0d4681bc979 100644
> --- a/lisp/emacs-lisp/backquote.el
> +++ b/lisp/emacs-lisp/backquote.el
> @@ -172,6 +172,30 @@ backquote-process
> (backquote-delay-process s (1- level))))
> ((eq (car s) backquote-backquote-symbol)
> (backquote-delay-process s (1+ level)))
> + ;; Process a (lambda ...) form specially, since otherwise the
> + ;; lambda symbol would get separated from its introductory (,
> + ;; preventing this processing from being done elsewhere in macro
> + ;; expansion.
> + ((and (eq (car s) 'lambda)
> + (symbol-with-pos-p (car s))
> + (listp (car-safe (cdr s))))
> + (let ((kdr (backquote-process (cdr s) level))
> + (lambda-pos (symbol-with-pos-pos (car s)))
> + )
> + (if (null byte-compile-in-progress)
> + (setcar s 'lambda)) ; Strip the position.
> + (cond
> + ((= (car kdr) 0)
> + (cons (car kdr)
> + (list 'quote
> + (byte-run-posify-lambda-form
> + (cons (car s) (car (cdr (cdr kdr)))) ; Two cdr's to strip 'quote.
> + lambda-pos))))
> + (t
> + (cons 1
> + (list 'byte-run-posify-lambda-form
> + (list 'cons (list 'quote (car s)) (cdr kdr))
> + lambda-pos))))))
> (t
> (let ((rest s)
> item firstlist list lists expression)
> - Testing `byte-compile-in-progress` can't be right. Do you to test
> whether the result of this backquote will be byte-compiled or do you
> really mean to test whether this backquote happens to be executed
> during compilation (which could be because the compiler ends up
> loading code while executing `eval-when-compile` or `require`)?
Quite simply, during compilation, all symbols (except nil) get read with
position, so to strip their positions here would be wrong. Outside of
compilation (an evaluation of a defun, for example), only defined
symbols get positioned, and these symbols with position would interfere
with Emacs's working. So they get stripped of their positions as soon
as possible after the info has been transferred to the doc string. For
example, SWPs cannot be dumped by the portable dumper.
> - My gut tells me that changing backquote can't be right.
I tend to agree. I put the code into backquote-process when having
problems with things like:
(mapatoms #'(lambda (,(car spec)) ,@body)
in cl-macs.el, where it's impossible to know where the doc string (if
any) is until after the expansion of the backquotes, or even at run time
(as here). In the latter case, rather than "posifying" the doc string
at macro expansion time, we have to generate code to do it at run time.
> (lambda (f) ..) *can* appear within a backquote without it being an
> actual lambda expression.
> What alternatives have you considered?
Not a lot of them, as yet. Maybe testing for (function (lambda ...))
would be safer. But I think I should try and find some other way of
solving these problems than altering backquote.el, which should be
easier now that the code is basically working.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 9 Mar 2024 21:37:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 09 16:37:51 2024
Received: from localhost ([127.0.0.1]:34925 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rj4O7-0002jV-55
for submit <at> debbugs.gnu.org; Sat, 09 Mar 2024 16:37:51 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:40630)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rj4O3-0002jH-ND
for 67455 <at> debbugs.gnu.org; Sat, 09 Mar 2024 16:37:49 -0500
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id D6467100170;
Sat, 9 Mar 2024 16:37:05 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1710020223;
bh=aVEYQlb5wgJGI2359fAdvjT0E2hrKhLaZepwmW8GXC4=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=OtrF/o4X0z0Uc4ScRhWDusMuRTKi/TQqe1NvvXcRFlzN+/M6UNw+1RhH9rgJfDlpK
2fm4jZ/gU+GgR2ISSRvmSGJpNP2sORJ37FsyI+O/nvpDHYKMvcsCYILfHilsG0J9SG
Wiuw/hCXswy6gFF7lONvKEyUSNgXNNRvKcAvRguiVoKK8jotpG+xZfQaQi3u/4KW/G
PypHCCdz56p6t/UMTP6amlI1/2FUaYofx/P19mk8P/e2xt0PE8z9m9Bu9V/Z3fqXDx
dFVCjKEQz+9j7nx7gyI172cKvdAwsaH3sz4tNj/MR1COjDE6ArXKjGe+Pmn4W3PmhJ
D8w9H7mNB9M6w==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 83CB310005D;
Sat, 9 Mar 2024 16:37:03 -0500 (EST)
Received: from pastel (unknown [69.165.147.56])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3BA8212059D;
Sat, 9 Mar 2024 16:37:03 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: (Record source position, etc., in doc strings, and
use this in *Help* and backtraces.)
In-Reply-To: <ZeXq5qUN5blZ8IVT@ACM> (Alan Mackenzie's message of "Mon, 4 Mar
2024 15:38:14 +0000")
Message-ID: <jwv1q8jume6.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM>
<handler.67455.B.170100905232659.ack <at> debbugs.gnu.org>
<ZeXq5qUN5blZ8IVT@ACM>
Date: Sat, 09 Mar 2024 16:36:57 -0500
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> I've just pushed a large commit to feature/positioned-lambdas, a work
> in progress commit for bug#67455, putting source position information at
> the start of doc strings. master was merged into it just before the
> commit.
I barely started to look at the code, but regarding the following hunk:
diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
index 6917128d70a..0d4681bc979 100644
--- a/lisp/emacs-lisp/backquote.el
+++ b/lisp/emacs-lisp/backquote.el
@@ -172,6 +172,30 @@ backquote-process
(backquote-delay-process s (1- level))))
((eq (car s) backquote-backquote-symbol)
(backquote-delay-process s (1+ level)))
+ ;; Process a (lambda ...) form specially, since otherwise the
+ ;; lambda symbol would get separated from its introductory (,
+ ;; preventing this processing from being done elsewhere in macro
+ ;; expansion.
+ ((and (eq (car s) 'lambda)
+ (symbol-with-pos-p (car s))
+ (listp (car-safe (cdr s))))
+ (let ((kdr (backquote-process (cdr s) level))
+ (lambda-pos (symbol-with-pos-pos (car s)))
+ )
+ (if (null byte-compile-in-progress)
+ (setcar s 'lambda)) ; Strip the position.
+ (cond
+ ((= (car kdr) 0)
+ (cons (car kdr)
+ (list 'quote
+ (byte-run-posify-lambda-form
+ (cons (car s) (car (cdr (cdr kdr)))) ; Two cdr's to strip 'quote.
+ lambda-pos))))
+ (t
+ (cons 1
+ (list 'byte-run-posify-lambda-form
+ (list 'cons (list 'quote (car s)) (cdr kdr))
+ lambda-pos))))))
(t
(let ((rest s)
item firstlist list lists expression)
- Testing `byte-compile-in-progress` can't be right. Do you to test
whether the result of this backquote will be byte-compiled or do you
really mean to test whether this backquote happens to be executed
during compilation (which could be because the compiler ends up
loading code while executing `eval-when-compile` or `require`)?
- My gut tells me that changing backquote can't be right.
(lambda (f) ..) *can* appear within a backquote without it being an
actual lambda expression.
What alternatives have you considered?
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 4 Mar 2024 15:38:59 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 04 10:38:58 2024 Received: from localhost ([127.0.0.1]:44510 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rhAP1-0001DZ-25 for submit <at> debbugs.gnu.org; Mon, 04 Mar 2024 10:38:58 -0500 Received: from mail.muc.de ([193.149.48.3]:18410) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rhAOw-0001DF-Dx for 67455 <at> debbugs.gnu.org; Mon, 04 Mar 2024 10:38:54 -0500 Received: (qmail 37545 invoked by uid 3782); 4 Mar 2024 16:38:15 +0100 Received: from acm.muc.de (p4fe15ee5.dip0.t-ipconnect.de [79.225.94.229]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 04 Mar 2024 16:38:14 +0100 Received: (qmail 16868 invoked by uid 1000); 4 Mar 2024 15:38:14 -0000 Date: Mon, 4 Mar 2024 15:38:14 +0000 To: 67455 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Message-ID: <ZeXq5qUN5blZ8IVT@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <handler.67455.B.170100905232659.ack <at> debbugs.gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <handler.67455.B.170100905232659.ack <at> debbugs.gnu.org> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Eli and Stefan. On Sun, Nov 26, 2023 at 14:31:02 +0000, GNU bug Tracking System wrote: > Thank you for filing a new bug report with debbugs.gnu.org. [ .... ] I've just pushed a large commit to feature/positioned-lambdas, a work in progress commit for bug#67455, putting source position information at the start of doc strings. master was merged into it just before the commit. The main topic of the commit is putting position information into interpreted functions, and into lambda forms created at run time. Still missing is position information for defvars and defconsts, along with the same for cl-defmethods (which are complicated because the doc string has no fixed position). Also missing is the handling of Oclosures. There are also some diagnostic functions still in byte-run.el, as they are still useful. As yet, I'm not doing anything with this information, though I anticipate that will be quite easy. This commit, although it works, is unfinished, and still contains quite a few of my private change annotations, and lots of temporary changes to enable useful backtraces to be produced. make check runs without any errors. -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Stefan Kangas <stefankangas@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 15 Dec 2023 23:12:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Dec 15 18:12:53 2023
Received: from localhost ([127.0.0.1]:53736 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rEHMT-0002C1-DB
for submit <at> debbugs.gnu.org; Fri, 15 Dec 2023 18:12:53 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:25387)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rEHMR-0002Bm-9C
for 67455 <at> debbugs.gnu.org; Fri, 15 Dec 2023 18:12:51 -0500
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4B5B3444BC9;
Fri, 15 Dec 2023 18:12:45 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1702681963;
bh=ILel/5mtuiJEj50Ui1Xv8cfaOHrMxgrbJXPwgFjEWOo=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=Gz+QK41d7WAvEZwi3NGFEvKBSte5QPx3Mv3Tgb4nzTehPCF8YcZjGBhFldhHIDQcY
yHbQ9r3jWgLGaiboJkoWycipezOGYK5N/1RCRkQxIzTLJ0sjr6jWXxgeuf/U5HQ0dm
ePo5qT1Kee/CBblQ69W0ucXaPxWbI0hwDh4G2Mla/iOKyG+IAqhtO0HEhUvBJHulGN
ZH9IQQne2zDdUxo80Lu1qKTkOgY67l5fSTK8lPELaj2Kh2dO5ATRfSw42A8D38aeq7
EzqdLkYbWA+SDAUxWC43iiBniXIMb00dcKZJPEadZ44mneg2M+BUqPwQL5baIvUE9l
ocEuQ7Dav0nlw==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id CEC7D444BC2;
Fri, 15 Dec 2023 18:12:43 -0500 (EST)
Received: from pastel (65-110-221-238.cpe.pppoe.ca [65.110.221.238])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A8A191201CA;
Fri, 15 Dec 2023 18:12:43 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: Record source position, etc., in doc strings, and
use this in *Help* and backtraces.
In-Reply-To: <ZXyZrb5eK6a0Q1f-@ACM> (Alan Mackenzie's message of "Fri, 15 Dec
2023 18:23:41 +0000")
Message-ID: <jwvv88z3wgg.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM>
<jwv1qc1n80o.fsf-monnier+emacs@HIDDEN> <ZXyZrb5eK6a0Q1f-@ACM>
Date: Fri, 15 Dec 2023 18:12:42 -0500
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.167 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> The problem that has thrown me is the use of the doc string in oclosures
> for other purposes. For example, in position 2 of a lambda form,
> appears something like
>
> (:documentation 'oclosure-accessor)
Hmm...
> .. My current code is expecting, on encountering (:documentation ...),
> for the cadr to be a string,
`:documentation` is supposed to be followed by an expression (whose
evaluation should usually return a string, although indeed we abuse it
in the case of OClosures to return a symbol instead).
Regardless of OClosures, the `cadr` of a `:documentation` will very
rarely be a mere string since you don't need `:documentation` when the
docstring can be written as a literal.
> onto which it can add a concat form which
> will prefix the position information.
For the non-OClosure case you should be able to use the equivalent of:
(:documentation
(concat <YOURINFOHERE> <THEORIGINALEXPRESSIONHERE>))
is that what you mean?
> A solution to this problem would be to move the above symbol to element
> 2 of the list, something like
>
> (:documentation nil 'oclosure-accessor)
Here you're talking about the source code, right?
I think the question is rather what representation to use in the values:
OClosures store their type name (symbol) in the slot that normally holds
the docstring, so I think the best short term answer is to say that
OClosures aren't supported by your new feature.
OClosures are a mix of functions and structs, so while it would be
nice to make them enjoy your new feature, I think it's perfectly OK
to say that in this respect they're more like structs than like
functions.
Also they come with more introspection features than normal
functions, so it's usually easier to figure out where they come from
(you can find their type, and then look for the `oclosure-lambda`s
which built OClosures of this type. For most OClosure types, there's
only one such lambda anyway).
> , so that my new code could place its information in the now vacant
> position 1, giving something like
>
> (:documentation ";POS\36\0\0\0 [ .... ]\n" 'oclosure-accessor)
>
> .. What do you think of this idea, and have you any better ideas for a
> solution to the problem?
If it works it might be a good option. I don't know how this turns into
when we build the corresponding bytecode object, so it's hard to judge.
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 15 Dec 2023 18:23:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Dec 15 13:23:50 2023
Received: from localhost ([127.0.0.1]:53437 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rECqk-00040N-AT
for submit <at> debbugs.gnu.org; Fri, 15 Dec 2023 13:23:50 -0500
Received: from mail.muc.de ([193.149.48.3]:61245)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1rECqj-00040A-96
for 67455 <at> debbugs.gnu.org; Fri, 15 Dec 2023 13:23:49 -0500
Received: (qmail 9432 invoked by uid 3782); 15 Dec 2023 19:23:42 +0100
Received: from acm.muc.de (p4fe15b06.dip0.t-ipconnect.de [79.225.91.6]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Fri, 15 Dec 2023 19:23:42 +0100
Received: (qmail 5552 invoked by uid 1000); 15 Dec 2023 18:23:41 -0000
Date: Fri, 15 Dec 2023 18:23:41 +0000
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#67455: Record source position, etc., in doc strings, and use
this in *Help* and backtraces.
Message-ID: <ZXyZrb5eK6a0Q1f-@ACM>
References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM>
<jwv1qc1n80o.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwv1qc1n80o.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67455
Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan.
On Mon, Dec 04, 2023 at 13:33:25 -0500, Stefan Monnier wrote:
> > I've committed the first stage of this implementation in the new git
> > branch feature/positioned-lambdas.
> Haven't had enough time to look closely, so just some quick early comment.
.....
I'm making steady, if not rapid, progress on the use of doc strings to
store position information. There have been several unexpected
problems (which is only to be expected), all of which, bar one, I've
been able to solve.
The problem that has thrown me is the use of the doc string in oclosures
for other purposes. For example, in position 2 of a lambda form,
appears something like
(:documentation 'oclosure-accessor)
.. My current code is expecting, on encountering (:documentation ...),
for the cadr to be a string, onto which it can add a concat form which
will prefix the position information.
A solution to this problem would be to move the above symbol to element
2 of the list, something like
(:documentation nil 'oclosure-accessor)
, so that my new code could place its information in the now vacant
position 1, giving something like
(:documentation ";POS\36\0\0\0 [ .... ]\n" 'oclosure-accessor)
.. What do you think of this idea, and have you any better ideas for a
solution to the problem?
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 4 Dec 2023 23:00:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 04 18:00:24 2023
Received: from localhost ([127.0.0.1]:36064 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rAHvM-00005a-9E
for submit <at> debbugs.gnu.org; Mon, 04 Dec 2023 18:00:24 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:41127)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rAHvK-00005J-PQ
for 67455 <at> debbugs.gnu.org; Mon, 04 Dec 2023 18:00:23 -0500
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 8320D80663;
Mon, 4 Dec 2023 18:00:06 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1701730805;
bh=a2IWF1vz3uRLCX1lx9dwEzToD8TM/XB/WKJxo7CMdr0=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=HuGl7iadcIRKoLOQwuoQVPjsAiVJcY6c+NhwWXbB64rrjS7uDvX45L+Q6J8gE98jd
hWF9ke/c51b1YqKYO4SP9Z+C2R8U8MZLrQCfBOX1qtkfpUwR2gQ4ICl5dwWEU4reMp
si5OgGZSExyKJC3XQOaZot/2GGonphmSgEDHKjPP8iw984STqYOl5R+2sC4iMFSkq1
Pg7QFrBv69fp2ElMyL5pmEGnBzwGJ9k+UemPaDNkzxVvDE82tfuuCSwfF3ubSnasB0
Aqh+SXkp+/Lsp/RMMHzvw+soa5yuKuvm7Ta+2lDGBvSZbuHO5g7maOwJ/m0p7W2v6C
L8X3+cH/KU1Dg==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 6D1268054F;
Mon, 4 Dec 2023 18:00:05 -0500 (EST)
Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 52848120388;
Mon, 4 Dec 2023 18:00:05 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: Record source position, etc., in doc strings, and
use this in *Help* and backtraces.
In-Reply-To: <ZW5TB-6nnwXWkLfb@ACM> (Alan Mackenzie's message of "Mon, 4 Dec
2023 22:30:31 +0000")
Message-ID: <jwvfs0hk2zw.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM>
<jwv1qc1n80o.fsf-monnier+emacs@HIDDEN> <ZW5FY57AbZiixJ-0@ACM>
<jwv34whlk4t.fsf-monnier+emacs@HIDDEN> <ZW5TB-6nnwXWkLfb@ACM>
Date: Mon, 04 Dec 2023 17:59:46 -0500
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.092 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Maybe, though, it would be useful in some circumstances to record the
> buffer the definition was loaded from if it's not a file. But it's
> getting rather late here, so I haven't thought this through at all.
I think if it comes from a buffer (and one that's not associated with
a file, to boot), it's usually not byte-compiled, so I suspect you'll be
better off confining that handling to the interpreted case.
>> >> > the position of the defining symbol in the file, and the position of
>> >> > the lambda (should we be in one) in the file.
>> >> Why two positions?
>> >> Why not use the same position info as used in `byte-compile-warn-x`?
>> > I'm not sure, yet, but I suspect both positions will be useful.
>> Can you give an example scenario?
> Something about a lambda form, when we need to find the lambda itself,
> but also its containing form. Sorry I can't be more definite, yet.
I'm having a hard time imagining why you'd need two positions. Instead,
I want to go to *the* position, which should be "the best we can come up with".
And that's also what `byte-compile-warn-x` wants and it has access to
the same information.
IOW if you think you can do better for lambdas-with-pos, then please try
and retro fit it into `byte-compile-warn-x`.
>> Ah, got it. I like the way you can refer to args by name.
>> So I guess all the (defining-symbol 1) could similarly be replaced with
>> (defining-symbol THE-ARG-NAME).
> Yes, I hadn't really thought of that. But (defining-symbol 1) matches
> similar forms like (docstring 3) which are used somewhere.
(defining-symbol N) is nice as well.
> Ah, the debug spec; yet another difficult domain specific language. ;-(
Yup. It's even worse, because at this stage I don't think anyone
truly understands its semantics.
[ I just wish we had a "similar" DSL that served /all/ those purposes,
i.e. would let the macro writer express where the docstring is (if any),
where the defining names are, how to indent each part, how to instrument
each part, ...
And of course, with a well-defined and well-understood semantics,
a simple implementation (that works efficiently for all those uses),
and a simple&concise syntax.
Oh and while at it, it would also be used during macro-expansion to
provide meaningful error messages when the macro is used
incorrectly.
Clearly, [Fortifying macros](https://dl.acm.org/doi/10.1145/1863543.1863577)
can provide useful inspiration. ]
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 4 Dec 2023 22:30:51 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 04 17:30:51 2023 Received: from localhost ([127.0.0.1]:36013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rAHSl-0007cM-8o for submit <at> debbugs.gnu.org; Mon, 04 Dec 2023 17:30:51 -0500 Received: from mail.muc.de ([193.149.48.3]:24296) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rAHSi-0007c9-Re for 67455 <at> debbugs.gnu.org; Mon, 04 Dec 2023 17:30:49 -0500 Received: (qmail 89250 invoked by uid 3782); 4 Dec 2023 23:30:32 +0100 Received: from acm.muc.de (pd953aae0.dip0.t-ipconnect.de [217.83.170.224]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 04 Dec 2023 23:30:31 +0100 Received: (qmail 13735 invoked by uid 1000); 4 Dec 2023 22:30:31 -0000 Date: Mon, 4 Dec 2023 22:30:31 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: Record source position, etc., in doc strings, and use this in *Help* and backtraces. Message-ID: <ZW5TB-6nnwXWkLfb@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM> <jwv1qc1n80o.fsf-monnier+emacs@HIDDEN> <ZW5FY57AbZiixJ-0@ACM> <jwv34whlk4t.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwv34whlk4t.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Mon, Dec 04, 2023 at 16:56:41 -0500, Stefan Monnier wrote: > >> Why include the file name? Presumably we will rely on a `(FILE . POS)` > >> to find that docstring so including FILE in there is rather redundant > >> (and in addition to that, this file name can be wrong if it's absolute > >> since files often get moved between compilation and use). > > No, the (FILE . POS) is the .elc file, the one I'm putting into the new > > info is the source file. > Until now, Emacs has always lived with the `.elc` file names (e.g. in > `load-history`) and managed to find the corresponding `.el` file (when > you try to jump to the source via `M-.` or by clicking in `C-h f`). > It comes with its share of problems, but we've learned to live with them. Yes. I've come to the same conclusion in the last hour or so that, no doubt, early Emacs hackers reached forty years ago. We can find the ..elc full filename from load-history (via symbol-file), but there's no more useful way of finding the corresponding source than removing the 'c' from the end of foo.elc. And if it's not there, it's nowhere. > Is there any reason you think this new functionality should go through > the extra trouble to record the `.el` name (and thus develop its own > set of workarounds for the cases where the `.el` doesn't live where we > think it should)? Not any more, having thought it through. Maybe, though, it would be useful in some circumstances to record the buffer the definition was loaded from if it's not a file. But it's getting rather late here, so I haven't thought this through at all. > >> > the position of the defining symbol in the file, and the position of > >> > the lambda (should we be in one) in the file. > >> Why two positions? > >> Why not use the same position info as used in `byte-compile-warn-x`? > > I'm not sure, yet, but I suspect both positions will be useful. > Can you give an example scenario? Something about a lambda form, when we need to find the lambda itself, but also its containing form. Sorry I can't be more definite, yet. When I start hacking out the code to use all this information, it will become clearer what we actually need. It's very easy to change at this stage. > >> > or (defining-symbol (if (consp struct) (car struct) struct)) > >> [ Haven't seen that yet. ] > > Have a look at cl-defgeneric and cl-defmethod in cl-generic.el. > Ah, got it. I like the way you can refer to args by name. > So I guess all the (defining-symbol 1) could similarly be replaced with > (defining-symbol THE-ARG-NAME). Yes, I hadn't really thought of that. But (defining-symbol 1) matches similar forms like (docstring 3) which are used somewhere. > [ I also notice that this extension is somewhat incompatible with the > use I proposed for `font-lock`. :-( ] > [ As you noted, this info is also related to the `&define` debug spec. > I wish we could unify that information. ] Ah, the debug spec; yet another difficult domain specific language. ;-( > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 4 Dec 2023 21:57:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 04 16:57:24 2023
Received: from localhost ([127.0.0.1]:35974 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rAGwO-0006fE-5n
for submit <at> debbugs.gnu.org; Mon, 04 Dec 2023 16:57:24 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:1404)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rAGwM-0006f2-58
for 67455 <at> debbugs.gnu.org; Mon, 04 Dec 2023 16:57:22 -0500
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AAF3D44447D;
Mon, 4 Dec 2023 16:57:05 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1701727020;
bh=G/fkIM6F5zW3O90JBC531aFzoZ/F1Uhe0vO4lpHl/GI=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=mefNrHWk2AArvMPh8rexl+LnVNjkiEKCLyFOs94TpBcohqI2IBqO4S6cctcY/QNxV
ERnazDZ/yEJe2XrCDvhAcpATvb87sSMbjjjwagOWQfogaEoEzsubiytG7ma+wbsoSi
tnDqM1wzQMXPAqjK4LS4KltMv+IW/STqg2brua6pMuWqvLWJs2tyqUtFJsSF2BqZN/
fTsIcmqKFwezS5ia9oWC6HiYAnQo0yTg8Ez/imH+CsDY+J5SShCzjLwm4h6fMhn1nA
LfbNmjp2XBXzXP16nCSf9KCn6zBRN+ef2GQm1FUAPluBvkfNEupMSjIOuhsnm4inhr
BWOZ8HjXPH2+A==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4EBA9444449;
Mon, 4 Dec 2023 16:57:00 -0500 (EST)
Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3B5E11203A1;
Mon, 4 Dec 2023 16:57:00 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: Record source position, etc., in doc strings, and
use this in *Help* and backtraces.
In-Reply-To: <ZW5FY57AbZiixJ-0@ACM> (Alan Mackenzie's message of "Mon, 4 Dec
2023 21:32:19 +0000")
Message-ID: <jwv34whlk4t.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM>
<jwv1qc1n80o.fsf-monnier+emacs@HIDDEN> <ZW5FY57AbZiixJ-0@ACM>
Date: Mon, 04 Dec 2023 16:56:41 -0500
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.119 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> Why include the file name? Presumably we will rely on a `(FILE . POS)`
>> to find that docstring so including FILE in there is rather redundant
>> (and in addition to that, this file name can be wrong if it's absolute
>> since files often get moved between compilation and use).
> No, the (FILE . POS) is the .elc file, the one I'm putting into the new
> info is the source file.
Until now, Emacs has always lived with the `.elc` file names (e.g. in
`load-history`) and managed to find the corresponding `.el` file (when
you try to jump to the source via `M-.` or by clicking in `C-h f`).
It comes with its share of problems, but we've learned to live with them.
Is there any reason you think this new functionality should go through
the extra trouble to record the `.el` name (and thus develop its own
set of workarounds for the cases where the `.el` doesn't live where we
think it should)?
>> > the position of the defining symbol in the file, and the position of
>> > the lambda (should we be in one) in the file.
>> Why two positions?
>> Why not use the same position info as used in `byte-compile-warn-x`?
> I'm not sure, yet, but I suspect both positions will be useful.
Can you give an example scenario?
>> > or (defining-symbol (if (consp struct) (car struct) struct))
>> [ Haven't seen that yet. ]
> Have a look at cl-defgeneric and cl-defmethod in cl-generic.el.
Ah, got it. I like the way you can refer to args by name.
So I guess all the (defining-symbol 1) could similarly be replaced with
(defining-symbol THE-ARG-NAME).
[ I also notice that this extension is somewhat incompatible with the
use I proposed for `font-lock`. :-( ]
[ As you noted, this info is also related to the `&define` debug spec.
I wish we could unify that information. ]
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 4 Dec 2023 21:32:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 04 16:32:40 2023 Received: from localhost ([127.0.0.1]:35933 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rAGYR-0005th-JJ for submit <at> debbugs.gnu.org; Mon, 04 Dec 2023 16:32:39 -0500 Received: from mail.muc.de ([193.149.48.3]:22632) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rAGYP-0005tS-JQ for 67455 <at> debbugs.gnu.org; Mon, 04 Dec 2023 16:32:38 -0500 Received: (qmail 23399 invoked by uid 3782); 4 Dec 2023 22:32:20 +0100 Received: from acm.muc.de (pd953aae0.dip0.t-ipconnect.de [217.83.170.224]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 04 Dec 2023 22:32:20 +0100 Received: (qmail 13521 invoked by uid 1000); 4 Dec 2023 21:32:19 -0000 Date: Mon, 4 Dec 2023 21:32:19 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#67455: Record source position, etc., in doc strings, and use this in *Help* and backtraces. Message-ID: <ZW5FY57AbZiixJ-0@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM> <jwv1qc1n80o.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwv1qc1n80o.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: acm@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. Thanks for such a quick response. On Mon, Dec 04, 2023 at 13:33:25 -0500, Stefan Monnier wrote: > > I've committed the first stage of this implementation in the new git > > branch feature/positioned-lambdas. > Haven't had enough time to look closely, so just some quick early comment. > > It creates for compiled functions (but not yet for interpreted ones) doc > > strings starting with a specially formatted string containing the > > defining symbol, the source file name, > Why include the file name? Presumably we will rely on a `(FILE . POS)` > to find that docstring so including FILE in there is rather redundant > (and in addition to that, this file name can be wrong if it's absolute > since files often get moved between compilation and use). No, the (FILE . POS) is the .elc file, the one I'm putting into the new info is the source file. But you're right about an absolute name (as it currently is) not being the Right Thing. A lot of Emacs users will be using binaries compiled by somebody else on a different machine, so an absolute name is unhelpful, as well as divulging information about the builder's file structure. Maybe it should be lisp/emacs-lisp/bytecomp.el rather than /home/acm/emacs/emacs.git/sub-master-a/lisp/emacs-lisp/bytecomp.el. At least it would be shorter. > > the position of the defining symbol in the file, and the position of > > the lambda (should we be in one) in the file. > Why two positions? > Why not use the same position info as used in `byte-compile-warn-x`? I'm not sure, yet, but I suspect both positions will be useful. If not, it's trivial to change the contents of the new info, at least it is now. > > To instruct a defining macro to set a defining-symbol, it is only > > necessary to add a clause like (defining-symbol 1) > Sounds good. > [ Hopefully we can eventually use that info for `font-lock` so we can > get rid of ad-hoc rules for `defun` and friends. It would require an > additional piece of info (not only the position but also the kind of > definition, to distinguish `defun` from `defvar`). ] > > or (defining-symbol (if (consp struct) (car struct) struct)) > [ Haven't seen that yet. ] Have a look at cl-defgeneric and cl-defmethod in cl-generic.el. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
Received: (at 67455) by debbugs.gnu.org; 4 Dec 2023 18:34:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 04 13:34:02 2023
Received: from localhost ([127.0.0.1]:35693 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1rADla-0006g1-GW
for submit <at> debbugs.gnu.org; Mon, 04 Dec 2023 13:34:02 -0500
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:20268)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1rADlZ-0006fZ-CT
for 67455 <at> debbugs.gnu.org; Mon, 04 Dec 2023 13:34:01 -0500
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 43E25100068;
Mon, 4 Dec 2023 13:33:45 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1701714824;
bh=D2CtDjg1nFCQxwN1bKAyoQ8U52PBE9ESuGs2dgE1Mtk=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=mp+fLueb0ODA+g3GU4E/uQUctcBE4Ncw/0Mu/K6+hTbnE9eetkz9kyBcwfkS7jvYv
8dsWObXbFcML9jVn8iQ/WWDVACNS+4bUptO5g2MBWM5nDLtH5FBTZAqUZOOKdbtuON
ty0s1yyhc+ZCMbPnQOxJ8gmXgaHxOmS3T5FGI+JWwAjCHMTiW2QY4wmKBfJC4cmFrw
LD3afEtVv/M3ttkno6DAUFP2Lmc7XtyKRk9I64tSAJk6EcqfIySMAKFoQ75pby4thx
OF7u0gUnb7Z+2HhVLeZKqlU8WZjKr66VrSnYSUJGe/StEZnjHHPj0YqtSY18wlIaKa
JZsvq4gLBBLwQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 9A7F8100033;
Mon, 4 Dec 2023 13:33:44 -0500 (EST)
Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 83CF91203A2;
Mon, 4 Dec 2023 13:33:44 -0500 (EST)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#67455: Record source position, etc., in doc strings, and
use this in *Help* and backtraces.
In-Reply-To: <ZW4OJNN4W1rP-c96@ACM> (Alan Mackenzie's message of "Mon, 4 Dec
2023 17:36:36 +0000")
Message-ID: <jwv1qc1n80o.fsf-monnier+emacs@HIDDEN>
References: <ZWNWgUmzYrCRZ65G@ACM> <ZW4OJNN4W1rP-c96@ACM>
Date: Mon, 04 Dec 2023 13:33:25 -0500
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.101 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain T_SCC_BODY_TEXT_LINE -0.01 -
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67455
Cc: Eli Zaretskii <eliz@HIDDEN>, 67455 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> I've committed the first stage of this implementation in the new git
> branch feature/positioned-lambdas.
Haven't had enough time to look closely, so just some quick early comment.
> It creates for compiled functions (but not yet for interpreted ones) doc
> strings starting with a specially formatted string containing the
> defining symbol, the source file name,
Why include the file name? Presumably we will rely on a `(FILE . POS)`
to find that docstring so including FILE in there is rather redundant
(and in addition to that, this file name can be wrong if it's absolute
since files often get moved between compilation and use).
> the position of the defining symbol in the file, and the position of
> the lambda (should we be in one) in the file.
Why two positions?
Why not use the same position info as used in `byte-compile-warn-x`?
> To instruct a defining macro to set a defining-symbol, it is only
> necessary to add a clause like (defining-symbol 1)
Sounds good.
[ Hopefully we can eventually use that info for `font-lock` so we can
get rid of ad-hoc rules for `defun` and friends. It would require an
additional piece of info (not only the position but also the kind of
definition, to distinguish `defun` from `defvar`). ]
> or (defining-symbol (if (consp struct) (car struct) struct))
[ Haven't seen that yet. ]
Stefan
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at 67455) by debbugs.gnu.org; 4 Dec 2023 17:36:56 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 04 12:36:56 2023 Received: from localhost ([127.0.0.1]:35596 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rACsK-00059A-44 for submit <at> debbugs.gnu.org; Mon, 04 Dec 2023 12:36:56 -0500 Received: from mail.muc.de ([193.149.48.3]:15852) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1rACsH-00058y-Vt for 67455 <at> debbugs.gnu.org; Mon, 04 Dec 2023 12:36:54 -0500 Received: (qmail 53292 invoked by uid 3782); 4 Dec 2023 18:36:37 +0100 Received: from acm.muc.de (pd953aae0.dip0.t-ipconnect.de [217.83.170.224]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 04 Dec 2023 18:36:36 +0100 Received: (qmail 21416 invoked by uid 1000); 4 Dec 2023 17:36:36 -0000 Date: Mon, 4 Dec 2023 17:36:36 +0000 To: 67455 <at> debbugs.gnu.org Subject: Re: bug#67455: Record source position, etc., in doc strings, and use this in *Help* and backtraces. Message-ID: <ZW4OJNN4W1rP-c96@ACM> References: <ZWNWgUmzYrCRZ65G@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <ZWNWgUmzYrCRZ65G@ACM> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67455 Cc: Eli Zaretskii <eliz@HIDDEN>, Stefan Monnier <monnier@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan and Eli. On Sun, Nov 26, 2023 at 14:30:25 +0000, Alan Mackenzie wrote: > Hello, Emacs. > The reasons for this bug and the ways to solve it have been discussed > extensively in the thread for bug#66750. > We will, in functions' and macros' doc strings record the file name and > position of the source code, and possibly other items, in a machine > parseable format (which has yet to be decided). > This is particularly intended for lambda functions, which currently > appear in backtraces and *Help* buffers with insufficient information to > identify them. These displays will be enhanced to identify these lambda > functions satisfactorally. > The functions in Emacs which currently use doc strings will be modified > so as not to be negatively affected by the new information written into > them. I've committed the first stage of this implementation in the new git branch feature/positioned-lambdas. It creates for compiled functions (but not yet for interpreted ones) doc strings starting with a specially formatted string containing the defining symbol, the source file name, the position of the defining symbol in the file, and the position of the lambda (should we be in one) in the file. To instruct a defining macro to set a defining-symbol, it is only necessary to add a clause like (defining-symbol 1) or (defining-symbol (if (consp struct) (car struct) struct)) to the macro's declare form. This is how defun now works. (Thank you Stefan for the handy "cl.el feature" in byte-run--parse-declarations. ;-) I scanned our Lisp code for debug-spec &define keywords, and added defining-symbol declare clauses to (most of) the ones I found. As yet, no use is made of the new position information (beyond stripping it off for other uses of the doc string). > -- > Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.Received: (at submit) by debbugs.gnu.org; 26 Nov 2023 14:30:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Nov 26 09:30:52 2023 Received: from localhost ([127.0.0.1]:40984 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1r7G9r-0008Uh-VU for submit <at> debbugs.gnu.org; Sun, 26 Nov 2023 09:30:52 -0500 Received: from lists.gnu.org ([2001:470:142::17]:43810) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1r7G9o-0008UQ-13 for submit <at> debbugs.gnu.org; Sun, 26 Nov 2023 09:30:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <acm@HIDDEN>) id 1r7G9Z-0000Fq-EX for bug-gnu-emacs@HIDDEN; Sun, 26 Nov 2023 09:30:33 -0500 Received: from mail.muc.de ([193.149.48.3]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <acm@HIDDEN>) id 1r7G9W-0004fe-DJ for bug-gnu-emacs@HIDDEN; Sun, 26 Nov 2023 09:30:33 -0500 Received: (qmail 81305 invoked by uid 3782); 26 Nov 2023 15:30:26 +0100 Received: from acm.muc.de (p4fe15a0f.dip0.t-ipconnect.de [79.225.90.15]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 26 Nov 2023 15:30:25 +0100 Received: (qmail 8920 invoked by uid 1000); 26 Nov 2023 14:30:25 -0000 Date: Sun, 26 Nov 2023 14:30:25 +0000 To: bug-gnu-emacs@HIDDEN Subject: Record source position, etc., in doc strings, and use this in *Help* and backtraces. Message-ID: <ZWNWgUmzYrCRZ65G@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN Received-SPF: pass client-ip=193.149.48.3; envelope-from=acm@HIDDEN; helo=mail.muc.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.1 (/) Hello, Emacs. The reasons for this bug and the ways to solve it have been discussed extensively in the thread for bug#66750. We will, in functions' and macros' doc strings record the file name and position of the source code, and possibly other items, in a machine parseable format (which has yet to be decided). This is particularly intended for lambda functions, which currently appear in backtraces and *Help* buffers with insufficient information to identify them. These displays will be enhanced to identify these lambda functions satisfactorally. The functions in Emacs which currently use doc strings will be modified so as not to be negatively affected by the new information written into them. -- Alan Mackenzie (Nuremberg, Germany).
Alan Mackenzie <acm@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#67455; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.