Received: (at 73725) by debbugs.gnu.org; 9 Nov 2024 12:42:36 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 09 07:42:36 2024 Received: from localhost ([127.0.0.1]:53827 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t9knT-0002Dt-Ru for submit <at> debbugs.gnu.org; Sat, 09 Nov 2024 07:42:36 -0500 Received: from mail.muc.de ([193.149.48.3]:27438) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t9knR-0002DY-4o for 73725 <at> debbugs.gnu.org; Sat, 09 Nov 2024 07:42:34 -0500 Received: (qmail 54658 invoked by uid 3782); 9 Nov 2024 13:42:26 +0100 Received: from muc.de (pd953a3cb.dip0.t-ipconnect.de [217.83.163.203]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 09 Nov 2024 13:42:26 +0100 Received: (qmail 10049 invoked by uid 1000); 9 Nov 2024 12:42:26 -0000 Date: Sat, 9 Nov 2024 12:42:26 +0000 To: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <Zy9YspWyf4hzvr7z@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <jwvh6967oir.fsf-monnier+emacs@HIDDEN> <ZxefBoBT1hGRf08W@HIDDEN> <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN> <ZxjcbhlkHhE2_-Dn@HIDDEN> <26A5812D-9C23-4DB8-8E8D-BD958730AF6C@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <26A5812D-9C23-4DB8-8E8D-BD958730AF6C@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Stefan Monnier <monnier@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Mattias. On Wed, Oct 23, 2024 at 19:49:13 +0200, Mattias Engdegård wrote: > 23 okt. 2024 kl. 13.22 skrev Alan Mackenzie <acm@HIDDEN>: > > Hello, Stefan and Mattias. > > I've incorporated your (Stafan's) suggestions from yesterday into the > > code. See the patch below. > Thank you, I'll try to get some time to look at it .... Ping! The patch for this bug, accepted as satisfactory by Stefan M, has been ready for over two weeks. Do you actually have any comments on it? -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 23 Oct 2024 19:03:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 23 15:03:42 2024 Received: from localhost ([127.0.0.1]:60835 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t3gdy-0002of-8x for submit <at> debbugs.gnu.org; Wed, 23 Oct 2024 15:03:42 -0400 Received: from mail.muc.de ([193.149.48.3]:43201) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t3gdw-0002oO-Ey for 73725 <at> debbugs.gnu.org; Wed, 23 Oct 2024 15:03:41 -0400 Received: (qmail 62353 invoked by uid 3782); 23 Oct 2024 21:03:03 +0200 Received: from muc.de (pd953aa34.dip0.t-ipconnect.de [217.83.170.52]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 23 Oct 2024 21:03:03 +0200 Received: (qmail 1870 invoked by uid 1000); 23 Oct 2024 19:03:03 -0000 Date: Wed, 23 Oct 2024 19:03:02 +0000 To: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <ZxlIZndsmmQdWknZ@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <jwvh6967oir.fsf-monnier+emacs@HIDDEN> <ZxefBoBT1hGRf08W@HIDDEN> <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN> <ZxjcbhlkHhE2_-Dn@HIDDEN> <26A5812D-9C23-4DB8-8E8D-BD958730AF6C@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <26A5812D-9C23-4DB8-8E8D-BD958730AF6C@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Stefan Monnier <monnier@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Mattias. On Wed, Oct 23, 2024 at 19:49:13 +0200, Mattias Engdegård wrote: > 23 okt. 2024 kl. 13.22 skrev Alan Mackenzie <acm@HIDDEN>: > > I've incorporated your (Stafan's) suggestions from yesterday into > > the code. See the patch below. > Thank you, I'll try to get some time to look at it -- a couple of > quick notes: > - Isn't the code mutating the return value of a macro? I may be > mistaken but surely it could be a constant. Not really. It's preserving a label on the code between the source and the code generated by the macro. It has no effect whatsoever on the code generated. It merely records the source position for the display of warning and error messages. Whether that label is regarded as part of the "value" seems to be rather an academic question. > - The unknown free variable warning which was the motivation of this > issue should really be moved from codegen to the frontend (ie, cconv > at present). Unless someone can think of a reason why it couldn't (I > can't). That seems unrelated to the two bugs here. Regardless of where the warnings/errors are generated, the correct source file position needs to be attached to the pertinent source form. > Sorry if these were mentioned in your discussion; I haven't read it > through yet. -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 23 Oct 2024 17:50:48 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 23 13:50:48 2024 Received: from localhost ([127.0.0.1]:60711 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t3fVP-0007t9-R2 for submit <at> debbugs.gnu.org; Wed, 23 Oct 2024 13:50:48 -0400 Received: from mail-lf1-f52.google.com ([209.85.167.52]:53474) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mattias.engdegard@HIDDEN>) id 1t3fVN-0007sw-KH; Wed, 23 Oct 2024 13:50:46 -0400 Received: by mail-lf1-f52.google.com with SMTP id 2adb3069b0e04-53b13ea6b78so53693e87.2; Wed, 23 Oct 2024 10:50:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729705755; x=1730310555; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=tt3b3f5kDlzn/rEPo8oE3aS3s0wbtKf4MkmYyBdnXvg=; b=mKQVqtlJjBtL3LCcPpNNEVidLUvz5I4fFXF+uzuW16u8gVhJGapolvxd9iCu8uUAKG OVQhAry6d5Dgi6qkcuH0xZvQNmOWO3O3VsQHgZVd0azYKFKnha7vumm9ETDs5h2h3btz CSZrD7ymjmXmgM+11qKI1bMOWz23Q7VKWeIR3xIyr3CqmmYPRT+SqvDz/BSwbiEbWfCU yd6zmXajRCjmdIr3hkGjk/YkI/jPMph9PNRsVbftbra3T5/oxX7QsQ1NmsOUU82l6dSo Geo2KmXbXNvQG6fg7s96+HTy9WzzGsdNIdcRPejhE0Wwx9tzh5MgiaIsVH3MjLtDLAHH GMHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729705755; x=1730310555; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=tt3b3f5kDlzn/rEPo8oE3aS3s0wbtKf4MkmYyBdnXvg=; b=jj5AJ5sIOYg3bDzb7w7lt8vK02NBDWDWCEYCMRdTZPKIjyb4Mx6k/o/I/GXOS7G7qT HgC+Y0whv8qtsKN7gnWILOySeiGg/1P8mcosBLBGojuYEqqshnQDnv9FIIu2P/dHOZZU wULogsbO81VaUY+90IhOo//c14NNatTD/kv5Yor8B81EW21CtWeSP4y3I5WHAZ4sZKap C9X1AV7ZYnI3GbXYO7vk4NFLY/RpworkBsS0hwPehT2uaBaR+TH7b760iXE/YCQ7O6Hc dhf9+4efJ1tJdMBEyfYck4aoON1AE83bkCeLDTFxZylhzdpHyDst+/lbaug0RaFqaJ1N 0XyA== X-Forwarded-Encrypted: i=1; AJvYcCVoeIvP4kjOBMv0k3jwghdJ6zj19TvhizgupNlNkxLhD1XZos4z/dxOR+6z0RNVZM5G1eNhQ64=@debbugs.gnu.org, AJvYcCX3nNBys5KHzHsnWSJq3ZAByOqsUgPsO5DP+msnjrHyMM/CJ/Qj/dW2WDoLbQGYNj6sFDGmvA==@debbugs.gnu.org X-Gm-Message-State: AOJu0YwyYF2oF/SgLMXwmQcqaT64e1uC9z22YlwXaqYhW7QQIhKe/faS RQuZmnUEUT8VC/kFpmB6IOHiB1lTxbRkwZfpsz47OYJGdQIB7Nc8 X-Google-Smtp-Source: AGHT+IHh+yz2FLF+YV+2haIqzzFDaHHAtdUB5artIMD1cxZQ2WxmTeuJ/BSA5kuD+APTiqkLGAETsg== X-Received: by 2002:a05:6512:334d:b0:53b:1fea:baa9 with SMTP id 2adb3069b0e04-53b1feac190mr1508554e87.8.1729705754753; Wed, 23 Oct 2024 10:49:14 -0700 (PDT) Received: from smtpclient.apple (c188-150-183-180.bredband.tele2.se. [188.150.183.180]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-53a223e5365sm1129240e87.48.2024.10.23.10.49.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Oct 2024 10:49:14 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= <mattias.engdegard@HIDDEN> In-Reply-To: <ZxjcbhlkHhE2_-Dn@HIDDEN> Date: Wed, 23 Oct 2024 19:49:13 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <26A5812D-9C23-4DB8-8E8D-BD958730AF6C@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <jwvh6967oir.fsf-monnier+emacs@HIDDEN> <ZxefBoBT1hGRf08W@HIDDEN> <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN> <ZxjcbhlkHhE2_-Dn@HIDDEN> To: Alan Mackenzie <acm@HIDDEN> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) 23 okt. 2024 kl. 13.22 skrev Alan Mackenzie <acm@HIDDEN>: > Hello, Stefan and Mattias. >=20 > I've incorporated your (Stafan's) suggestions from yesterday into the > code. See the patch below. Thank you, I'll try to get some time to look at it -- a couple of quick = notes: - Isn't the code mutating the return value of a macro? I may be mistaken = but surely it could be a constant. - The unknown free variable warning which was the motivation of this = issue should really be moved from codegen to the frontend (ie, cconv at = present). Unless someone can think of a reason why it couldn't (I = can't). Sorry if these were mentioned in your discussion; I haven't read it = through yet.
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 23 Oct 2024 17:45:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 23 13:45:03 2024 Received: from localhost ([127.0.0.1]:60690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t3fPq-0007YC-TG for submit <at> debbugs.gnu.org; Wed, 23 Oct 2024 13:45:03 -0400 Received: from mail.muc.de ([193.149.48.3]:20723) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t3fPn-0007XR-Tt for 73725 <at> debbugs.gnu.org; Wed, 23 Oct 2024 13:45:01 -0400 Received: (qmail 71261 invoked by uid 3782); 23 Oct 2024 19:44:22 +0200 Received: from muc.de (pd953aa34.dip0.t-ipconnect.de [217.83.170.52]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 23 Oct 2024 19:44:22 +0200 Received: (qmail 700 invoked by uid 1000); 23 Oct 2024 17:44:22 -0000 Date: Wed, 23 Oct 2024 17:44:22 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <Zxk19m0Gn60zMl4t@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <jwvh6967oir.fsf-monnier+emacs@HIDDEN> <ZxefBoBT1hGRf08W@HIDDEN> <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN> <ZxjcbhlkHhE2_-Dn@HIDDEN> <jwva5ev0xn6.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <jwva5ev0xn6.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Wed, Oct 23, 2024 at 09:27:12 -0400, Stefan Monnier wrote: > LGTM. Further nitpicks below, > Stefan > > diff --git a/lisp/emacs-lisp/byte-opt.el b/lisp/emacs-lisp/byte-opt.el > > index d8dbfa62bf9..5cfc3528492 100644 > > --- a/lisp/emacs-lisp/byte-opt.el > > +++ b/lisp/emacs-lisp/byte-opt.el > > @@ -509,8 +509,12 @@ byte-optimize-one-form > > (defun byte-optimize-form (form &optional for-effect) > > (while > > (progn > > - ;; First, optimize all sub-forms of this one. > > - (setq form (byte-optimize-form-code-walker form for-effect)) > > + ;; First, optimize all sub-forms of this one. Note that > > + ;; `byte-optimize-form-code-walker' sometimes changes the car of > > + ;; `form', hence the `macroexp-preserve-posification'. > > + (setq form > > + (macroexp-preserve-posification > > + form (byte-optimize-form-code-walker form for-effect))) > We know it can change the `car`, that's not the question I think the > comment should answer. The comment should instead explain why > `byte-optimize-form-code-walker` doesn't use > `macroexp-preserve-posification` in those few places where it can change > the `car`s. > IIUC the answer is something like "because it was more work". Well, we could argue for some time as to whether 5 items from 20 is "few" or not. But I've amended the comment to say the invocation to m-p-posification is here for "source code economy". As I think I've already said, the slowdown on a make bootstrap is around 0.5%, barely measureable without using perf. > > @@ -524,7 +603,9 @@ macroexpand-all > > definitions to shadow the loaded ones for use in file byte-compilation." > > (let ((macroexpand-all-environment environment) > > (macroexp--dynvars macroexp--dynvars)) > > - (macroexp--expand-all form))) > > + (macroexp-preserve-posification > > + form > > + (macroexp--expand-all form)))) > I missed this one earlier: why do we need this one? Very good point. I vaguely remember going through this macro--expand-all when I was introducing the SWPs in the first place. However, there's one point in the function where the SWP isn't necessarily preserved, and that's near the end where compiler macros get handled. Here, we have less control over what the handler does. So I put an invocation of macroexp-preserve-posification into macroexp--compiler-macro to fix this, and took the one out of macroexpand-all as not needed, as you suggested. > Doesn't `macroexp--expand-all` already take care of > preserving positions? > [ You don't need to answer here: better answer in the code 🙂 ] I've done both! But I'm now confident enough about the patch not to send it to you for a third time. Thanks for all the feedback. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 23 Oct 2024 13:31:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 23 09:31:04 2024
Received: from localhost ([127.0.0.1]:59037 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t3bS4-00049b-2w
for submit <at> debbugs.gnu.org; Wed, 23 Oct 2024 09:31:04 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:62891)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>)
id 1t3bS1-00049A-DT; Wed, 23 Oct 2024 09:31:01 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 967414407CB;
Wed, 23 Oct 2024 09:30:26 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1729690225;
bh=j7QrK91pfGswVPmE1b8l/+6qpVtYuOGpL7WuOFxNL/E=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=kfRPI+kLefrUEqLFKoTLNIxX/hz2Bx097+EpxJvuX7NFJZDB69PnX7mSOBq9ZrS9q
SEc32+2it9d20zTgYn+LXJ5wQfqPZJrgHTQgS04f4seJbDMkmC+E8Ed38QmRVI3Gns
qDXB3itvEhR0NX0eGPtDo+ajMg5hTS1QOyEhtXooi4AaTdZyTDQncwg6VkJeeVPyVg
EmE/QEVsF6rPKkiPNjakOUemur1naRtkIyEmH3JEe4XECqvbOpGGU2HogWvQ12Gwq2
vOKB7+8Lbxxufjvf97XwCCZZs/KCaazxFScSoF6wGuAg4ETO/o/BddbE1oFMmLeDY1
l1MOclxnZkT8w==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4F4D4440A97;
Wed, 23 Oct 2024 09:30:25 -0400 (EDT)
Received: from pastel (69-196-161-60.dsl.teksavvy.com [69.196.161.60])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 11C18120BC6;
Wed, 23 Oct 2024 09:30:25 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
In-Reply-To: <ZxgW0YjpLqlgndcI@HIDDEN> (Alan Mackenzie's message of
"Tue, 22 Oct 2024 21:19:13 +0000")
Message-ID: <jwv4j530wzs.fsf-monnier+emacs@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
<jwvttdii56q.fsf-monnier+emacs@HIDDEN>
<8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN>
<ZxUQCUY25Ek3XhmS@HIDDEN>
<jwvh6967oir.fsf-monnier+emacs@HIDDEN>
<ZxefBoBT1hGRf08W@HIDDEN>
<jwv5xpk47d3.fsf-monnier+emacs@HIDDEN>
<ZxgW0YjpLqlgndcI@HIDDEN>
Date: Wed, 23 Oct 2024 09:30:23 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.012 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 73725
Cc: 73746 <at> debbugs.gnu.org,
Mattias =?windows-1252?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>,
73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> > I remember removing it, but can't remember exactly why. When I byte
>> > compile the code, I don't get an undeclared variable warning for it, for
>> > some reason.
>> [ Interesting. I'll try to remember to track down this sucker later. ]
>
> I think I understand this, now. It's a "use-mention" confusion. ;-).
> When macroexp-macroexpand is getting compiled, the compiler calls (the
> loaded or compiled version of) that function. Running the function
> `setq's macroexpanded, hence binds it.
Ah, right, yes. That makes sense, thanks for saving me the trouble to
track it down.
> I'm not sure there's much that can be done about this "bug". I'm even
> less sure that it's worth the trouble. Maybe there could be a more
> rigorous check of the variable's existence than (boundp var). But, as I
> say, it's likely not worth the trouble. Maybe there are other free
> variables hiding in the compiler, though.
Yeah, it's possible, but I won't lose too much sleep over it.
Stefan
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 23 Oct 2024 13:27:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 23 09:27:55 2024
Received: from localhost ([127.0.0.1]:59024 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t3bP0-0003xW-Ms
for submit <at> debbugs.gnu.org; Wed, 23 Oct 2024 09:27:55 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:23539)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>)
id 1t3bOy-0003xF-A3; Wed, 23 Oct 2024 09:27:53 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 6A6BB809CE;
Wed, 23 Oct 2024 09:27:15 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1729690034;
bh=gZkaWb9K25Kh66wC25/IY7qK/2VJjzCTzBM1+Uz/HjM=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=Jjl1fScKLaV44WWM+0j4+4GWQL+rVlULzNYxg2/Vd+Opk17+fw4f32ykbeU6MKV72
BTASulccracnFCa7jfgdUJb6eXeDaDmOz+JiWcplalFN8r7+5ihZ/Vw6jpFjBcK9z4
OVSlU+GY9qrgNYwD+TyhIOXaIlAdxdhxRXx5ruTzh224Iq7+zVUNyU5uI2LWbFJa5w
xB0M2M9yrV8KWkS5QAL82EcXESHn54Xq7u/Tmr3FO2vhsyNO5YpEW1hTXQfQpgQroV
8WxEeMikkBywKzi5/5EDjnGeIwQPwxQ8QGL/vuR4df5H9bk5edOrte06Z07K0rvHmE
X5ORbg0jFD+aQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7C25B807C9;
Wed, 23 Oct 2024 09:27:14 -0400 (EDT)
Received: from pastel (69-196-161-60.dsl.teksavvy.com [69.196.161.60])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 47D601201BF;
Wed, 23 Oct 2024 09:27:14 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
In-Reply-To: <ZxjcbhlkHhE2_-Dn@HIDDEN> (Alan Mackenzie's message of
"Wed, 23 Oct 2024 11:22:22 +0000")
Message-ID: <jwva5ev0xn6.fsf-monnier+emacs@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
<jwvttdii56q.fsf-monnier+emacs@HIDDEN>
<8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN>
<ZxUQCUY25Ek3XhmS@HIDDEN>
<jwvh6967oir.fsf-monnier+emacs@HIDDEN>
<ZxefBoBT1hGRf08W@HIDDEN>
<jwv5xpk47d3.fsf-monnier+emacs@HIDDEN>
<ZxjcbhlkHhE2_-Dn@HIDDEN>
Date: Wed, 23 Oct 2024 09:27:12 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL 0.000 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 73725
Cc: 73746 <at> debbugs.gnu.org,
Mattias =?windows-1252?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>,
73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
LGTM. Further nitpicks below,
Stefan
> diff --git a/lisp/emacs-lisp/byte-opt.el b/lisp/emacs-lisp/byte-opt.el
> index d8dbfa62bf9..5cfc3528492 100644
> --- a/lisp/emacs-lisp/byte-opt.el
> +++ b/lisp/emacs-lisp/byte-opt.el
> @@ -509,8 +509,12 @@ byte-optimize-one-form
> (defun byte-optimize-form (form &optional for-effect)
> (while
> (progn
> - ;; First, optimize all sub-forms of this one.
> - (setq form (byte-optimize-form-code-walker form for-effect))
> + ;; First, optimize all sub-forms of this one. Note that
> + ;; `byte-optimize-form-code-walker' sometimes changes the car of
> + ;; `form', hence the `macroexp-preserve-posification'.
> + (setq form
> + (macroexp-preserve-posification
> + form (byte-optimize-form-code-walker form for-effect)))
We know it can change the `car`, that's not the question I think the
comment should answer. The comment should instead explain why
`byte-optimize-form-code-walker` doesn't use
`macroexp-preserve-posification` in those few places where it can change
the `car`s.
IIUC the answer is something like "because it was more work".
> @@ -524,7 +603,9 @@ macroexpand-all
> definitions to shadow the loaded ones for use in file byte-compilation."
> (let ((macroexpand-all-environment environment)
> (macroexp--dynvars macroexp--dynvars))
> - (macroexp--expand-all form)))
> + (macroexp-preserve-posification
> + form
> + (macroexp--expand-all form))))
I missed this one earlier: why do we need this one?
Doesn't `macroexp--expand-all` already take care of
preserving positions?
[ You don't need to answer here: better answer in the code =F0=9F=99=82 ]
Stefan
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 23 Oct 2024 11:23:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 23 07:23:10 2024
Received: from localhost ([127.0.0.1]:58805 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t3ZSH-0006ku-FM
for submit <at> debbugs.gnu.org; Wed, 23 Oct 2024 07:23:10 -0400
Received: from mail.muc.de ([193.149.48.3]:52708)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1t3ZSC-0006k4-AP
for 73725 <at> debbugs.gnu.org; Wed, 23 Oct 2024 07:23:05 -0400
Received: (qmail 31955 invoked by uid 3782); 23 Oct 2024 13:22:23 +0200
Received: from muc.de (pd953aa34.dip0.t-ipconnect.de [217.83.170.52]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Wed, 23 Oct 2024 13:22:23 +0200
Received: (qmail 10533 invoked by uid 1000); 23 Oct 2024 11:22:23 -0000
Date: Wed, 23 Oct 2024 11:22:22 +0000
To: Stefan Monnier <monnier@HIDDEN>,
Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
Message-ID: <ZxjcbhlkHhE2_-Dn@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
<jwvttdii56q.fsf-monnier+emacs@HIDDEN>
<8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN>
<ZxUQCUY25Ek3XhmS@HIDDEN>
<jwvh6967oir.fsf-monnier+emacs@HIDDEN>
<ZxefBoBT1hGRf08W@HIDDEN>
<jwv5xpk47d3.fsf-monnier+emacs@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 73725
Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, 73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Stefan and Mattias.
I've incorporated your (Stafan's) suggestions from yesterday into the
code. See the patch below.
As for timings, these changes slow down a make bootstrap by around 0.5%,
something barely measurable.
On Tue, Oct 22, 2024 at 09:33:02 -0400, Stefan Monnier wrote:
[ .... ]
> >> > -(defun macroexpand-all (form &optional environment)
> >> > +(defun macroexpand-all (form &optional environment keep-pos)
> >> Any reason why we need this new argument?
> >> Can't we just always try to preserve the positions?
> > There are lots of calls to macroexpand-all (around 37), and I was
> > concerned about possible accidental side effects in these.
> I think we should assume that those potential side effects would more
> likely be beneficial than harmful.
> Another way to look at it is to look at the doc you provided:
> KEEP-POS, if non-nil, specifies that any symbol-with-position for
> FORM should be preserved, later to be usable by
> `byte-compile--warning-source-offset'.
> Even when `keep-pos` is nil, `macroexpand-all` will preserve most of the
> SWPs, so the doc is misleading.
OK, I've removed that extra optional parameter.
> > Also, always trying to preserve the position might slow down
> > compilation, but I haven't measured that yet.
> I think the calls where you currently ask for `keep-pos` account for the
> vast majority of the time spent macro-expanding, so I'd be surprised if
> doing it in all cases would make it measurably worse.
As said above, the slow down this bug fix causes is only just measurable.
Here is the current state of the patch. I propose committing it to
master. What do you say, Mattias?
diff --git a/lisp/emacs-lisp/byte-opt.el b/lisp/emacs-lisp/byte-opt.el
index d8dbfa62bf9..5cfc3528492 100644
--- a/lisp/emacs-lisp/byte-opt.el
+++ b/lisp/emacs-lisp/byte-opt.el
@@ -509,8 +509,12 @@ byte-optimize-one-form
(defun byte-optimize-form (form &optional for-effect)
(while
(progn
- ;; First, optimize all sub-forms of this one.
- (setq form (byte-optimize-form-code-walker form for-effect))
+ ;; First, optimize all sub-forms of this one. Note that
+ ;; `byte-optimize-form-code-walker' sometimes changes the car of
+ ;; `form', hence the `macroexp-preserve-posification'.
+ (setq form
+ (macroexp-preserve-posification
+ form (byte-optimize-form-code-walker form for-effect)))
;; If a form-specific optimizer is available, run it and start over
;; until a fixpoint has been reached.
@@ -519,7 +523,8 @@ byte-optimize-form
(let ((opt (byte-opt--fget (car form) 'byte-optimizer)))
(and opt
(let ((old form)
- (new (funcall opt form)))
+ (new (macroexp-preserve-posification
+ form (funcall opt form))))
(byte-compile-log " %s\t==>\t%s" old new)
(setq form new)
(not (eq new old))))))))
diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el
index 053db927b67..1224223993e 100644
--- a/lisp/emacs-lisp/macroexp.el
+++ b/lisp/emacs-lisp/macroexp.el
@@ -238,22 +238,101 @@ macroexpand-1
form))))))))
(t form)))
+(defun macroexp--posify-form-1 (form call-pos depth)
+ "The recursive part of `macroexp--posify-form'.
+It modifies a single symbol to a symbol with position, or does nothing.
+FORM and CALL-POS are as in that function. DEPTH is a small integer,
+decremented at each recursive call, to prevent infinite recursion.
+
+Return the form with a symbol with position in the canonical position
+for that form, either the one that was already there or CALL-POS; return
+nil if this isn't possible.
+"
+ (let (new-form)
+ (cond
+ ((zerop depth) nil)
+ ((and (consp form)
+ (symbolp (car form))
+ (car form))
+ (unless (symbol-with-pos-p (car form))
+ (setcar form (position-symbol (car form) call-pos)))
+ form)
+ ((consp form)
+ (or (when (setq new-form (macroexp--posify-form-1
+ (car form) call-pos (1- depth)))
+ (setcar form new-form)
+ form)
+ (when (setq new-form (macroexp--posify-form-1
+ (cdr form) call-pos (1- depth)))
+ (setcdr form new-form)
+ form)))
+ ((symbolp form)
+ (if form ; Don't position nil!
+ (if (symbol-with-pos-p form)
+ form
+ (position-symbol form call-pos))))
+ ((and (or (vectorp form) (recordp form)))
+ (let ((len (length form))
+ (i 0)
+ )
+ (while (and (< i len)
+ (not (setq new-form (macroexp--posify-form-1
+ (aref form i) call-pos (1- depth)))))
+ (setq i (1+ i)))
+ (when (< i len)
+ (aset form i new-form)
+ form))))))
+
+(defun macroexp--posify-form (form call-pos)
+ "Try to apply the position CALL-POS to the form FORM, if needed.
+CALL-POS is a buffer position, a number. FORM may be any lisp form,
+and is typically the output form returned by a macro expansion.
+
+Apply CALL-POS to FORM as a symbol with position, such that
+`byte-compile--first-symbol-with-pos' can later return it. If there is
+already a symbol with position in a \"canonical\" position for that
+function, leave it unchanged and do nothing. Return the possibly
+modified FORM."
+ (let ((new-form (macroexp--posify-form-1 form call-pos 10)))
+ (or new-form form)))
+
+(defmacro macroexp-preserve-posification (pos-form &rest body)
+ "Evaluate BODY..., posifying the result with POS-FORM's position, if any.
+If the result of body happens to have a position already, we do not
+change this."
+ (declare (debug (sexp body)) (indent 1))
+ `(let ((call-pos (cond
+ ((consp ,pos-form)
+ (and (symbol-with-pos-p (car ,pos-form))
+ (symbol-with-pos-pos (car ,pos-form))))
+ ((symbol-with-pos-p ,pos-form)
+ (symbol-with-pos-pos ,pos-form))))
+ (new-value (progn ,@body)))
+ (if (and call-pos
+ (not (or (and (consp new-value)
+ (symbol-with-pos-p (car new-value)))
+ (and (symbol-with-pos-p new-value)))))
+ (macroexp--posify-form new-value call-pos)
+ new-value)))
+
(defun macroexp-macroexpand (form env)
"Like `macroexpand' but checking obsolescence."
(let* ((macroexpand-all-environment env)
new-form)
- (while (not (eq form (setq new-form (macroexpand-1 form env))))
- (let ((fun (car-safe form)))
- (setq form
- (if (and fun (symbolp fun)
- (get fun 'byte-obsolete-info))
- (macroexp-warn-and-return
- (macroexp--obsolete-warning
- fun (get fun 'byte-obsolete-info)
- (if (symbolp (symbol-function fun)) "alias" "macro"))
- new-form (list 'obsolete fun) nil fun)
- new-form))))
- form))
+ (macroexp-preserve-posification
+ form
+ (while (not (eq form (setq new-form (macroexpand-1 form env))))
+ (let ((fun (car-safe form)))
+ (setq form
+ (if (and fun (symbolp fun)
+ (get fun 'byte-obsolete-info))
+ (macroexp-warn-and-return
+ (macroexp--obsolete-warning
+ fun (get fun 'byte-obsolete-info)
+ (if (symbolp (symbol-function fun)) "alias" "macro"))
+ new-form (list 'obsolete fun) nil fun)
+ new-form))))
+ form)))
(defun macroexp--unfold-lambda (form &optional name)
(or name (setq name "anonymous lambda"))
@@ -524,7 +603,9 @@ macroexpand-all
definitions to shadow the loaded ones for use in file byte-compilation."
(let ((macroexpand-all-environment environment)
(macroexp--dynvars macroexp--dynvars))
- (macroexp--expand-all form)))
+ (macroexp-preserve-posification
+ form
+ (macroexp--expand-all form))))
;; This function is like `macroexpand-all' but for use with top-level
;; forms. It does not dynbind `macroexp--dynvars' because we want
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 22 Oct 2024 21:19:58 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 22 17:19:58 2024 Received: from localhost ([127.0.0.1]:57654 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t3MII-0000Jb-3T for submit <at> debbugs.gnu.org; Tue, 22 Oct 2024 17:19:58 -0400 Received: from mail.muc.de ([193.149.48.3]:59643) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t3MIE-0000Iy-02 for 73725 <at> debbugs.gnu.org; Tue, 22 Oct 2024 17:19:54 -0400 Received: (qmail 57441 invoked by uid 3782); 22 Oct 2024 23:19:14 +0200 Received: from muc.de (p4fe15a4c.dip0.t-ipconnect.de [79.225.90.76]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 22 Oct 2024 23:19:13 +0200 Received: (qmail 9293 invoked by uid 1000); 22 Oct 2024 21:19:13 -0000 Date: Tue, 22 Oct 2024 21:19:13 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <ZxgW0YjpLqlgndcI@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <jwvh6967oir.fsf-monnier+emacs@HIDDEN> <ZxefBoBT1hGRf08W@HIDDEN> <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. On Tue, Oct 22, 2024 at 09:33:02 -0400, Stefan Monnier wrote: [ .... ] > >> > (defun macroexp-macroexpand (form env) > >> > "Like `macroexpand' but checking obsolescence." > >> > (let* ((macroexpand-all-environment env) > >> > new-form) > >> > + (macroexp-preserve-posification > >> > + form > >> > (while (not (eq form (setq new-form (macroexpand-1 form env)))) > >> > + (setq macroexpanded t) > >> > (let ((fun (car-safe form))) > >> > (setq form > >> > (if (and fun (symbolp fun) > >> This `(setq macroexpanded t)` looks like some leftover code, at least > >> I couldn't find this var declared or used elsewhere. > > I remember removing it, but can't remember exactly why. When I byte > > compile the code, I don't get an undeclared variable warning for it, for > > some reason. > [ Interesting. I'll try to remember to track down this sucker later. ] I think I understand this, now. It's a "use-mention" confusion. ;-). When macroexp-macroexpand is getting compiled, the compiler calls (the loaded or compiled version of) that function. Running the function `setq's macroexpanded, hence binds it. Later, the compiler calls byte-compile-free-vars-warn. This fails to do anything because of the clause (boundp var) in the enclosing `unless' form. VAR aka macroexpanded is indeed now bound, see previous paragraph. I'm not sure there's much that can be done about this "bug". I'm even less sure that it's worth the trouble. Maybe there could be a more rigorous check of the variable's existence than (boundp var). But, as I say, it's likely not worth the trouble. Maybe there are other free variables hiding in the compiler, though. What do you think? > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 22 Oct 2024 13:33:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 22 09:33:43 2024
Received: from localhost ([127.0.0.1]:55084 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t3F15-0003GV-1e
for submit <at> debbugs.gnu.org; Tue, 22 Oct 2024 09:33:43 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:65499)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>)
id 1t3F12-0003GB-Ir; Tue, 22 Oct 2024 09:33:41 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id DAA244417A9;
Tue, 22 Oct 2024 09:33:05 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1729603983;
bh=Lyw4dfTRcO9z/hQarKXrBigs2fwVGkXdKLBAAQQaHVA=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=IBrZvCJYHxtLSiR0DETPgbevemJaik3TZdXj08F1MLeT9oexGLoYd7czXdJmv4aIM
FbYX7nUfqFoeFx3TbwYy1Nj1Od6I+xucj1/YR53IPIlUxyqMjypPbd5mU9/yE4gLsJ
RlUJ+tPExB9fOX9NOaS4cW68SyjsfseD6eRY+POQHUN3iZ6HAYJaxMwn+TCFR3Awfm
M49TlW6/2lTwH4YndUWmaXDCtc8yB7S+i8W5c5iTfEkdsR8BomTtYkBxoQbDcRBquw
tQYcCfaTvJdedwwMxg+ACvpY2hJ8dIf9N6urRLIRXCSnoixs9lhT9kvv1mvQylBsNJ
Pe8LsBKYIa2vQ==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C456C4410A9;
Tue, 22 Oct 2024 09:33:03 -0400 (EDT)
Received: from pastel (69-196-161-60.dsl.teksavvy.com [69.196.161.60])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 94CD7120EB4;
Tue, 22 Oct 2024 09:33:03 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
In-Reply-To: <ZxefBoBT1hGRf08W@HIDDEN> (Alan Mackenzie's message of
"Tue, 22 Oct 2024 12:48:06 +0000")
Message-ID: <jwv5xpk47d3.fsf-monnier+emacs@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
<jwvttdii56q.fsf-monnier+emacs@HIDDEN>
<8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN>
<ZxUQCUY25Ek3XhmS@HIDDEN>
<jwvh6967oir.fsf-monnier+emacs@HIDDEN>
<ZxefBoBT1hGRf08W@HIDDEN>
Date: Tue, 22 Oct 2024 09:33:02 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.014 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 73725
Cc: 73746 <at> debbugs.gnu.org,
Mattias =?windows-1252?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>,
73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
>> Is this needed? I'd expect we need `macroexp-preserve-posification`
>> only at those places where an optimization returns a form whose `car` is
>> different from that of FORM. So I expect this to happen in more
>> specific places inside byte-opt.el than here.
> Yes, it is needed. byte-optimize-form-code-walker sometimes does return
> a form with a different car. For example, a progn form with a single
> sub-form comes back without the progn.
This should usually be harmless since the subform should come with its own
position information.
> Even if there are several sub-forms, the code uses macroexp-progn to
> substitute a different progn symbol without the position. I don't
> know why it does this.
This is more annoying, indeed. Ideally, we'd move the
`macroexp-preserve-posification` to the place(s) where this remove+readd
`progn` happens.
> One of my ideas was to fix byte-optimize-form-code-walker by fixing each
> individual bit where the SWP got lost. In the end I gave that up as too
> much work, and too difficult to test.
Hmm... I see. We can leave for a later optimization of
`byte-optimize-form-code-walker`.
>> IOW, this deserves a clear comment explaining why we need it here,
>> probably with some kind of example.
> OK, I'll put one in, along the lines of the above, but less wordy.
Thanks.
> Good point. The doc string of that function is a little clumsy, as it
> refers to the doc string of the next function macroexp--posify-form,
That part did not bother me. These are internal functions so it's OK if
the docstring is a bit less "self contained".
> But I'll write somewhere that we modify "a single symbol", or something
> like that.
Thanks.
>> > (defun macroexp-macroexpand (form env)
>> > "Like `macroexpand' but checking obsolescence."
>> > (let* ((macroexpand-all-environment env)
>> > new-form)
>> > + (macroexp-preserve-posification
>> > + form
>> > (while (not (eq form (setq new-form (macroexpand-1 form env))))
>> > + (setq macroexpanded t)
>> > (let ((fun (car-safe form)))
>> > (setq form
>> > (if (and fun (symbolp fun)
>
>> This `(setq macroexpanded t)` looks like some leftover code, at least
>> I couldn't find this var declared or used elsewhere.
>
> I remember removing it, but can't remember exactly why. When I byte
> compile the code, I don't get an undeclared variable warning for it, for
> some reason.
[ Interesting. I'll try to remember to track down this sucker later. ]
>> > -(defun macroexpand-all (form &optional environment)
>> > +(defun macroexpand-all (form &optional environment keep-pos)
>> Any reason why we need this new argument?
>> Can't we just always try to preserve the positions?
> There are lots of calls to macroexpand-all (around 37), and I was
> concerned about possible accidental side effects in these.
I think we should assume that those potential side effects would more
likely be beneficial than harmful.
Another way to look at it is to look at the doc you provided:
KEEP-POS, if non-nil, specifies that any symbol-with-position for
FORM should be preserved, later to be usable by
`byte-compile--warning-source-offset'.
Even when `keep-pos` is nil, `macroexpand-all` will preserve most of the
SWPs, so the doc is misleading.
> Also, always trying to preserve the position might slow down
> compilation, but I haven't measured that yet.
I think the calls where you currently ask for `keep-pos` account for the
vast majority of the time spent macro-expanding, so I'd be surprised if
doing it in all cases would make it measurably worse.
Stefan
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 22 Oct 2024 12:48:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 22 08:48:46 2024 Received: from localhost ([127.0.0.1]:54983 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t3EJZ-00011u-G5 for submit <at> debbugs.gnu.org; Tue, 22 Oct 2024 08:48:46 -0400 Received: from mail.muc.de ([193.149.48.3]:63474) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t3EJW-00011e-Fk for 73725 <at> debbugs.gnu.org; Tue, 22 Oct 2024 08:48:43 -0400 Received: (qmail 62759 invoked by uid 3782); 22 Oct 2024 14:48:07 +0200 Received: from muc.de (p4fe15a4c.dip0.t-ipconnect.de [79.225.90.76]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 22 Oct 2024 14:48:06 +0200 Received: (qmail 11185 invoked by uid 1000); 22 Oct 2024 12:48:06 -0000 Date: Tue, 22 Oct 2024 12:48:06 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <ZxefBoBT1hGRf08W@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <jwvh6967oir.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvh6967oir.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. Thanks for giving my patch such a thorough review. On Sun, Oct 20, 2024 at 12:35:46 -0400, Stefan Monnier wrote: > > I haven't heard anything back in over a week, so I assume my patch for > > bug#73725 and bug#73746 is OK. I've committed it. > Sorry, got pushed too far down my todo while I was busy and forgot about it. > Side note: thinking some more about the problem you're trying to fix, > I can't help find it is somewhat funny: it's exactly the kind of > "position-preservation" work we were hoping to avoid having to do by > using SWPs. Well, we always knew that macros would be problematic. > Anyway, the general approach seems about right. > See comments below: > > @@ -510,7 +510,9 @@ byte-optimize-form > > (while > > (progn > > ;; First, optimize all sub-forms of this one. > > - (setq form (byte-optimize-form-code-walker form for-effect)) > > + (setq form > > + (macroexp-preserve-posification > > + form (byte-optimize-form-code-walker form for-effect))) > > > > ;; If a form-specific optimizer is available, run it and start over > > ;; until a fixpoint has been reached. > Is this needed? I'd expect we need `macroexp-preserve-posification` > only at those places where an optimization returns a form whose `car` is > different from that of FORM. So I expect this to happen in more > specific places inside byte-opt.el than here. Yes, it is needed. byte-optimize-form-code-walker sometimes does return a form with a different car. For example, a progn form with a single sub-form comes back without the progn. Even if there are several sub-forms, the code uses macroexp-progn to substitute a different progn symbol without the position. I don't know why it does this. One of my ideas was to fix byte-optimize-form-code-walker by fixing each individual bit where the SWP got lost. In the end I gave that up as too much work, and too difficult to test. > IOW, this deserves a clear comment explaining why we need it here, > probably with some kind of example. OK, I'll put one in, along the lines of the above, but less wordy. > > @@ -519,7 +521,8 @@ byte-optimize-form > > (let ((opt (byte-opt--fget (car form) 'byte-optimizer))) > > (and opt > > (let ((old form) > > - (new (funcall opt form))) > > + (new (macroexp-preserve-posification > > + form (funcall opt form)))) > > (byte-compile-log " %s\t==>\t%s" old new) > > (setq form new) > > (not (eq new old)))))))) > E.g. this is the kind of place where it makes sense to me. > > +(defun sub-macroexp--posify-form (form call-pos depth) > Please don't eat up namespace gratuitously. > IOW stick to the "macroexp-" prefix. Sorry, I wasn't thinking straight when I named that. I've already corrected it in my sources, it will appear in the next patch or commit I submit. > > + "Try to apply the transformation of `macroexp--posify-form' to FORM. > > +FORM and CALL-POS are as in that function. DEPTH is a small integer, > > +decremented at each recursive call, to prevent infinite recursion. > > +Return the changed form, or nil if no change happened." > Somewhere in the doc or in a comment we should document the "design", > which AFAICT is to try and find "one" place where we can put the > `call-pos` info: the docstring suggests we'll apply it to all the > symbols within `depth` (which would be both costly and undesirable), > whereas we actually stop (more or less) as soon as we find a good spot. Good point. The doc string of that function is a little clumsy, as it refers to the doc string of the next function macroexp--posify-form, since it is the recursive part of that function. Maybe I should just write out that of the first function in full, even though that would duplicate a lot of stuff. But I'll write somewhere that we modify "a single symbol", or something like that. > > + (let (new-form) > > + (cond > > + ((zerop depth) nil) > > + ((and (consp form) > > + (symbolp (car form)) > > + (car form)) > > + (setcar form (position-symbol (car form) call-pos)) > > + form) > AFAICT this can and occasionally will throw away valuable > position information: if `(car form)` is an SWP we don't know for sure > here that `call-pos` is always a better position info than the one > already carried by `(car form)`. Thanks. Such occasions will be when the form returned by the macro has as a car a SWP whose position is inside the macro. Such a position would indeed be a better position for the warning message that the position of the macro call. > We could consider combining position information (but this would make > the whole system more complex: when printing warnings/errors we'd have > to find a way to make use of the various recorded positions, ...). > But a simpler choice is to check (not (symbol-with-pos-p (car form))). I think the simpler choice is the one to go with, here. ;-) > > + ((symbolp form) > > + (if form ; Don't position nil! > > + (position-symbol form call-pos))) > Same here. Yes. > > +(defmacro macroexp-preserve-posification (pos-form &rest body) > > + "Evaluate BODY..., posifying the result with POS-FORM's position, if any." > This lacks a `declare` with `debug` and `indent` specs. Yes. Sorry, I'll fix these. > > + `(let ((call-pos (cond > > + ((consp ,pos-form) > > + (and (symbol-with-pos-p (car ,pos-form)) > > + (symbol-with-pos-pos (car ,pos-form)))) > > + ((symbol-with-pos-p ,pos-form) > > + (symbol-with-pos-pos ,pos-form)))) > > + (new-value (progn ,@body))) > > + (if call-pos > > + (macroexp--posify-form new-value call-pos) > > + new-value))) > > + > > (defun macroexp-macroexpand (form env) > > "Like `macroexpand' but checking obsolescence." > > (let* ((macroexpand-all-environment env) > > new-form) > > + (macroexp-preserve-posification > > + form > > (while (not (eq form (setq new-form (macroexpand-1 form env)))) > > + (setq macroexpanded t) > > (let ((fun (car-safe form))) > > (setq form > > (if (and fun (symbolp fun) > This `(setq macroexpanded t)` looks like some leftover code, at least > I couldn't find this var declared or used elsewhere. I remember removing it, but can't remember exactly why. When I byte compile the code, I don't get an undeclared variable warning for it, for some reason. > But yes, I suspect it would make a fair bit of sense to perform the > "preserve-posification" only when `macroexpand-1` did return a new form. > Did you try to do it (as suggested by this leftover code) and found it > was not worth the trouble or that it didn't work? I think that with the variable `macroexpand', the code was getting a bit bulky. But the check for macroexpand-1 returning a new form could easily be performed in the macro macroexp-preserve-posification. So I'll try that, and see what sort of effect it has on the timings. > If so, please add a comment recording it. > > @@ -253,7 +316,7 @@ > > (if (symbolp (symbol-function fun)) "alias" "macro")) > > new-form (list 'obsolete fun) nil fun) > > new-form)))) > > - form)) > > + new-form))) > Why? I can't remember. The change doesn't seem to make sense. > > -(defun macroexpand-all (form &optional environment) > > +(defun macroexpand-all (form &optional environment keep-pos) > Any reason why we need this new argument? > Can't we just always try to preserve the positions? There are lots of calls to macroexpand-all (around 37), and I was concerned about possible accidental side effects in these. Also, always trying to preserve the position might slow down compilation, but I haven't measured that yet. > Stefan -- Alan Mackenzie (Nuremberg, Germany)
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 20 Oct 2024 16:36:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 20 12:36:25 2024
Received: from localhost ([127.0.0.1]:48267 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t2Yun-0000mc-1H
for submit <at> debbugs.gnu.org; Sun, 20 Oct 2024 12:36:25 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:48280)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>)
id 1t2Yuk-0000lm-QC; Sun, 20 Oct 2024 12:36:23 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 83A278093F;
Sun, 20 Oct 2024 12:35:50 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1729442149;
bh=Wr/64k+jRnzAXlarjcUAdwVFuhco4pSk2b/+uDDBP5k=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=aLYH13ew1SJHKBtoO+O96pbL1Ud6t05RGZnr5Ws1RKdsYuNx155Tg8R+66vfhWqqU
XOEcS3JLyuLRbqPzmC/RfWq2eCR1ym4kSYeohpI9pa/tClu8YNRRqGqUfBdQy4o2n+
MsJd97dXedLrRJJydxZn4AC2+B1cmPk+P07jH4i9Ml+h4hZ8Y8vh5Bp2fhDe+Cux7F
6dUNSet13Vd2auUXGS85AAmEMNOOWo0yT3amnQJN5BXMryBznBbP5M6tDDCOZm7+QR
O2sKQOAJpmZNFEo9qwrR7JVcFFoACOuMNtypNUD3I6qhB+vpwUZiZ2bAQwFIJVgWtM
IbCxPZuA2tTaA==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 4218B80088;
Sun, 20 Oct 2024 12:35:49 -0400 (EDT)
Received: from pastel (69-196-161-60.dsl.teksavvy.com [69.196.161.60])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 0BADC12029A;
Sun, 20 Oct 2024 12:35:49 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
In-Reply-To: <ZxUQCUY25Ek3XhmS@HIDDEN> (Alan Mackenzie's message of
"Sun, 20 Oct 2024 14:13:29 +0000")
Message-ID: <jwvh6967oir.fsf-monnier+emacs@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
<jwvttdii56q.fsf-monnier+emacs@HIDDEN>
<8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN>
<ZxUQCUY25Ek3XhmS@HIDDEN>
Date: Sun, 20 Oct 2024 12:35:46 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 73725
Cc: 73746 <at> debbugs.gnu.org,
Mattias =?windows-1252?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>,
73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> I haven't heard anything back in over a week, so I assume my patch for
> bug#73725 and bug#73746 is OK. I've committed it.
Sorry, got pushed too far down my todo while I was busy and forgot about it.
Side note: thinking some more about the problem you're trying to fix,
I can't help find it is somewhat funny: it's exactly the kind of
"position-preservation" work we were hoping to avoid having to do by
using SWPs.
Anyway, the general approach seems about right.
See comments below:
> @@ -510,7 +510,9 @@ byte-optimize-form
> (while
> (progn
> ;; First, optimize all sub-forms of this one.
> - (setq form (byte-optimize-form-code-walker form for-effect))
> + (setq form
> + (macroexp-preserve-posification
> + form (byte-optimize-form-code-walker form for-effect)))
>
> ;; If a form-specific optimizer is available, run it and start over
> ;; until a fixpoint has been reached.
Is this needed? I'd expect we need `macroexp-preserve-posification`
only at those places where an optimization returns a form whose `car` is
different from that of FORM. So I expect this to happen in more
specific places inside byte-opt.el than here.
IOW, this deserves a clear comment explaining why we need it here,
probably with some kind of example.
> @@ -519,7 +521,8 @@ byte-optimize-form
> (let ((opt (byte-opt--fget (car form) 'byte-optimizer)))
> (and opt
> (let ((old form)
> - (new (funcall opt form)))
> + (new (macroexp-preserve-posification
> + form (funcall opt form))))
> (byte-compile-log " %s\t==>\t%s" old new)
> (setq form new)
> (not (eq new old))))))))
E.g. this is the kind of place where it makes sense to me.
> +(defun sub-macroexp--posify-form (form call-pos depth)
Please don't eat up namespace gratuitously.
IOW stick to the "macroexp-" prefix.
> + "Try to apply the transformation of `macroexp--posify-form' to FORM.
> +FORM and CALL-POS are as in that function. DEPTH is a small integer,
> +decremented at each recursive call, to prevent infinite recursion.
> +Return the changed form, or nil if no change happened."
Somewhere in the doc or in a comment we should document the "design",
which AFAICT is to try and find "one" place where we can put the
`call-pos` info: the docstring suggests we'll apply it to all the
symbols within `depth` (which would be both costly and undesirable),
whereas we actually stop (more or less) as soon as we find a good spot.
> + (let (new-form)
> + (cond
> + ((zerop depth) nil)
> + ((and (consp form)
> + (symbolp (car form))
> + (car form))
> + (setcar form (position-symbol (car form) call-pos))
> + form)
AFAICT this can and occasionally will throw away valuable
position information: if `(car form)` is an SWP we don't know for sure
here that `call-pos` is always a better position info than the one
already carried by `(car form)`.
We could consider combining position information (but this would make
the whole system more complex: when printing warnings/errors we'd have
to find a way to make use of the various recorded positions, ...).
But a simpler choice is to check (not (symbol-with-pos-p (car form))).
> + ((symbolp form)
> + (if form ; Don't position nil!
> + (position-symbol form call-pos)))
Same here.
> +(defmacro macroexp-preserve-posification (pos-form &rest body)
> + "Evaluate BODY..., posifying the result with POS-FORM's position, if any."
This lacks a `declare` with `debug` and `indent` specs.
> + `(let ((call-pos (cond
> + ((consp ,pos-form)
> + (and (symbol-with-pos-p (car ,pos-form))
> + (symbol-with-pos-pos (car ,pos-form))))
> + ((symbol-with-pos-p ,pos-form)
> + (symbol-with-pos-pos ,pos-form))))
> + (new-value (progn ,@body)))
> + (if call-pos
> + (macroexp--posify-form new-value call-pos)
> + new-value)))
> +
> (defun macroexp-macroexpand (form env)
> "Like `macroexpand' but checking obsolescence."
> (let* ((macroexpand-all-environment env)
> new-form)
> + (macroexp-preserve-posification
> + form
> (while (not (eq form (setq new-form (macroexpand-1 form env))))
> + (setq macroexpanded t)
> (let ((fun (car-safe form)))
> (setq form
> (if (and fun (symbolp fun)
This `(setq macroexpanded t)` looks like some leftover code, at least
I couldn't find this var declared or used elsewhere.
But yes, I suspect it would make a fair bit of sense to perform the
"preserve-posification" only when `macroexpand-1` did return a new form.
Did you try to do it (as suggested by this leftover code) and found it
was not worth the trouble or that it didn't work?
If so, please add a comment recording it.
> @@ -253,7 +316,7 @@
> (if (symbolp (symbol-function fun)) "alias" "macro"))
> new-form (list 'obsolete fun) nil fun)
> new-form))))
> - form))
> + new-form)))
Why?
> -(defun macroexpand-all (form &optional environment)
> +(defun macroexpand-all (form &optional environment keep-pos)
Any reason why we need this new argument?
Can't we just always try to preserve the positions?
Stefan
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 20 Oct 2024 15:39:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 20 11:39:34 2024 Received: from localhost ([127.0.0.1]:48144 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t2Y1m-0006Nh-DP for submit <at> debbugs.gnu.org; Sun, 20 Oct 2024 11:39:34 -0400 Received: from mail.muc.de ([193.149.48.3]:15913) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t2Y1j-0006NA-1B for 73725 <at> debbugs.gnu.org; Sun, 20 Oct 2024 11:39:31 -0400 Received: (qmail 56128 invoked by uid 3782); 20 Oct 2024 17:38:54 +0200 Received: from muc.de (pd953a8ba.dip0.t-ipconnect.de [217.83.168.186]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 20 Oct 2024 17:38:54 +0200 Received: (qmail 9867 invoked by uid 1000); 20 Oct 2024 15:38:53 -0000 Date: Sun, 20 Oct 2024 15:38:53 +0000 To: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <ZxUkDch_zR8ord12@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> <B61322D4-D86B-40CD-AC2C-5AEC43126A09@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <B61322D4-D86B-40CD-AC2C-5AEC43126A09@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Stefan Monnier <monnier@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Mattias. On Sun, Oct 20, 2024 at 17:06:01 +0200, Mattias Engdegård wrote: > 20 okt. 2024 kl. 16.13 skrev Alan Mackenzie <acm@HIDDEN>: > > I haven't heard anything back in over a week, so I assume my patch for > > bug#73725 and bug#73746 is OK. I've committed it. [ .... ] > (Are 73725 and 73746 about different problems? They looked confusingly > similar.) As explained in the text for 73746, there is a "," different. This has a huge effect on the optimisation which is carried out on the macro. -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 20 Oct 2024 15:07:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 20 11:07:32 2024 Received: from localhost ([127.0.0.1]:48094 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t2XWl-0004im-TW for submit <at> debbugs.gnu.org; Sun, 20 Oct 2024 11:07:32 -0400 Received: from mail-lf1-f47.google.com ([209.85.167.47]:51294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mattias.engdegard@HIDDEN>) id 1t2XWj-0004iQ-S9; Sun, 20 Oct 2024 11:07:30 -0400 Received: by mail-lf1-f47.google.com with SMTP id 2adb3069b0e04-53a0c160b94so1514682e87.2; Sun, 20 Oct 2024 08:07:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729436764; x=1730041564; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=3erE8EEZDZG4AALlVqWFYqhYfJOvNvQ95sNRcszmon8=; b=lP52qr3vX6Ns/PJo2DJUsp2WKOdXWS3LE46Ysu+zN86cM7Je0dbXJ9H3HNtdtC1sM7 uNF2wxn1vLq558iXQr47doEpOWmcg6T3LTFcvnpESflSPt5CyDd7RaojXXxLvDWO80ze T/DWyFziTrqx2i1/gvsjg7s5lGDtuHHvx4e34A5WnHDaQk6Cpid68dj9JNq/sU1dNXqV d4cRHLdZKCiDzr07ARN4Of62s32N+OOF9skVs5ABqIabNRxUbk5juFwd2LgRxVKQKOJY feCRFwYusvZ5R33glrJaSDYTus8E7uwcYUUtlA2XJgxWe41OgQ+K/51FFdGxOFIb8aZh UIBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729436764; x=1730041564; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=3erE8EEZDZG4AALlVqWFYqhYfJOvNvQ95sNRcszmon8=; b=cIU+nN5q/9r6V4SrpE7m0w15sOgKk5YkyJaYZRqobzi/Dwt3hvyqC33Up3p1wTOuCd XWYr9lAvfRqvRfrk9i0y2FoVkI7HobYj2oZJr6ej/IKhPcq7eUqlvA/LnQ5ee+TYi1Yl iQVWppnyh39bSILNIIIDqlhhIh/VTVQO8qGdwOT1OCtZPFRiEstfdNgu0dU3O6CPyYFD SKT/jjgXyQlTE7DP9csYiKsn+Fi4ogPtzyDHqkM8fd0LQBfNAVWlB/qjHpKojbBbsmV1 ens4h0Y5uvaSNRDm8TbrkCz5SKK/kNI999JSmJG2soe791pf8d2KyJ62sImB9t9A6dET r2Qg== X-Forwarded-Encrypted: i=1; AJvYcCU9q7Zy7kHmBpncYA6LkWyt4Ry08cw/uA+Pjl33zURE1HZ3m/gJtbwn/tq7y3IGvGKYG1gxVtg=@debbugs.gnu.org, AJvYcCUqQRRoEwu8fY7hrFB1p5m7w+Kn9F7pOSIgwXh1S69Y7cdR9MIltTNoCC7aL0aIbv/DVbLIEA==@debbugs.gnu.org X-Gm-Message-State: AOJu0YzVF4AMn2hfNLGbzCe5mZ6OUQ0mv9f94e82Ol7l6sXsSA7pnQDt CB9TRHxDZHBQV5LNXZv3GczhHoddUyh1IR9qdTuKlOipdUUwBV7r X-Google-Smtp-Source: AGHT+IGRVWQeZ8dslCLJYbJ24bPRMb1wFBL3DTkBPsXxbOp4Vdy/3IUqZOfD04FYro3+G8sVAEgzsw== X-Received: by 2002:a05:6512:ea2:b0:535:6942:29ea with SMTP id 2adb3069b0e04-53a1520bd89mr4498016e87.11.1729436763277; Sun, 20 Oct 2024 08:06:03 -0700 (PDT) Received: from smtpclient.apple (c188-150-183-180.bredband.tele2.se. [188.150.183.180]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-53a2243ef6asm242721e87.287.2024.10.20.08.06.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 20 Oct 2024 08:06:02 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= <mattias.engdegard@HIDDEN> In-Reply-To: <ZxUQCUY25Ek3XhmS@HIDDEN> Date: Sun, 20 Oct 2024 17:06:01 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <B61322D4-D86B-40CD-AC2C-5AEC43126A09@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> <ZxUQCUY25Ek3XhmS@HIDDEN> To: Alan Mackenzie <acm@HIDDEN> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) 20 okt. 2024 kl. 16.13 skrev Alan Mackenzie <acm@HIDDEN>: > I haven't heard anything back in over a week, so I assume my patch for > bug#73725 and bug#73746 is OK. I've committed it. Sorry about the delay, but you can't make that assumption. Now reverted. I promise I'll look at your report soon, but let's not make your = original patch a fait accompli. (Are 73725 and 73746 about different problems? They looked confusingly = similar.)
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 20 Oct 2024 14:14:06 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 20 10:14:06 2024 Received: from localhost ([127.0.0.1]:47936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t2Wh3-000244-MJ for submit <at> debbugs.gnu.org; Sun, 20 Oct 2024 10:14:05 -0400 Received: from mail.muc.de ([193.149.48.3]:49107) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t2Wh1-00023O-KT for 73725 <at> debbugs.gnu.org; Sun, 20 Oct 2024 10:14:04 -0400 Received: (qmail 59964 invoked by uid 3782); 20 Oct 2024 16:13:30 +0200 Received: from muc.de (pd953a8ba.dip0.t-ipconnect.de [217.83.168.186]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 20 Oct 2024 16:13:30 +0200 Received: (qmail 4689 invoked by uid 1000); 20 Oct 2024 14:13:29 -0000 Date: Sun, 20 Oct 2024 14:13:29 +0000 To: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <ZxUQCUY25Ek3XhmS@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: 73746 <at> debbugs.gnu.org, acm@HIDDEN, Stefan Monnier <monnier@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Mattias and Stefan. On Sat, Oct 12, 2024 at 14:01:20 +0200, Mattias Engdegård wrote: > 12 okt. 2024 kl. 01.45 skrev Stefan Monnier <monnier@HIDDEN>: > > That sounds potentially costly > I'm also concerned about costs but won't have time to look at it for at > least a few days. > However since it touches code that I'm revamping (although that process > has been demoted to power-saving mode at the moment) I'd like to give > it a look-over when I can. I haven't heard anything back in over a week, so I assume my patch for bug#73725 and bug#73746 is OK. I've committed it. If I don't hear anything more soon, I will close these two bugs as fixed. -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 13 Oct 2024 15:32:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 13 11:32:24 2024 Received: from localhost ([127.0.0.1]:52111 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1t00a0-00015n-9A for submit <at> debbugs.gnu.org; Sun, 13 Oct 2024 11:32:24 -0400 Received: from mail.muc.de ([193.149.48.3]:18718) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1t00Zv-00015N-Fy for 73725 <at> debbugs.gnu.org; Sun, 13 Oct 2024 11:32:22 -0400 Received: (qmail 19334 invoked by uid 3782); 13 Oct 2024 17:31:56 +0200 Received: from muc.de (p4fe15e31.dip0.t-ipconnect.de [79.225.94.49]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 13 Oct 2024 17:31:56 +0200 Received: (qmail 1875 invoked by uid 1000); 13 Oct 2024 15:31:55 -0000 Date: Sun, 13 Oct 2024 15:31:55 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <Zwvn6wPHssh82hJl@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvttdii56q.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: acm@HIDDEN, Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello again, Stefan. On Fri, Oct 11, 2024 at 19:45:18 -0400, Stefan Monnier wrote: > > (i) Create the following file, bad-error-position2.el: > > ######################################################################### > > ;; -*- lexical-binding:t -*- > > (eval-and-compile > > (defmacro increase () > > `(let ((foo (point-max))) > > (cond > > ((consp foo) > > (message "consp %s" foo) > > foo) > > ((numberp foo) > > (1+ fooo)) ; Note the misspelling. > > (t (message "Something else: %s" foo)))))) > > (defun call-increase (bar) > > (cond > > ((not (or (consp bar) > > (numberp bar))) > > bar) > > (t (increase)))) > > ######################################################################### > > Note the misspelling of `foo' as `fooo' on line 10. > > (ii) emacs -Q > > (iii) M-x byte-compile-file /path/to/bad-error-position2.el. > > (iv) This gives the warning message: > > bad-error-position2.el:14:4: Warning: reference to free variable `fooo' > > (v) The position 14:4 is wrong. That is the position of the `cond' > > symbol in `call-increase'. > > (vi) The correct message should be: > > bad-error-position2.el:18:8: Warning: reference to free variable `fooo' > > .. 18:8 is the position of `increase' on the last line of the file. > Nitpick: one could also argue that the "correct" message should point to > line 8 where is the `fooo` typo. In this case, no. The function from which the macro is called might have a locally bound variable called `fooo'. ;-) But there are surely errors in macros which aren't like that which might be picked up by compiling the macro. Then warnings containing the line in the macro could be given. That would be a non-trivial exercise. > > +(defun sub-macroexp--posify-form (form call-pos depth) > > + "Try to apply the transformation of `macroexp--posify-form' to FORM. > > +FORM and CALL-POS are as in that function. DEPTH is a small integer, > > +decremented at each recursive call, to prevent infinite recursion. > > +Return the changed form, or nil if no change happened." > > + (let (new-form > > + ) > > + (cond > > + ((zerop depth) nil) > > + ((and (consp form) > > + (symbolp (car form)) > > + (car form)) > > + (setcar form (position-symbol (car form) call-pos)) > > + form) > > + ((consp form) > > + (or (when (setq new-form (sub-macroexp--posify-form > > + (car form) call-pos (1- depth))) > > + (setcar form new-form) > > + form) > > + (when (setq new-form (sub-macroexp--posify-form > > + (cdr form) call-pos (1- depth))) > > + (setcdr form new-form) > > + form))) > > + ((symbolp form) > > + (if form ; Don't position nil! > > + (position-symbol form call-pos))) > > + ((and (or (vectorp form) (recordp form))) > > + (let ((len (length form)) > > + (i 0) > > + ) > > + (while (and (< i len) > > + (not (setq new-form (sub-macroexp--posify-form > > + (aref form i) call-pos (1- depth))))) > > + (setq i (1+ i))) > > + (when (< i len) > > + (aset form i new-form) > > + form)))))) > That sounds potentially costly Have you tried to measure the > performance impact in some "typical" cases? I've fixed this bug together with bug#73746, and submitted the patch for that other bug which fixes this bug, too. I've just done $ time make -j25 bootstrap on a branch with the bug#73746 patch, and a (more or less) standard master. The bug#73746 branch took 1min 58sec, the standard branch to 1min 51sec. That's a difference of ~6% in the bootstrap. It will of course be a bigger difference in the compilation. If that was felt to be too much, it would be possible instead to go round macroexp--expand-all and byte-optimize-form-code-walker and all the functions like byte-optimize-letX, putting in custom `position-symbol' calls. This would be a lot of work, though. > While reading your description I was naively thinking: we can > probably fix it with a trivial patch like: > diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el > index 19daa57b207..5cc471e32f6 100644 > --- a/lisp/emacs-lisp/macroexp.el > +++ b/lisp/emacs-lisp/macroexp.el > @@ -246,6 +246,7 @@ macroexp-macroexpand > (let* ((macroexpand-all-environment env) > new-form) > (while (not (eq form (setq new-form (macroexpand-1 form env)))) > + (push form byte-compile-form-stack) > (let ((fun (car-safe form))) > (setq form > (if (and fun (symbolp fun) > Have you tried something like this? > If it doesn't work, do you happen to know why? Like you said in another post, it might well not work for the byte-opt.el functionality. I think I was looking around for that sort of solution, and decided it wouldn't work, for a reason I've forgotten, before coming up with the idea in my patch > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 12 Oct 2024 13:54:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 12 09:54:20 2024
Received: from localhost ([127.0.0.1]:44381 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1szcZX-0000Fc-MB
for submit <at> debbugs.gnu.org; Sat, 12 Oct 2024 09:54:19 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:50567)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1szcZV-0000FL-ED
for 73725 <at> debbugs.gnu.org; Sat, 12 Oct 2024 09:54:18 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id E2E9E1000C3;
Sat, 12 Oct 2024 09:53:57 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1728741237;
bh=T4Q3P7lFggOrj8Cpvg+cQhWweGnTIPf2uzEYJ9cHNyE=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=E/skFrRwg0mKyd4jMx7xQfNLVbXD61N923oCj2EkGfZ0LXXgZuVqRUnmhYBML3foD
fi6I/9uCO+4im3Lc3YBgp4n7CkROIUA6n6PBNhS/cTJw98OcIlh1LOQg6D1v2yuQ3x
wPj4bq6VgTOITtrxMS2STJ4XqoKfjX7gjgD5fcKuNSMVvEPlEfRfgbBmqt3wyQZxAs
l7YhyIoIrsa2+kY+9dPryCWPSkFBFO39jJH8i9fSTx51oy1f6jhY5d1nzjljmzeQ0e
2XFZh/NICN115B+Ap6HxHiaosub/QoHFPcrltokKkJ4CiL/NxcvUpP5CPj0REpMWfs
ejm6s5OH6+Rkg==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 1266110002E;
Sat, 12 Oct 2024 09:53:57 -0400 (EDT)
Received: from pastel (104-195-209-82.cpe.teksavvy.com [104.195.209.82])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id DA14C120588;
Sat, 12 Oct 2024 09:53:56 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
In-Reply-To: <ZwpT2cqTwdUBldr3@HIDDEN> (Alan Mackenzie's message of
"Sat, 12 Oct 2024 10:47:53 +0000")
Message-ID: <jwv7cadigcq.fsf-monnier+emacs@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
<jwvttdii56q.fsf-monnier+emacs@HIDDEN>
<ZwpT2cqTwdUBldr3@HIDDEN>
Date: Sat, 12 Oct 2024 09:53:55 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.063 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 73725
Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>,
73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> I haven't, no. It might well work. But I think the `push' form should
> go outside of the loop,
Really? I thought it made more sense to put it inside the loop, so
a macro X1 that expands to macro X2 ... will get all of the
corresponding locations pushed, thus preserving more of the info about
where things are coming from.
> I'm also a little wary about pushing an item onto stack when it
> doesn't get popped again after the form(s) it is "guarding".
The patch seemed like an easier way to describe the intent than trying
to phrase it in prose, but it's definitely not meant to be used as-is.
I agree that we need to make sure things get popped as needed (tho I got
the impression that it should indeed already occur (at least in the
call-path I looked at)).
>> If it doesn't work, do you happen to know why?
> I'm not sure whether it would work for the (similar) bug, bug#73746,
> where the symbols with position were getting lost in byte-opt.el.
Oh, indeed my patch won't work at all when the warning/error message is
emitted later on: we need to preserve the info in the macro-expanded
code itself rather than in the locations-info stack.
Stefan
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 12 Oct 2024 12:02:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 12 08:02:49 2024 Received: from localhost ([127.0.0.1]:40295 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1szapc-0001Mb-Vb for submit <at> debbugs.gnu.org; Sat, 12 Oct 2024 08:02:49 -0400 Received: from mail-lf1-f42.google.com ([209.85.167.42]:42110) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mattias.engdegard@HIDDEN>) id 1szapa-0001MG-2I for 73725 <at> debbugs.gnu.org; Sat, 12 Oct 2024 08:02:47 -0400 Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-539e4b7409fso721808e87.0 for <73725 <at> debbugs.gnu.org>; Sat, 12 Oct 2024 05:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728734485; x=1729339285; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=vnSflh5yHUfvsVFQYJmjB8HqWb1zq8T8M2feaKjwqTc=; b=mRMoaCt8gWxEi4BPrEUbrhULC6+NJZpNXJQfrdUge3bEXWmmn6YSCasx6yte/4nj8r axA3MFNjwamR2faDp1e4Ekd/sD7nUoFMHbJGyo2C0NYDRBLh2aWsRQOHjh7mvtt5SsKJ /r1cQh6oF21Wo5hoKWh+DxCC9h+FieDtGkkepNIJ5/L1XanLi5J51HZoYAe9WawkWdaF 1kNuaPP79DQ0BTm5aJK5LQuuwCUT1ZE2e/KbLkkTiPfTQDSffIh7b9sSFyPUDPTeizdP 2mqPT4i66anWjHhzM3TJahmtGuEYHeixMxxEjgM3dh0yLo9CfoDMuc1FbLIpAH4KNcKW ohHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728734485; x=1729339285; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=vnSflh5yHUfvsVFQYJmjB8HqWb1zq8T8M2feaKjwqTc=; b=aBUidGFdPBVT5Mct8WW2q9ML36ofKQEGJIJeDQUSvxgOQWS8WAQa4DajxgwkTwX8sA ZUCcWGWWVqA7F+IVSTRxph2V9YWP3uTPVQYBJpjBxHeyh9AKDpGRh9jx4MScs4L9PhbR BTTfeWKJbVUjOqqZExoa62TT+dRbtUU5TbzFFN+VbVaPuaUIvdnU+96IYnl1ntdETOEB kbtjYmx/y5pWLbjyV6/XAtGJh9Zxy+lwmlxhq/j3natLtULAgxQmV89ULkoVc6Gplql1 1QT3jyLRd/RWOqZ07RYXWEIaHH688+1hTQEJJjyiS/sqydAzauu6FMEWo/kBOAvLJen6 lpIg== X-Forwarded-Encrypted: i=1; AJvYcCWJsDfVTNM8Pn2IGna/8iUkJiA/EcHSStjVv0M3SERJ/LyI10FNhwqpsdkHeltsuk7gBMaMkw==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyqPIU2ztkqkrIONOzfNDBbZIU/vJcs/qZ/V6hK2VXzEjT3cWWL 4V3ASUQGZAon4voPyY2VucjWB7GZmbCHLoSCU/1MJ3UUph8Qn/iG X-Google-Smtp-Source: AGHT+IGbSECusj5hi5RYM0TcEKOBBWH/wmt0DpDmrY464ztI/TbN+fLiK0DTyBUmNOtW3KaiQ1Kn9g== X-Received: by 2002:a05:6512:3b96:b0:539:e453:d90c with SMTP id 2adb3069b0e04-539e453da3amr929515e87.2.1728734484417; Sat, 12 Oct 2024 05:01:24 -0700 (PDT) Received: from smtpclient.apple ([2a02:1406:3:6dc3:8d7f:9bc7:63a4:2ec7]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-539cb6c8a48sm910630e87.108.2024.10.12.05.01.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 12 Oct 2024 05:01:23 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= <mattias.engdegard@HIDDEN> In-Reply-To: <jwvttdii56q.fsf-monnier+emacs@HIDDEN> Date: Sat, 12 Oct 2024 14:01:20 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <8A05D4D2-C77B-4089-B82C-4DB3E88F1276@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> To: Stefan Monnier <monnier@HIDDEN> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: Alan Mackenzie <acm@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) 12 okt. 2024 kl. 01.45 skrev Stefan Monnier <monnier@HIDDEN>: > That sounds potentially costly I'm also concerned about costs but won't have time to look at it for at = least a few days. However since it touches code that I'm revamping (although that process = has been demoted to power-saving mode at the moment) I'd like to give it = a look-over when I can.
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.Received: (at 73725) by debbugs.gnu.org; 12 Oct 2024 10:55:16 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 12 06:55:16 2024 Received: from localhost ([127.0.0.1]:38222 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1szZmG-0005KL-0i for submit <at> debbugs.gnu.org; Sat, 12 Oct 2024 06:55:16 -0400 Received: from mail.muc.de ([193.149.48.3]:60226) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1szZfU-0004qz-R7 for 73725 <at> debbugs.gnu.org; Sat, 12 Oct 2024 06:48:17 -0400 Received: (qmail 47348 invoked by uid 3782); 12 Oct 2024 12:47:55 +0200 Received: from muc.de (p4fe156e7.dip0.t-ipconnect.de [79.225.86.231]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 12 Oct 2024 12:47:54 +0200 Received: (qmail 6318 invoked by uid 1000); 12 Oct 2024 10:47:53 -0000 Date: Sat, 12 Oct 2024 10:47:53 +0000 To: Stefan Monnier <monnier@HIDDEN> Subject: Re: bug#73725: Master: Wrong position for byte compiler warning message. Message-ID: <ZwpT2cqTwdUBldr3@HIDDEN> References: <Zweq-fxF2cNXL2nA@HIDDEN> <jwvttdii56q.fsf-monnier+emacs@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <jwvttdii56q.fsf-monnier+emacs@HIDDEN> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 73725 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>, 73725 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Stefan. Thanks for the reply! On Fri, Oct 11, 2024 at 19:45:18 -0400, Stefan Monnier wrote: > > (i) Create the following file, bad-error-position2.el: > > ######################################################################### > > ;; -*- lexical-binding:t -*- > > (eval-and-compile > > (defmacro increase () > > `(let ((foo (point-max))) > > (cond > > ((consp foo) > > (message "consp %s" foo) > > foo) > > ((numberp foo) > > (1+ fooo)) ; Note the misspelling. > > (t (message "Something else: %s" foo)))))) > > (defun call-increase (bar) > > (cond > > ((not (or (consp bar) > > (numberp bar))) > > bar) > > (t (increase)))) > > ######################################################################### > > Note the misspelling of `foo' as `fooo' on line 10. > > (ii) emacs -Q > > (iii) M-x byte-compile-file /path/to/bad-error-position2.el. > > (iv) This gives the warning message: > > bad-error-position2.el:14:4: Warning: reference to free variable `fooo' > > (v) The position 14:4 is wrong. That is the position of the `cond' > > symbol in `call-increase'. > > (vi) The correct message should be: > > bad-error-position2.el:18:8: Warning: reference to free variable `fooo' > > .. 18:8 is the position of `increase' on the last line of the file. > Nitpick: one could also argue that the "correct" message should point to > line 8 where is the `fooo` typo. > > +(defun sub-macroexp--posify-form (form call-pos depth) > > + "Try to apply the transformation of `macroexp--posify-form' to FORM. > > +FORM and CALL-POS are as in that function. DEPTH is a small integer, > > +decremented at each recursive call, to prevent infinite recursion. > > +Return the changed form, or nil if no change happened." > > + (let (new-form > > + ) > > + (cond > > + ((zerop depth) nil) > > + ((and (consp form) > > + (symbolp (car form)) > > + (car form)) > > + (setcar form (position-symbol (car form) call-pos)) > > + form) > > + ((consp form) > > + (or (when (setq new-form (sub-macroexp--posify-form > > + (car form) call-pos (1- depth))) > > + (setcar form new-form) > > + form) > > + (when (setq new-form (sub-macroexp--posify-form > > + (cdr form) call-pos (1- depth))) > > + (setcdr form new-form) > > + form))) > > + ((symbolp form) > > + (if form ; Don't position nil! > > + (position-symbol form call-pos))) > > + ((and (or (vectorp form) (recordp form))) > > + (let ((len (length form)) > > + (i 0) > > + ) > > + (while (and (< i len) > > + (not (setq new-form (sub-macroexp--posify-form > > + (aref form i) call-pos (1- depth))))) > > + (setq i (1+ i))) > > + (when (< i len) > > + (aset form i new-form) > > + form)))))) > That sounds potentially costly Have you tried to measure the > performance impact in some "typical" cases? I haven't measured it, no, but I doubt it will be costly - the recursion in sub-macroexp--posify-form will very rarely occur. Virtually every form passed to it will by a macro invocation or a symbol, I think. > While reading your description I was naively thinking: we can > probably fix it with a trivial patch like: > diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el > index 19daa57b207..5cc471e32f6 100644 > --- a/lisp/emacs-lisp/macroexp.el > +++ b/lisp/emacs-lisp/macroexp.el > @@ -246,6 +246,7 @@ macroexp-macroexpand > (let* ((macroexpand-all-environment env) > new-form) > (while (not (eq form (setq new-form (macroexpand-1 form env)))) > + (push form byte-compile-form-stack) > (let ((fun (car-safe form))) > (setq form > (if (and fun (symbolp fun) > Have you tried something like this? I haven't, no. It might well work. But I think the `push' form should go outside of the loop, I'm also a little wary about pushing an item onto stack when it doesn't get popped again after the form(s) it is "guarding". > If it doesn't work, do you happen to know why? I'm not sure whether it would work for the (similar) bug, bug#73746, where the symbols with position were getting lost in byte-opt.el. But I'll give it a try. I'm a bit busy in the next few days, so it might be next week before I manage it. > Stefan -- Alan Mackenzie (Nuremberg, Germany).
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at 73725) by debbugs.gnu.org; 11 Oct 2024 23:45:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 11 19:45:42 2024
Received: from localhost ([127.0.0.1]:35690 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1szPKI-0002vV-8X
for submit <at> debbugs.gnu.org; Fri, 11 Oct 2024 19:45:42 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:32493)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <monnier@HIDDEN>) id 1szPKG-0002vH-2d
for 73725 <at> debbugs.gnu.org; Fri, 11 Oct 2024 19:45:40 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 1E4C1100055;
Fri, 11 Oct 2024 19:45:21 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
s=mail; t=1728690319;
bh=hLoiia8p+YNSqk4s6KM2kAYL+DBkOQJZxzkTXSIIi2o=;
h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
b=Dz7zfEvlsd5Ib5rb2LW8PKl1arMrZ3tBwI6e0fonlrggy8R0xACqz/K57HYcDFz0z
8iuNU7latLTQKdc1FuoN3Em18HcdhPl092iC0D00XMK6bkdhccynd9CUeMBz1athjC
1SdZS7KvgFGzlj7q9DwHRjcySM5bHrDdS0+eL7M1w/+9uz+p968krZUrojRNEq5VlB
vrP6i3Vjk/8P/tXGM3Rk0WipsvBt5saJlLSiaSOx/q6aVTJDVlb8XDVFcOAhU9kYK2
TeSpfhzeOg6bAgHUtc/Mxo5G3toeG1XZlJt//U0qkEzVVCYOesHntmzaHf07fzfalj
+UVP13co2gOhw==
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 2AF15100042;
Fri, 11 Oct 2024 19:45:19 -0400 (EDT)
Received: from pastel (104-195-209-82.cpe.teksavvy.com [104.195.209.82])
by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id F187B120849;
Fri, 11 Oct 2024 19:45:18 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Alan Mackenzie <acm@HIDDEN>
Subject: Re: bug#73725: Master: Wrong position for byte compiler warning
message.
In-Reply-To: <Zweq-fxF2cNXL2nA@HIDDEN> (Alan Mackenzie's message of
"Thu, 10 Oct 2024 10:22:49 +0000")
Message-ID: <jwvttdii56q.fsf-monnier+emacs@HIDDEN>
References: <Zweq-fxF2cNXL2nA@HIDDEN>
Date: Fri, 11 Oct 2024 19:45:18 -0400
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results: 0
ALL_TRUSTED -1 Passed through trusted hosts only via SMTP
AWL -0.070 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature,
not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
domain
X-SPAM-LEVEL:
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 73725
Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= <mattias.engdegard@HIDDEN>,
73725 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> (i) Create the following file, bad-error-position2.el:
>
> #########################################################################
> ;; -*- lexical-binding:t -*-
> (eval-and-compile
> (defmacro increase ()
> `(let ((foo (point-max)))
> (cond
> ((consp foo)
> (message "consp %s" foo)
> foo)
> ((numberp foo)
> (1+ fooo)) ; Note the misspelling.
> (t (message "Something else: %s" foo))))))
>
> (defun call-increase (bar)
> (cond
> ((not (or (consp bar)
> (numberp bar)))
> bar)
> (t (increase))))
> #########################################################################
>
> Note the misspelling of `foo' as `fooo' on line 10.
>
> (ii) emacs -Q
> (iii) M-x byte-compile-file /path/to/bad-error-position2.el.
> (iv) This gives the warning message:
>
> bad-error-position2.el:14:4: Warning: reference to free variable `fooo'
>
> (v) The position 14:4 is wrong. That is the position of the `cond'
> symbol in `call-increase'.
> (vi) The correct message should be:
>
> bad-error-position2.el:18:8: Warning: reference to free variable `fooo'
>
> .. 18:8 is the position of `increase' on the last line of the file.
Nitpick: one could also argue that the "correct" message should point to
line 8 where is the `fooo` typo.
> +(defun sub-macroexp--posify-form (form call-pos depth)
> + "Try to apply the transformation of `macroexp--posify-form' to FORM.
> +FORM and CALL-POS are as in that function. DEPTH is a small integer,
> +decremented at each recursive call, to prevent infinite recursion.
> +Return the changed form, or nil if no change happened."
> + (let (new-form
> + )
> + (cond
> + ((zerop depth) nil)
> + ((and (consp form)
> + (symbolp (car form))
> + (car form))
> + (setcar form (position-symbol (car form) call-pos))
> + form)
> + ((consp form)
> + (or (when (setq new-form (sub-macroexp--posify-form
> + (car form) call-pos (1- depth)))
> + (setcar form new-form)
> + form)
> + (when (setq new-form (sub-macroexp--posify-form
> + (cdr form) call-pos (1- depth)))
> + (setcdr form new-form)
> + form)))
> + ((symbolp form)
> + (if form ; Don't position nil!
> + (position-symbol form call-pos)))
> + ((and (or (vectorp form) (recordp form)))
> + (let ((len (length form))
> + (i 0)
> + )
> + (while (and (< i len)
> + (not (setq new-form (sub-macroexp--posify-form
> + (aref form i) call-pos (1- depth)))))
> + (setq i (1+ i)))
> + (when (< i len)
> + (aset form i new-form)
> + form))))))
That sounds potentially costly Have you tried to measure the
performance impact in some "typical" cases?
While reading your description I was naively thinking: we can
probably fix it with a trivial patch like:
diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el
index 19daa57b207..5cc471e32f6 100644
--- a/lisp/emacs-lisp/macroexp.el
+++ b/lisp/emacs-lisp/macroexp.el
@@ -246,6 +246,7 @@ macroexp-macroexpand
(let* ((macroexpand-all-environment env)
new-form)
(while (not (eq form (setq new-form (macroexpand-1 form env))))
+ (push form byte-compile-form-stack)
(let ((fun (car-safe form)))
(setq form
(if (and fun (symbolp fun)
Have you tried something like this?
If it doesn't work, do you happen to know why?
Stefan
bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 10 Oct 2024 10:23:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 10 06:23:20 2024
Received: from localhost ([127.0.0.1]:58764 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1syqKF-0000IZ-O2
for submit <at> debbugs.gnu.org; Thu, 10 Oct 2024 06:23:20 -0400
Received: from lists.gnu.org ([209.51.188.17]:45804)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1syqKC-0000IR-No
for submit <at> debbugs.gnu.org; Thu, 10 Oct 2024 06:23:18 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <acm@HIDDEN>) id 1syqK1-0003Ju-87
for bug-gnu-emacs@HIDDEN; Thu, 10 Oct 2024 06:23:05 -0400
Received: from mail.muc.de ([193.149.48.3])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <acm@HIDDEN>) id 1syqJy-0003oh-Tw
for bug-gnu-emacs@HIDDEN; Thu, 10 Oct 2024 06:23:05 -0400
Received: (qmail 23986 invoked by uid 3782); 10 Oct 2024 12:22:50 +0200
Received: from muc.de (p4fe15213.dip0.t-ipconnect.de [79.225.82.19]) (using
STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP;
Thu, 10 Oct 2024 12:22:50 +0200
Received: (qmail 11006 invoked by uid 1000); 10 Oct 2024 10:22:49 -0000
Date: Thu, 10 Oct 2024 10:22:49 +0000
To: bug-gnu-emacs@HIDDEN
Subject: Master: Wrong position for byte compiler warning message.
Message-ID: <Zweq-fxF2cNXL2nA@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-DEBBUGS-CC: Stefan Monnier =?iso-8859-1?B?PG1vbm5pZXJA?=
=?iso-8859-1?Q?iro=2Eumontreal=2Eca=3E=2C_Mattias_Engdeg=E5r?=
=?iso-8859-1?Q?d?= <mattias.engdegard@HIDDEN>
X-Submission-Agent: TMDA/1.3.x (Ph3nix)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
Received-SPF: pass client-ip=193.149.48.3; envelope-from=acm@HIDDEN;
helo=mail.muc.de
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)
Hello, Emacs.
On the master branch.
(i) Create the following file, bad-error-position2.el:
#########################################################################
;; -*- lexical-binding:t -*-
(eval-and-compile
(defmacro increase ()
`(let ((foo (point-max)))
(cond
((consp foo)
(message "consp %s" foo)
foo)
((numberp foo)
(1+ fooo)) ; Note the misspelling.
(t (message "Something else: %s" foo))))))
(defun call-increase (bar)
(cond
((not (or (consp bar)
(numberp bar)))
bar)
(t (increase))))
#########################################################################
Note the misspelling of `foo' as `fooo' on line 10.
(ii) emacs -Q
(iii) M-x byte-compile-file /path/to/bad-error-position2.el.
(iv) This gives the warning message:
bad-error-position2.el:14:4: Warning: reference to free variable `fooo'
(v) The position 14:4 is wrong. That is the position of the `cond'
symbol in `call-increase'.
(vi) The correct message should be:
bad-error-position2.el:18:8: Warning: reference to free variable `fooo'
.. 18:8 is the position of `increase' on the last line of the file.
#########################################################################
Diagnosis
---------
When the byte compiler expands the invocation of `increase' on the last
line, `increase' is a symbol with position. The form returned by the
macro expander, however, has no position on its first symbol, the `let'.
Thus when the warning is being processed, the pertinent entry on
byte-compile-form-stack has no symbols with position, so the code takes
a SWP from a deeper nested form, the `cond' form on line 14, and derives
the warning position from it.
#########################################################################
Proposed fix
------------
To fix this I propose amending the macro expander, such that if the
first symbol in a form being expanded is a SWP, the position is applied
to the expanded form, typically on the car of that form, but possibly
elsewhere if that expanded form isn't a list.
The following patch appears to fix the bug:
diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 29e7882c851..e0a59d19d4e 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -2582,14 +2582,17 @@ byte-compile-flush-pending
byte-compile-jump-tables nil))))
(defun byte-compile-preprocess (form &optional _for-effect)
- (setq form (macroexpand-all form byte-compile-macro-environment))
+ (let ((call-pos (and (consp form)
+ (symbol-with-pos-p (car form))
+ (symbol-with-pos-pos (car form)))))
+ (setq form (macroexpand-all form byte-compile-macro-environment call-pos))
;; FIXME: We should run byte-optimize-form here, but it currently does not
;; recurse through all the code, so we'd have to fix this first.
;; Maybe a good fix would be to merge byte-optimize-form into
;; macroexpand-all.
;; (if (memq byte-optimize '(t source))
;; (setq form (byte-optimize-form form for-effect)))
- (cconv-closure-convert form byte-compile-bound-variables))
+ (cconv-closure-convert form byte-compile-bound-variables)))
;; byte-hunk-handlers cannot call this!
(defun byte-compile-toplevel-file-form (top-level-form)
diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el
index 4524eccc7ef..44d49bd7757 100644
--- a/lisp/emacs-lisp/macroexp.el
+++ b/lisp/emacs-lisp/macroexp.el
@@ -237,11 +237,63 @@ macroexpand-1
form))))))))
(t form)))
+(defun sub-macroexp--posify-form (form call-pos depth)
+ "Try to apply the transformation of `macroexp--posify-form' to FORM.
+FORM and CALL-POS are as in that function. DEPTH is a small integer,
+decremented at each recursive call, to prevent infinite recursion.
+Return the changed form, or nil if no change happened."
+ (let (new-form
+ )
+ (cond
+ ((zerop depth) nil)
+ ((and (consp form)
+ (symbolp (car form))
+ (car form))
+ (setcar form (position-symbol (car form) call-pos))
+ form)
+ ((consp form)
+ (or (when (setq new-form (sub-macroexp--posify-form
+ (car form) call-pos (1- depth)))
+ (setcar form new-form)
+ form)
+ (when (setq new-form (sub-macroexp--posify-form
+ (cdr form) call-pos (1- depth)))
+ (setcdr form new-form)
+ form)))
+ ((symbolp form)
+ (if form ; Don't position nil!
+ (position-symbol form call-pos)))
+ ((and (or (vectorp form) (recordp form)))
+ (let ((len (length form))
+ (i 0)
+ )
+ (while (and (< i len)
+ (not (setq new-form (sub-macroexp--posify-form
+ (aref form i) call-pos (1- depth)))))
+ (setq i (1+ i)))
+ (when (< i len)
+ (aset form i new-form)
+ form))))))
+
+(defun macroexp--posify-form (form call-pos)
+ "Try to apply the position CALL-POS to the form FORM.
+CALL-POS is a buffer position, a number. FORM may be any lisp form,
+and is typically the output form returned by macro expansion.
+Apply CALL-POS to FORM as a symbol with position, such that
+`byte-compile--first-symbol-with-pos' can later return it. Return
+the possibly modified FORM."
+ (let ((new-form (sub-macroexp--posify-form form call-pos 10)))
+ (or new-form form)))
+
(defun macroexp-macroexpand (form env)
"Like `macroexpand' but checking obsolescence."
(let* ((macroexpand-all-environment env)
- new-form)
+ (call-pos (and (consp form)
+ (symbol-with-pos-p (car form))
+ (symbol-with-pos-pos (car form))))
+ macroexpanded new-form)
(while (not (eq form (setq new-form (macroexpand-1 form env))))
+ (setq macroexpanded t)
(let ((fun (car-safe form)))
(setq form
(if (and fun (symbolp fun)
@@ -252,6 +304,8 @@ macroexp-macroexpand
(if (symbolp (symbol-function fun)) "alias" "macro"))
new-form (list 'obsolete fun) nil fun)
new-form))))
+ (when (and macroexpanded call-pos)
+ (setq form (macroexp--posify-form form call-pos)))
form))
(defun macroexp--unfold-lambda (form &optional name)
@@ -516,14 +570,22 @@ macroexp--expand-all
(_ form))))))
;;;###autoload
-(defun macroexpand-all (form &optional environment)
+(defun macroexpand-all (form &optional environment call-pos)
"Return result of expanding macros at all levels in FORM.
If no macros are expanded, FORM is returned unchanged.
The second optional arg ENVIRONMENT specifies an environment of macro
-definitions to shadow the loaded ones for use in file byte-compilation."
- (let ((macroexpand-all-environment environment)
- (macroexp--dynvars macroexp--dynvars))
- (macroexp--expand-all form)))
+definitions to shadow the loaded ones for use in file byte-compilation.
+CALL-POS, if non-nil, is a buffer position which will be incorporated
+into the result form as a symbol with position, later to be usable by
+`byte-compile--warning-source-offset'."
+ (let*
+ ((macroexpand-all-environment environment)
+ (macroexp--dynvars macroexp--dynvars)
+ (new-form
+ (macroexp--expand-all form)))
+ (if call-pos
+ (macroexp--posify-form new-form call-pos)
+ new-form)))
;; This function is like `macroexpand-all' but for use with top-level
;; forms. It does not dynbind `macroexp--dynvars' because we want
--
Alan Mackenzie (Nuremberg, Germany).
Alan Mackenzie <acm@HIDDEN>:monnier@HIDDEN, mattias.engdegard@HIDDEN, bug-gnu-emacs@HIDDEN.
Full text available.monnier@HIDDEN, mattias.engdegard@HIDDEN, bug-gnu-emacs@HIDDEN:bug#73725; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.