GNU logs - #55137, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#55137: Different result when interpreted and when evaluating byte-compiled code
Resent-From: Paul Pogonyshev <pogonyshev@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 26 Apr 2022 22:18:01 +0000
Resent-Message-ID: <handler.55137.B.165101142110340 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 55137 <at> debbugs.gnu.org
MIME-Version: 1.0
From: Paul Pogonyshev <pogonyshev@HIDDEN>
Date: Wed, 27 Apr 2022 00:16:42 +0200
Message-ID: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
Content-Type: multipart/alternative; boundary="000000000000d4b4ca05dd960c7f"
Received-SPF: pass client-ip=2a00:1450:4864:20::529;
 envelope-from=pogonyshev@HIDDEN; helo=mail-ed1-x529.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

--000000000000d4b4ca05dd960c7f
Content-Type: text/plain; charset="UTF-8"

I'm not absolutely sure if it is a bug or a "feature", but it is
extremely confusing.

To reproduce: store this as file `test.el':

    (defvar special-variable nil)

    (defmacro is-special-as-macro ()
      (special-variable-p 'special-variable))

    (defun is-special-as-function ()
      (is-special-as-macro))

    (print (is-special-as-function))
    (print (eval '(is-special-as-macro)))

Now, from command line:

    $ rm -f test.elc; emacs --batch -l test.el

This loads the file as interpreted Elisp and prints `t' two times,
i.e. always recognizes the variable as special.

Next, byte-compile the file before loading:

    $ rm -f test.elc; emacs --batch --eval "(byte-compile-file
\"test.el\")"; emacs --batch -l test.elc

This prints `nil' and `t', i.e. variable is now considered non-special
by `is-special-as-function'.  From investigating `.elc' file, it is
apparent that this is encoded into it as a macroexpanded constant.
Note that `is-special-as-macro' still works fine, the problem appears
only when it gets macroexpanded during byte-compilation.

So, apparently `defvar' form is sort of "skipped without paying
attention" during byte-compilation.  If I put it into an
`eval-and-compile' form, then macroexpansion does produce expected
result.

I noticed this with code using `iter2' library.  When `iter2-defun'
generator functions get macroexpanded during compilation, built-in
`special-variable-p' is called, because for generator functions this
is important to distinguish between local and dynamic variables.  The
example above is derived from debugging real failure.

Standard `generator' package is also affected.  Here is example code:

    ;;; -*- lexical-binding: t -*-

    (require 'generator)

    (defvar special-variable nil)

    (defun get-special-variable ()
      special-variable)

    (iter-defun buggy ()
      (let ((special-variable t))
        (iter-yield (get-special-variable))))

    (print (iter-next (buggy)))

As before, save as `test.el' and execute the same two commands.
Produced output is different depending on whether the file has been
byte-compiled or not.

Is `eval-and-compile' the proper workaround?  Can things be made less
confusing by noticing declared special variables during
byte-compilation?

Paul

--000000000000d4b4ca05dd960c7f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I&#39;m not absolutely sure if it is a bug or a &quot;feat=
ure&quot;, but it is<br>extremely confusing.<br><br>To reproduce: store thi=
s as file `test.el&#39;:<br><br>=C2=A0 =C2=A0 (defvar special-variable nil)=
<br><br>=C2=A0 =C2=A0 (defmacro is-special-as-macro ()<br>=C2=A0 =C2=A0 =C2=
=A0 (special-variable-p &#39;special-variable))<br><br>=C2=A0 =C2=A0 (defun=
 is-special-as-function ()<br>=C2=A0 =C2=A0 =C2=A0 (is-special-as-macro))<b=
r><br>=C2=A0 =C2=A0 (print (is-special-as-function))<br>=C2=A0 =C2=A0 (prin=
t (eval &#39;(is-special-as-macro)))<br><br>Now, from command line:<br><br>=
=C2=A0 =C2=A0 $ rm -f test.elc; emacs --batch -l test.el<br><br>This loads =
the file as interpreted Elisp and prints `t&#39; two times,<br>i.e. always =
recognizes the variable as special.<br><br>Next, byte-compile the file befo=
re loading:<br><br>=C2=A0 =C2=A0 $ rm -f test.elc; emacs --batch --eval &qu=
ot;(byte-compile-file \&quot;test.el\&quot;)&quot;; emacs --batch -l test.e=
lc<br><br>This prints `nil&#39; and `t&#39;, i.e. variable is now considere=
d non-special<br>by `is-special-as-function&#39;.=C2=A0 From investigating =
`.elc&#39; file, it is<br>apparent that this is encoded into it as a macroe=
xpanded constant.<br>Note that `is-special-as-macro&#39; still works fine, =
the problem appears<br>only when it gets macroexpanded during byte-compilat=
ion.<br><br>So, apparently `defvar&#39; form is sort of &quot;skipped witho=
ut paying<br>attention&quot; during byte-compilation.=C2=A0 If I put it int=
o an<br>`eval-and-compile&#39; form, then macroexpansion does produce expec=
ted<br>result.<br><br>I noticed this with code using `iter2&#39; library.=
=C2=A0 When `iter2-defun&#39;<br>generator functions get macroexpanded duri=
ng compilation, built-in<br>`special-variable-p&#39; is called, because for=
 generator functions this<br>is important to distinguish between local and =
dynamic variables.=C2=A0 The<br>example above is derived from debugging rea=
l failure.<br><br><div>Standard `generator&#39; package is also affected.=
=C2=A0 Here is example code:<br><br>=C2=A0 =C2=A0 ;;; -*- lexical-binding: =
t -*-<br><br>=C2=A0 =C2=A0 (require &#39;generator)<br><br>=C2=A0 =C2=A0 (d=
efvar special-variable nil)<br><br>=C2=A0 =C2=A0 (defun get-special-variabl=
e ()<br>=C2=A0 =C2=A0 =C2=A0 special-variable)<br><br>=C2=A0 =C2=A0 (iter-d=
efun buggy ()<br>=C2=A0 =C2=A0 =C2=A0 (let ((special-variable t))<br>=C2=A0=
 =C2=A0 =C2=A0 =C2=A0 (iter-yield (get-special-variable))))<br><br>=C2=A0 =
=C2=A0 (print (iter-next (buggy)))<br><br>As before, save as `test.el&#39; =
and execute the same two commands.<br>Produced output is different dependin=
g on whether the file has been<br>byte-compiled or not.<br><br>Is `eval-and=
-compile&#39; the proper workaround?=C2=A0 Can things be made less<br>confu=
sing by noticing declared special variables during<br>byte-compilation?<br>=
<br>Paul<br></div></div>

--000000000000d4b4ca05dd960c7f--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Paul Pogonyshev <pogonyshev@HIDDEN>
Subject: bug#55137: Acknowledgement (Different result when interpreted and
 when evaluating byte-compiled code)
Message-ID: <handler.55137.B.165101142110340.ack <at> debbugs.gnu.org>
References: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
X-Gnu-PR-Message: ack 55137
X-Gnu-PR-Package: emacs
Reply-To: 55137 <at> debbugs.gnu.org
Date: Tue, 26 Apr 2022 22:18:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 55137 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
55137: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D55137
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#55137: Different result when interpreted and when evaluating byte-compiled code
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Wed, 27 Apr 2022 02:28:02 +0000
Resent-Message-ID: <handler.55137.B55137.16510264651951 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Paul Pogonyshev <pogonyshev@HIDDEN>
Cc: 55137 <at> debbugs.gnu.org
Date: Wed, 27 Apr 2022 05:27:25 +0300
Message-Id: <83k0bbl0qa.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
 (message from Paul Pogonyshev on Wed, 27 Apr 2022 00:16:42 +0200)
References: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

> From: Paul Pogonyshev <pogonyshev@HIDDEN>
> Date: Wed, 27 Apr 2022 00:16:42 +0200
> 
> I'm not absolutely sure if it is a bug or a "feature", but it is
> extremely confusing.

Thanks, but please tell in what version of Emacs is this, and
preferably show the information about your build that's collected by
"M-x report-emacs-bug".

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#55137: Different result when interpreted and when evaluating byte-compiled code
Resent-From: Paul Pogonyshev <pogonyshev@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Wed, 27 Apr 2022 09:20:02 +0000
Resent-Message-ID: <handler.55137.B55137.165105117610576 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 55137 <at> debbugs.gnu.org
MIME-Version: 1.0
References: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
 <83k0bbl0qa.fsf@HIDDEN>
In-Reply-To: <83k0bbl0qa.fsf@HIDDEN>
From: Paul Pogonyshev <pogonyshev@HIDDEN>
Date: Wed, 27 Apr 2022 11:19:17 +0200
Message-ID: <CAG7BpaoK6oBtRLmDtjeLTybRfB=G4xHa3zdK5VDnRYPb8+vc=Q@HIDDEN>
Content-Type: multipart/alternative; boundary="00000000000066fb2405dd9f4e2b"
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

--00000000000066fb2405dd9f4e2b
Content-Type: text/plain; charset="UTF-8"

Originally reproduced with 27.2. I now upgraded to 28.1. Pretty sure it is
the same with master.

In GNU Emacs 28.1.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 2.24.33,
cairo version 1.16.0)
 of 2022-04-27 built on gonzo
Repository revision: b511f5c05ced05fc7d1e403002bc5ba782879cba
Repository branch: emacs-28
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-x-toolkit=gtk2'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK2 ZLIB

Important settings:
  value of $LC_MONETARY: de_AT.UTF-8
  value of $LC_NUMERIC: ru_RU.UTF-8
  value of $LC_TIME: de_DE.UTF-8
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix

On Wed, 27 Apr 2022 at 04:27, Eli Zaretskii <eliz@HIDDEN> wrote:

> > From: Paul Pogonyshev <pogonyshev@HIDDEN>
> > Date: Wed, 27 Apr 2022 00:16:42 +0200
> >
> > I'm not absolutely sure if it is a bug or a "feature", but it is
> > extremely confusing.
>
> Thanks, but please tell in what version of Emacs is this, and
> preferably show the information about your build that's collected by
> "M-x report-emacs-bug".
>

--00000000000066fb2405dd9f4e2b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Originally reproduced with 27.2. I now upgraded to 28=
.1. Pretty sure it is the same with master.</div><div><br></div>In GNU Emac=
s 28.1.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 2.24.33, cairo versio=
n 1.16.0)<br>=C2=A0of 2022-04-27 built on gonzo<br>Repository revision: b51=
1f5c05ced05fc7d1e403002bc5ba782879cba<br>Repository branch: emacs-28<br>Win=
dowing system distributor &#39;The X.Org Foundation&#39;, version 11.0.1201=
4000<br>System Description: Debian GNU/Linux bookworm/sid<br><br>Configured=
 using:<br>=C2=A0&#39;configure --with-x-toolkit=3Dgtk2&#39;<br><br>Configu=
red features:<br>CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ=
 JPEG<br>LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUN=
D<br>THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK2 ZLIB<br><br>Imp=
ortant settings:<br>=C2=A0 value of $LC_MONETARY: de_AT.UTF-8<br>=C2=A0 val=
ue of $LC_NUMERIC: ru_RU.UTF-8<br>=C2=A0 value of $LC_TIME: de_DE.UTF-8<br>=
=C2=A0 value of $LANG: en_CA.UTF-8<br>=C2=A0 locale-coding-system: utf-8-un=
ix<br></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_=
attr">On Wed, 27 Apr 2022 at 04:27, Eli Zaretskii &lt;<a href=3D"mailto:eli=
z@HIDDEN">eliz@HIDDEN</a>&gt; wrote:<br></div><blockquote class=3D"gmail_=
quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=
204);padding-left:1ex">&gt; From: Paul Pogonyshev &lt;<a href=3D"mailto:pog=
onyshev@HIDDEN" target=3D"_blank">pogonyshev@HIDDEN</a>&gt;<br>
&gt; Date: Wed, 27 Apr 2022 00:16:42 +0200<br>
&gt; <br>
&gt; I&#39;m not absolutely sure if it is a bug or a &quot;feature&quot;, b=
ut it is<br>
&gt; extremely confusing.<br>
<br>
Thanks, but please tell in what version of Emacs is this, and<br>
preferably show the information about your build that&#39;s collected by<br=
>
&quot;M-x report-emacs-bug&quot;.<br>
</blockquote></div>

--00000000000066fb2405dd9f4e2b--

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#55137: Different result when interpreted and when evaluating byte-compiled code
Resent-From: Phil Sainty <psainty@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Wed, 27 Apr 2022 11:21:02 +0000
Resent-Message-ID: <handler.55137.B55137.165105844822780 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 55137
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Paul Pogonyshev <pogonyshev@HIDDEN>
Cc: 55137 <at> debbugs.gnu.org
Received: via spool by 55137-submit <at> debbugs.gnu.org id=B55137.165105844822780
          (code B ref 55137); Wed, 27 Apr 2022 11:21:02 +0000
Received: (at 55137) by debbugs.gnu.org; 27 Apr 2022 11:20:48 +0000
Received: from localhost ([127.0.0.1]:41344 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1njfiy-0005vM-Ek
	for submit <at> debbugs.gnu.org; Wed, 27 Apr 2022 07:20:48 -0400
Received: from smtp-3.orcon.net.nz ([60.234.4.44]:53007)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <psainty@HIDDEN>) id 1njfiv-0005vB-2p
 for 55137 <at> debbugs.gnu.org; Wed, 27 Apr 2022 07:20:46 -0400
Received: from [10.253.37.70] (port=35617 helo=webmail.orcon.net.nz)
 by smtp-3.orcon.net.nz with esmtpa (Exim 4.90_1)
 (envelope-from <psainty@HIDDEN>)
 id 1njfir-0000Xh-TZ; Wed, 27 Apr 2022 23:20:42 +1200
Received: from ip-139-180-65-103.kinect.net.nz ([139.180.65.103])
 via [10.253.37.253] by webmail.orcon.net.nz
 with HTTP (HTTP/1.1 POST); Wed, 27 Apr 2022 23:20:41 +1200
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Wed, 27 Apr 2022 23:20:41 +1200
From: Phil Sainty <psainty@HIDDEN>
In-Reply-To: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
References: <CAG7BpaqmWCYjdqDwQYZubgkYQdJH39wuh123+3G1P5oVhb5fJg@HIDDEN>
Message-ID: <8ddae25aa2488d550ca63e396015002e@HIDDEN>
X-Sender: psainty@HIDDEN
User-Agent: Orcon Webmail
X-GeoIP: --
X-Spam_score: -2.9
X-Spam_score_int: -28
X-Spam_bar: --
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 2022-04-27 10:16, Paul Pogonyshev wrote:
>     (defmacro is-special-as-macro ()
>       (special-variable-p 'special-variable))

This macro does not expand to code which calls `special-variable-p'.
Rather it calls `special-variable-p' at expansion time, and expands
to the return value of that call (nil or t).

If you byte-compile your code without loading it, then your would-be
`special-variable' doesn't exist, and hence your macro was expanding
to nil rather than t.

Try this:

      (defmacro is-special-as-macro ()
        '(special-variable-p 'special-variable))


-Phil

Last modified: Wed, 27 Apr 2022 11:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.