GNU bug report logs - #29165
26.0.90; can't use some code byte-compiled under emacs 24

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29165 in the body.
You can then email your comments to 29165 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> permabit.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.0.90; can't use some code byte-compiled under emacs 24
Date: Mon, 06 Nov 2017 01:57:25 -0500

Create a file with the contents:

    (require 'cl)
    (defun* foobar (init-fun test-fun cleanup-fun
                             &optional &key
                             (test #'eql)
                             (size 65))
      "Blah blah blah"
      (list 'x init-fun test-fun cleanup-fun test size))


Use Emacs 24.3 to byte-compile the file.

Start the Emacs 26 pretest, and load the .elc file.

Type into the scratch buffer:

    (foobar 1 2 3 :test 4)

And use eval-last-sexp to try to evaluate it.  You should get a list
back.  I get an invalid-function error instead.  With the debugger
enabled, I see:

    Debugger entered--Lisp error: (invalid-function #f(compiled-function (init-fun test-fun cleanup-fun &optional &rest --cl-rest--) "Blah blah blah" #<bytecode 0x501733>))
      foobar(1 2 3 :test 4)
      eval((foobar 1 2 3 :test 4) nil)
      elisp--eval-last-sexp(nil)
      eval-last-sexp(nil)
      funcall-interactively(eval-last-sexp nil)
      call-interactively(eval-last-sexp nil nil)
      command-execute(eval-last-sexp)

It appears that the eval.c code of 26.0.90 doesn't like to see &optional
and &rest together in the argument list of a compiled function.  But in
this case, the byte compiler from version 24 does generate that.  In
version 25, only &rest is put into the list.


In GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2017-10-27 built on just-testing.permabit.com
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description:	Ubuntu 14.04.4 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading /tmp/foo.elc...done
Entering debugger...
Back to top level

Configured using:
 'configure --prefix=/u1/ken/install/emacs-26.0.90
 --with-x-toolkit=lucid --enable-checking --without-sound --without-gif'

Configured features:
XAW3D XPM JPEG TIFF PNG RSVG DBUS GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS
LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11

Important settings:
  locale-coding-system: nil

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq dired
dired-loaddefs format-spec rfc822 mml mml-sec password-cache epa derived
epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils help-mode
easymenu cl-print byte-opt bytecomp byte-compile cconv debug cl gv
cl-loaddefs cl-lib elec-pair time-date mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind inotify dynamic-setting system-font-setting
font-render-setting x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 98378 10227)
 (symbols 48 20669 1)
 (miscs 40 45 197)
 (strings 32 29669 1021)
 (string-bytes 1 765521)
 (vectors 16 14315)
 (vector-slots 8 495072 6314)
 (floats 8 53 146)
 (intervals 56 314 138)
 (buffers 992 12)
 (heap 1024 32780 1041))

Message #8 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 06 Nov 2017 07:44:47 -0500

tags 29165 + notabug
quit

Ken Raeburn <raeburn <at> permabit.com> writes:

>     (defun* foobar (init-fun test-fun cleanup-fun
>                              &optional &key

> Use Emacs 24.3 to byte-compile the file.
> 
> Start the Emacs 26 pretest, and load the .elc file.

>     Debugger entered--Lisp error: (invalid-function #f(compiled-function (init-fun test-fun cleanup-fun &optional &rest --cl-rest--) "Blah blah blah" 

> It appears that the eval.c code of 26.0.90 doesn't like to see &optional
> and &rest together in the argument list of a compiled function.

Yes, see Bug#24912 and Bug#24913.

> But in this case, the byte compiler from version 24 does generate
> that.  In version 25, only &rest is put into the list.

It's not the byte compiler specifically, the defun* macro (aka cl-defun)
expands &key into &rest, so the expansion has &optional &rest.  At any
rate, using &optional is meaningless there, so the solution is just to
remove it.

We should perhaps put something about throwing error on '&option &rest'
into NEWS though.

Message #13 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>, Ken Raeburn
 <raeburn <at> permabit.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org
Subject: RE: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Mon, 6 Nov 2017 06:24:04 -0800 (PST)

> It's not the byte compiler specifically, the defun* macro (aka cl-defun)
> expands &key into &rest, so the expansion has &optional &rest.  At any
> rate, using &optional is meaningless there, so the solution is just to
> remove it.
> 
> We should perhaps put something about throwing error on '&option &rest'
> into NEWS though.

I don't understand.  In Common Lisp it is perfectly correct
to use both &optional and &rest.  If Emacs's emulation of
CL's `defun' is defective in this regard, is that not a bug?

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node64.html

Or are you saying that there was a bug in Emacs 24's handling
of the CL `defun' emulation, where it expanded &key into
&rest, producing multiple &optional or &rest, and that now
this has been fixed - hence Emacs 24 byte-code does not work
in Emacs 26?

If you do add something to NEWS, please make clear just what
the problem is/was.  Does Emacs now correctly emulate CL
`defun' wrt &* keywords?  Is it an error to repeat &optional
or &rest?  If not, what is the behavior in that case?

Whatever the explanation is, please make it very clear in
NEWS.  Thx.

Message #16 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: Ken Raeburn <raeburn <at> permabit.com>,
 Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 6 Nov 2017 09:35:31 -0500

On Mon, Nov 6, 2017 at 9:24 AM, Drew Adams <drew.adams <at> oracle.com> wrote:

>> We should perhaps put something about throwing error on '&option &rest'
>> into NEWS though.
>
> I don't understand.  In Common Lisp it is perfectly correct
> to use both &optional and &rest.

What's rejected is (&optional &rest other-vars), whereas (&optional
var1 &rest other-vars) is okay. Does CL accept the first form (and if
yes, what does it mean)? I couldn't tell from the page you linked to.

Message #19 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Ken Raeburn <raeburn <at> permabit.com>,
 Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org
Subject: RE: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Mon, 6 Nov 2017 06:40:58 -0800 (PST)

> >> We should perhaps put something about throwing error on '&option &rest'
> >> into NEWS though.
> >
> > I don't understand.  In Common Lisp it is perfectly correct
> > to use both &optional and &rest.
> 
> What's rejected is (&optional &rest other-vars), whereas (&optional
> var1 &rest other-vars) is okay. Does CL accept the first form (and if
> yes, what does it mean)? I couldn't tell from the page you linked to.

CL accepts a single variable after &rest. And there must be
a variable after &optional.  (&optional foo &rest bar) is OK.

(&optional &rest foo) is not OK.
(&optional foo &rest bar toto titi) is not OK.

Message #22 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: raeburn <at> permabit.com, p.stephani2 <at> gmail.com, 29165 <at> debbugs.gnu.org
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 06 Nov 2017 18:08:00 +0200

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Mon, 06 Nov 2017 07:44:47 -0500
> Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org
> 
> We should perhaps put something about throwing error on '&option &rest'
> into NEWS though.

Please do, and thanks.

Message #25 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: Ken Raeburn <raeburn <at> permabit.com>, 29165 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 06 Nov 2017 17:20:22 +0000

[Message part 1 (text/plain, inline)]

Drew Adams <drew.adams <at> oracle.com> schrieb am Mo., 6. Nov. 2017 um
15:41 Uhr:

> > >> We should perhaps put something about throwing error on '&option
> &rest'
> > >> into NEWS though.
> > >
> > > I don't understand.  In Common Lisp it is perfectly correct
> > > to use both &optional and &rest.
> >
> > What's rejected is (&optional &rest other-vars), whereas (&optional
> > var1 &rest other-vars) is okay. Does CL accept the first form (and if
> > yes, what does it mean)? I couldn't tell from the page you linked to.
>
> CL accepts a single variable after &rest. And there must be
> a variable after &optional.  (&optional foo &rest bar) is OK.
>
> (&optional &rest foo) is not OK.
> (&optional foo &rest bar toto titi) is not OK.
>

That should match the current behavior in Emacs Lisp now.

[Message part 2 (text/html, inline)]

Message #28 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> permabit.com>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Mon, 6 Nov 2017 12:25:24 -0500

On Nov 6, 2017, at 09:40, Drew Adams <drew.adams <at> oracle.com> wrote:

>>>> We should perhaps put something about throwing error on '&option &rest'
>>>> into NEWS though.
>>> 
>>> I don't understand.  In Common Lisp it is perfectly correct
>>> to use both &optional and &rest.
>> 
>> What's rejected is (&optional &rest other-vars), whereas (&optional
>> var1 &rest other-vars) is okay. Does CL accept the first form (and if
>> yes, what does it mean)? I couldn't tell from the page you linked to.
> 
> CL accepts a single variable after &rest. And there must be
> a variable after &optional.  (&optional foo &rest bar) is OK.
> 
> (&optional &rest foo) is not OK.
> (&optional foo &rest bar toto titi) is not OK.

Is this CL in general or a particular CL implementation? The web page you sent the URL for earlier reads like a specification, and from its use of “*” looks to me like it allows the (admittedly useless) form of &optional with no variables.

Ken

Message #31 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
 Drew Adams <drew.adams <at> oracle.com>,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 06 Nov 2017 19:10:30 +0100

On Nov 06 2017, Ken Raeburn <raeburn <at> permabit.com> wrote:

> On Nov 6, 2017, at 09:40, Drew Adams <drew.adams <at> oracle.com> wrote:
>
>>>>> We should perhaps put something about throwing error on '&option &rest'
>>>>> into NEWS though.
>>>> 
>>>> I don't understand.  In Common Lisp it is perfectly correct
>>>> to use both &optional and &rest.
>>> 
>>> What's rejected is (&optional &rest other-vars), whereas (&optional
>>> var1 &rest other-vars) is okay. Does CL accept the first form (and if
>>> yes, what does it mean)? I couldn't tell from the page you linked to.
>> 
>> CL accepts a single variable after &rest. And there must be
>> a variable after &optional.  (&optional foo &rest bar) is OK.
>> 
>> (&optional &rest foo) is not OK.
>> (&optional foo &rest bar toto titi) is not OK.
>
> Is this CL in general or a particular CL implementation? The web page you sent the URL for earlier reads like a specification, and from its use of “*” looks to me like it allows the (admittedly useless) form of &optional with no variables.

clisp accepts it.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #34 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> permabit.com>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
 Drew Adams <drew.adams <at> oracle.com>,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Mon, 6 Nov 2017 14:10:27 -0500

[Message part 1 (text/plain, inline)]

On Nov 6, 2017, at 13:10, Andreas Schwab <schwab <at> linux-m68k.org> wrote:

> On Nov 06 2017, Ken Raeburn <raeburn <at> permabit.com <mailto:raeburn <at> permabit.com>> wrote:
> 
>> On Nov 6, 2017, at 09:40, Drew Adams <drew.adams <at> oracle.com> wrote:
>> 
>>>>>> We should perhaps put something about throwing error on '&option &rest'
>>>>>> into NEWS though.
>>>>> 
>>>>> I don't understand.  In Common Lisp it is perfectly correct
>>>>> to use both &optional and &rest.
>>>> 
>>>> What's rejected is (&optional &rest other-vars), whereas (&optional
>>>> var1 &rest other-vars) is okay. Does CL accept the first form (and if
>>>> yes, what does it mean)? I couldn't tell from the page you linked to.
>>> 
>>> CL accepts a single variable after &rest. And there must be
>>> a variable after &optional.  (&optional foo &rest bar) is OK.
>>> 
>>> (&optional &rest foo) is not OK.
>>> (&optional foo &rest bar toto titi) is not OK.
>> 
>> Is this CL in general or a particular CL implementation? The web page you sent the URL for earlier reads like a specification, and from its use of “*” looks to me like it allows the (admittedly useless) form of &optional with no variables.
> 
> clisp accepts it.

It appears that the emacs-26 version of defun* is happy with it (the original Lisp code I posted, using &optional &key) as well, as long as I provide the source, or a byte-compiled file from Emacs 25 or 26; it’s the .elc file generated by the older Emacs that’s causing me a problem. The (new?) checks are incompatible with the the old compiled file, even though the Lisp code itself *appears* to be acceptable still.

[Message part 2 (text/html, inline)]

Message #37 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 6 Nov 2017 14:16:47 -0500

On Mon, Nov 6, 2017 at 2:10 PM, Ken Raeburn <raeburn <at> permabit.com> wrote:

> It appears that the emacs-26 version of defun* is happy with it (the
> original Lisp code I posted, using &optional &key) as well, as long as I
> provide the source, or a byte-compiled file from Emacs 25 or 26; it’s the
> .elc file generated by the older Emacs that’s causing me a problem. The
> (new?) checks are incompatible with the the old compiled file, even though
> the Lisp code itself *appears* to be acceptable still.

It looks like the cl-defun in newer Emacs throws away the &optional
for you in this case.

Message #40 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: RE: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Mon, 6 Nov 2017 11:31:53 -0800 (PST)

> > CL accepts a single variable after &rest. And there must be
> > a variable after &optional.  (&optional foo &rest bar) is OK.
> >
> > (&optional &rest foo) is not OK.
> > (&optional foo &rest bar toto titi) is not OK.
> 
> Is this CL in general or a particular CL implementation? The web page you
> sent the URL for earlier reads like a specification, and from its use of
> “*” looks to me like it allows the (admittedly useless) form of &optional
> with no variables.

What I wrote was from memory.  What counts is the definition
of CL.  If it doesn't forbid such constructions then an
implementation or emulation) of CL is free to support them.

If it doesn't define them then an implementation is free
to define them any way it wants.

Message #43 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 13 Nov 2017 13:06:42 -0500

[Message part 1 (text/plain, inline)]

On Mon, Nov 6, 2017 at 2:16 PM, Noam Postavsky
<npostavs <at> users.sourceforge.net> wrote:
> On Mon, Nov 6, 2017 at 2:10 PM, Ken Raeburn <raeburn <at> permabit.com> wrote:
>
>> It appears that the emacs-26 version of defun* is happy with it (the
>> original Lisp code I posted, using &optional &key) as well, as long as I
>> provide the source, or a byte-compiled file from Emacs 25 or 26
>
> It looks like the cl-defun in newer Emacs throws away the &optional
> for you in this case.

I think we should make cl-defun reject this kind of code, to be
consistent with plain defun. See attached.

[v1-0001-Mention-new-strictness-for-optional-rest-in-argli.patch (application/octet-stream, attachment)]

Message #46 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> permabit.com>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Mon, 13 Nov 2017 14:42:18 -0500

On Nov 13, 2017, at 13:06, Noam Postavsky <npostavs <at> users.sourceforge.net> wrote:

> On Mon, Nov 6, 2017 at 2:16 PM, Noam Postavsky
> <npostavs <at> users.sourceforge.net> wrote:
>> On Mon, Nov 6, 2017 at 2:10 PM, Ken Raeburn <raeburn <at> permabit.com> wrote:
>> 
>>> It appears that the emacs-26 version of defun* is happy with it (the
>>> original Lisp code I posted, using &optional &key) as well, as long as I
>>> provide the source, or a byte-compiled file from Emacs 25 or 26
>> 
>> It looks like the cl-defun in newer Emacs throws away the &optional
>> for you in this case.
> 
> I think we should make cl-defun reject this kind of code, to be
> consistent with plain defun. See attached.

I’m of two minds about it… it’s a useless but harmless degenerate case, and based on the link Drew posted and the test Andreas did, making it an error would be a gratuitous incompatibility with CL or at least one implementation.  And maybe it’s not entirely useless if it simplifies someone’s macro so that they can treat zero-or-more optional variables with a single, simple common code path.  But even if we do make it an error, isn’t there usually a stage where it’s just a warning?

(And if we’re going to make that sort of thing an error, we should probably check whether empty &key or &aux variable lists are similarly rejected.  I haven’t looked.)

Also, what the CL macros do going forward is arguably a separate question from whether byte-code processing should reject the byte code generated for the same construct by older releases.  That’s what we’ve got now — Emacs 26.0.90 accepts the forms for CL macros but rejects them in byte code, and Emacs 24 CL macros produced such forms in byte code when given such forms as input.  So currently-acceptable code, or at least code still treated as acceptable under Emacs 25, when compiled by an older release, is no longer accepted. If the source isn’t going to be rejected (e.g., if it’s quietly accepted or only produces a warning), then the byte-code for it probably ought not to be rejected.

Ken

Message #49 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 13 Nov 2017 15:05:09 -0500

On Mon, Nov 13, 2017 at 2:42 PM, Ken Raeburn <raeburn <at> permabit.com> wrote:

> But even if we do make it an error, isn’t there usually a stage where it’s just a warning?

Maybe. There hasn't been this time (for plain defun, I mean).

> (And if we’re going to make that sort of thing an error, we should probably check whether empty &key or &aux variable lists are similarly rejected.  I haven’t looked.)

I believe empty &key would be tested in my patch, though not &aux.

> If the source isn’t going to be rejected (e.g., if it’s quietly accepted or only produces a warning), then the byte-code for it probably ought not to be rejected.

Yes, that's why my patch rejects the source as well.

Message #52 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Mon, 27 Nov 2017 17:24:01 -0500

[Message part 1 (text/plain, inline)]

On Mon, Nov 13, 2017 at 3:05 PM, Noam Postavsky
<npostavs <at> users.sourceforge.net> wrote:
> On Mon, Nov 13, 2017 at 2:42 PM, Ken Raeburn <raeburn <at> permabit.com> wrote:
>
>> But even if we do make it an error, isn’t there usually a stage where it’s just a warning?
>
> Maybe. There hasn't been this time (for plain defun, I mean).

As another case, there wasn't any warning stage for changing setq to
only accept an even number of arguments.

>> (And if we’re going to make that sort of thing an error, we should probably check whether empty &key or &aux variable lists are similarly rejected.  I haven’t looked.)
>
> I believe empty &key would be tested in my patch, though not &aux.

Updated patch which handles &aux as well. I also tested a bootstrap
(doing this I found the previous patch messed up some positive cases).

[v2-0001-Mention-new-strictness-for-optional-rest-in-argli.patch (application/octet-stream, attachment)]

Message #55 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Ken Raeburn <raeburn <at> permabit.com>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Wed, 13 Dec 2017 17:36:01 -0500

close 29165 26.1
quit

On Mon, Nov 27, 2017 at 5:24 PM, Noam Postavsky
<npostavs <at> users.sourceforge.net> wrote:
> On Mon, Nov 13, 2017 at 3:05 PM, Noam Postavsky
> <npostavs <at> users.sourceforge.net> wrote:
>> On Mon, Nov 13, 2017 at 2:42 PM, Ken Raeburn <raeburn <at> permabit.com> wrote:
>>
>>> But even if we do make it an error, isn’t there usually a stage where it’s just a warning?
>>
>> Maybe. There hasn't been this time (for plain defun, I mean).
>
> As another case, there wasn't any warning stage for changing setq to
> only accept an even number of arguments.
>
>>> (And if we’re going to make that sort of thing an error, we should probably check whether empty &key or &aux variable lists are similarly rejected.  I haven’t looked.)
>>
>> I believe empty &key would be tested in my patch, though not &aux.
>
> Updated patch which handles &aux as well. I also tested a bootstrap
> (doing this I found the previous patch messed up some positive cases).

Pushed to emacs-26.

[1: e7b1111]: 2017-12-13 17:31:27 -0500
  Mention new strictness for &optional, &rest in arglists (Bug#29165)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=e7b1111155b3116d0c7b137e0e1d312db0f1ca80

Message #60 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Ken Raeburn <raeburn <at> permabit.com>, Andreas Schwab <schwab <at> linux-m68k.org>,
 Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Wed, 13 Dec 2017 18:39:52 -0500

>> (And if we’re going to make that sort of thing an error, we should
>> probably check whether empty &key or &aux variable lists are similarly
>> rejected.  I haven’t looked.)

I recently installed a patch to fix/improve the behavior of &aux with no
keyword variable (I bumped into it while working on some Elisp package,
tho I can't remember which right now).

I think it's usually worth the small extra effort to support &optional
not followed by any var (as well as &aux not followed by any var) since
it sometimes comes in handy.  But not if it costs extra at run-time.

> Updated patch which handles &aux as well.  I also tested a bootstrap
> (doing this I found the previous patch messed up some positive cases).

To the extent that &aux is only handled by macro-expansion, accepting an
empty &aux never costs anything at run-time, so I think rejecting it is
a disservice to our users.


        Stefan

Message #63 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Ken Raeburn <raeburn <at> permabit.com>, 29165 <at> debbugs.gnu.org,
 Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Thu, 14 Dec 2017 20:16:20 -0500

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>>> (And if we’re going to make that sort of thing an error, we should
>>> probably check whether empty &key or &aux variable lists are similarly
>>> rejected.  I haven’t looked.)
>
> I recently installed a patch to fix/improve the behavior of &aux with no
> keyword variable (I bumped into it while working on some Elisp package,
> tho I can't remember which right now).
>
> I think it's usually worth the small extra effort to support &optional
> not followed by any var (as well as &aux not followed by any var) since
> it sometimes comes in handy.  But not if it costs extra at run-time.

I don't think there is any performance penalty (when running
byte-compiled code anyway).  Although I see that &optional at the end of
the arglist has been a compile error for a long time (I tested back to
24.3).  E.g., (defun foo (&optional)) fails to compile.  (defun foo
(&optional &rest)) happened to work, though I wouldn't exactly call that
"support".

>> Updated patch which handles &aux as well.  I also tested a bootstrap
>> (doing this I found the previous patch messed up some positive cases).
>
> To the extent that &aux is only handled by macro-expansion, accepting an
> empty &aux never costs anything at run-time, so I think rejecting it is
> a disservice to our users.

I don't see the use case for empty &aux, but I don't mind reverting that
change.

Message #66 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Ken Raeburn <raeburn <at> permabit.com>, 29165 <at> debbugs.gnu.org,
 Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Thu, 14 Dec 2017 22:04:15 -0500

> I don't see the use case for empty &aux, but I don't mind reverting that
> change.

It's typically when you define a function from within a macro so you may
have something like

   `(...
     (cl-defun ,foo (,@args &aux ,@auxargs)
       ,@body)
     ...)

where `auxargs` can have any length including 0.  It's not
terribly hard to strip the &aux when `auxargs` is empty, but it's
convenient to be able to use the above form without having to worry
about a possibly empty `auxargs`.


        Stefan

Message #69 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> redhat.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#29165: 26.0.90; can't use some code byte-compiled under emacs
 24
Date: Fri, 15 Dec 2017 00:17:18 -0500

On Dec 14, 2017, at 22:04, Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:

>> I don't see the use case for empty &aux, but I don't mind reverting that
>> change.
> 
> It's typically when you define a function from within a macro so you may
> have something like
[…]

Why is this different from "… &optional ,@optargs” in a similar form, when optargs could have no elements?  Rejecting that was (part of) the point of Noam’s change (and the previous change for bug 24913).  I think I’ve already made my objection to that part plain, but it seems like rejecting an empty &optional and accepting an empty &aux would be inconsistent.

Ken

Message #72 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Ken Raeburn <raeburn <at> redhat.com>
Cc: 29165 <at> debbugs.gnu.org, Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>,
 Noam Postavsky <npostavs <at> users.sourceforge.net>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Fri, 15 Dec 2017 08:54:52 -0500

> Why is this different from "… &optional ,@optargs” in a similar form, when
> optargs could have no elements?  Rejecting that was (part of) the point of
> Noam’s change (and the previous change for bug 24913).

It's not, and I also think rejecting it is generally unadvisable unless
it has a performance impact.

> I think I’ve already made my objection to that part plain, but it
> seems like rejecting an empty &optional and accepting an empty &aux
> would be inconsistent.

Inconsistency is part of life, tho.


        Stefan

Message #75 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Ken Raeburn <raeburn <at> permabit.com>, 29165 <at> debbugs.gnu.org,
 Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Fri, 15 Dec 2017 11:48:47 -0500

Noam Postavsky wrote:

> Pushed to emacs-26.
>
> [1: e7b1111]: 2017-12-13 17:31:27 -0500
>   Mention new strictness for &optional, &rest in arglists (Bug#29165)
>   https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=e7b1111155b3116d0c7b137e0e1d312db0f1ca80

When attempting to merge emacs-26 to master, this conflicts with

commit cea0bca54f
Date:   Mon Nov 27 12:45:16 2017 -0500

    * lisp/emacs-lisp/cl-macs.el: Fix &key with no key arg

Please could someone indicate how this is to be resolved.

Message #78 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Ken Raeburn <raeburn <at> permabit.com>, Andreas Schwab <schwab <at> linux-m68k.org>,
 Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Fri, 15 Dec 2017 23:54:53 -0500

reopen 29165
quit

Glenn Morris <rgm <at> gnu.org> writes:

> Noam Postavsky wrote:
>
>> Pushed to emacs-26.
>>
>> [1: e7b1111]: 2017-12-13 17:31:27 -0500
>>   Mention new strictness for &optional, &rest in arglists (Bug#29165)
>>   https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=e7b1111155b3116d0c7b137e0e1d312db0f1ca80
>
> When attempting to merge emacs-26 to master, this conflicts with
>
> commit cea0bca54f
> Date:   Mon Nov 27 12:45:16 2017 -0500
>
>     * lisp/emacs-lisp/cl-macs.el: Fix &key with no key arg
>
> Please could someone indicate how this is to be resolved.

I've reverted my cl-lib code changes, so the conflict should be resolved
now.  As for this bug, it seems the best way forward is to relax the
checks so that an empty list of &optional variables is accepted.

Message #85 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Ken Raeburn <raeburn <at> permabit.com>, 29165 <at> debbugs.gnu.org,
 Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Sat, 20 Jan 2018 17:10:17 -0500

[Message part 1 (text/plain, inline)]

tags 29165 = patch
quit

Noam Postavsky <npostavs <at> users.sourceforge.net> writes:

> I've reverted my cl-lib code changes, so the conflict should be resolved
> now.  As for this bug, it seems the best way forward is to relax the
> checks so that an empty list of &optional variables is accepted.

Here's a patch.  Details of behaviour changes over versions are in the
commit message.

I probably should have got around to this before 26.0.91, but I think
this should still go to emacs-26 because the patch makes the behaviour
closer to what was in v25.

[v1-0001-Allow-rest-or-optional-without-following-variable.patch (text/plain, attachment)]

Message #90 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: Ken Raeburn <raeburn <at> permabit.com>, Glenn Morris <rgm <at> gnu.org>,
 Philipp Stephani <p.stephani2 <at> gmail.com>,
 Andreas Schwab <schwab <at> linux-m68k.org>, 29165 <at> debbugs.gnu.org
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Sat, 20 Jan 2018 22:02:06 -0500

> Here's a patch.  Details of behaviour changes over versions are in the
> commit message.

LGTM,


        Stefan

Message #93 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: rgm <at> gnu.org, 29165 <at> debbugs.gnu.org, raeburn <at> permabit.com,
 p.stephani2 <at> gmail.com, schwab <at> linux-m68k.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Sun, 21 Jan 2018 18:04:23 +0200

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Sat, 20 Jan 2018 17:10:17 -0500
> Cc: Ken Raeburn <raeburn <at> permabit.com>, Andreas Schwab <schwab <at> linux-m68k.org>,
> 	Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
> 
> I probably should have got around to this before 26.0.91, but I think
> this should still go to emacs-26 because the patch makes the behaviour
> closer to what was in v25.

Is it important enough to justify one more pretest, after 26.0.91, and
delay the release by a few more weeks?  Because that's what would be
needed if we install this on the release branch.

Besides, this change was introduced more than a year ago, and we did
that in clear understanding that it changes behavior.  So "closer to
Emacs 25" doesn't sound like a convincing argument in this case,
because we gave up that similarity up front.

Your call.

Message #96 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rgm <at> gnu.org, 29165 <at> debbugs.gnu.org, raeburn <at> permabit.com,
 p.stephani2 <at> gmail.com, schwab <at> linux-m68k.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Sun, 21 Jan 2018 11:30:02 -0500

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> Date: Sat, 20 Jan 2018 17:10:17 -0500
>> Cc: Ken Raeburn <raeburn <at> permabit.com>, Andreas Schwab <schwab <at> linux-m68k.org>,
>> 	Philipp Stephani <p.stephani2 <at> gmail.com>, 29165 <at> debbugs.gnu.org,
>> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
>> 
>> I probably should have got around to this before 26.0.91, but I think
>> this should still go to emacs-26 because the patch makes the behaviour
>> closer to what was in v25.
>
> Is it important enough to justify one more pretest, after 26.0.91, and
> delay the release by a few more weeks?  Because that's what would be
> needed if we install this on the release branch.

Hmm, no, I think these edge cases are not important enough to delay the
release.

> Besides, this change was introduced more than a year ago, and we did
> that in clear understanding that it changes behavior.  So "closer to
> Emacs 25" doesn't sound like a convincing argument in this case,
> because we gave up that similarity up front.

Well, the unfortunate thing about waiting is that we get more strict in
v26 and then turn around in v27 and (partially) reverse the decision.
But still, as I said above, not worth delaying the release over it.

Message #99 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: rgm <at> gnu.org, 29165 <at> debbugs.gnu.org, raeburn <at> permabit.com,
 p.stephani2 <at> gmail.com, schwab <at> linux-m68k.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Sun, 21 Jan 2018 20:01:16 +0200

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: rgm <at> gnu.org,  29165 <at> debbugs.gnu.org,  raeburn <at> permabit.com,  p.stephani2 <at> gmail.com,  schwab <at> linux-m68k.org,  monnier <at> iro.umontreal.ca
> Date: Sun, 21 Jan 2018 11:30:02 -0500
> 
> > Is it important enough to justify one more pretest, after 26.0.91, and
> > delay the release by a few more weeks?  Because that's what would be
> > needed if we install this on the release branch.
> 
> Hmm, no, I think these edge cases are not important enough to delay the
> release.
> 
> > Besides, this change was introduced more than a year ago, and we did
> > that in clear understanding that it changes behavior.  So "closer to
> > Emacs 25" doesn't sound like a convincing argument in this case,
> > because we gave up that similarity up front.
> 
> Well, the unfortunate thing about waiting is that we get more strict in
> v26 and then turn around in v27 and (partially) reverse the decision.
> But still, as I said above, not worth delaying the release over it.

OK, then please push to master, and thanks.

Message #102 received at 29165 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rgm <at> gnu.org, 29165 <at> debbugs.gnu.org,
 Noam Postavsky <npostavs <at> users.sourceforge.net>, raeburn <at> permabit.com,
 p.stephani2 <at> gmail.com, schwab <at> linux-m68k.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#29165: 26.0.90;
 can't use some code byte-compiled under emacs 24
Date: Sun, 25 Mar 2018 11:32:50 -0400

tags 29165 fixed
close 29165 27.1
quit

Eli Zaretskii <eliz <at> gnu.org> writes:
> OK, then please push to master, and thanks.

Right, done (finally).

[1: 1d47d777ef]: 2018-03-25 07:56:35 -0400
  Allow `&rest' or `&optional' without following variable (Bug#29165)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1d47d777ef24c0be9153b0a1c8ba21918fa1025a

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.