GNU bug report logs - #62020
Lisp reader: dotted pair notation not working when initial elements are omitted

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 62020 in the body.
You can then email your comments to 62020 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Federico Tedin <federicotedin <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Lisp reader: dotted pair notation not working when initial elements
 are omitted
Date: Tue, 7 Mar 2023 02:14:02 +0100

According to the Elisp docs
(https://www.gnu.org/software/emacs/manual/html_node/elisp/Dotted-Pair-Notation.html),
one should be able to evaluate e.g.:
    (. 1)
to:
    1

This works correctly in Emacs 28.1 However, in the emacs-29 branch
(bd07cec) this results in:
    *** Read error ***  Invalid read syntax: "."

Since the description in the docs are the same in the emacs-29 I
assume this is a bug.
I believe this may have been caused by the changes implementing the
nonrecursive Lisp reader.
I believe the fix would be roughly:
- in read0 (lread.c), when handling c == '.', ensure we handle not
only the case where the top of the read stack is RE_list but *also*
RE_list_start.
- if the top of the stack was effectively RE_list_start, then the top
of the stack needs to be manipulated somehow so that at the end of
read0 the correct value is returned, when emptying the stack.

Message #8 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Federico Tedin <federicotedin <at> gmail.com>,
 Mattias Engdegård <mattiase <at> acm.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 62020 <at> debbugs.gnu.org
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Tue, 07 Mar 2023 14:53:59 +0200

> From: Federico Tedin <federicotedin <at> gmail.com>
> Date: Tue, 7 Mar 2023 02:14:02 +0100
> 
> According to the Elisp docs
> (https://www.gnu.org/software/emacs/manual/html_node/elisp/Dotted-Pair-Notation.html),
> one should be able to evaluate e.g.:
>     (. 1)
> to:
>     1
> 
> This works correctly in Emacs 28.1 However, in the emacs-29 branch
> (bd07cec) this results in:
>     *** Read error ***  Invalid read syntax: "."
> 
> Since the description in the docs are the same in the emacs-29 I
> assume this is a bug.
> I believe this may have been caused by the changes implementing the
> nonrecursive Lisp reader.
> I believe the fix would be roughly:
> - in read0 (lread.c), when handling c == '.', ensure we handle not
> only the case where the top of the read stack is RE_list but *also*
> RE_list_start.
> - if the top of the stack was effectively RE_list_start, then the top
> of the stack needs to be manipulated somehow so that at the end of
> read0 the correct value is returned, when emptying the stack.

Stefan and Mattias, can you please see how to fix this regression?

Message #11 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 62020 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Federico Tedin <federicotedin <at> gmail.com>
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Tue, 7 Mar 2023 14:31:19 +0100

Eli, thank you for bringing this to our attention.

>> one should be able to evaluate e.g.:
>>    (. 1)
>> to:
>>    1

Federico, it would be very easy to change the reader into behaving that way and I'll do that if required, but before I or anyone else change code or docs, and above all much more important and interesting would be to hear exactly why it matters to you and how you were affected by this corner of the reader semantics.

Could be it that you saw the manual passage, decided to try it out -- which is good, we want more people to do that -- and observed that Emacs and the manual didn't agree on that point?

As far as I can tell while researching ahead of the previous changed, the documented (old) reader semantics was merely emergent behaviour of an under-constrained implementation, never a purposeful design for user convenience.

No other Lisp (Common Lisp, Scheme etc) implementation known to me provides such a reader 'feature', and no evidence of any use of it was found at the time. This is why your report is of such interest: did it actually break existing code, and if so, how exactly?

Message #14 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Federico Tedin <federicotedin <at> gmail.com>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 62020 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Tue, 7 Mar 2023 18:24:32 +0100

> Federico, it would be very easy to change the reader into behaving that way and I'll do that if required, but before I or anyone else change code or docs, and above all much more important and interesting would be to hear exactly why it matters to you and how you were affected by this corner of the reader semantics.
> Could be it that you saw the manual passage, decided to try it out -- which is good, we want more people to do that -- and observed that Emacs and the manual didn't agree on that point?
> As far as I can tell while researching ahead of the previous changed, the documented (old) reader semantics was merely emergent behaviour of an under-constrained implementation, never a purposeful design for user convenience.
> No other Lisp (Common Lisp, Scheme etc) implementation known to me provides such a reader 'feature', and no evidence of any use of it was found at the time. This is why your report is of such interest: did it actually break existing code, and if so, how exactly?

Hey Mattias. The reason I found this is actually a bit funny. I am
working on re-implementing the Elisp interpreter (and other parts of
Emacs) in Go, as a hobby/learning/fun project. My initial
implementation of the reader was based on the old version of Emacs'
reader (i.e. with read0 and read1 calling each other). Recently I
decided to port over the changes made to the Emacs reader
(nonrecursive reader), and immediately after I finished, I ran my test
suite and realized that some cases were failing. All of them were
related to the trailing dot notation, specifically when the initial
elements were omitted. The project itself can be found here
(https://github.com/federicotdn/pimacs), the reader is in read.go and
the test cases in interpreter_test.go. I implemented a fix in my code
in order to still be able to read these expressions (though it's a bit
ugly at the moment).

I agree that using the (. x) notation is not really needed, I haven't
used it in any of my code myself. And specially if no other Lisp
implements it, it would make sense to consider its removal! Though I
am not sure how these types of changes are handled in Emacs.

Message #17 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Federico Tedin <federicotedin <at> gmail.com>
Cc: 62020 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Wed, 8 Mar 2023 11:37:27 +0100

7 mars 2023 kl. 18.24 skrev Federico Tedin <federicotedin <at> gmail.com>:

> The reason I found this is actually a bit funny. I am
> working on re-implementing the Elisp interpreter (and other parts of
> Emacs) in Go, as a hobby/learning/fun project.

Delighted to hear about that! Best of luck, and do let us know about what else of interest you discover, whether you think they are bugs in Emacs or not.

The manual should include a formal syntax that describes what the Lisp reader accepts. It would benefit many users, not just those seeking to clone Emacs.

> I agree that using the (. x) notation is not really needed, I haven't
> used it in any of my code myself.

That's the general conclusion on this side as well: by disallowing the quirk we do the user a much greater service than by letting it pass, since any occurrence is far more likely to be a mistake than actually intended.

Let's correct the manual in Emacs 29 so that it corresponds to the code.

Message #20 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 62020 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, federicotedin <at> gmail.com
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Wed, 08 Mar 2023 16:01:31 +0200

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Wed, 8 Mar 2023 11:37:27 +0100
> Cc: Eli Zaretskii <eliz <at> gnu.org>, Stefan Monnier <monnier <at> iro.umontreal.ca>,
>         62020 <at> debbugs.gnu.org
> 
> Let's correct the manual in Emacs 29 so that it corresponds to the code.

Any specific correction you had in mind?

Message #23 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 62020 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, federicotedin <at> gmail.com
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Fri, 10 Mar 2023 11:11:35 +0100

8 mars 2023 kl. 15.01 skrev Eli Zaretskii <eliz <at> gnu.org>:

>> Let's correct the manual in Emacs 29 so that it corresponds to the code.
> 
> Any specific correction you had in mind?

Just removing the part where we say that (. X) is valid, and perhaps clarify that the dot must follow (and precede) a value. That is, if documentation fixes are still fine for emacs-29.

Message #26 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 62020 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, federicotedin <at> gmail.com
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Fri, 10 Mar 2023 13:55:02 +0200

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Fri, 10 Mar 2023 11:11:35 +0100
> Cc: federicotedin <at> gmail.com, monnier <at> iro.umontreal.ca, 62020 <at> debbugs.gnu.org
> 
> 8 mars 2023 kl. 15.01 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> >> Let's correct the manual in Emacs 29 so that it corresponds to the code.
> > 
> > Any specific correction you had in mind?
> 
> Just removing the part where we say that (. X) is valid, and perhaps clarify that the dot must follow (and precede) a value. That is, if documentation fixes are still fine for emacs-29.

Documentation changes are always fine for a release branch, but I'd
prefer to discuss a specific patch, if you don't mind posting one.

Thanks.

Message #29 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 62020 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, federicotedin <at> gmail.com
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Sat, 11 Mar 2023 10:32:48 +0100

[Message part 1 (text/plain, inline)]

10 mars 2023 kl. 12.55 skrev Eli Zaretskii <eliz <at> gnu.org>:

> Documentation changes are always fine for a release branch, but I'd
> prefer to discuss a specific patch, if you don't mind posting one.

Not at all. There's not much to see here -- the minimal change would just remove a paragraph:

[reader-dotted-doc.diff (application/octet-stream, attachment)]

Message #32 received at 62020 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 62020 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, federicotedin <at> gmail.com
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Sat, 11 Mar 2023 14:19:09 +0200

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sat, 11 Mar 2023 10:32:48 +0100
> Cc: federicotedin <at> gmail.com, monnier <at> iro.umontreal.ca, 62020 <at> debbugs.gnu.org
> 
> 10 mars 2023 kl. 12.55 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> > Documentation changes are always fine for a release branch, but I'd
> > prefer to discuss a specific patch, if you don't mind posting one.
> 
> Not at all. There's not much to see here -- the minimal change would just remove a paragraph:
> 
> diff --git a/doc/lispref/objects.texi b/doc/lispref/objects.texi
> index 99a3c073971..2fe7e6db560 100644
> --- a/doc/lispref/objects.texi
> +++ b/doc/lispref/objects.texi
> @@ -1007,13 +1007,6 @@ Dotted Pair Notation
>  @end example
>  @end ifnottex
>  
> -  As a somewhat peculiar side effect of @code{(a b . c)} and
> -@code{(a . (b . c))} being equivalent, for consistency this means
> -that if you replace @code{b} here with the empty sequence, then it
> -follows that @code{(a . c)} and @code{(a . ( . c))} are equivalent,
> -too.  This also means that @code{( .  c)} is equivalent to @code{c},
> -but this is seldom used.
> -

OK, thanks.

Message #37 received at 62020-done <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 62020-done <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Federico Tedin <federicotedin <at> gmail.com>
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Sun, 12 Mar 2023 18:10:39 +0100

11 mars 2023 kl. 13.19 skrev Eli Zaretskii <eliz <at> gnu.org>:

>>> Documentation changes are always fine for a release branch, but I'd
>>> prefer to discuss a specific patch, if you don't mind posting one.
>> 
>> Not at all. There's not much to see here -- the minimal change would just remove a paragraph:
> 
> OK, thanks.

Pushed to emacs-29.

Message #40 received at 62020-done <at> debbugs.gnu.org (full text, mbox):

From: Federico Tedin <federicotedin <at> gmail.com>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 62020-done <at> debbugs.gnu.org
Subject: Re: bug#62020: Lisp reader: dotted pair notation not working when
 initial elements are omitted
Date: Sun, 12 Mar 2023 21:35:59 +0100

> Delighted to hear about that! Best of luck, and do let us know about what else of interest you discover, whether you think they are bugs in Emacs or not.

Yes no problem, will do!

> >>> Documentation changes are always fine for a release branch, but I'd
> >>> prefer to discuss a specific patch, if you don't mind posting one.
> >>
> >> Not at all. There's not much to see here -- the minimal change would just remove a paragraph:
> >
> > OK, thanks.
>
> Pushed to emacs-29.

Thanks for the fix !

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.