GNU bug report logs -
#63288
30.0.50; Emacs 30 packages fail to build with native comp on some machines
Previous Next
Reported by: Brian Leung <leungbk <at> posteo.net>
Date: Fri, 5 May 2023 04:00:02 UTC
Severity: normal
Found in version 30.0.50
Done: Pip Cet <pipcet <at> protonmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 63288 in the body.
You can then email your comments to 63288 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Fri, 05 May 2023 04:00:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Brian Leung <leungbk <at> posteo.net>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Fri, 05 May 2023 04:00:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
In https://github.com/nix-community/emacs-overlay/issues/318, we
received a report about failed external-package builds when using
Nix, which has some additional plumbing including
https://github.com/nixos/nixpkgs/blob/abcc3146aeb06b8d2ded75bddac3a63544a6be7e/pkgs/build-support/emacs/melpa2nix.el#L26-L32.
It was observed that reverting
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=ea9831bb3cb4878273f6f848051c9b8c3c76d5f1
would resolve our issue.
On my own machine, I don't always encounter the build failure (happened
to me once a few weeks ago, and then again today).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Fri, 05 May 2023 05:18:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> From: Brian Leung <leungbk <at> posteo.net>
> Date: Fri, 05 May 2023 03:59:22 +0000
>
> In https://github.com/nix-community/emacs-overlay/issues/318, we
> received a report about failed external-package builds when using
> Nix, which has some additional plumbing including
> https://github.com/nixos/nixpkgs/blob/abcc3146aeb06b8d2ded75bddac3a63544a6be7e/pkgs/build-support/emacs/melpa2nix.el#L26-L32.
>
> It was observed that reverting
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=ea9831bb3cb4878273f6f848051c9b8c3c76d5f1
> would resolve our issue.
>
> On my own machine, I don't always encounter the build failure (happened
> to me once a few weeks ago, and then again today).
Adding Mattias.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Fri, 05 May 2023 14:13:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 63288 <at> debbugs.gnu.org (full text, mbox):
>> In https://github.com/nix-community/emacs-overlay/issues/318, we
>> received a report about failed external-package builds when using
>> Nix, which has some additional plumbing including
>> https://github.com/nixos/nixpkgs/blob/abcc3146aeb06b8d2ded75bddac3a63544a6be7e/pkgs/build-support/emacs/melpa2nix.el#L26-L32.
>>
>> It was observed that reverting
>> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=ea9831bb3cb4878273f6f848051c9b8c3c76d5f1
>> would resolve our issue.
>>
>> On my own machine, I don't always encounter the build failure (happened
>> to me once a few weeks ago, and then again today).
That's remarkable -- not only is it difficult to see anything wrong with that change (ea9831bb3c), it actually reverts byte-code generation for `ignore` forms in general so that the bytecode generated for `package-read-from-string` on master is again identical to that on emacs-29.
Andrea, does the native compiler somehow miscompile package-read-from-string?
Since the failure appears to be nondeterministic, perhaps there is a native-comp build race involved?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 10 May 2023 10:02:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Mattias Engdegård <mattiase <at> acm.org> writes:
>>> In https://github.com/nix-community/emacs-overlay/issues/318, we
>>> received a report about failed external-package builds when using
>>> Nix, which has some additional plumbing including
>>> https://github.com/nixos/nixpkgs/blob/abcc3146aeb06b8d2ded75bddac3a63544a6be7e/pkgs/build-support/emacs/melpa2nix.el#L26-L32.
>>>
>>> It was observed that reverting
>>> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=ea9831bb3cb4878273f6f848051c9b8c3c76d5f1
>>> would resolve our issue.
>>>
>>> On my own machine, I don't always encounter the build failure (happened
>>> to me once a few weeks ago, and then again today).
>
> That's remarkable -- not only is it difficult to see anything wrong
> with that change (ea9831bb3c), it actually reverts byte-code
> generation for `ignore` forms in general so that the bytecode
> generated for `package-read-from-string` on master is again identical
> to that on emacs-29.
>
> Andrea, does the native compiler somehow miscompile package-read-from-string?
Hi, (sorry for being late) not that I'm aware! But in this case the
failer should be deterministic no?
> Since the failure appears to be nondeterministic, perhaps there is a native-comp build race involved?
Mmmh, or maybe related to the presence or not of some eln?
Best Regards
Andrea
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Thu, 11 May 2023 10:34:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 63288 <at> debbugs.gnu.org (full text, mbox):
>> Andrea, does the native compiler somehow miscompile package-read-from-string?
>
> Hi, (sorry for being late) not that I'm aware! But in this case the
> failer should be deterministic no?
Well I haven't studied the native compiler enough to know how many dice it rolls...
>> Since the failure appears to be nondeterministic, perhaps there is a native-comp build race involved?
>
> Mmmh, or maybe related to the presence or not of some eln?
Could be. I'm going out on a limb here and declare the byte-code compiler completely innocent in this case.
Would you work with the original reporters so that they don't have to maintain some terrible hack that may be hiding a deeper bug? If anything turns out to be pointing in my direction I'll be happy to help out.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 15 Jan 2025 07:13:03 GMT)
Full text and
rfc822 format available.
Message #20 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hello,
I'm currently reproducing this bug, is there anyway I can help? I can
confirm that calling `package-read-from-string' fails when starting
emacs. After manually evaluating the defun again, it works okay.
Br,
Damien
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 15 Jan 2025 15:05:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> Date: Tue, 14 Jan 2025 20:38:07 +0100
> From: damien <at> merenne.me
>
> I'm currently reproducing this bug, is there anyway I can help? I can confirm that calling
> `package-read-from-string' fails when starting emacs. After manually evaluating the defun again, it works
> okay.
The file etc/DEBUG has some advice about debugging native-compiled
Lisp, under "Debugging problems with native-compiled Lisp". Could you
please follow that and post here the results?
I'm adding Andrea to the discussion.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 15 Jan 2025 20:16:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 63288 <at> debbugs.gnu.org (full text, mbox):
damien <at> merenne.me writes:
> Hello,
>
> I'm currently reproducing this bug, is there anyway I can help? I can confirm that calling `package-read-from-string'
> fails when starting emacs. After manually evaluating the defun again, it works okay.
Could you provide a recipe to reproduce the bug starting from a 'emacs
-Q'?
Andrea
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 25 Jan 2025 09:04:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> Cc: 63288 <at> debbugs.gnu.org
> From: Andrea Corallo <acorallo <at> gnu.org>
> Date: Wed, 15 Jan 2025 15:14:57 -0500
>
> damien <at> merenne.me writes:
>
> > Hello,
> >
> > I'm currently reproducing this bug, is there anyway I can help? I can confirm that calling `package-read-from-string'
> > fails when starting emacs. After manually evaluating the defun again, it works okay.
>
> Could you provide a recipe to reproduce the bug starting from a 'emacs
> -Q'?
Ping! Damien, could you please provide us with such a recipe?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 25 Jan 2025 17:09:03 GMT)
Full text and
rfc822 format available.
Message #32 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
To reproduce, I simply have to start `emacs -Q` and eval
```
(require 'package)
(package-read-from-string "((emacs \"25.1\"))")
```
in the scratch buffer:
```
Debugger entered--Lisp error: (error "Can’t read whole string")
error("Can't read whole string")
package-read-from-string("((emacs \"25.1\"))")
(progn (package-read-from-string "((emacs \"25.1\"))"))
eval((progn (package-read-from-string "((emacs \"25.1\"))")) t)
elisp--eval-last-sexp(nil)
```
Again, after re-evaluating the defun, it's working ok.
I found something while messing around with recompiling the file.
If I rebuild it manually it works ok, but then rebuilding the whole emacs source tree, and it fails again.
So I found this: building with `make -j48` (I have a 24 core CPU) triggers the problem while building
with `make -j 1` does not. I attach the build.sh I used to configure the source tree, the configure and
build logs, the elc file with the problem and a good version.
I don't think this is some memory corruption happening under load, I compile a big c++ code base
daily and I never encountered any problem with the compiled binaries. Also I'm not the only one with
this exact problem so... But who knows...
Hope this helps!
Damien
On 2025-01-25T10:03:32.000+01:00, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > Cc: 63288 <at> debbugs.gnu.org
> > From: Andrea Corallo <acorallo <at> gnu.org>
> > Date: Wed, 15 Jan 2025 15:14:57 -0500
> >
> > damien <at> merenne.me writes:
> >
> >
> > > Hello,
> > >
> > > I'm currently reproducing this bug, is there anyway I can help? I can confirm that calling `package-read-from-string'
> > > fails when starting emacs. After manually evaluating the defun again, it works okay.
> >
> > Could you provide a recipe to reproduce the bug starting from a 'emacs
> > -Q'?
>
> Ping! Damien, could you please provide us with such a recipe?
[logs.tar.gz (application/gzip, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 25 Jan 2025 17:27:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 63288 <at> debbugs.gnu.org (full text, mbox):
<damien <at> merenne.me> writes:
> To reproduce, I simply have to start `emacs -Q` and eval
>
> ```
> (require 'package)
> (package-read-from-string "((emacs \"25.1\"))")
> ```
> in the scratch buffer:
>
> ```
> Debugger entered--Lisp error: (error "Can’t read whole string")
> error("Can't read whole string")
> package-read-from-string("((emacs \"25.1\"))")
> (progn (package-read-from-string "((emacs \"25.1\"))"))
> eval((progn (package-read-from-string "((emacs \"25.1\"))")) t)
> elisp--eval-last-sexp(nil)
> ```
> Again, after re-evaluating the defun, it's working ok.
>
> I found something while messing around with recompiling the file.
> If I rebuild it manually it works ok, but then rebuilding the whole emacs source tree, and it fails again.
> So I found this: building with `make -j48` (I have a 24 core CPU) triggers the problem while building
> with `make -j 1` does not. I attach the build.sh I used to configure the source tree, the configure and
> build logs, the elc file with the problem and a good version.
Ouch. It's known that make -j produces different output sometimes
(defsubst gets inlined sometimes, sometimes it doesn't), but that's the
first time I heard about an actual build breaking in identical settings
depending on it!
No "pure space overflow" message in the logs you sent, so it's probably
not that.
Few people have 24 cores, so it might just be that that particular
constellation results in a weird build order.
-(defalias 'package-read-from-string #[257 "\300!\211\242\243\3011 \300\"\210\302\303!0\207\210\207" [read-from-string (end-of-file) error "Can't read whole string"] 7 "Read a Lisp expression from STR.\nSignal an error if the entire string was not used.\n\n(fn STR)"])
+(defalias 'package-read-from-string #[257 "\300!\211\242\243\3011 \302\303!0\207\210\207" [read-from-string (end-of-file) error "Can't read whole string"] 6 "Read a Lisp expression from STR.\nSignal an error if the entire string was not used.\n\n(fn STR)"])
That seems to be the problematic code. For some weird reason, I'm
seeing problems with either definition when running M-x disassemble.
That's very strange, but it might also be a local bug in my current
session.
Disassembling by reading the byte string is possible, but it'll take a
while.
(I was going to suggest this might be a nativecomp bug, and that maybe
removing comp--type-check-optim from comp.el here:
(defconst comp-passes '(comp--spill-lap
comp--limplify
comp--fwprop
comp--call-optim
comp--ipa-pure
comp--add-cstrs
comp--fwprop
comp--type-check-optim
comp--tco
comp--fwprop
comp--remove-type-hints
comp--sanitizer
comp--compute-function-types
comp--final)
"Passes to be executed in order.")
might help. However, that seems less likely now so I really wouldn't
bother at this point. Disassembling the byte code is the next logical
thing to do, and then it's more likely to be a byte-opt bug that only
happens in weird nativecomp setups because the byte compiler options
differ there.)
> I don't think this is some memory corruption happening under load, I compile a big c++ code base
> daily and I never encountered any problem with the compiled binaries. Also I'm not the only one with
> this exact problem so... But who knows...
Unlikely to be HW error. It's reproducible, right?
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 25 Jan 2025 19:14:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> Date: Sat, 25 Jan 2025 18:08:10 +0100
> From: damien <at> merenne.me
> Cc: Andrea Corallo <acorallo <at> gnu.org>, 63288 <at> debbugs.gnu.org
>
> To reproduce, I simply have to start `emacs -Q` and eval
>
> ```
> (require 'package)
> (package-read-from-string "((emacs \"25.1\"))")
> ```
> in the scratch buffer:
>
> ```
> Debugger entered--Lisp error: (error "Can’t read whole string")
> error("Can't read whole string")
> package-read-from-string("((emacs \"25.1\"))")
> (progn (package-read-from-string "((emacs \"25.1\"))"))
> eval((progn (package-read-from-string "((emacs \"25.1\"))")) t)
> elisp--eval-last-sexp(nil)
> ```
I cannot reproduce this, neither in a build with native compilation
nor without it.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 25 Jan 2025 20:08:02 GMT)
Full text and
rfc822 format available.
Message #41 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
> <damien <at> merenne.me> writes:
>
>> To reproduce, I simply have to start `emacs -Q` and eval
>>
>> ```
>> (require 'package)
>> (package-read-from-string "((emacs \"25.1\"))")
>> ```
>> in the scratch buffer:
>>
>> ```
>> Debugger entered--Lisp error: (error "Can’t read whole string")
>> error("Can't read whole string")
>> package-read-from-string("((emacs \"25.1\"))")
>> (progn (package-read-from-string "((emacs \"25.1\"))"))
>> eval((progn (package-read-from-string "((emacs \"25.1\"))")) t)
>> elisp--eval-last-sexp(nil)
>> ```
>> Again, after re-evaluating the defun, it's working ok.
>>
>> I found something while messing around with recompiling the file.
>> If I rebuild it manually it works ok, but then rebuilding the whole emacs source tree, and it fails again.
>> So I found this: building with `make -j48` (I have a 24 core CPU) triggers the problem while building
>> with `make -j 1` does not. I attach the build.sh I used to configure the source tree, the configure and
>> build logs, the elc file with the problem and a good version.
>
> Ouch. It's known that make -j produces different output sometimes
> (defsubst gets inlined sometimes, sometimes it doesn't), but that's the
> first time I heard about an actual build breaking in identical settings
> depending on it!
>
> No "pure space overflow" message in the logs you sent, so it's probably
> not that.
>
> Few people have 24 cores, so it might just be that that particular
> constellation results in a weird build order.
>
> -(defalias 'package-read-from-string #[257 "\300!\211\242\243\3011\300\"\210\302\303!0\207\210\207" [read-from-string (end-of-file) error "Can't read whole string"] 7 "Read a Lisp expression from STR.\nSignal an error if the entire string was not used.\n\n(fn STR)"])
> +(defalias 'package-read-from-string #[257 "\300!\211\242\243\3011\302\303!0\207\210\207" [read-from-string (end-of-file) error "Can't read whole string"] 6 "Read a Lisp expression from STR.\nSignal an error if the entire string was not used.\n\n(fn STR)"])
(These bytecode objects are incorrect; I was in a comint buffer and ^A,
one of the most common bytecodes, was lost).
The actual disassembly of the correct code is:
byte code for package-read-from-string:
doc: Read a Lisp expression from STR. ...
args: (arg1)
0 constant read-from-string
1 stack-ref 1
2 call 1
3 dup
4 car-safe
5 stack-ref 1
6 cdr-safe
7 constant (end-of-file)
8 pushconditioncase 1
11 constant read-from-string
12 stack-ref 4
13 stack-ref 2
14 call 2
15 discard
16 constant error
17 constant "Can't read whole string"
18 call 1
19 pophandler
20 return
21:1 discard
22 stack-ref 1
23 return
That's good and proper and should do the right thing. The incorrect
code:
0 constant read-from-string
1 stack-ref 1
2 call 1
3 dup
4 car-safe
5 stack-ref 1
6 cdr-safe
7 constant (end-of-file)
8 pushconditioncase 1
11 constant error
12 constant "Can't read whole string"
13 call 1
14 pophandler
15 return
16:1 discard
17 stack-ref 1
18 return
omits the second call to read-string.
Lisp code:
;;;; Inferring package from current buffer
(defun package-read-from-string (str)
"Read a Lisp expression from STR.
Signal an error if the entire string was not used."
(pcase-let ((`(,expr . ,offset) (read-from-string str)))
(condition-case ()
;; The call to `ignore' suppresses a compiler warning.
(progn (ignore (read-from-string str offset))
(error "Can't read whole string"))
(end-of-file expr))))
read-from-string is side-effect-free, but not
side-effect-and-error-free. Both the warning and the compiled code look
like the byte compiler assumed it was the latter.
I tried
(put 'read-from-string 'side-effect-free 'error-free)
and recompiled the Lisp function, and the result was identical to the
bad code.
This could be caused by byte-compile-delete-errors: setting that to t
also produces the incorrect code.
We need to figure out what was compiled in which order. Can you
ls -lR --full-time
in the "emacs" directory so we get precise timestamps?
Thanks!
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Mon, 27 Jan 2025 08:28:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 2025-01-25T21:07:12.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
> We need to figure out what was compiled in which order. Can you
>
> ls -lR --full-time
>
> in the "emacs" directory so we get precise timestamps?
Attached to this mail!
Br,
Damien
[ls.log.gz (application/gzip, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Tue, 28 Jan 2025 16:47:01 GMT)
Full text and
rfc822 format available.
Message #47 received at 63288 <at> debbugs.gnu.org (full text, mbox):
<damien <at> merenne.me> writes:
> On 2025-01-25T21:07:12.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
>> We need to figure out what was compiled in which order. Can you
>>
>> ls -lR --full-time
>>
>> in the "emacs" directory so we get precise timestamps?
>
> Attached to this mail!
Thanks! Can you let me know the precise git revision that this is from?
I tried building the elc files in the order they appeared on your
machine, but no luck so far...
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Tue, 28 Jan 2025 17:12:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 63288 <at> debbugs.gnu.org (full text, mbox):
On 2025-01-28T17:46:38.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
> Can you let me know the precise git revision that this is from?
> I tried building the elc files in the order they appeared on your
> machine, but no luck so far...
Sure!
commit b981889e9ee0a37f1bc8e2c9b90a5d154c1d032e on the emacs-30 branch
Damien
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 29 Jan 2025 18:47:02 GMT)
Full text and
rfc822 format available.
Message #53 received at 63288 <at> debbugs.gnu.org (full text, mbox):
<damien <at> merenne.me> writes:
> On 2025-01-28T17:46:38.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
>> Can you let me know the precise git revision that this is from?
>> I tried building the elc files in the order they appeared on your
>> machine, but no luck so far...
>
> Sure!
>
> commit b981889e9ee0a37f1bc8e2c9b90a5d154c1d032e on the emacs-30 branch
Thanks! I rebuilt with that branch, trying to order things just like you
did (this is hard given that the .elcs are written only when they are
done compiling, so there may well be reorderings beyond the ones there
appear to be).
This is the list of .elc files that differ in size still:
2 ./cedet/semantic/db-ref.elc
2 ./cedet/semantic/grm-wy-boot.elc
2 ./cedet/semantic/idle.elc
2 ./cedet/semantic/tag.elc
2 ./emacs-lisp/bytecomp.elc
2 ./emacs-lisp/multisession.elc
2 ./emacs-lisp/package.elc
2 ./emacs-lisp/warnings.elc
2 ./erc/erc-button.elc
2 ./erc/erc-common.elc
2 ./erc/erc-fill.elc
2 ./erc/erc-nicks.elc
2 ./erc/erc-notify.elc
2 ./erc/erc-pcomplete.elc
2 ./erc/erc-stamp.elc
2 ./erc/erc-track.elc
2 ./erc/erc.elc
2 ./eshell/esh-util.elc
2 ./gnus/mml-sec.elc
2 ./gnus/nnatom.elc
2 ./gnus/nnfeed.elc
2 ./mail/rmail.elc
2 ./org/ob-core.elc
2 ./org/org-compat.elc
2 ./org/org-fold-core.elc
2 ./org/org-list.elc
2 ./org/org-src.elc
2 ./org/org-table.elc
2 ./progmodes/cc-awk.elc
2 ./progmodes/cc-cmds.elc
2 ./progmodes/cc-mode.elc
2 ./progmodes/compile.elc
2 ./progmodes/ebrowse.elc
2 ./progmodes/eglot.elc
2 ./progmodes/flymake-proc.elc
2 ./progmodes/flymake.elc
2 ./progmodes/opascal.elc
2 ./progmodes/python.elc
2 ./progmodes/verilog-mode.elc
2 ./textmodes/bibtex.elc
2 ./textmodes/enriched.elc
2 ./url/url-handlers.elc
2 ./use-package/use-package-core.elc
2 ./vc/log-edit.elc
2 ./vc/pcvs.elc
without going into too much detail, I think bytecomp.elc is not what it
should be. Would it be possible for you to provide the 184350-byte
version you've seen in the broken build, and the (possibly 184350-byte)
version that produced a working Emacs? The differences might be very
interesting. Note that it is the .elc files that are interesting, not
their .el sources, and Emacs ignores the .elc extension when tab
completing by default.
(Those files are long; if you cat them together and pipe through zstd
-22 --ultra --long, the result should be short enough to send).
If you sill have time, warnings.elc may also be interesting.
Thanks!
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 01 Feb 2025 11:12:01 GMT)
Full text and
rfc822 format available.
Message #56 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 2025-01-29T19:46:04.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
> without going into too much detail, I think bytecomp.elc is not what it
> should be. Would it be possible for you to provide the 184350-byte
> version you've seen in the broken build, and the (possibly 184350-byte)
> version that produced a working Emacs? The differences might be very
> interesting. Note that it is the .elc files that are interesting, not
> their .el sources, and Emacs ignores the .elc extension when tab
> completing by default.
>
> (Those files are long; if you cat them together and pipe through zstd
> -22 --ultra --long, the result should be short enough to send).
>
> If you sill have time, warnings.elc may also be interesting.
>
> Thanks!
>
> Pip
Here you are!
[elc-files.tar.zst (application/zstd, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 01 Feb 2025 12:17:01 GMT)
Full text and
rfc822 format available.
Message #59 received at 63288 <at> debbugs.gnu.org (full text, mbox):
<damien <at> merenne.me> writes:
> On 2025-01-29T19:46:04.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
>> without going into too much detail, I think bytecomp.elc is not what it
>> should be. Would it be possible for you to provide the 184350-byte
>> version you've seen in the broken build, and the (possibly 184350-byte)
>> version that produced a working Emacs? The differences might be very
>> interesting. Note that it is the .elc files that are interesting, not
>> their .el sources, and Emacs ignores the .elc extension when tab
>> completing by default.
>>
>> (Those files are long; if you cat them together and pipe through zstd
>> -22 --ultra --long, the result should be short enough to send).
>>
>> If you sill have time, warnings.elc may also be interesting.
>>
>> Thanks!
>>
>> Pip
>
> Here you are!
>
> [2. application/zstd; elc-files.tar.zst]...
Thanks, but...
-rw-r--r-- 1 pip pip 184350 Jan 27 08:18 bad-bytecomp.elc
-rw-r--r-- 1 pip pip 11853 Feb 1 11:08 bad-warnings.elc
-rw-r--r-- 1 pip pip 184350 Feb 1 10:54 good-bytecomp.elc
-rw-r--r-- 1 pip pip 11853 Feb 1 10:57 good-warnings.elc
I hope that's just the "bad" files, because I have "good" files myself.
Thanks agani!
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 01 Feb 2025 13:05:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 63288 <at> debbugs.gnu.org (full text, mbox):
<damien <at> merenne.me> writes:
> On 2025-01-29T19:46:04.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
>> without going into too much detail, I think bytecomp.elc is not what it
>> should be. Would it be possible for you to provide the 184350-byte
>> version you've seen in the broken build, and the (possibly 184350-byte)
>> version that produced a working Emacs? The differences might be very
>> interesting. Note that it is the .elc files that are interesting, not
>> their .el sources, and Emacs ignores the .elc extension when tab
>> completing by default.
>>
>> (Those files are long; if you cat them together and pipe through zstd
>> -22 --ultra --long, the result should be short enough to send).
>>
>> If you sill have time, warnings.elc may also be interesting.
>>
>> Thanks!
>>
>> Pip
>
> Here you are!
>
> [2. application/zstd; elc-files.tar.zst]...
Thanks, but...
-rw-r--r-- 1 pip pip 184350 Jan 27 08:18 bad-bytecomp.elc
-rw-r--r-- 1 pip pip 11853 Feb 1 11:08 bad-warnings.elc
-rw-r--r-- 1 pip pip 184350 Feb 1 10:54 good-bytecomp.elc
-rw-r--r-- 1 pip pip 11853 Feb 1 10:57 good-warnings.elc
I hope that's just the "bad" files, because I have "good" files myself.
Thanks agani!
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sun, 02 Feb 2025 09:35:01 GMT)
Full text and
rfc822 format available.
Message #65 received at 63288 <at> debbugs.gnu.org (full text, mbox):
<damien <at> merenne.me> writes:
> On 2025-01-29T19:46:04.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
>> without going into too much detail, I think bytecomp.elc is not what it
>> should be. Would it be possible for you to provide the 184350-byte
>> version you've seen in the broken build, and the (possibly 184350-byte)
>> version that produced a working Emacs? The differences might be very
>> interesting. Note that it is the .elc files that are interesting, not
>> their .el sources, and Emacs ignores the .elc extension when tab
>> completing by default.
>>
>> (Those files are long; if you cat them together and pipe through zstd
>> -22 --ultra --long, the result should be short enough to send).
>>
>> If you sill have time, warnings.elc may also be interesting.
>>
>> Thanks!
>>
>> Pip
>
> Here you are!
So I was in luck and the files were two copies of the "bad" files.
Unfortunately, while package.elc is clearly incorrect, it's less obvious
in these cases, because both versions seem correct: in bytecomp.elc, an
(inline ...) form is treated as progn in one build and inlined using
byte-optimize-inline-handler in the other. I can reproduce that
difference by forcing byte-opt.el not to be compiled before bytecomp is.
Andrea, I'd like to carefully suggest that this is possibly bug#74771.
I still don't understand how reproducible this bug (bug#63288) is,
particularly without a 24-core CPU is, but my next suggestion would be
to disable nativecomp optimizations to see whether we're miscompiling
bytecomp.el.
But I'd like to make sure I'm not missing a more obvious explanation
here, so please let me know whether it's better to leave this bug to you
for now.
Thanks!
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 15 Feb 2025 11:22:01 GMT)
Full text and
rfc822 format available.
Message #68 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Ping! Can we please make some progress in this matter?
> Date: Sun, 02 Feb 2025 09:34:34 +0000
> From: Pip Cet <pipcet <at> protonmail.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, Andrea Corallo <acorallo <at> gnu.org>, 63288 <at> debbugs.gnu.org
>
> <damien <at> merenne.me> writes:
>
> > On 2025-01-29T19:46:04.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
> >> without going into too much detail, I think bytecomp.elc is not what it
> >> should be. Would it be possible for you to provide the 184350-byte
> >> version you've seen in the broken build, and the (possibly 184350-byte)
> >> version that produced a working Emacs? The differences might be very
> >> interesting. Note that it is the .elc files that are interesting, not
> >> their .el sources, and Emacs ignores the .elc extension when tab
> >> completing by default.
> >>
> >> (Those files are long; if you cat them together and pipe through zstd
> >> -22 --ultra --long, the result should be short enough to send).
> >>
> >> If you sill have time, warnings.elc may also be interesting.
> >>
> >> Thanks!
> >>
> >> Pip
> >
> > Here you are!
>
> So I was in luck and the files were two copies of the "bad" files.
> Unfortunately, while package.elc is clearly incorrect, it's less obvious
> in these cases, because both versions seem correct: in bytecomp.elc, an
> (inline ...) form is treated as progn in one build and inlined using
> byte-optimize-inline-handler in the other. I can reproduce that
> difference by forcing byte-opt.el not to be compiled before bytecomp is.
>
> Andrea, I'd like to carefully suggest that this is possibly bug#74771.
> I still don't understand how reproducible this bug (bug#63288) is,
> particularly without a 24-core CPU is, but my next suggestion would be
> to disable nativecomp optimizations to see whether we're miscompiling
> bytecomp.el.
>
> But I'd like to make sure I'm not missing a more obvious explanation
> here, so please let me know whether it's better to leave this bug to you
> for now.
>
> Thanks!
>
> Pip
>
>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sun, 16 Feb 2025 00:15:02 GMT)
Full text and
rfc822 format available.
Message #71 received at 63288 <at> debbugs.gnu.org (full text, mbox):
"Eli Zaretskii" <eliz <at> gnu.org> writes:
> Ping! Can we please make some progress in this matter?
Okay, I've got it. The problem is present on master as well, so we
should fix it.
This code in eieio-core.el is the culprit:
(progn
;; Arrange for field access not to bother checking if the access is indeed
;; made to an eieio--class object.
(eval-when-compile (cl-declaim (optimize (safety 0))))
(cl-defstruct (eieio--class
(:constructor nil)
(:constructor eieio--class-make (name))
(:include cl--class)
(:copier nil))
children
initarg-tuples ;; initarg tuples list
(class-slots nil :type (vector-of eieio--slot))
class-allocation-values ;; class allocated value vector
default-object-cache ;; what a newly created object would look like.
; This will speed up instantiation time as
; only a `copy-sequence' will be needed, instead of
; looping over all the values and setting them from
; the default.
options ;; storage location of tagged class option
; Stored outright without modifications or stripping
)
;; Set it back to the default value. NOTE: Using the default
;; `safety' value does NOT give the default
;; `byte-compile-delete-errors' value. Therefore limit this (and
;; the above `cl-declaim') to compile time so that we don't affect
;; code which only loads this library.
(eval-when-compile (cl-declaim (optimize (safety 1)))))
If package.el is compiled before eieio.el is (which can happen with 48
threads!), the above code is executed when package.el loads eieio-core
(via url-handlers -> url-parse -> auth-source -> eieio), leaving
cl--optimize-safety at 1 and byte-compile-delete-errors at t.
However, the maximum value of cl--optimize-safety is 3, which is the
only level at which byte-compile-delete-errors is set to nil.
IOW, this single-character patch fixes it:
From 5a5080e7a75867916c877f4763f94dcf9dd6ef81 Mon Sep 17 00:00:00 2001
From: Pip Cet <pipcet <at> protonmail.com>
Subject: [PATCH] Fix compilation errors caused by reduced safety level
(bug#63288)
* lisp/emacs-lisp/eieio-core.el: Restore 'safety' to 3, not 1, so
byte-compile-delete-errors is reset to nil.
---
lisp/emacs-lisp/eieio-core.el | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lisp/emacs-lisp/eieio-core.el b/lisp/emacs-lisp/eieio-core.el
index f95fd65fa5c..865906131fe 100644
--- a/lisp/emacs-lisp/eieio-core.el
+++ b/lisp/emacs-lisp/eieio-core.el
@@ -106,7 +106,7 @@ eieio-default-superclass
;; `byte-compile-delete-errors' value. Therefore limit this (and
;; the above `cl-declaim') to compile time so that we don't affect
;; code which only loads this library.
- (eval-when-compile (cl-declaim (optimize (safety 1)))))
+ (eval-when-compile (cl-declaim (optimize (safety 3)))))
(eval-and-compile
--
2.48.1
Stefan, it seems strange to me that cl--optimize-safety at 1 sometimes
corresponds to byte errors being deleted, and sometimes it doesn't. I
suspect we might want this change instead:
From d79b21f213f52fd8ce70943e7f9128d05b88ce59 Mon Sep 17 00:00:00 2001
From: Pip Cet <pipcet <at> protonmail.com>
Subject: [PATCH] Don't allow the byte compiler to delete errors at default
safety (bug#63288)
* lisp/emacs-lisp/cl-macs.el (cl--do-proclaim): Change
'byte-compile-delete-errors' to take effect only at 'safety' level 0.
---
lisp/emacs-lisp/cl-macs.el | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index caaffcf19be..42830ca9f7a 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -2701,7 +2701,7 @@ cl--do-proclaim
(let ((speed (assq (nth 1 (assq 'speed (cdr spec)))
'((0 nil) (1 t) (2 t) (3 t))))
(safety (assq (nth 1 (assq 'safety (cdr spec)))
- '((0 t) (1 t) (2 t) (3 nil)))))
+ '((0 t) (1 nil) (2 nil) (3 nil)))))
(if speed (setq cl--optimize-speed (car speed)
byte-optimize (nth 1 speed)))
(if safety (setq cl--optimize-safety (car safety)
--
2.48.1
This would synchronize byte-compile-delete-errors to have the correct
default value at the (default?) safety level 1.
To reproduce the problem on master:
1. Build the tree
2. rm -f lisp/emacs-lisp/eieio-core.elc
3. rm -f lisp/emacs-lisp/package.elc
4. make -C lisp emacs-lisp/package.elc
5. inspect the newly-built lisp/emacs-lisp/package.elc to see that it
has the incorrect compilation output for package-read-from-string.
As a tangent, it would be nice to randomize elc compilation order to
find such errors, but it's not fully automatable, because even now,
parallel builds aren't reproducible.
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sun, 16 Feb 2025 04:34:03 GMT)
Full text and
rfc822 format available.
Message #74 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
> index caaffcf19be..42830ca9f7a 100644
> --- a/lisp/emacs-lisp/cl-macs.el
> +++ b/lisp/emacs-lisp/cl-macs.el
> @@ -2701,7 +2701,7 @@ cl--do-proclaim
> (let ((speed (assq (nth 1 (assq 'speed (cdr spec)))
> '((0 nil) (1 t) (2 t) (3 t))))
> (safety (assq (nth 1 (assq 'safety (cdr spec)))
> - '((0 t) (1 t) (2 t) (3 nil)))))
> + '((0 t) (1 nil) (2 nil) (3 nil)))))
> (if speed (setq cl--optimize-speed (car speed)
> byte-optimize (nth 1 speed)))
> (if safety (setq cl--optimize-safety (car safety)
LGTM. I'd also deprecate this machinery.
For the case at hand, I think a better option is to add keyword
arguments to `cl-defstruct` to control whether accessors need to check
the type of their arg.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Mon, 17 Feb 2025 15:17:02 GMT)
Full text and
rfc822 format available.
Message #77 received at 63288 <at> debbugs.gnu.org (full text, mbox):
"Stefan Monnier" <monnier <at> iro.umontreal.ca> writes:
>> diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
>> index caaffcf19be..42830ca9f7a 100644
>> --- a/lisp/emacs-lisp/cl-macs.el
>> +++ b/lisp/emacs-lisp/cl-macs.el
>> @@ -2701,7 +2701,7 @@ cl--do-proclaim
>> (let ((speed (assq (nth 1 (assq 'speed (cdr spec)))
>> '((0 nil) (1 t) (2 t) (3 t))))
>> (safety (assq (nth 1 (assq 'safety (cdr spec)))
>> - '((0 t) (1 t) (2 t) (3 nil)))))
>> + '((0 t) (1 nil) (2 nil) (3 nil)))))
>> (if speed (setq cl--optimize-speed (car speed)
>> byte-optimize (nth 1 speed)))
>> (if safety (setq cl--optimize-safety (car safety)
>
> LGTM. I'd also deprecate this machinery.
I really don't understand this code, I'm afraid. In particular,
(cl--do-proclaim) often affects global variables, but we still defer
them until we load cl-macs.el. That looks to me like another similar
accident waiting to happen (debugging this one took a while, and there
was a good amount of luck involved).
So I'll commit the minimal change above, and hope that the rest of the
problem goes away when you have the time to look at it next :-)
> For the case at hand, I think a better option is to add keyword
> arguments to `cl-defstruct` to control whether accessors need to check
> the type of their arg.
That makes sense to me. Setting byte-compile-delete-errors means you
cannot compile any "normal" Elisp code, not just skipped type checks.
It's -Ofast, not -DNDEBUG; you should never use the former, only the
latter.
In any case, the indentation in eieio-core.el is also confusing, IMHO
:-)
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Mon, 17 Feb 2025 22:21:02 GMT)
Full text and
rfc822 format available.
Message #80 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:
>> diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
>> index caaffcf19be..42830ca9f7a 100644
>> --- a/lisp/emacs-lisp/cl-macs.el
>> +++ b/lisp/emacs-lisp/cl-macs.el
>> @@ -2701,7 +2701,7 @@ cl--do-proclaim
>> (let ((speed (assq (nth 1 (assq 'speed (cdr spec)))
>> '((0 nil) (1 t) (2 t) (3 t))))
>> (safety (assq (nth 1 (assq 'safety (cdr spec)))
>> - '((0 t) (1 t) (2 t) (3 nil)))))
>> + '((0 t) (1 nil) (2 nil) (3 nil)))))
>> (if speed (setq cl--optimize-speed (car speed)
>> byte-optimize (nth 1 speed)))
>> (if safety (setq cl--optimize-safety (car safety)
>
> LGTM. I'd also deprecate this machinery.
+1
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Tue, 18 Feb 2025 20:56:02 GMT)
Full text and
rfc822 format available.
Message #83 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> I really don't understand this code, I'm afraid.
Don't worry: I don't think anybody does (I sure don't).
It's code inherited from *many* years ago, where it approximated the
desired semantics via approximate hacks (like delaying assignments
until some package is loaded).
We could try and go back to the CLHS doc to figure out what it "should"
do, and then implement it better. But seeing how little it's used,
I doubt it's worth the trouble (my impression is that its intended
semantics doesn't align very well with our implementation(s), so it
would take work and/or ugly hacks to make it work "right").
> So I'll commit the minimal change above, and hope that the rest of the
> problem goes away when you have the time to look at it next :-)
Looking at it doesn't make me very happy, so I don't have any plan to
look at it.
If you want to do me a favor you could try and mark it (both `proclaim`
and `declaim`) as deprecated.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Tue, 18 Feb 2025 21:33:02 GMT)
Full text and
rfc822 format available.
Message #86 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:
> If you want to do me a favor you could try and mark it (both `proclaim`
> and `declaim`) as deprecated.
What about cl-declare?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Tue, 18 Feb 2025 21:37:01 GMT)
Full text and
rfc822 format available.
Message #89 received at 63288 <at> debbugs.gnu.org (full text, mbox):
>> If you want to do me a favor you could try and mark it (both `proclaim`
>> and `declaim`) as deprecated.
> What about cl-declare?
🙂
Stefan
Information forwarded
to
monnier <at> iro.umontreal.ca, pipcet <at> protonmail.com, eliz <at> gnu.org, acorallo <at> gnu.org, bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 01:09:02 GMT)
Full text and
rfc822 format available.
Message #92 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:
>> I really don't understand this code, I'm afraid.
>
> Don't worry: I don't think anybody does (I sure don't).
>
> It's code inherited from *many* years ago, where it approximated the
> desired semantics via approximate hacks (like delaying assignments
> until some package is loaded).
>
> We could try and go back to the CLHS doc to figure out what it "should"
> do, and then implement it better. But seeing how little it's used,
> I doubt it's worth the trouble (my impression is that its intended
> semantics doesn't align very well with our implementation(s), so it
> would take work and/or ugly hacks to make it work "right").
>
>> So I'll commit the minimal change above, and hope that the rest of the
>> problem goes away when you have the time to look at it next :-)
>
> Looking at it doesn't make me very happy, so I don't have any plan to
> look at it.
>
> If you want to do me a favor you could try and mark it (both `proclaim`
> and `declaim`) as deprecated.
(This is a spin-off from Bug#63288.)
The code for the cl-lib declarations, i.e. (info "(cl) Declarations"),
is hacky, hard to maintain, and not well-understood by anyone. It
doesn't seem to provide any benefits over using standard Emacs Lisp
facilities such as `declare`, `defvar`, and setting byte-compiler
variables.
The attached patch marks cl-proclaim, cl-declaim, and cl-declare as
obsolete.
[0001-Mark-cl-proclaim-cl-declaim-and-cl-declare-as-obsole.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 04:43:01 GMT)
Full text and
rfc822 format available.
Message #95 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Stefan Kangas <stefankangas <at> gmail.com> writes:
> Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>
>>> I really don't understand this code, I'm afraid.
>>
>> Don't worry: I don't think anybody does (I sure don't).
>>
>> It's code inherited from *many* years ago, where it approximated the
>> desired semantics via approximate hacks (like delaying assignments
>> until some package is loaded).
>>
>> We could try and go back to the CLHS doc to figure out what it "should"
>> do, and then implement it better. But seeing how little it's used,
>> I doubt it's worth the trouble (my impression is that its intended
>> semantics doesn't align very well with our implementation(s), so it
>> would take work and/or ugly hacks to make it work "right").
>>
>>> So I'll commit the minimal change above, and hope that the rest of the
>>> problem goes away when you have the time to look at it next :-)
>>
>> Looking at it doesn't make me very happy, so I don't have any plan to
>> look at it.
>>
>> If you want to do me a favor you could try and mark it (both `proclaim`
>> and `declaim`) as deprecated.
>
> (This is a spin-off from Bug#63288.)
>
> The code for the cl-lib declarations, i.e. (info "(cl) Declarations"),
> is hacky, hard to maintain, and not well-understood by anyone. It
> doesn't seem to provide any benefits over using standard Emacs Lisp
> facilities such as `declare`, `defvar`, and setting byte-compiler
> variables.
No, in some cases you must use (cl-declare (special ...)), and no you
can't use defvar in such cases, so if you intend to remove/deprecate
cl-declare ensure you provide the feature in declare.
Don't know for cl-proclaim and cl-declaim, I don't use them.
Thanks.
>
> The attached patch marks cl-proclaim, cl-declaim, and cl-declare as
> obsolete.
>
>
--
Thierry
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 11:56:01 GMT)
Full text and
rfc822 format available.
Message #98 received at 63288 <at> debbugs.gnu.org (full text, mbox):
"Stefan Kangas" <stefankangas <at> gmail.com> writes:
> Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>
>>> I really don't understand this code, I'm afraid.
>>
>> Don't worry: I don't think anybody does (I sure don't).
>>
>> It's code inherited from *many* years ago, where it approximated the
>> desired semantics via approximate hacks (like delaying assignments
>> until some package is loaded).
>>
>> We could try and go back to the CLHS doc to figure out what it "should"
>> do, and then implement it better. But seeing how little it's used,
>> I doubt it's worth the trouble (my impression is that its intended
>> semantics doesn't align very well with our implementation(s), so it
>> would take work and/or ugly hacks to make it work "right").
>>
>>> So I'll commit the minimal change above, and hope that the rest of the
>>> problem goes away when you have the time to look at it next :-)
>>
>> Looking at it doesn't make me very happy, so I don't have any plan to
>> look at it.
>>
>> If you want to do me a favor you could try and mark it (both `proclaim`
>> and `declaim`) as deprecated.
>
> (This is a spin-off from Bug#63288.)
>
> The code for the cl-lib declarations, i.e. (info "(cl) Declarations"),
> is hacky, hard to maintain, and not well-understood by anyone. It
> doesn't seem to provide any benefits over using standard Emacs Lisp
> facilities such as `declare`, `defvar`, and setting byte-compiler
> variables.
>
> The attached patch marks cl-proclaim, cl-declaim, and cl-declare as
> obsolete.
Hooray!
(I'm not sure about cl-declare, but I'm pretty sure the way cl-proclaim
and cl-declaim are implemented would lead to further bugs if they were
used more.)
> diff --git a/doc/misc/cl.texi b/doc/misc/cl.texi
> index da73aeb6387..e1e67e2c0e7 100644
> --- a/doc/misc/cl.texi
> +++ b/doc/misc/cl.texi
> @@ -60,7 +60,7 @@ Top
> * Predicates:: Type predicates and equality predicates.
> * Control Structure:: Assignment, conditionals, blocks, looping.
> * Macros:: Destructuring, compiler macros.
> -* Declarations:: @code{cl-proclaim}, @code{cl-declare}, etc.
> +* Declarations:: @code{cl-the}.
> * Symbols:: Property lists, creating symbols.
> * Numbers:: Predicates, functions, random numbers.
> * Sequences:: Mapping, functions, searching, sorting.
> @@ -2745,46 +2745,12 @@ Declarations
> mechanism that allows you to give the compiler special hints
> about the types of data that will be stored in particular variables,
> and about the ways those variables and functions will be used. This
> -package defines versions of all the Common Lisp declaration forms:
> -@code{declare}, @code{locally}, @code{proclaim}, @code{declaim},
> -and @code{the}.
> +package defines a versions of only one Common Lisp declaration forms:
> +@code{the}.
"version", and "form".
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 14:25:02 GMT)
Full text and
rfc822 format available.
Message #101 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> No, in some cases you must use (cl-declare (special ...)), and no you
> can't use defvar in such cases, so if you intend to remove/deprecate
Could you show us an example?
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 16:26:02 GMT)
Full text and
rfc822 format available.
Message #104 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>> No, in some cases you must use (cl-declare (special ...)), and no you
>> can't use defvar in such cases, so if you intend to remove/deprecate
>
> Could you show us an example?
I used it in helm--completion-in-region to pass PROMPT and
REQUIRE-MATCH, don't know if it still needed, however
removing/deprecating cl-declare remove a nice feature IMHO like the
ability to do things like this:
(defmacro my-with-progress (&rest body)
`(let ((reporter (make-progress-reporter "Updating...")))
(progn
,@body)))
(defun my-updating ()
(cl-declare (special reporter))
(my-with-progress
(cl-loop for elm in '(a b c d e f g)
do (progn
(sit-for 1)
(progress-reporter-update reporter))
collect elm)))
of course you can use a defvar in this case, but it is more handy to do
like this.
Sorry I don't have other examples in mind right now.
--
Thierry
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 17:54:02 GMT)
Full text and
rfc822 format available.
Message #107 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Pip Cet [2025-02-17 15:16 +0000] wrote:
> "Stefan Monnier" <monnier <at> iro.umontreal.ca> writes:
>
>> For the case at hand, I think a better option is to add keyword
>> arguments to `cl-defstruct` to control whether accessors need to check
>> the type of their arg.
+1.
> That makes sense to me. Setting byte-compile-delete-errors means you
> cannot compile any "normal" Elisp code, not just skipped type checks.
> It's -Ofast, not -DNDEBUG; you should never use the former, only the
> latter.
Does setting byte-compile-delete-errors actually have the same effect?
I couldn't get it to behave like (cl-declaim (optimize (safety 0)))
for cl-defstruct accessors.
Thanks,
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 17:54:03 GMT)
Full text and
rfc822 format available.
Message #110 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Stefan Kangas [2025-02-19 01:08 +0000] wrote:
> The code for the cl-lib declarations, i.e. (info "(cl) Declarations"),
> is hacky, hard to maintain, and not well-understood by anyone. It
> doesn't seem to provide any benefits over using standard Emacs Lisp
> facilities such as `declare`, `defvar`, and setting byte-compiler
> variables.
Do we need the optimize/safety flag for cl-defstruct that Stefan M.
mentioned before we can declare cl-proclaim and cl-declaim obsolete?
Just to be clear, I mean that I'm not aware of an alternative to the
following:
;; -*- lexical-binding: t -*-
(require 'cl-lib)
(cl-declaim (optimize (safety 0)))
(cl-defstruct foo a b)
which removes foo-p checks from the accessors foo-a and foo-b.
Either way, (info "(cl) Structures") may also need updating.
Thanks,
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 20:07:02 GMT)
Full text and
rfc822 format available.
Message #113 received at 63288 <at> debbugs.gnu.org (full text, mbox):
"Basil L. Contovounesios" <basil <at> contovou.net> writes:
> Pip Cet [2025-02-17 15:16 +0000] wrote:
>
>> "Stefan Monnier" <monnier <at> iro.umontreal.ca> writes:
>>
>>> For the case at hand, I think a better option is to add keyword
>>> arguments to `cl-defstruct` to control whether accessors need to check
>>> the type of their arg.
>
> +1.
>
>> That makes sense to me. Setting byte-compile-delete-errors means you
>> cannot compile any "normal" Elisp code, not just skipped type checks.
>> It's -Ofast, not -DNDEBUG; you should never use the former, only the
>> latter.
>
> Does setting byte-compile-delete-errors actually have the same effect?
> I couldn't get it to behave like (cl-declaim (optimize (safety 0)))
> for cl-defstruct accessors.
I'm not sure I understand. byte-compile-delete-errors will cause the
compilation errors described, but I don't think it would change the code
generated by cl-defstruct....
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 21:25:03 GMT)
Full text and
rfc822 format available.
Message #116 received at 63288 <at> debbugs.gnu.org (full text, mbox):
>>> No, in some cases you must use (cl-declare (special ...)), and no you
>>> can't use defvar in such cases, so if you intend to remove/deprecate
>> Could you show us an example?
> I used it in helm--completion-in-region to pass PROMPT and
> REQUIRE-MATCH,
Actually, no, you didn't. 🙁
I mean, yes you did use it, but no it did not "pass PROMPT and
REQUIRE-MATCH".
> (defmacro my-with-progress (&rest body)
> `(let ((reporter (make-progress-reporter "Updating...")))
> (progn
> ,@body)))
>
> (defun my-updating ()
> (cl-declare (special reporter))
> (my-with-progress
> (cl-loop for elm in '(a b c d e f g)
> do (progn
> (sit-for 1)
> (progress-reporter-update reporter))
> collect elm)))
I don't see what is the use of (cl-declare (special reporter)) here.
What do you think it changes to the behavior of this code?
> of course you can use a defvar in this case,
Exactly: (defvar reporter) would have the same effect.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 21:34:01 GMT)
Full text and
rfc822 format available.
Message #119 received at 63288 <at> debbugs.gnu.org (full text, mbox):
> Does setting byte-compile-delete-errors actually have the same effect?
Not at all, no. What it does is tell the compiler that it's OK to erase
code whose return value is unused even if the code could signal
an error at run time, as in:
(let ((x (/ 5 n)))
(+ n 1))
where the division usually can't be optimized away because it should
signal an error when `n` is zero.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Wed, 19 Feb 2025 21:35:02 GMT)
Full text and
rfc822 format available.
Message #122 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Pip Cet [2025-02-19 20:06 +0000] wrote:
> "Basil L. Contovounesios" <basil <at> contovou.net> writes:
>> Pip Cet [2025-02-17 15:16 +0000] wrote:
>>> "Stefan Monnier" <monnier <at> iro.umontreal.ca> writes:
>>>
>>>> For the case at hand, I think a better option is to add keyword
>>>> arguments to `cl-defstruct` to control whether accessors need to check
>>>> the type of their arg.
>>
>>> That makes sense to me. Setting byte-compile-delete-errors means you
>>> cannot compile any "normal" Elisp code, not just skipped type checks.
>>> It's -Ofast, not -DNDEBUG; you should never use the former, only the
>>> latter.
>>
>> Does setting byte-compile-delete-errors actually have the same effect?
>> I couldn't get it to behave like (cl-declaim (optimize (safety 0)))
>> for cl-defstruct accessors.
>
> I'm not sure I understand. byte-compile-delete-errors will cause the
> compilation errors described, but I don't think it would change the code
> generated by cl-defstruct....
Then I misunderstood your message, sorry.
[ You mentioned byte-compile-delete-errors in a reply to cl-defstruct
accessors, and the 'optimize' entry under (info "(cl) Declarations")
ties b-c-d-e to the 'safety' quality, which is the same quality which
affects cl-defstruct accessor generation, so I assumed the same link
in your reply. ]
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Thu, 20 Feb 2025 04:00:02 GMT)
Full text and
rfc822 format available.
Message #125 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>>>> No, in some cases you must use (cl-declare (special ...)), and no you
>>>> can't use defvar in such cases, so if you intend to remove/deprecate
>>> Could you show us an example?
>> I used it in helm--completion-in-region to pass PROMPT and
>> REQUIRE-MATCH,
>
> Actually, no, you didn't. 🙁
> I mean, yes you did use it, but no it did not "pass PROMPT and
> REQUIRE-MATCH".
It used to work (circa 26?), but yes now it is broken, since when I
don't know, I rarely use CRM (I saw just yesterday PROMPT was no more
passing, thanks to this thread). So because this is no more working, I
understand you don't want to bother maintaining this code which looks
complex indeed.
>> (defmacro my-with-progress (&rest body)
>> `(let ((reporter (make-progress-reporter "Updating...")))
>> (progn
>> ,@body)))
>>
>> (defun my-updating ()
>> (cl-declare (special reporter))
>> (my-with-progress
>> (cl-loop for elm in '(a b c d e f g)
>> do (progn
>> (sit-for 1)
>> (progress-reporter-update reporter))
>> collect elm)))
>
> I don't see what is the use of (cl-declare (special reporter)) here.
> What do you think it changes to the behavior of this code?
It declare reporter as a variable and prevent byte compile warnings,
same as (defvar reporter) as you have mentioned below.
>> of course you can use a defvar in this case,
>
> Exactly: (defvar reporter) would have the same effect.
>
>
> Stefan
--
Thierry
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Thu, 20 Feb 2025 04:22:02 GMT)
Full text and
rfc822 format available.
Message #128 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Thierry Volpiatto <thievol <at> posteo.net> writes:
> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>
>>>>> No, in some cases you must use (cl-declare (special ...)), and no you
>>>>> can't use defvar in such cases, so if you intend to remove/deprecate
>>>> Could you show us an example?
>>> I used it in helm--completion-in-region to pass PROMPT and
>>> REQUIRE-MATCH,
>>
>> Actually, no, you didn't. 🙁
>> I mean, yes you did use it, but no it did not "pass PROMPT and
>> REQUIRE-MATCH".
>
> It used to work (circa 26?), but yes now it is broken, since when I
> don't know, I rarely use CRM (I saw just yesterday PROMPT was no more
> passing, thanks to this thread). So because this is no more working, I
> understand you don't want to bother maintaining this code which looks
> complex indeed.
>
>
>>> (defmacro my-with-progress (&rest body)
>>> `(let ((reporter (make-progress-reporter "Updating...")))
>>> (progn
>>> ,@body)))
>>>
>>> (defun my-updating ()
>>> (cl-declare (special reporter))
>>> (my-with-progress
>>> (cl-loop for elm in '(a b c d e f g)
>>> do (progn
>>> (sit-for 1)
>>> (progress-reporter-update reporter))
>>> collect elm)))
>>
>> I don't see what is the use of (cl-declare (special reporter)) here.
>> What do you think it changes to the behavior of this code?
>
> It declare reporter as a variable and prevent byte compile warnings,
> same as (defvar reporter) as you have mentioned below.
In fact such code is reporting warnings when using (defvar reporter)
instead of cl-declare:
helm-all-the-icons.el:32:9: Warning: global/dynamic var ‘reporter’ lacks a prefix
helm-all-the-icons.el:42:75: Warning: Lexical argument shadows the dynamic variable reporter
So for this to work I would have to give a prefix to reporter and a nil
value and initialize it with setq, then disable the reporter and
reinitialize the var to nil when done (optional), less handy that cl-declare.
>>> of course you can use a defvar in this case,
>>
>> Exactly: (defvar reporter) would have the same effect.
>>
>>
>> Stefan
--
Thierry
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Thu, 20 Feb 2025 04:44:01 GMT)
Full text and
rfc822 format available.
Message #131 received at 63288 <at> debbugs.gnu.org (full text, mbox):
>>> (defmacro my-with-progress (&rest body)
>>> `(let ((reporter (make-progress-reporter "Updating...")))
>>> (progn
>>> ,@body)))
>>>
>>> (defun my-updating ()
>>> (cl-declare (special reporter))
>>> (my-with-progress
>>> (cl-loop for elm in '(a b c d e f g)
>>> do (progn
>>> (sit-for 1)
>>> (progress-reporter-update reporter))
>>> collect elm)))
>>
>> I don't see what is the use of (cl-declare (special reporter)) here.
>> What do you think it changes to the behavior of this code?
>
> It declare reporter as a variable and prevent byte compile warnings,
Really? AFAICT the code should work just fine, and without warnings, if
you just remove the `cl-declare` and leave the variable be lexically scoped.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Thu, 20 Feb 2025 14:32:02 GMT)
Full text and
rfc822 format available.
Message #134 received at 63288 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>>>> (defmacro my-with-progress (&rest body)
>>>> `(let ((reporter (make-progress-reporter "Updating...")))
>>>> (progn
>>>> ,@body)))
>>>>
>>>> (defun my-updating ()
>>>> (cl-declare (special reporter))
>>>> (my-with-progress
>>>> (cl-loop for elm in '(a b c d e f g)
>>>> do (progn
>>>> (sit-for 1)
>>>> (progress-reporter-update reporter))
>>>> collect elm)))
>>>
>>> I don't see what is the use of (cl-declare (special reporter)) here.
>>> What do you think it changes to the behavior of this code?
>>
>> It declare reporter as a variable and prevent byte compile warnings,
>
> Really? AFAICT the code should work just fine, and without warnings, if
> you just remove the `cl-declare` and leave the variable be lexically scoped.
Hu yes indeed, I haven't tried this beeing sure it would produce
warnings but it doesn't!
Thanks.
--
Thierry
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 01 Mar 2025 12:10:02 GMT)
Full text and
rfc822 format available.
Message #137 received at 63288 <at> debbugs.gnu.org (full text, mbox):
Ping! Ping! Any further progress here?
> Cc: damien <at> merenne.me, 63288 <at> debbugs.gnu.org
> Date: Sat, 15 Feb 2025 13:20:52 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
>
> Ping! Can we please make some progress in this matter?
>
> > Date: Sun, 02 Feb 2025 09:34:34 +0000
> > From: Pip Cet <pipcet <at> protonmail.com>
> > Cc: Eli Zaretskii <eliz <at> gnu.org>, Andrea Corallo <acorallo <at> gnu.org>, 63288 <at> debbugs.gnu.org
> >
> > <damien <at> merenne.me> writes:
> >
> > > On 2025-01-29T19:46:04.000+01:00, Pip Cet <pipcet <at> protonmail.com> wrote:
> > >> without going into too much detail, I think bytecomp.elc is not what it
> > >> should be. Would it be possible for you to provide the 184350-byte
> > >> version you've seen in the broken build, and the (possibly 184350-byte)
> > >> version that produced a working Emacs? The differences might be very
> > >> interesting. Note that it is the .elc files that are interesting, not
> > >> their .el sources, and Emacs ignores the .elc extension when tab
> > >> completing by default.
> > >>
> > >> (Those files are long; if you cat them together and pipe through zstd
> > >> -22 --ultra --long, the result should be short enough to send).
> > >>
> > >> If you sill have time, warnings.elc may also be interesting.
> > >>
> > >> Thanks!
> > >>
> > >> Pip
> > >
> > > Here you are!
> >
> > So I was in luck and the files were two copies of the "bad" files.
> > Unfortunately, while package.elc is clearly incorrect, it's less obvious
> > in these cases, because both versions seem correct: in bytecomp.elc, an
> > (inline ...) form is treated as progn in one build and inlined using
> > byte-optimize-inline-handler in the other. I can reproduce that
> > difference by forcing byte-opt.el not to be compiled before bytecomp is.
> >
> > Andrea, I'd like to carefully suggest that this is possibly bug#74771.
> > I still don't understand how reproducible this bug (bug#63288) is,
> > particularly without a 24-core CPU is, but my next suggestion would be
> > to disable nativecomp optimizations to see whether we're miscompiling
> > bytecomp.el.
> >
> > But I'd like to make sure I'm not missing a more obvious explanation
> > here, so please let me know whether it's better to leave this bug to you
> > for now.
> >
> > Thanks!
> >
> > Pip
> >
> >
>
>
>
>
Reply sent
to
Pip Cet <pipcet <at> protonmail.com>:
You have taken responsibility.
(Sat, 01 Mar 2025 13:12:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Brian Leung <leungbk <at> posteo.net>:
bug acknowledged by developer.
(Sat, 01 Mar 2025 13:12:03 GMT)
Full text and
rfc822 format available.
Message #142 received at 63288-done <at> debbugs.gnu.org (full text, mbox):
"Eli Zaretskii" <eliz <at> gnu.org> writes:
> Ping! Ping! Any further progress here?
Fixed on master, further discussion at bug#76523 (waiting for Andrea's
nput on whether to cherry-pick the fix to 30).
Closing this one as the root issue has been fixed.
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#63288; Package
emacs.
(Sat, 01 Mar 2025 13:17:02 GMT)
Full text and
rfc822 format available.
Message #145 received at 63288-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Just wanted to thank you all. I've been using emacs since my uni years
(I guess 1998, time flies), contributing here and there but it was
enlightening to see the dedication of you all to fix this issue.
Thanks for keeping my favorite editor more alive then ever.
Damien
Le samedi 1 mars 2025 à 14:11, Pip Cet <pipcet <at> protonmail.com> a
écrit :
> "Eli Zaretskii" <eliz <at> gnu.org> writes:
>
>> Ping! Ping! Any further progress here?
>
> Fixed on master, further discussion at bug#76523 (waiting for Andrea's
> nput on whether to cherry-pick the fix to 30).
>
> Closing this one as the root issue has been fixed.
>
> Pip
[Message part 2 (text/html, inline)]
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 30 Mar 2025 11:24:38 GMT)
Full text and
rfc822 format available.
This bug report was last modified 39 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.