GNU bug report logs -
#66864
emacs-build-system builds .eln-files with mismatching path-hashes
Previous Next
To reply to this bug, email your comments to 66864 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Wed, 01 Nov 2023 00:39:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Mekeor Melire <mekeor <at> posteo.de>
:
New bug report received and forwarded. Copy sent to
bug-guix <at> gnu.org
.
(Wed, 01 Nov 2023 00:39:01 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
BUG EXPLANATION
Emacs's natively-compiled .eln-files have a basename following the pattern "{feature-name}-{path-hash}-{content-hash}.eln". [0]
Guix' emacs-build-system is used to build Emacs-related packages. By
default, it uses the "emacs-minimal" package during build, which
does not support native-compilation. But if you replace the
"emacs-minimal" input with "emacs-no-x", e.g. by using
--with-input=emacs-minimal=emacs-no-x, then emacs-build-system
will make use of emacs-no-x' support of native-compilation [1]:
The build will contain .eln-files.
Hereby I'd like to report the bug that consists of mismatched path-hashes in the .eln-files that builds of Emacs-related packages contain when build with emacs-no-x (or any other Emacs that supports native compilation).
BUG REPRODUCTION
To reproduce this bug follow the following steps. Please note that guix-shell seems to leak .eln-files. (This should be reported as
another bug.) That why the reproduction steps avoid guix-shell.
Instead, we'll work with the current user profile.
Delete Emacs' eln-cache (so that we can later see if new
.eln-files have been generated):
rm -rf ~/.emacs.d/eln-cache
Remove all Emacs- and Emacs-related packages from Guix profile:
guix package -I | cut -f 4 | grep emacs | xargs guix remove
Install Emacs and emacs-unfill, as exemplary package, while
replacing input "emacs-minimal" with "emacs", so that .eln-files
are generated during the build:
guix install emacs emacs-unfill
--with-input=emacs-minimal=emacs
Launch the freshly installed Emacs and load the "unfill" package.
If the .eln-files that the emacs-unfill package provides match
Emacs' expectations (path- and content-hash), it'll use it;
otherwise, Emacs will compile a new .eln-file and save it into
~/.emacs.d/eln-cache/*/unfill-{path-hash}-{content-hash}.eln.
emacs -q --eval "(require 'unfill)"
Close Emacs after some seconds. Now determine the path-hash from
Guix' build:
basename
~/.guix-profile/lib/emacs/native-site-lisp/*/unfill-*.eln \
| cut -d - -f 2
Determine the path-hash from Emacs' native-compilation, which
apparently has happened:
basename ~/.emacs.d/eln-cache/*/unfill*.eln \
| cut -d - -f 2
The path-hashes from the last two steps are not equal.
BUG SOLUTION HINTS
In the #guix:libera.chat IRC channel, jpoiret pointed out: "the .eln file hash problem is due to grafts, grafts change the
final output name, but they can't also update the file hashes...
we'd need to modify emacs' behavior for this to work".
CITATIONS
[0]: Emacs' source code documents the meaning of the two hashes here: https://git.sv.gnu.org/cgit/emacs.git/tree/src/comp.c?h=194a8f5c1406dd7e762376bdfde78d1b7d01b6b1#n4405
[1]: Here you can see that emacs-no-x supports native-compilation unlike emacs-minimal: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/emacs.scm?h=92913703448c8e1a488ab066f60741262cdbf923#n294
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Wed, 01 Nov 2023 11:54:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 66864 <at> debbugs.gnu.org (full text, mbox):
Am Dienstag, dem 31.10.2023 um 23:49 +0000 schrieb Mekeor Melire:
> BUG EXPLANATION
>
> Emacs's natively-compiled .eln-files have a basename following the
> pattern "{feature-name}-{path-hash}-{content-hash}.eln". [0]
>
> Guix' emacs-build-system is used to build Emacs-related packages. By
> default, it uses the "emacs-minimal" package during build, which
> does not support native-compilation. But if you replace the
> "emacs-minimal" input with "emacs-no-x", e.g. by using
> --with-input=emacs-minimal=emacs-no-x, then emacs-build-system
> will make use of emacs-no-x' support of native-compilation [1]:
> The build will contain .eln-files.
>
> Hereby I'd like to report the bug that consists of mismatched path-
> hashes in the .eln-files that builds of Emacs-related packages
> contain when build with emacs-no-x (or any other Emacs that supports
> native compilation).
>
> BUG REPRODUCTION
>
> To reproduce this bug follow the following steps. Please note that
> guix-shell seems to leak .eln-files. (This should be reported as
> another bug.)
What do you mean by "leaks .eln-files"?
> That why the reproduction steps avoid guix-shell. Instead, we'll
> work with the current user profile.
>
> Delete Emacs' eln-cache (so that we can later see if new
> .eln-files have been generated):
>
> rm -rf ~/.emacs.d/eln-cache
>
> Remove all Emacs- and Emacs-related packages from Guix profile:
>
> guix package -I | cut -f 4 | grep emacs | xargs guix remove
>
> Install Emacs and emacs-unfill, as exemplary package, while
> replacing input "emacs-minimal" with "emacs", so that .eln-files
> are generated during the build:
>
> guix install emacs emacs-unfill
> --with-input=emacs-minimal=emacs
Just deleting the eln-cache should be enough for a MWE. When doing an
MWE, make sure that its actually minimal :)
> Launch the freshly installed Emacs and load the "unfill" package.
> If the .eln-files that the emacs-unfill package provides match
> Emacs' expectations (path- and content-hash), it'll use it;
> otherwise, Emacs will compile a new .eln-file and save it into
> ~/.emacs.d/eln-cache/*/unfill-{path-hash}-{content-hash}.eln.
>
> emacs -q --eval "(require 'unfill)"
>
> Close Emacs after some seconds. Now determine the path-hash from
> Guix' build:
>
> basename
> ~/.guix-profile/lib/emacs/native-site-lisp/*/unfill-*.eln \
> | cut -d - -f 2
>
> Determine the path-hash from Emacs' native-compilation, which
> apparently has happened:
>
> basename ~/.emacs.d/eln-cache/*/unfill*.eln \
> | cut -d - -f 2
This is already the bug. There should not be a file written to the
eln-cache (save for the trampolines that we still write there, which is
also a known bug among those who care).
> The path-hashes from the last two steps are not equal.
>
> BUG SOLUTION HINTS
>
> In the #guix:libera.chat IRC channel, jpoiret pointed out: "the .eln
> file hash problem is due to grafts, grafts change the
> final output name, but they can't also update the file hashes...
> we'd need to modify emacs' behavior for this to work".
As jpoiret points out, this has to do with the file naming choices of
Emacs, not with emacs-build-system per se. We would need to get rid of
a lot of hashes if we wanted interoperable native-compiled Emacs
libraries. I wonder what upstream has to say about this.
Cheers
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Wed, 01 Nov 2023 13:16:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 66864 <at> debbugs.gnu.org (full text, mbox):
2023-11-01 12:52 liliana.prikler <at> gmail.com:
> Am Dienstag, dem 31.10.2023 um 23:49 +0000 schrieb Mekeor
> Melire:
> > To reproduce this bug follow the following steps. Please note
> > that
> > guix-shell seems to leak .eln-files. (This should be reported
> > as
> > another bug.)
> What do you mean by "leaks .eln-files"?
To be honest, I can't reproduce the leakage right now. I'll create another bug report if I can.
> > That why the reproduction steps avoid guix-shell. Instead,
> > we'll
> > work with the current user profile.
> >
> > Delete Emacs' eln-cache (so that we can later see if new
> > .eln-files have been generated):
> >
> > rm -rf ~/.emacs.d/eln-cache
> >
> > Remove all Emacs- and Emacs-related packages from Guix
> > profile:
> >
> > guix package -I | cut -f 4 | grep emacs | xargs guix
> > remove
> >
> > Install Emacs and emacs-unfill, as exemplary package, while
> > replacing input "emacs-minimal" with "emacs", so that
> > .eln-files
> > are generated during the build:
> >
> > guix install emacs emacs-unfill
> > --with-input=emacs-minimal=emacs
> Just deleting the eln-cache should be enough for a MWE. When
> doing an
> MWE, make sure that its actually minimal :)
I wanted to make sure that an Emacs-related package is installed, and specifically with the --with-input=emacs-minimal=emacs transformation because otherwise .eln-files won't be built. The MRE is minimal in that sense that it ensures what's needed; only one Emacs-related package is installed; and commands are kept simple.
> > Launch the freshly installed Emacs and load the "unfill"
> > package.
> > If the .eln-files that the emacs-unfill package provides match
> > Emacs' expectations (path- and content-hash), it'll use it;
> > otherwise, Emacs will compile a new .eln-file and save it into
> > ~/.emacs.d/eln-cache/*/unfill-{path-hash}-{content-hash}.eln.
> >
> > emacs -q --eval "(require 'unfill)"
> >
> > Close Emacs after some seconds. Now determine the path-hash
> > from
> > Guix' build:
> >
> > basename
> > ~/.guix-profile/lib/emacs/native-site-lisp/*/unfill-*.eln
> > \
> > | cut -d - -f 2
> >
> > Determine the path-hash from Emacs' native-compilation, which
> > apparently has happened:
> >
> > basename ~/.emacs.d/eln-cache/*/unfill*.eln \
> > | cut -d - -f 2
> This is already the bug. There should not be a file written to
> the
> eln-cache (save for the trampolines that we still write there,
> which is
> also a known bug among those who care).
Yes, this is already the bug. The reason for the eln-cache to be created is that the two path-hashes do not equal.
> > The path-hashes from the last two steps are not equal.
> >
> > BUG SOLUTION HINTS
> >
> > In the #guix:libera.chat IRC channel, jpoiret pointed out:
> > "the .eln
> > file hash problem is due to grafts, grafts change the
> > final output name, but they can't also update the file
> > hashes...
> > we'd need to modify emacs' behavior for this to work".
> As jpoiret points out, this has to do with the file naming
> choices of
> Emacs, not with emacs-build-system per se. We would need to get
> rid of
> a lot of hashes if we wanted interoperable native-compiled Emacs
> libraries. I wonder what upstream has to say about this.
The problem is that the .el-file-path that is passed to the Emacs function comp-el-to-eln-filename during build [1] does not equal to the
.el-file-path when Emacs is invoked. Personally, I do not
understand how grafting causes this. But I can confirm that when
--no-grafts is passed to "guix install emacs emacs-unfill
--with-input=emacs-minimal=emacs", then no eln-cache is created.
[1]: See these lines of code:
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/emacs-build-system.scm?h=92913703448c8e1a488ab066f60741262cdbf923#n133
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/emacs-utils.scm?h=92913703448c8e1a488ab066f60741262cdbf923#n149
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Wed, 01 Nov 2023 14:18:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 66864 <at> debbugs.gnu.org (full text, mbox):
Am Mittwoch, dem 01.11.2023 um 13:03 +0000 schrieb Mekeor Melire:
> The problem is that the .el-file-path that is passed to the Emacs
> function comp-el-to-eln-filename during build [1] does not equal to
> the .el-file-path when Emacs is invoked. Personally, I do not
> understand how grafting causes this. But I can confirm that when
> --no-grafts is passed to "guix install emacs emacs-unfill
> --with-input=emacs-minimal=emacs", then no eln-cache is created.
I think Emacs might be calculating its own hash at runtime rather than
baking in the value at build time. I would need to investigate this,
however.
Cheers
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Thu, 02 Nov 2023 08:17:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 66864 <at> debbugs.gnu.org (full text, mbox):
2023-11-01 15:16 liliana.prikler <at> gmail.com:
> I think Emacs might be calculating its own hash at runtime
> rather than
> baking in the value at build time.
Exactly. That's what I was trying to express.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Thu, 09 Nov 2023 10:56:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 66864 <at> debbugs.gnu.org (full text, mbox):
Am Donnerstag, dem 02.11.2023 um 08:13 +0000 schrieb Mekeor Melire:
>
> 2023-11-01 15:16 liliana.prikler <at> gmail.com:
>
> > I think Emacs might be calculating its own hash at runtime
> > rather than baking in the value at build time.
>
> Exactly. That's what I was trying to express.
I'm not sure whether this is reproducible. On my system
$ guix build emacs-dash --with-input=emacs-minimal=emacs
/gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1
$ ls /gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1/lib/emacs/native-site-lisp
29.1-e9e5c1ce
$ emacs --batch --eval='(message "%s" comp-abi-hash)'
e9e5c1ce
Looks like everything's alright?
Cheers
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Thu, 09 Nov 2023 11:23:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 66864 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi,
Liliana Marie Prikler <liliana.prikler <at> gmail.com> writes:
> Am Donnerstag, dem 02.11.2023 um 08:13 +0000 schrieb Mekeor Melire:
>>
>> 2023-11-01 15:16 liliana.prikler <at> gmail.com:
>>
>> > I think Emacs might be calculating its own hash at runtime
>> > rather than baking in the value at build time.
>>
>> Exactly. That's what I was trying to express.
> I'm not sure whether this is reproducible. On my system
> $ guix build emacs-dash --with-input=emacs-minimal=emacs
> /gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1
> $ ls /gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1/lib/emacs/native-site-lisp
> 29.1-e9e5c1ce
> $ emacs --batch --eval='(message "%s" comp-abi-hash)'
> e9e5c1ce
> Looks like everything's alright?
It's the .eln file itself that has the hash of the .el's path in its
name. That's computed by `comp-el-to-eln-filename`.
Best,
--
Josselin Poiret
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#66864
; Package
guix
.
(Thu, 09 Nov 2023 12:05:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 66864 <at> debbugs.gnu.org (full text, mbox):
Am Donnerstag, dem 09.11.2023 um 12:21 +0100 schrieb Josselin Poiret:
> Hi,
>
> Liliana Marie Prikler <liliana.prikler <at> gmail.com> writes:
>
> > Am Donnerstag, dem 02.11.2023 um 08:13 +0000 schrieb Mekeor Melire:
> > >
> > > 2023-11-01 15:16 liliana.prikler <at> gmail.com:
> > >
> > > > I think Emacs might be calculating its own hash at runtime
> > > > rather than baking in the value at build time.
> > >
> > > Exactly. That's what I was trying to express.
> > I'm not sure whether this is reproducible. On my system
> > $ guix build emacs-dash --with-input=emacs-minimal=emacs
> > /gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1
> > $ ls /gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-
> > 2.19.1/lib/emacs/native-site-lisp
> > 29.1-e9e5c1ce
> > $ emacs --batch --eval='(message "%s" comp-abi-hash)'
> > e9e5c1ce
> > Looks like everything's alright?
>
> It's the .eln file itself that has the hash of the .el's path in its
> name. That's computed by `comp-el-to-eln-filename`.
Does this still occur on the emacs-team branch, where we compile
everything from the build directory?
Cheers
This bug report was last modified 1 year and 115 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.