GNU bug report logs - #44775
[PATCH] WIP: Add gccemacs

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 44775 in the body.
You can then email your comments to 44775 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Soo <jsoo1 <at> asu.edu>
To: guix-patches <at> gnu.org
Subject: [PATCH] WIP: Add gccemacs
Date: Sat, 21 Nov 2020 01:15:40 -0800

[Message part 1 (text/plain, inline)]

Hi Guix,

I was curious how fast this gccemacs branch might be, so I got it to
compile.

It feels fast but there are bugs and the closure is huge. Also these
patches do not use any of the parameterization machinery.

I just thought I would share my work.

Thanks,

John

[0001-gnu-Add-emacs-next-no-x.patch (text/x-patch, attachment)]

[0002-gnu-Add-gccemacs.patch (text/x-patch, attachment)]

[0003-gnu-Add-gcceamcs-no-x.patch (text/x-patch, attachment)]

Message #8 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: John Soo <jsoo1 <at> asu.edu>, 44775 <at> debbugs.gnu.org
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Mon, 23 Nov 2020 23:22:55 +0100

Hi John,

On Sat, 21 Nov 2020 at 01:15, John Soo <jsoo1 <at> asu.edu> wrote:

> I was curious how fast this gccemacs branch might be, so I got it to
> compile.

Thanks!  This motivates me to resume what had been discussed here [1]:
be able to somehow run:

--8<---------------cut here---------------start------------->8---
 guix build emacs-magit emacs-foo emacs-bar --with-input=emacs=gccemacs
--8<---------------cut here---------------end--------------->8---

At least, have a package transformation that allow to rebuild all the
Emacs packages with the ’gccemacs’ VM instead of the current Emacs 27 one.


> It feels fast but there are bugs and the closure is huge. Also these
> patches do not use any of the parameterization machinery.

What do you mean by “parameterization machinery”?  The new
’with-parameter’ introduced here [2] or the package transformation that
replaces all the dependencies (explicit and implicit).


1: <https://yhetil.org/guix-bugs/868scwtt34.fsf <at> gmail.com/>
2: <https://yhetil.org/guix-devel/87eeku8trb.fsf <at> gnu.org>


All the best,
simon

Message #11 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: John Soo <jsoo1 <at> asu.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 44775 <at> debbugs.gnu.org
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Tue, 24 Nov 2020 07:06:46 -0800

Hi zimoun,

zimoun <zimon.toutoune <at> gmail.com> writes:

> Thanks!  This motivates me to resume what had been discussed here [1]:
> be able to somehow run:
>
>  guix build emacs-magit emacs-foo emacs-bar --with-input=emacs=gccemacs
>
> At least, have a package transformation that allow to rebuild all the
> Emacs packages with the ’gccemacs’ VM instead of the current Emacs 27 one.
>
>
>> It feels fast but there are bugs and the closure is huge. Also these
>> patches do not use any of the parameterization machinery.
>
> What do you mean by “parameterization machinery”?  The new
> ’with-parameter’ introduced here [2] or the package transformation that
> replaces all the dependencies (explicit and implicit).
>
>
> 1: <https://yhetil.org/guix-bugs/868scwtt34.fsf <at> gmail.com/>
> 2: <https://yhetil.org/guix-devel/87eeku8trb.fsf <at> gnu.org>

I am referring to https://yhetil.org/guix-devel/87eeku8trb.fsf <at> gnu.org

--with-parameter=gccemacs or similar seem to be required since the
native-comp branch requires a different source, configure flags, and
probably native-search-paths at least.

There appears to be a separate compiled artifact directory under
$out/lib/emacs/$version/native-lisp/$version-triple which has the
compiled native libraries (.eln files). That directory seems to not be
in the search path.  That appears to be causing the first error
I see. I am not sure which env variable would be tweaked to pick those
paths up.

Hope that helps!

John

Message #14 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: John Soo <jsoo1 <at> asu.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 44775 <at> debbugs.gnu.org
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Tue, 24 Nov 2020 07:28:23 -0800

[Message part 1 (text/plain, inline)]

Hi Guix and zimoun,

I fixed a bug in the build process here.  Still no big progress on the
other bugs.

- John

[0001-gnu-Add-gccemacs.patch (text/x-patch, attachment)]

Message #17 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: Morgan Smith <Morgan.J.Smith <at> outlook.com>
To: John Soo <jsoo1 <at> asu.edu>
Cc: 44775 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Sat, 28 Nov 2020 21:14:42 -0500

Have you seen this gccemacs package? I haven't looked into it myself but
it might help you.

https://github.com/flatwhatson/guix-channel

Message #20 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: John Soo <jsoo1 <at> asu.edu>
To: Morgan Smith <Morgan.J.Smith <at> outlook.com>
Cc: 44775 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Fri, 11 Dec 2020 12:40:36 -0800

Hi Morgan,

Thank you! I will take a look. If I end up using some of their code,
should I consult them about it and see if they want a copyright line?
How is that supposed to work?

Thank you again,

John

Message #23 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: Morgan Smith <Morgan.J.Smith <at> outlook.com>
To: John Soo <jsoo1 <at> asu.edu>, whatson <at> gmail.com
Cc: 44775 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Thu, 17 Dec 2020 12:26:19 -0500

Hi John,

(And hi Andrew. We're discussing including your code into the main Guix
repository. You can view the full conversation here:
https://issues.guix.gnu.org/44775)

The code I linked to is already licensed under GPLv3+ meaning that you
are free to integrate their code into the Guix code-base (Going to put
here that I'm not a lawyer and that this is not legal advice and might
not even be correct). You should indeed put a little copyright line for
Andrew in the code. The exact copyright line you should use seems to be
this (pulled from their git repo):

;;; Copyright © 2020 Andrew Whatson <whatson <at> gmail.com>

You can do all this without contacting the author, but hopefully Andrew
responds to us with lots of love.

Thanks,

Morgan

On 12/11/20 3:40 PM, John Soo wrote:
> Hi Morgan,
> 
> Thank you! I will take a look. If I end up using some of their code,
> should I consult them about it and see if they want a copyright line?
> How is that supposed to work?
> 
> Thank you again,
> 
> John
> 

Message #26 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: Andrew Whatson <whatson <at> gmail.com>
To: Morgan Smith <Morgan.J.Smith <at> outlook.com>
Cc: Andrea Corallo <akrl <at> sdf.org>, 44775 <at> debbugs.gnu.org,
 John Soo <jsoo1 <at> asu.edu>, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Fri, 18 Dec 2020 10:25:09 +1000

Hi everyone,

Yes, I'm very happy for my work to be included in Guix in whatever
form.  I've detailed my work below, and also CC'd Andrea who is
responsible for developing this feature in case he has anything to
add.

1. Enable the `--with-native-comp` configure flag.  Self explanatory!

2. Set the `NATIVE_FULL_AOT=1` make flag, maybe.  This tells the build
process to native-compile *all* of the Elisp that ships with Emacs.
Otherwise only the minimal set of Elisp packages included in the pdump
will be native-compiled, with the rest to be compiled on-demand at
runtime.  This has a significant impact on compilation time, so makes
more sense if the package will be built once and installed many times,
and less sense if everyone is building the package themselves.

3. Extend `LIBRARY_PATH` so libgccjit works at compile time.  This is
necessary for a functioning libgccjit and to pass the "smoke test" in
the configure script.  I think this should probably be exported by the
libgccjit package as a search path instead of requiring all dependents
to handle it manually.

4. Patch the `comp-native-driver-options` in `comp.el`.  This is
necessary to have libgccjit functioning at runtime.  Originally I was
setting LIBRARY_PATH in the emacs wrapper to achieve this, but that
had the undesirable effect of leaking to any process launched from
Emacs.  Setting the necessary paths via the driver options is a much
more targeted solution.

5. Remove the shell-script wrappers from eln files.  The
`glib-or-gtk-build-system` aggressively wraps anything which smells
like an executable, preventing the native-compiled Elisp from being
loaded as shared libraries at runtime.  This is based on the
`restore-emacs-pdmp` phase in the base Emacs package which works
around the same problem.

6. Use a newer `libgccjit` based on `gcc-10`.  This is not strictly
necessary, but if I recall correctly libgccjit-9 suffers from a bug
related to the inlining of constant strings which was fixed in
libgccjit-10, and this has some impact on the performance of
native-compiled Elisp.  The `libgccjit` package in Guix is only
defined for gcc-9, so I created a package factory to define libgccjit
packages based on an arbitrary gcc, and use gcc-10 and libgccjit-10 as
dependencies for the emacs-native-comp package.  I haven't tried to
build using gcc-9 and libgccjit-10, so I'm not sure if there's some
interdependency.  I think it would be best to upstream libgccjit-10
alongside an emacs-native-comp package.

I hope this all makes sense, happy to answer any questions.

Cheers,
Andrew

On Fri, 18 Dec 2020 at 03:26, Morgan Smith <Morgan.J.Smith <at> outlook.com> wrote:
>
> Hi John,
>
> (And hi Andrew. We're discussing including your code into the main Guix
> repository. You can view the full conversation here:
> https://issues.guix.gnu.org/44775)
>
> The code I linked to is already licensed under GPLv3+ meaning that you
> are free to integrate their code into the Guix code-base (Going to put
> here that I'm not a lawyer and that this is not legal advice and might
> not even be correct). You should indeed put a little copyright line for
> Andrew in the code. The exact copyright line you should use seems to be
> this (pulled from their git repo):
>
> ;;; Copyright © 2020 Andrew Whatson <whatson <at> gmail.com>
>
> You can do all this without contacting the author, but hopefully Andrew
> responds to us with lots of love.
>
> Thanks,
>
> Morgan
>
> On 12/11/20 3:40 PM, John Soo wrote:
> > Hi Morgan,
> >
> > Thank you! I will take a look. If I end up using some of their code,
> > should I consult them about it and see if they want a copyright line?
> > How is that supposed to work?
> >
> > Thank you again,
> >
> > John
> >

Message #29 received at 44775 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: John Soo <jsoo1 <at> asu.edu>
Cc: 44775 <at> debbugs.gnu.org
Subject: Re: [bug#44775] [PATCH] WIP: Add gccemacs
Date: Fri, 18 Dec 2020 03:50:18 +0100

Hi John,

On Tue, 24 Nov 2020 at 07:06, John Soo <jsoo1 <at> asu.edu> wrote:

>> Thanks!  This motivates me to resume what had been discussed here [1]:
>> be able to somehow run:
>>
>>  guix build emacs-magit emacs-foo emacs-bar --with-input=emacs=gccemacs
>>
>> At least, have a package transformation that allow to rebuild all the
>> Emacs packages with the ’gccemacs’ VM instead of the current Emacs 27 one.
>>
>>> It feels fast but there are bugs and the closure is huge. Also these
>>> patches do not use any of the parameterization machinery.
>>
>> What do you mean by “parameterization machinery”?  The new
>> ’with-parameter’ introduced here [2] or the package transformation that
>> replaces all the dependencies (explicit and implicit).

> I am referring to https://yhetil.org/guix-devel/87eeku8trb.fsf <at> gnu.org
>
> --with-parameter=gccemacs or similar seem to be required since the
> native-comp branch requires a different source, configure flags, and
> probably native-search-paths at least.

I am not convinced that the “parameterization machinery” could help here
(aside it is far to be ready :-)) because in this case, “gccemacs” is
not a «parameter» but an «argument» of the Emacs build system.

Maybe I miss something.


> There appears to be a separate compiled artifact directory under
> $out/lib/emacs/$version/native-lisp/$version-triple which has the
> compiled native libraries (.eln files). That directory seems to not be
> in the search path.  That appears to be causing the first error
> I see. I am not sure which env variable would be tweaked to pick those
> paths up.

Thanks for the explanations.

All the best,
simon

Message #34 received at 44775-done <at> debbugs.gnu.org (full text, mbox):

From: Jelle Licht <jlicht <at> fsfe.org>
To: John Soo <jsoo1 <at> asu.edu>
Cc: 44775-done <at> debbugs.gnu.org
Subject: Re: bug#44775: [PATCH] WIP: Add gccemacs
Date: Mon, 29 May 2023 14:06:15 +0200

Most versions of Emacs available on the guix master branch make use of
this work to make everything blazing fast, so thanks. Closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.