GNU bug report logs - #79074
31.0.50; Increase gc-cons-percentage in Makefiles

Previous Next

To reply to this bug, email your comments to 79074 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Helmut Eller <eller.helmut <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; Increase gc-cons-percentage in Makefiles
Date: Tue, 22 Jul 2025 17:27:11 +0200

It seems that the default value for gc-cons-percentage (10%) is very
conservative and hurts performance.

I did the following experiment: first, I built the master branch (with
-O2 and without checking); second, I executed this script:

for percentage in 0.1 0.75 1.5 ; do
    find lisp/ -name '*.elc' -exec rm -v {} +
    /usr/bin/time -o make-$percentage.time make -l3.2 -j4 \
BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $percentage)'"\
lisp
done

The intention is to measure the time it takes to compile all the .el
files in the lisp directory for different values of gc-cons-percentage.
I get these results:

* for gc-cons-percentage = 0.1:
1697.81user 57.67system 12:08.57elapsed 240%CPU (0avgtext+0avgdata 202268maxresident)k
12424inputs+173840outputs (29major+9304710minor)pagefaults 0swaps

* for gc-cons-percentage = 0.75
1034.16user 61.77system 7:55.26elapsed 230%CPU (0avgtext+0avgdata 216888maxresident)k
42864inputs+172280outputs (157major+10152739minor)pagefaults 0swaps

* for gc-cons-percentage = 1.5
823.91user 65.62system 6:27.09elapsed 229%CPU (0avgtext+0avgdata 246948maxresident)k
118640inputs+171184outputs (106major+12651774minor)pagefaults 0swaps

We see that increasing gc-cons-percentage to 0.75 reduces the elapsed
real time considerably (from ~12 minutes to ~8); it increases maxrss from
203MB to 216MB, which seems quite modest.

So I think we should add something to BYTE_COMPILE_FLAGS to increase
gc-cons-percentage; or perhaps increase it in batch-byte-compile.

Message #8 received at 79074 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Helmut Eller <eller.helmut <at> gmail.com>
Cc: 79074 <at> debbugs.gnu.org
Subject: Re: bug#79074: 31.0.50; Increase gc-cons-percentage in Makefiles
Date: Tue, 22 Jul 2025 20:02:29 +0300

> From: Helmut Eller <eller.helmut <at> gmail.com>
> Date: Tue, 22 Jul 2025 17:27:11 +0200
> 
> It seems that the default value for gc-cons-percentage (10%) is very
> conservative and hurts performance.
> 
> I did the following experiment: first, I built the master branch (with
> -O2 and without checking); second, I executed this script:
> 
> for percentage in 0.1 0.75 1.5 ; do
>     find lisp/ -name '*.elc' -exec rm -v {} +
>     /usr/bin/time -o make-$percentage.time make -l3.2 -j4 \
> BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $percentage)'"\
> lisp
> done
> 
> The intention is to measure the time it takes to compile all the .el
> files in the lisp directory for different values of gc-cons-percentage.
> I get these results:
> 
> * for gc-cons-percentage = 0.1:
> 1697.81user 57.67system 12:08.57elapsed 240%CPU (0avgtext+0avgdata 202268maxresident)k
> 12424inputs+173840outputs (29major+9304710minor)pagefaults 0swaps
> 
> * for gc-cons-percentage = 0.75
> 1034.16user 61.77system 7:55.26elapsed 230%CPU (0avgtext+0avgdata 216888maxresident)k
> 42864inputs+172280outputs (157major+10152739minor)pagefaults 0swaps
> 
> * for gc-cons-percentage = 1.5
> 823.91user 65.62system 6:27.09elapsed 229%CPU (0avgtext+0avgdata 246948maxresident)k
> 118640inputs+171184outputs (106major+12651774minor)pagefaults 0swaps
> 
> We see that increasing gc-cons-percentage to 0.75 reduces the elapsed
> real time considerably (from ~12 minutes to ~8); it increases maxrss from
> 203MB to 216MB, which seems quite modest.
> 
> So I think we should add something to BYTE_COMPILE_FLAGS to increase
> gc-cons-percentage; or perhaps increase it in batch-byte-compile.

We could certainly try that; patches welcome.

P.S. The above benchmark should be repeated with native-compilation,
IMO.

Thanks.

Message #11 received at 79074 <at> debbugs.gnu.org (full text, mbox):

From: Helmut Eller <eller.helmut <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79074 <at> debbugs.gnu.org
Subject: Re: bug#79074: 31.0.50; Increase gc-cons-percentage in Makefiles
Date: Wed, 23 Jul 2025 14:03:44 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jul 22 2025, Eli Zaretskii wrote:

[...]
>> So I think we should add something to BYTE_COMPILE_FLAGS to increase
>> gc-cons-percentage; or perhaps increase it in batch-byte-compile.
>
> We could certainly try that; patches welcome.

Here is a one:

[0001-Increase-gc-cons-percentage-when-building-.elc-files.patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

> P.S. The above benchmark should be repeated with native-compilation,
> IMO.

I repeated it for "make bootstrap" because I don't know how to only
rebuild the lisp directory with native-compilation.

So basically I did:

for percentage in 0.1 0.75 1.5 ; do
    /usr/bin/time -o make-$percentage.time make -l3.2 -j4 \
 BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $percentage)'" \
 FAST=true -l3.2 -j4 bootstrap
done

The results are:

* for gc-cons-percentage = 0.1:
2331.71user 86.23system 14:38.08elapsed 275%CPU (0avgtext+0avgdata 641880maxresident)k
4432inputs+742056outputs (18major+14952231minor)pagefaults 0swaps

* for gc-cons-percentage = 0.75:
1223.32user 85.96system 8:06.30elapsed 269%CPU (0avgtext+0avgdata 706240maxresident)k
0inputs+742064outputs (2major+15873839minor)pagefaults 0swaps

* for gc-cons-percentage = 1.5:
961.91user 94.57system 6:32.58elapsed 269%CPU (0avgtext+0avgdata 766348maxresident)k
16inputs+742064outputs (0major+18617902minor)pagefaults 0swaps

[BTW, boostrapping the igc branch requires:
1039.27user 181.75system 7:19.31elapsed 277%CPU (0avgtext+0avgdata 929092maxresident)k
38960inputs+682528outputs (8major+27973694minor)pagefaults 0swaps ]

For fun, I also recorded the times for individual compilation processes.
Here a scatter plot of the result:

[elc.svgz (image/svg+xml, attachment)]

[Message part 5 (text/plain, inline)]

Some statistics:

 gc-cons-percentage │ variable  mean       min      median   max    
────────────────────┼───────────────────────────────────────────────
                0.1 │ real      1.3452     0.12     0.48     162.52 
                0.1 │ rssmax    51622.8    42692    45388.0  641880 
		    │
               0.75 │ real      0.660551   0.1      0.285    69.69  
               0.75 │ rssmax    54213.1    43032    47932.0  706240 
	            │
                1.5 │ real      0.507625   0.08     0.22     62.93  
                1.5 │ rssmax    60135.5    43060    55508.0  766348 

and here the full tables, if somebody is interested:

[elc-0.1.csv.gz (application/gzip, attachment)]

[elc-0.75.csv.gz (application/gzip, attachment)]

[elc-1.5.csv.gz (application/gzip, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.