GNU bug report logs - #28882
[bug-gnu-emacs] emacs-26.0.90 build feedback

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28882 in the body.
You can then email your comments to 28882 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
To: bug-gnu-emacs <at> gnu.org
Cc: beebe <at> math.utah.edu
Subject: [bug-gnu-emacs] emacs-26.0.90 build feedback
Date: Tue, 17 Oct 2017 17:31:17 -0600

I now have more than 230 build attempts for emacs-26.0.90 in our
growing test lab with about 180 flavors of Unix(-like) operating
systems.

Of the initial 160 automated builds, 57 completed, and all that
I had to do manually was the "make install" step (at my site,
that is hidden in wrapper so that the installed executable is
named nemacs, for new emacs, so as not to overwrite any existing
version called emacs).

Unfortunately, a further 77 automated builds were broken by this
obnoxious failure:

>> ...
>>     configure: error: The following required libraries were not found:
>> 	 gnutls
>>     Maybe some development libraries/packages are missing?
>>     If you don't want to link with them give
>> 	 --with-gnutls=no
>>     as options to configure
>> ...

My suspicion is that the gnutls package is needed only for editing
files across a network, which few users ever do.  Thus, my strong view
is that, while it is fine for configure to test for the availability
of gnutls, it should NOT be a show-stopper that requires a manual
override and a fresh build attempt with that extra option.

I've expressed the view before on this list that the same should apply
for the various options for libraries that allow emacs to view
bitmap-graphics files, again a feature that few people are ever likely
to use.

With as many test systems as I have, it is utterly impractical to know
in advance which of the --with-XXX=no options are needed to avoid
configure-time termination.  Even if I wrote down once what those
options are for each system, subsequent system updates would likely
invalidate those choices, as would different versions of emacs.  On
several of my systems, there are GIF, JPEG, PNG, RSVG, TIFF,
... libraries installed, but configure may still reject them because
of version-number tests, or feature tests.

I strongly recommend that configure.ac be revised so that all
--with-XXX=[no|yes] options allow configuration and building to
proceed, regardless of their settings.  It is just too painful to have
to spend several unnecessary hours restarting builds to work around
the current undesirable behavior.

I still have a few systems where emacs-26.0.90 has not yet
successfully been built and installed, and I'm continuing to work on
finding solutions.  However, I wanted to get this report posted now
that I've done 125 successful installs.

On our old CentOS 5 systems (packages frozen by Red Hat), I got a
surprise message that has never appeared with any earlier version of
emacs (I have almost 6000 logs for emacs builds going back as far as
20-Aug-1998):

>> ...
>> loading /local/build/cc/emacs-26.0.90/lisp/leim/leim-list.el (source)...
>> Finding pointers to doc strings...
>> Finding pointers to doc strings...done
>> Dumping under the name emacs
>> **************************************************
>> Warning: Your system has a gap between BSS and the
>> heap (258966655 bytes).  This usually means that exec-shield
>> or something similar is in effect.  The dump may
>> fail because of this.  See the section about
>> exec-shield in etc/PROBLEMS for more information.
>> **************************************************
>> make[1]: *** [bootstrap-emacs] Segmentation fault (core dumped)
>> ...

I'm doubtful that CentOS 5 had any protection against stack/heap/bss
collision, because such discussions have only shown up on security
lists in the last several months.  Thus, it may be that the emacs test
for the unexpected gap is faulty.

The one class of operating systems for which emacs is known not to
work out-of-the-box from standard GNU distribution channels is Alpine
Linux. I have 5 VMs running various versions of that O/S.  Alpine uses
muscl instead of glibc, and has a different memory model that breaks
emacs, and also the tcsh shell.  Some Alpine versions have a patched
emacs in the package system, but the latest ones do not.  Thus,
emacs-26.0.90 will not build at all on Alpine Linux.

In no case has a build been stopped by emacs source code errors, but
some builds were stopped by link-time failures to find particular
functions.  In most cases, I could later get a successful build by
adding a suitable --with-XXX=no option to suppress use of a particular
library.

One final point: as far as I recall, in emacs-25 and earlier, the
Makefiles were carefully written to be usable by traditional Unix
make.  Sadly, that is no longer the case, and I think the changes that
now mandate GNU make for building emacs should be rescinded.

Even though most Unix-(like) systems other than GNU Hurd and GNU/Linux
have a gmake package, when bootstrapping a new system, usually the
first GNU package that I'm likely to build is GNU tar, and the second,
GNU emacs.  That means that both need to be buildable with traditional
make, and both are far too complex to do a build by manually running
cc.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe <at> math.utah.edu  -
- 155 S 1400 E RM 233                       beebe <at> acm.org  beebe <at> computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

Message #8 received at 28882 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
Cc: 28882 <at> debbugs.gnu.org
Subject: Re: bug#28882: [bug-gnu-emacs] emacs-26.0.90 build feedback
Date: Tue, 17 Oct 2017 23:41:07 -0400

[Message part 1 (text/plain, inline)]

"Nelson H. F. Beebe" <beebe <at> math.utah.edu> writes:

> Unfortunately, a further 77 automated builds were broken by this
> obnoxious failure:
>
>>> ...
>>>     configure: error: The following required libraries were not found:
>>> 	 gnutls
>>>     Maybe some development libraries/packages are missing?
>>>     If you don't want to link with them give
>>> 	 --with-gnutls=no
>>>     as options to configure
>>> ...
>
> My suspicion is that the gnutls package is needed only for editing
> files across a network, which few users ever do.  Thus, my strong view
> is that, while it is fine for configure to test for the availability
> of gnutls, it should NOT be a show-stopper that requires a manual
> override and a fresh build attempt with that extra option.

It's also used for downloading packages from repositories.  It is
optional there, but in recent times not using https for this is
considered unacceptable (e.g. [1]).  You might call this mass hysteria,
or you might call it sanity being restored, but either way, I don't
think it's accurate that few users will want gnutls.

[1]: https://glyph.twistedmatrix.com/2015/11/editor-malware.html

> I've expressed the view before on this list that the same should apply
> for the various options for libraries that allow emacs to view
> bitmap-graphics files, again a feature that few people are ever likely
> to use.

Yes, in Bug#25317 [2].  I was going to to add an option to make that
possible, but I didn't get around to finishing it at the time.

[2]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25317#13

> I strongly recommend that configure.ac be revised so that all
> --with-XXX=[no|yes] options allow configuration and building to
> proceed, regardless of their settings.  It is just too painful to have
> to spend several unnecessary hours restarting builds to work around
> the current undesirable behavior.

Here's a patch now, which should give the behaviour you want when
passing --with-all=maybe to ./configure.  Unfortunately, it's probably
too late to get this in for Emacs 26, but hopefully we can smooth things
out for version 27.

[v1-0001-Support-.-configure-with-all-maybe.patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

And here is the equivalent for configure, in case you don't have
autoconf handy:

[configure.diff (text/x-diff, attachment)]

[Message part 5 (text/plain, inline)]

> On our old CentOS 5 systems (packages frozen by Red Hat), I got a
> surprise message that has never appeared with any earlier version of
> emacs (I have almost 6000 logs for emacs builds going back as far as
> 20-Aug-1998):
>
>>> ...
>>> loading /local/build/cc/emacs-26.0.90/lisp/leim/leim-list.el (source)...
>>> Finding pointers to doc strings...
>>> Finding pointers to doc strings...done
>>> Dumping under the name emacs
>>> **************************************************
>>> Warning: Your system has a gap between BSS and the
>>> heap (258966655 bytes).  This usually means that exec-shield
>>> or something similar is in effect.  The dump may
>>> fail because of this.  See the section about
>>> exec-shield in etc/PROBLEMS for more information.
>>> **************************************************
>>> make[1]: *** [bootstrap-emacs] Segmentation fault (core dumped)
>>> ...
>
> I'm doubtful that CentOS 5 had any protection against stack/heap/bss
> collision, because such discussions have only shown up on security
> lists in the last several months.  Thus, it may be that the emacs test
> for the unexpected gap is faulty.

Hmm, a quick web search does seem to indicate that exec-shield only is
only set by default in CentOS 6 and up.  On the other hand, since you
got a segfault, the test for the gap is probably not faulty (just that
the warning text is imprecise: "exec-shield or something").

> The one class of operating systems for which emacs is known not to
> work out-of-the-box from standard GNU distribution channels is Alpine
> Linux. I have 5 VMs running various versions of that O/S.  Alpine uses
> muscl instead of glibc, and has a different memory model that breaks
> emacs, and also the tcsh shell.  Some Alpine versions have a patched
> emacs in the package system, but the latest ones do not.  Thus,
> emacs-26.0.90 will not build at all on Alpine Linux.

I was under the impression that Emacs 26 should be able to work with
muscl now, see Bug#22086[3] (which specifically mentions "Alpine
Linux").

[3]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22086

> One final point: as far as I recall, in emacs-25 and earlier, the
> Makefiles were carefully written to be usable by traditional Unix
> make.  Sadly, that is no longer the case, and I think the changes that
> now mandate GNU make for building emacs should be rescinded.

I think this is unlikely to happen:

    The whole thing is a bit of a mess, and the mess is largely caused by
    Emacs using Automake, and Automake is needed only because of Gnulib
    (because Gnulib assumes Automake for portability to older systems
    lacking GNU Make). I'll look into fixing this so that Gnulib no longer
    assumes Automake, so that Emacs can stop relying on Automake. Since
    Emacs assumes GNU Make, it doesn't need Automake (except for the Gnulib
    code, which I think I can fix).

http://lists.gnu.org/archive/html/emacs-devel/2017-01/msg00097.html

Message #11 received at 28882 <at> debbugs.gnu.org (full text, mbox):

From: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
To: 28882 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: beebe <at> math.utah.edu
Subject: Re: bug#28882: [bug-gnu-emacs] emacs-26.0.90 build feedback
Date: Wed, 18 Oct 2017 07:31:10 -0600

Thanks, Noam, for your extensive response to my long report yesterday
of my build experience for emacs-26.0.90 on, initially, 160 different
operating systems in our test lab.

Since that report, with manual tweaks, such as hiding /usr/local
during builds, and restoring it before installs, I've increased the
number of successes to 135 systems.

This reply addresses the issue of Alpine Linux.  This morning, I read
the section in etc/PROBLEMS on address space layout randomization
(ASLR), and on our Alpine systems, tried the trick (as root) of

	echo 0 > /proc/sys/kernel/randomize_va_space

before doing a build with 

	env CONFIG_SITE= ./configure  --prefix=$L --with-pop && make all check

[L is a personal variable for our default prefix on newer systems].

I'm pleased to report that the builds on Alpine Linux succeeded, and
emacs is now fully functional on all of them, which report their
versions in /etc/alpine-release as 3.4.6, 3.5.2, 3.6.0, and 3.6.2.

I'm now looking into similar issues on Hardened BSD 11.1-STABLE-HBSD
and 12.0-CURRENT, where the build succeeds up to the point of the
memory dump, and then fails with

	Dumping under the name emacs
	11323200 of 33554432 static heap bytes used
	gmake[1]: *** [Makefile:738: bootstrap-emacs] Segmentation fault

I'll report back on this list if/when I learn more.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe <at> math.utah.edu  -
- 155 S 1400 E RM 233                       beebe <at> acm.org  beebe <at> computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

Message #14 received at 28882 <at> debbugs.gnu.org (full text, mbox):

From: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
To: 28882 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: beebe <at> math.utah.edu
Subject: Re: bug#28882: [bug-gnu-emacs] emacs-26.0.90 build feedback
Date: Wed, 18 Oct 2017 08:17:14 -0600

The emacs-26.0.90 etc/PROBLEMS file suggests another approach to work
around kernel memory layout randomizations for Red Hat systems.
Curiously, I had no difficulty building emacs-26.0.90 on CentOS 6 and
7 on x86-64 systems, and CentOS 5 IA-64 (Itanium), but on CentOS 5 x86
and x86-64, I get

	Dumping under the name emacs
	**************************************************
	Warning: Your system has a gap between BSS and the
	heap (8467071 bytes).  This usually means that exec-shield
	or something similar is in effect.  The dump may
	fail because of this.  See the section about
	exec-shield in etc/PROBLEMS for more information.
	**************************************************
	make[1]: *** [bootstrap-emacs] Segmentation fault

Today, as root, I ran 

	# cat /proc/sys/kernel/exec-shield
	1

	# echo 0 > /proc/sys/kernel/exec-shield

	# cat /proc/sys/kernel/exec-shield
	0

and then as an unprivileged user, restarted the make.  Unlike on
Alpine Linux, the kernel symbol-value change had no effect: the build
still gets a segmentation fault.

I then tried

	% ./src/temacs
	Loading loadup.el (source)...
	Using load-path (/local/build/cc/emacs-26.0.90/lisp)
	Loading emacs-lisp/byte-run...
	...
	Finding pointers to doc strings...done
	Pure-hashed: 15477 strings, 4055 vectors, 40214 conses, 3919 bytecodes, 175 others

At that point, a normal emacs X11 window appears on my screen, so most
of emacs is working.

However, the dumped executable is unusable:

	% ls -l src/*emacs
	-rwxrwxr-x 1 beebe sysstaff 84149492 Oct 18 07:40 src/emacs
	-rwxrwxr-x 1 beebe sysstaff 48352600 Oct 18 07:40 src/temacs

	% file src/emacs
	src/emacs: data

	% ./src/emacs
	src/emacs: Exec format error. Binary file not executable.

	% ldd ./src/emacs
        not a dynamic executable

In CentOS 5 and 6, the exec-shield variable is 1 by default; it does
not exist on CentOS 7.  The latter instead has
/proc/sys/kernel/randomize_va_space set to 2, but a successful dump to
create src/emacs does not require changing that variable.

Next, I tried another suggestion from etc/PROBLEMS:

	% rm src/emacs src/temacs
	$ bash
	$ export PATH=/bin:/usr/bin:/sbin:/usr/sbin
	$ setarch $(uname -m ) -R make

That one succeeded in making a usable src/emacs executable on both x86
and x86-64 CentOS 5 systems, with /usr/local hidden; I then restored
/usr/local and successfully installed emacs-26.0.90 on those servers.

Thus, the build problems for emacs-26.0.90 on CentOS 5 are resolved,
and my count of successes has increased to 138.



-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe <at> math.utah.edu  -
- 155 S 1400 E RM 233                       beebe <at> acm.org  beebe <at> computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

Message #17 received at 28882 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
Cc: Noam Postavsky <npostavs <at> users.sourceforge.net>, 28882 <at> debbugs.gnu.org
Subject: Re: bug#28882: emacs-26.0.90 build feedback
Date: Wed, 18 Oct 2017 16:24:12 +0200

On Okt 18 2017, "Nelson H. F. Beebe" <beebe <at> math.utah.edu> wrote:

> I'm now looking into similar issues on Hardened BSD 11.1-STABLE-HBSD
> and 12.0-CURRENT, where the build succeeds up to the point of the
> memory dump, and then fails with
>
> 	Dumping under the name emacs
> 	11323200 of 33554432 static heap bytes used
> 	gmake[1]: *** [Makefile:738: bootstrap-emacs] Segmentation fault

Does it help to add -no-pie to LDFLAGS?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #20 received at 28882 <at> debbugs.gnu.org (full text, mbox):

From: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
To: 28882 <at> debbugs.gnu.org, Noam Postavsky <npostavs <at> users.sourceforge.net>,
 Andreas Schwab <schwab <at> suse.de>
Cc: beebe <at> math.utah.edu
Subject: Re: bug#28882: [bug-gnu-emacs] emacs-26.0.90 build feedback
Date: Wed, 18 Oct 2017 11:58:11 -0600

Andreas Schwab asks about my build attempts for emacs-26.0.90 on
HardenedBSD:

>> Does it help to add -no-pie to LDFLAGS?

For HardenedBSD 11.1-STABLE-HBSD (FreeBSD 11.1-STABLE-HBSD #0), that
worked:

	rm src/emacs src/temacs
	gmake LDFLAGS=-no-pie

I installed that version.

I then tried the same on HardenedBSD 12.0-CURRENT (FreeBSD
12.0-CURRENT #0).

It complained that the -no-pie option was unrecognized, so I retried
with --no-pie: that was accepted, but there was still a segmentation
fault at dump time.

Next, based on the advice for NetBSD in etc/PROBLEMS about kernel
parameters that control address-space layout randomization (ASLR), 
I looked to see what HardenedBSD had:

	# sysctl -a | grep -i aslr
	kern.features.hbsd_aslr: 1
	options PAX_ASLR
	hardening.pax.aslr.status: 2

With those defaults, I get failure like this in the emacs build:

	Dumping under the name emacs
	11323200 of 33554432 static heap bytes used
	gmake[1]: *** [Makefile:738: bootstrap-emacs] Segmentation fault

In another window, as root, I then ran

	# sysctl kern.features.hbsd_aslr:0
	sysctl: oid 'kern.features.hbsd_aslr' is read only

	# sysctl hardening.pax.aslr.status:0
	hardening.pax.aslr.status: 2 -> 0

Then, back in the emacs build window, I ran

	% \rm src/emacs src/temacs
	% gmake
	...
	./temacs --batch  --load loadup bootstrap
	Loading loadup.el (source)...
	Using load-path (/local/build/cc/emacs-26.0.90/lisp /local/build/cc/emacs-26.0.90/lisp/emacs-lisp
	/local/build/cc/emacs-26.0.90/lisp/language /local/build/cc/emacs-26.0.90/lisp/international
	/local/build/cc/emacs-26.0.90/lisp/textmodes /local/build/cc/emacs-26.0.90/lisp/vc)
	Loading emacs-lisp/byte-run...
	Loading emacs-lisp/backquote...
	...
	Finding pointers to doc strings...
	Finding pointers to doc strings...done
	Dumping under the name emacs
	11323200 of 33554432 static heap bytes used
	96055 pure bytes used
	mv -f emacs bootstrap-emacs
	gmake -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
	gmake[2]: Entering directory '/local/build/cc/emacs-26.0.90/lisp'
	gmake[2]: Nothing to be done for 'compile-first'.
	gmake[2]: Leaving directory '/local/build/cc/emacs-26.0.90/lisp'
	gmake -C ../admin/unidata all EMACS="../../src/bootstrap-emacs"
	gmake[2]: Entering directory '/local/build/cc/emacs-26.0.90/admin/unidata'
	  ELC      uvs.elc
	elf_load_section: truncated ELF file
	gmake[2]: *** [Makefile:72: uvs.elc] Abort trap

If I try to run the dumped emacs, I get

	% src/bootstrap-emacs --version
	elf_load_section: truncated ELF file
	Abort

Thus, disabling the ASLR feature in HardenedBSD 12.0 DOES let temacs
run to completion, but the result dumped emacs does not run correctly.

As I final experiment, I ported over the 11.1 emacs installation
directories to 12.0, and after installing some missing packages, and
creating some symlinks to missing older library versions, I was able
to get a usable emacs-26.0.90 on 12.0.  However, that has to be viewed
as a temporary stopgap.

The number of dependent shared libraries is frighteningly large

	% ldd $B/nemacs |wc -l
	96

so the import from 11.1 to 12.0 is fragile, and likely to break with
future system updates on the 12.0 system.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe <at> math.utah.edu  -
- 155 S 1400 E RM 233                       beebe <at> acm.org  beebe <at> computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

Message #23 received at 28882 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Nelson H. F. Beebe" <beebe <at> math.utah.edu>
Cc: Noam Postavsky <npostavs <at> users.sourceforge.net>, 28882 <at> debbugs.gnu.org
Subject: Re: bug#28882: [bug-gnu-emacs] emacs-26.0.90 build feedback
Date: Sat, 22 Aug 2020 18:19:05 +0200

"Nelson H. F. Beebe" <beebe <at> math.utah.edu> writes:

> I now have more than 230 build attempts for emacs-26.0.90 in our
> growing test lab with about 180 flavors of Unix(-like) operating
> systems.

Cool!  Is it possible to see the status of these builds anywhere?

> This reply addresses the issue of Alpine Linux.  This morning, I read
> the section in etc/PROBLEMS on address space layout randomization
> (ASLR), and on our Alpine systems, tried the trick (as root) of
>
> 	echo 0 > /proc/sys/kernel/randomize_va_space
>
> before doing a build with 
>
> 	env CONFIG_SITE= ./configure  --prefix=$L --with-pop && make all check

I think that newer Emacs versions should build with address
randomisation just fine, so this should no longer be necessary.

The other concrete issue in this bug report was that you have to say
./configure --with-gnutls=no (or ifavailable, these days), otherwise
Emacs will refuse to build (if you have no gnutls libraries).  I think
that's a the correct choice, and leads to a whole lot fewer bug reports
about packages etc not working.

So I'm closing this bug report; if there's anything more to be done
here, please respond to the debbugs mail address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.