GNU bug report logs - #47125
28.0.50; pdumper assumes compile time page size remains valid

Previous Next

Package: emacs;

Reported by: Pip Cet <pipcet <at> gmail.com>

Date: Sat, 13 Mar 2021 21:39:02 UTC

Severity: normal

Tags: fixed, patch

Found in version 28.0.50

Fixed in version 28.1

Done: Lars Ingebrigtsen <larsi <at> gnus.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 47125 in the body.
You can then email your comments to 47125 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sat, 13 Mar 2021 21:39:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Pip Cet <pipcet <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Sat, 13 Mar 2021 21:39:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; pdumper assumes compile time page size remains valid
Date: Sat, 13 Mar 2021 21:38:16 +0000
I'm running Debian GNU/Linux (the Linux part is not provided by
Debian) on an Apple M1-based machine. This currently involves running
a kernel compiled with a 16 KB page size (the only fully functional
kernel is currently available as a binary as recompilation of the
alleged source fails to produce a fully working kernel).

The Debian-packaged Emacs version does not start. Compiling from
scratch works fine.

After some investigation, this is because pdumper assumes that an
address aligned according to the page size at build time is
sufficiently aligned for mmap to work with the MAP_FIXED flag, when it
comes to loading the dump. That's not true because the Debian Emacs
was apparently built with a 4 KB page size, so it will not run on a
system with a 16 KB page size.

I've confirmed that I get the same error on current master if I modify
getpagesize to return 4096 rather than the correct value.

I think it would be best to handle this case gracefully, and I thought
pdumper already did that, but it appears to simply fail.

There are good reasons for increasing the page size, so this is likely
to happen more often and on other architectures with varying page
sizes. We're currently enforcing a page size of 64 KB on Windows, so
maybe it already has.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 14 Mar 2021 05:38:02 GMT) Full text and rfc822 format available.

Message #8 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pip Cet <pipcet <at> gmail.com>
Cc: Daniel Colascione <dancol <at> dancol.org>, 47125 <at> debbugs.gnu.org
Subject: Re: bug#47125: 28.0.50;
 pdumper assumes compile time page size remains valid
Date: Sun, 14 Mar 2021 07:37:07 +0200
> From: Pip Cet <pipcet <at> gmail.com>
> Date: Sat, 13 Mar 2021 21:38:16 +0000
> 
> I'm running Debian GNU/Linux (the Linux part is not provided by
> Debian) on an Apple M1-based machine. This currently involves running
> a kernel compiled with a 16 KB page size (the only fully functional
> kernel is currently available as a binary as recompilation of the
> alleged source fails to produce a fully working kernel).
> 
> The Debian-packaged Emacs version does not start. Compiling from
> scratch works fine.
> 
> After some investigation, this is because pdumper assumes that an
> address aligned according to the page size at build time is
> sufficiently aligned for mmap to work with the MAP_FIXED flag, when it
> comes to loading the dump. That's not true because the Debian Emacs
> was apparently built with a 4 KB page size, so it will not run on a
> system with a 16 KB page size.
> 
> I've confirmed that I get the same error on current master if I modify
> getpagesize to return 4096 rather than the correct value.
> 
> I think it would be best to handle this case gracefully, and I thought
> pdumper already did that, but it appears to simply fail.
> 
> There are good reasons for increasing the page size, so this is likely
> to happen more often and on other architectures with varying page
> sizes.

CC'ing Daniel, in case he has comments and/or suggestions.

> We're currently enforcing a page size of 64 KB on Windows

We do? can you point me to the code which does that?

> so maybe it already has.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 14 Mar 2021 05:44:02 GMT) Full text and rfc822 format available.

Message #11 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>, Pip Cet <pipcet <at> gmail.com>
Cc: 47125 <at> debbugs.gnu.org
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sat, 13 Mar 2021 21:43:07 -0800
On 3/13/21 9:37 PM, Eli Zaretskii wrote:

>> From: Pip Cet <pipcet <at> gmail.com>
>> Date: Sat, 13 Mar 2021 21:38:16 +0000
>>
>> I'm running Debian GNU/Linux (the Linux part is not provided by
>> Debian) on an Apple M1-based machine. This currently involves running
>> a kernel compiled with a 16 KB page size (the only fully functional
>> kernel is currently available as a binary as recompilation of the
>> alleged source fails to produce a fully working kernel).
>>
>> The Debian-packaged Emacs version does not start. Compiling from
>> scratch works fine.
>>
>> After some investigation, this is because pdumper assumes that an
>> address aligned according to the page size at build time is
>> sufficiently aligned for mmap to work with the MAP_FIXED flag, when it
>> comes to loading the dump. That's not true because the Debian Emacs
>> was apparently built with a 4 KB page size, so it will not run on a
>> system with a 16 KB page size.
>>
>> I've confirmed that I get the same error on current master if I modify
>> getpagesize to return 4096 rather than the correct value.
>>
>> I think it would be best to handle this case gracefully, and I thought
>> pdumper already did that, but it appears to simply fail.
>>
>> There are good reasons for increasing the page size, so this is likely
>> to happen more often and on other architectures with varying page
>> sizes.
> CC'ing Daniel, in case he has comments and/or suggestions.

We should modify this function to do what the doc comment says then. :-) 
I'm not sure if there's any reliable way to know what the worst case 
allocation granularity actually is: a quick grep through /usr/include 
didn't turn up anything. Maybe we should just use a minimum of 16kB 
here? It's not as if we'd be wasting a ton of RAM doing so.

/* Worst-case allocation granularity on any system that might load
   this dump.  */
static int
dump_get_page_size (void)
{
#if defined (WINDOWSNT) || defined (CYGWIN)
  return 64 * 1024;  /* Worst-case allocation granularity.  */
#else
  return getpagesize ();
#endif
}





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 14 Mar 2021 07:35:02 GMT) Full text and rfc822 format available.

Message #14 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 47125 <at> debbugs.gnu.org,
 Pip Cet <pipcet <at> gmail.com>
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sun, 14 Mar 2021 08:34:39 +0100
On Mär 13 2021, Daniel Colascione wrote:

> Maybe we should just use a minimum of 16kB here?

64kB would be better.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 14 Mar 2021 07:37:01 GMT) Full text and rfc822 format available.

Message #17 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 47125 <at> debbugs.gnu.org,
 Pip Cet <pipcet <at> gmail.com>
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sat, 13 Mar 2021 23:36:27 -0800
On 3/13/21 11:34 PM, Andreas Schwab wrote:
> On Mär 13 2021, Daniel Colascione wrote:
>
>> Maybe we should just use a minimum of 16kB here?
> 64kB would be better.

Sure.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 14 Mar 2021 08:18:01 GMT) Full text and rfc822 format available.

Message #20 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 47125 <at> debbugs.gnu.org
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sun, 14 Mar 2021 08:16:21 +0000
On Sun, Mar 14, 2021 at 5:43 AM Daniel Colascione <dancol <at> dancol.org> wrote:
> On 3/13/21 9:37 PM, Eli Zaretskii wrote:
>
> >> From: Pip Cet <pipcet <at> gmail.com>
> >> Date: Sat, 13 Mar 2021 21:38:16 +0000
> >>
> >> I'm running Debian GNU/Linux (the Linux part is not provided by
> >> Debian) on an Apple M1-based machine. This currently involves running
> >> a kernel compiled with a 16 KB page size (the only fully functional
> >> kernel is currently available as a binary as recompilation of the
> >> alleged source fails to produce a fully working kernel).
> >>
> >> The Debian-packaged Emacs version does not start. Compiling from
> >> scratch works fine.
> >>
> >> After some investigation, this is because pdumper assumes that an
> >> address aligned according to the page size at build time is
> >> sufficiently aligned for mmap to work with the MAP_FIXED flag, when it
> >> comes to loading the dump. That's not true because the Debian Emacs
> >> was apparently built with a 4 KB page size, so it will not run on a
> >> system with a 16 KB page size.
> >>
> >> I've confirmed that I get the same error on current master if I modify
> >> getpagesize to return 4096 rather than the correct value.
> >>
> >> I think it would be best to handle this case gracefully, and I thought
> >> pdumper already did that, but it appears to simply fail.
> >>
> >> There are good reasons for increasing the page size, so this is likely
> >> to happen more often and on other architectures with varying page
> >> sizes.
> > CC'ing Daniel, in case he has comments and/or suggestions.
>
> We should modify this function to do what the doc comment says then. :-)
> I'm not sure if there's any reliable way to know what the worst case
> allocation granularity actually is: a quick grep through /usr/include
> didn't turn up anything. Maybe we should just use a minimum of 16kB
> here? It's not as if we'd be wasting a ton of RAM doing so.

Linux also offers 64KB pages, so I believe Andreas is correct, that
would be better.

Should we verify that getpagesize isn't problematic when loading the dump?

Thanks
Pip




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 28 Mar 2021 15:47:02 GMT) Full text and rfc822 format available.

Message #23 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Pip Cet <pipcet <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Daniel Colascione <dancol <at> dancol.org>,
 47125 <at> debbugs.gnu.org
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sun, 28 Mar 2021 17:46:09 +0200
Pip Cet <pipcet <at> gmail.com> writes:

> Linux also offers 64KB pages, so I believe Andreas is correct, that
> would be better.
>
> Should we verify that getpagesize isn't problematic when loading the dump?

This was two weeks ago, and there was no followup here.  Everybody
seemed to agree that we should use 64K here...  So is the following
patch correct?

diff --git a/src/pdumper.c b/src/pdumper.c
index 337742fda4..bdaba0269f 100644
--- a/src/pdumper.c
+++ b/src/pdumper.c
@@ -162,11 +162,7 @@ ptrdiff_t_to_dump_off (ptrdiff_t value)
 static int
 dump_get_page_size (void)
 {
-#if defined (WINDOWSNT) || defined (CYGWIN)
   return 64 * 1024;  /* Worst-case allocation granularity.  */
-#else
-  return getpagesize ();
-#endif
 }
 
 #define dump_offsetof(type, member)                             \


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Added tag(s) patch. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Sun, 28 Mar 2021 15:47:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 28 Mar 2021 16:17:02 GMT) Full text and rfc822 format available.

Message #28 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: dancol <at> dancol.org, 47125 <at> debbugs.gnu.org, pipcet <at> gmail.com
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sun, 28 Mar 2021 19:16:36 +0300
> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: Daniel Colascione <dancol <at> dancol.org>,  Eli Zaretskii <eliz <at> gnu.org>,
>   47125 <at> debbugs.gnu.org
> Date: Sun, 28 Mar 2021 17:46:09 +0200
> 
> This was two weeks ago, and there was no followup here.  Everybody
> seemed to agree that we should use 64K here...  So is the following
> patch correct?

Yes, I think so.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#47125; Package emacs. (Sun, 28 Mar 2021 17:14:02 GMT) Full text and rfc822 format available.

Message #31 received at 47125 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dancol <at> dancol.org, 47125 <at> debbugs.gnu.org, pipcet <at> gmail.com
Subject: Re: bug#47125: 28.0.50; pdumper assumes compile time page size
 remains valid
Date: Sun, 28 Mar 2021 19:13:20 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

>> This was two weeks ago, and there was no followup here.  Everybody
>> seemed to agree that we should use 64K here...  So is the following
>> patch correct?
>
> Yes, I think so.

OK; pushed to Emacs 28 now, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Added tag(s) fixed. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Sun, 28 Mar 2021 17:14:02 GMT) Full text and rfc822 format available.

bug marked as fixed in version 28.1, send any further explanations to 47125 <at> debbugs.gnu.org and Pip Cet <pipcet <at> gmail.com> Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Sun, 28 Mar 2021 17:14:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 26 Apr 2021 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 337 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.