Package: guile;
To reply to this bug, email your comments to 61476 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-guile <at> gnu.org:bug#61476; Package guile.
(Mon, 13 Feb 2023 14:35:02 GMT) Full text and rfc822 format available.Torrekie <me <at> torrekie.dev>:bug-guile <at> gnu.org.
(Mon, 13 Feb 2023 14:35:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Torrekie <me <at> torrekie.dev> To: bug-guile <at> gnu.org Subject: mmap() with RWX cannot correctly invalidate cache Date: Mon, 13 Feb 2023 17:09:54 +0800
[Message part 1 (text/plain, inline)]
I'm building Guile 3.0.9 on iOS 14.5.1 (Darwin 20.4.0) which is suffering some system API differences within pthread_jit* stuff.
cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc chars.doc control.doc continuations.doc debug.doc deprecated.doc deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=0 ../meta/build-env guild snarf-check-and-output-texi > guile-procedures.texi || { rm guile-procedures.texi; false; }
cat: write error: Broken pipe
After inspecting the crash logs, it might be some unexpected freeing of mmap cache, the librecompat.0.dylib includes a sys_icache_invalidate wrapper for __clear_cache function (as the same implementation with macOS one since they are all come from LLVM) which has not been compiled to system libcompiler_rt.dylib for iOS.
Exception Type: EXC_BAD_ACCESS (SIGBUS)
Exception Subtype: KERN_PROTECTION_FAILURE at 0x0000000104ccb9d8
VM Region Info: 0x104ccb9d8 is in 0x104cc4000-0x104ccc000; bytes after start: 31192 bytes before end: 1575
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
__LINKEDIT 104bc0000-104cc4000 [ 1040K] r--/rw- SM=COW ...e-3.0.1.dylib
---> __TEXT 104cc4000-104ccc000 [ 32K] r--/rw- SM=COW ...ompat.0.dylib
__DATA_CONST 104ccc000-104cd0000 [ 16K] rw-/rw- SM=COW ...ompat.0.dylib
Termination Signal: Bus error: 10
Termination Reason: Namespace SIGNAL, Code 0xa
Terminating Process: exc handler [9957]
Triggered by Thread: 0
Thread 0 name: Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0 + librecompat.0.dylib 0x104ccb9d8 0x104cc4000 + 0x79d8 // __clear_cache + 0x0
1 libguile-3.0.1.dylib 0x104b4b728 0x104a44000 + 0x107728 // /buildroot/guile-3.0.9/libguile/./lightening/lightening/aarch64.c:217
2 libguile-3.0.1.dylib 0x104b4b3b0 0x104a44000 + 0x1073b0 // /buildroot/guile-3.0.9/libguile/./lightening/lightening/lightening.c:241
3 libguile-3.0.1.dylib 0x104aa12e0 0x104a44000 + 0x5d2e0 // /buildroot/guile-3.0.9/libguile/jit.c:1424
4 libguile-3.0.1.dylib 0x104aa0c38 0x104a44000 + 0x5cc38 // /buildroot/guile-3.0.9/libguile/jit.c:5895
5 libsystem_pthread.dylib 0x1def01f60 0x1deeff000 + 0x2f60 // __pthread_once_handler + 0x50
6 libsystem_platform.dylib 0x1deefb964 0x1deef7000 + 0x4964 // _os_once_callout + 0x20
7 libsystem_pthread.dylib 0x1def01ef4 0x1deeff000 + 0x2ef4 // pthread_once + 0x64
8 libguile-3.0.1.dylib 0x104aa0558 0x104a44000 + 0x5c558 // /buildroot/guile-3.0.9/libguile/jit.c:0
9 libguile-3.0.1.dylib 0x104aa0410 0x104a44000 + 0x5c410 // /buildroot/guile-3.0.9/libguile/jit.c:6031
10 libguile-3.0.1.dylib 0x104b35774 0x104a44000 + 0xf1774 // /buildroot/guile-3.0.9/libguile/./vm-engine.c:370
11 libguile-3.0.1.dylib 0x104b323d0 0x104a44000 + 0xee3d0 // /buildroot/guile-3.0.9/libguile/vm.c:1615
12 libguile-3.0.1.dylib 0x104a6776c 0x104a44000 + 0x2376c // /buildroot/guile-3.0.9/libguile/eval.c:496
13 libguile-3.0.1.dylib 0x104af9e20 0x104a44000 + 0xb5e20 // /buildroot/guile-3.0.9/libguile/read.c:1553
14 libguile-3.0.1.dylib 0x104abb158 0x104a44000 + 0x77158 // /buildroot/guile-3.0.9/libguile/load.c:124
15 libguile-3.0.1.dylib 0x104abc3d4 0x104a44000 + 0x783d4 // /buildroot/guile-3.0.9/libguile/load.c:1267
16 libguile-3.0.1.dylib 0x104abcb28 0x104a44000 + 0x78b28 // /buildroot/guile-3.0.9/libguile/load.c:1275
17 libguile-3.0.1.dylib 0x104a8e0ec 0x104a44000 + 0x4a0ec // /buildroot/guile-3.0.9/libguile/init.c:230
18 libguile-3.0.1.dylib 0x104a8e458 0x104a44000 + 0x4a458 // /buildroot/guile-3.0.9/libguile/init.c:510
19 libguile-3.0.1.dylib 0x104b2a89c 0x104a44000 + 0xe689c // /buildroot/guile-3.0.9/libguile/threads.c:578
20 libguile-3.0.1.dylib 0x104b2d47c 0x104a44000 + 0xe947c // /buildroot/guile-3.0.9/libguile/threads.c:642
21 + libgc.1.dylib 0x1053688e4 0x105354000 + 0x148e4 // /buildroot/gc-8.2.2/misc.c:2173
22 libguile-3.0.1.dylib 0x104b2a97c 0x104a44000 + 0xe697c // /buildroot/guile-3.0.9/libguile/threads.c:692
23 libguile-3.0.1.dylib 0x104b2a930 0x104a44000 + 0xe6930 // /buildroot/guile-3.0.9/libguile/threads.c:698
24 libguile-3.0.1.dylib 0x104a8e190 0x104a44000 + 0x4a190 // /buildroot/guile-3.0.9/libguile/init.c:295
25 guile (*) 0x104a37d90 0x104a30000 + 0x7d90 // /buildroot/guile-3.0.9/libguile/guile.c:94
26 libdyld.dylib 0x193c92cf8 0x193c91000 + 0x1cf8 // start + 0x4
But according to the JIT allocation problems which has been met by other projects:
https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-7-j <at> getutm.app/#23841029 <https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-7-j <at> getutm.app/#23841029>
I attempted to patch libguile/jit.c like that:
--- libguile/jit.c 1676274604.835054871
+++ libguile/jit.c.old 1676204975.415324076
@@ -47,8 +47,17 @@
#include <sys/mman.h>
#endif
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
#include <libkern/OSCacheControl.h>
+#include "tcg-apple-jit.h"
+
+static void tb_exec_change(bool locked)
+{
+ if (jit_write_protect_supported()) {
+ jit_write_protect(locked);
+ }
+}
+
#endif
#include "jit.h"
@@ -1414,8 +1435,9 @@
uint8_t *ret = jit_address (j->jit);
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
- pthread_jit_write_protect_np(0);
+#if defined __APPLE__
+// pthread_jit_write_protect_np(0);
+ tb_exec_change(0);
#endif
emit (j);
@@ -1423,10 +1445,11 @@
size_t size;
if (!jit_has_overflow (j->jit) && jit_end (j->jit, &size))
{
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
/* protect previous code arena. leave unprotected after emit()
since jit_end() also writes to code arena. */
- pthread_jit_write_protect_np(1);
+// pthread_jit_write_protect_np(1);
+ tb_exec_change(1);
sys_icache_invalidate(arena->base, arena->size);
#endif
ASSERT (size <= (arena->size - arena->used));
@@ -1442,9 +1465,10 @@
}
else
{
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
/* protect previous code arena */
- pthread_jit_write_protect_np(1);
+// pthread_jit_write_protect_np(1);
+ tb_exec_change(1);
sys_icache_invalidate(arena->base, arena->size);
#endif
jit_reset (j->jit);
Which replaces the previous `pthread_jit_write_protect_np` with another iOS capable implementation that provided in the link above:
/*
* Apple Silicon functions for JIT handling
*
* Copyright (c) 2020 osy
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TCG_APPLE_JIT_H
#define TCG_APPLE_JIT_H
/*
* APRR handling
* Credits to: https://siguza.github.io/APRR/
* Reversed from /usr/lib/system/libsystem_pthread.dylib
*/
#if defined(__aarch64__) && defined(__APPLE__)
#define _COMM_PAGE_START_ADDRESS (0x0000000FFFFFC000ULL) /* In TTBR0 */
#define _COMM_PAGE_APRR_SUPPORT (_COMM_PAGE_START_ADDRESS + 0x10C)
#define _COMM_PAGE_APPR_WRITE_ENABLE (_COMM_PAGE_START_ADDRESS + 0x110)
#define _COMM_PAGE_APRR_WRITE_DISABLE (_COMM_PAGE_START_ADDRESS + 0x118)
static __attribute__((__always_inline__)) bool jit_write_protect_supported(void)
{
/* Access shared kernel page at fixed memory location. */
uint8_t aprr_support = *(volatile uint8_t *)_COMM_PAGE_APRR_SUPPORT;
return aprr_support > 0;
}
/* write protect enable = write disable */
static __attribute__((__always_inline__)) void jit_write_protect(int enabled)
{
/* Access shared kernel page at fixed memory location. */
uint8_t aprr_support = *(volatile uint8_t *)_COMM_PAGE_APRR_SUPPORT;
if (aprr_support == 0 || aprr_support > 3) {
return;
} else if (aprr_support == 1) {
__asm__ __volatile__ (
"mov x0, %0\n"
"ldr x0, [x0]\n"
"msr S3_4_c15_c2_7, x0\n"
"isb sy\n"
:: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE
: _COMM_PAGE_APPR_WRITE_ENABLE)
: "memory", "x0"
);
} else {
__asm__ __volatile__ (
"mov x0, %0\n"
"ldr x0, [x0]\n"
"msr S3_6_c15_c1_5, x0\n"
"isb sy\n"
:: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE
: _COMM_PAGE_APPR_WRITE_ENABLE)
: "memory", "x0"
);
}
}
#else /* defined(__aarch64__) && defined(__APPLE__) */
static __attribute__((__always_inline__)) bool jit_write_protect_supported(void)
{
return false;
}
static __attribute__((__always_inline__)) void jit_write_protect(int enabled)
{
}
#endif
#endif /* define TCG_APPLE_JIT_H */
Then we have a successful jit allocation in the first process, but while it comes to stage 0, every allocation fails like that:
make[3]: Entering directory '/buildroot/guile-3.0.9/libguile'
cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc chars.doc control.doc continuations.doc debug.doc deprecated.doc deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=0 ../meta/build-env guild snarf-check-and-output-texi > guile-procedures.texi || { rm guile-procedures.texi; false; }
make[3]: Leaving directory '/buildroot/guile-3.0.9/libguile'
make[2]: Leaving directory '/buildroot/guile-3.0.9/libguile'
Making all in module
make[2]: Entering directory '/buildroot/guile-3.0.9/module'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/buildroot/guile-3.0.9/module'
Making all in stage0
make[2]: Entering directory '/buildroot/guile-3.0.9/stage0'
GUILE_BOOTSTRAP_STAGE=stage0 ../meta/build-env guild compile --target="aarch64-apple-darwin20.4.0" -W0 -O1 -L "/buildroot/guile-3.0.9/module" -o "ice-9/eval.go" "../module/ice-9/eval.scm"
allocating JIT code buffer failed: Invalid argument
JIT failed due to resource exhaustion
disabling automatic JIT compilation
The issue #47429 mentions a similar issue related with JIT allocation under Darwin20+, but alongside to the implementation of `allocate_code_arena` method in libguile/jit.c in 3.0.5 we don't have any processes regarding to the Darwin20+ JIT write protections, the culprit might different.
Now what I can confirm is that
- MAP_JIT is working on both macOS and iOS
- If not providing an alternative method of pthread_write_protect_np, we then should properly handle the flags that passing to mmaped pointers to make sure RWX does not applied same time when doing W/X operations.
- Current Guile does not have a working JIT on arm64 iOS if there's no HAVE_PTHREAD_JIT_WRITE_PROTECT_NP defined
[Message part 2 (text/html, inline)]
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.