GNU bug report logs - #44631
28.0.50; Byte compilation fails if destination file is a mount point

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 44631 in the body.
You can then email your comments to 44631 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Byte compilation fails if destination file is a mount point
Date: Sat, 14 Nov 2020 13:39:56 +0100

$ mkdir /tmp/mount
$ touch /tmp/mount/{file.el,file.elc,out.elc}
$ sudo mount --bind /tmp/mount/{out,file}.elc
$ emacs -Q -batch -f batch-byte-compile /tmp/mount/file.el
>>Error occurred processing /tmp/mount/file.el: File error (("Renaming" "Device or resource busy" "/tmp/mount/file.elc91nN38" "/tmp/mount/file.elc"))
Removing old name: Device or resource busy, /tmp/mount/file.elc

Granted, this is rather exotic.  Nevertheless, byte-compile-file could
fall back to a direct write-region without temporary file + rename, or
rename-file could fall back to copy + remove on EBUSY (like EXDEV).


In GNU Emacs 28.0.50 (build 123, x86_64-pc-linux-gnu, GTK+ Version 3.24.22, cairo version 1.16.0)
 of 2020-11-14
Repository revision: b13e87c35bb0215c211a456b8bc3184bf16eb5de
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux rodete

Configured using:
 'configure --enable-gcc-warnings=warn-only
 --enable-gtk-deprecation-warnings --without-pop --with-mailutils
 --enable-checking=all --enable-check-lisp-object-type --with-modules
 'CFLAGS=-O0 -ggdb3''

Configured features:
XPM JPEG TIFF GIF PNG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY
LIBSELINUX GNUTLS FREETYPE HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS GTK3 X11
XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER

Important settings:
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc dired dired-loaddefs rfc822
mml easymenu mml-sec epa epg epg-config gnus-util rmail rmail-loaddefs
time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils phst skeleton derived edmacro kmacro pcase ffap
thingatpt url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json map url-vars
mailcap rx gnutls puny dbus xml subr-x seq byte-opt gv bytecomp
byte-compile cconv compile text-property-search comint ansi-color ring
cl-loaddefs cl-lib tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 69605 6950)
 (symbols 48 8580 1)
 (strings 32 24388 1373)
 (string-bytes 1 790285)
 (vectors 16 13889)
 (vector-slots 8 188851 5441)
 (floats 8 26 30)
 (intervals 56 212 0)
 (buffers 992 11))

-- 
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich.  Falls Sie diese fälschlicherweise erhalten haben
sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie
alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail
an die falsche Person gesendet wurde.

This e-mail is confidential.  If you received this communication by mistake,
please don’t forward it to anyone else, please erase all copies and
attachments, and please let me know that it has gone to the wrong person.

Message #8 received at 44631 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 44631 <at> debbugs.gnu.org
Subject: Re: bug#44631: 28.0.50; Byte compilation fails if destination file
 is a mount point
Date: Sat, 14 Nov 2020 17:04:04 +0100

Philipp Stephani <p.stephani2 <at> gmail.com> writes:

> Granted, this is rather exotic.

It's really exotic.  :-)  I'm just wondering -- did you happen upon this
while trying to do something, or is it just designed to trigger the bug?

> Nevertheless, byte-compile-file could
> fall back to a direct write-region without temporary file + rename, or
> rename-file could fall back to copy + remove on EBUSY (like EXDEV).

Hm...  the latter sounds like an easy fix, but aren't there more commond
cases where you get EBUSY where copy + remove would fail, too?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #11 received at 44631 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 44631 <at> debbugs.gnu.org
Subject: Re: bug#44631: 28.0.50; Byte compilation fails if destination file is
 a mount point
Date: Sun, 22 Nov 2020 22:03:17 +0100

Am Sa., 14. Nov. 2020 um 17:04 Uhr schrieb Lars Ingebrigtsen <larsi <at> gnus.org>:
>
> Philipp Stephani <p.stephani2 <at> gmail.com> writes:
>
> > Granted, this is rather exotic.
>
> It's really exotic.  :-)  I'm just wondering -- did you happen upon this
> while trying to do something, or is it just designed to trigger the bug?

I've been experimenting with creating a sandbox for Emacs using
https://developers.google.com/sandboxed-api/. That works by (a)
installing a Linux kernel syscall filter to limit the allowed
syscalls, and (b) enabling user and mount namespaces. Using the mount
namespaces, the sandbox can be restricted exactly to the files that
Emacs needs to access. In the case of byte compilation, it needs to
write exactly one file, the byte-compiled output, so it's possible to
mount exactly that file. This bug makes this approach impossible.
(That's not a really onerous restriction; one can e.g. mount an empty
directory, and then have the output file written there. It would just
be nicer if it worked without such workarounds.)

>
> > Nevertheless, byte-compile-file could
> > fall back to a direct write-region without temporary file + rename, or
> > rename-file could fall back to copy + remove on EBUSY (like EXDEV).
>
> Hm...  the latter sounds like an easy fix, but aren't there more commond
> cases where you get EBUSY where copy + remove would fail, too?

Probably, but I guess the situation wouldn't become worse, as
presumably these cases already fail today.

Message #14 received at 44631 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 44631 <at> debbugs.gnu.org
Subject: Re: bug#44631: 28.0.50; Byte compilation fails if destination file
 is a mount point
Date: Tue, 24 Nov 2020 07:16:24 +0100

Philipp Stephani <p.stephani2 <at> gmail.com> writes:

> I've been experimenting with creating a sandbox for Emacs using
> https://developers.google.com/sandboxed-api/. That works by (a)
> installing a Linux kernel syscall filter to limit the allowed
> syscalls, and (b) enabling user and mount namespaces. Using the mount
> namespaces, the sandbox can be restricted exactly to the files that
> Emacs needs to access. In the case of byte compilation, it needs to
> write exactly one file, the byte-compiled output, so it's possible to
> mount exactly that file. This bug makes this approach impossible.

Hm, interesting...

>> > Nevertheless, byte-compile-file could
>> > fall back to a direct write-region without temporary file + rename, or
>> > rename-file could fall back to copy + remove on EBUSY (like EXDEV).
>>
>> Hm...  the latter sounds like an easy fix, but aren't there more commond
>> cases where you get EBUSY where copy + remove would fail, too?
>
> Probably, but I guess the situation wouldn't become worse, as
> presumably these cases already fail today.

No, I can't think of any cases where this would lead to a worse result
than the current situation, but it's somewhat scary making a low-level
general change like this in `rename-file' -- I have no idea whether
EBUSY is used for other oddball failure cases like this.

Making the change in byte-compile-file sounds safer, but would require
more work.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #17 received at 44631 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 44631 <at> debbugs.gnu.org
Subject: Re: bug#44631: 28.0.50; Byte compilation fails if destination file is
 a mount point
Date: Sat, 12 Dec 2020 16:19:27 +0100

Am Di., 24. Nov. 2020 um 07:16 Uhr schrieb Lars Ingebrigtsen <larsi <at> gnus.org>:

> >> > Nevertheless, byte-compile-file could
> >> > fall back to a direct write-region without temporary file + rename, or
> >> > rename-file could fall back to copy + remove on EBUSY (like EXDEV).
> >>
> >> Hm...  the latter sounds like an easy fix, but aren't there more commond
> >> cases where you get EBUSY where copy + remove would fail, too?
> >
> > Probably, but I guess the situation wouldn't become worse, as
> > presumably these cases already fail today.
>
> No, I can't think of any cases where this would lead to a worse result
> than the current situation, but it's somewhat scary making a low-level
> general change like this in `rename-file' -- I have no idea whether
> EBUSY is used for other oddball failure cases like this.
>
> Making the change in byte-compile-file sounds safer, but would require
> more work.

I guess it also depends on what the semantics are that
byte-compile-file guarantees. If it attempts to guarantee atomicity,
then only a intra-filesystem rename (or similar alternatives such as
O_TMPFILE + linkat) are acceptable, and technically, not even the
current fallback on EXDEV (which makes rename-file nonatomic) is OK.
If atomic writes are best-effort, then we could always fall back to
copy-file + delete-file on any file-error.

Message #20 received at 44631 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 44631 <at> debbugs.gnu.org
Subject: Re: bug#44631: 28.0.50; Byte compilation fails if destination file
 is a mount point
Date: Sun, 13 Dec 2020 13:32:37 +0100

Philipp Stephani <p.stephani2 <at> gmail.com> writes:

> I guess it also depends on what the semantics are that
> byte-compile-file guarantees. If it attempts to guarantee atomicity,
> then only a intra-filesystem rename (or similar alternatives such as
> O_TMPFILE + linkat) are acceptable, and technically, not even the
> current fallback on EXDEV (which makes rename-file nonatomic) is OK.
> If atomic writes are best-effort, then we could always fall back to
> copy-file + delete-file on any file-error.

It's definitely best-effort, so altering byte-compile-file to try 
copy-file + delete-file sounds like the best solution to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #25 received at 44631-done <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 44631-done <at> debbugs.gnu.org
Subject: Re: bug#44631: 28.0.50; Byte compilation fails if destination file is
 a mount point
Date: Sun, 13 Dec 2020 17:41:55 +0100

Am So., 13. Dez. 2020 um 13:32 Uhr schrieb Lars Ingebrigtsen <larsi <at> gnus.org>:
>
> Philipp Stephani <p.stephani2 <at> gmail.com> writes:
>
> > I guess it also depends on what the semantics are that
> > byte-compile-file guarantees. If it attempts to guarantee atomicity,
> > then only a intra-filesystem rename (or similar alternatives such as
> > O_TMPFILE + linkat) are acceptable, and technically, not even the
> > current fallback on EXDEV (which makes rename-file nonatomic) is OK.
> > If atomic writes are best-effort, then we could always fall back to
> > copy-file + delete-file on any file-error.
>
> It's definitely best-effort, so altering byte-compile-file to try
> copy-file + delete-file sounds like the best solution to me.

I've now pushed a slightly different fix to master (commit
fe50a8b9ba79b4ac14a3a352d8bf84eaee4f2122).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.