GNU bug report logs -
#25930
optimize mv for multiple bind mounts
Previous Next
Reported by: Ruediger Meier <sweet_f_a <at> gmx.de>
Date: Thu, 2 Mar 2017 09:52:01 UTC
Severity: normal
Done: Assaf Gordon <assafgordon <at> gmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25930 in the body.
You can then email your comments to 25930 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Thu, 02 Mar 2017 09:52:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Ruediger Meier <sweet_f_a <at> gmx.de>
:
New bug report received and forwarded. Copy sent to
bug-coreutils <at> gnu.org
.
(Thu, 02 Mar 2017 09:52:03 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Hi,
I have two bind mounts of the same filesystem
$ grep "/tmp" /etc/fstab
/dev/vg0/tmpdirs /mnt/tmpdirs ext4 acl,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 1 2
/mnt/tmpdirs/tmp /tmp none bind 0 0
/mnt/tmpdirs/var/tmp /var/tmp none bind 0 0
Using mv to move files between the bind mounts makes a slow copy:
$ time mv /var/tmp/BIGFILE /tmp/
real 0m0.622s
user 0m0.001s
sys 0m0.621s
In theory mv could know somehow that's the same filesystem
and do it like this:
$ time mv /mnt/tmpdirs/tmp/BIGFILE /mnt/tmpdirs/var/tmp/
real 0m0.004s
user 0m0.001s
sys 0m0.003s
Would it be possible to optimize mv regarding bind mounts? Or perhaps
are there other filesystems which do this better than ext4? Specially
the case that the original whole filesystem (/mnt/tmpdirs) is umounted
should work too somewhow.
cu,
Rudi
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Thu, 02 Mar 2017 10:07:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 25930 <at> debbugs.gnu.org (full text, mbox):
Am 02.03.2017 um 10:51 schrieb Ruediger Meier:
> I have two bind mounts of the same filesystem
>
> $ grep "/tmp" /etc/fstab
> /dev/vg0/tmpdirs /mnt/tmpdirs ext4
> acl,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0
> 1 2
> /mnt/tmpdirs/tmp /tmp none bind 0 0
> /mnt/tmpdirs/var/tmp /var/tmp none bind 0 0
>
>
> Using mv to move files between the bind mounts makes a slow copy:
>
> $ time mv /var/tmp/BIGFILE /tmp/
>
> real 0m0.622s
> user 0m0.001s
> sys 0m0.621s
>
>
> In theory mv could know somehow that's the same filesystem
> and do it like this:
>
> $ time mv /mnt/tmpdirs/tmp/BIGFILE /mnt/tmpdirs/var/tmp/
>
> real 0m0.004s
> user 0m0.001s
> sys 0m0.003s
>
>
> Would it be possible to optimize mv regarding bind mounts?
Apparently not, according to the rename(2) manpage:
,----
| EXDEV oldpath and newpath are not on the same mounted filesystem.
| (Linux permits a filesystem to be mounted at multiple
| points, but rename() does not work across different mount
| points, even if the same filesystem is mounted on both.)
`----
Cheers,
Sven
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Thu, 02 Mar 2017 21:18:01 GMT)
Full text and
rfc822 format available.
Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):
Sven Joachim wrote:
> ,----
> | EXDEV oldpath and newpath are not on the same mounted filesystem.
> | (Linux permits a filesystem to be mounted at multiple
> | points, but rename() does not work across different mount
> | points, even if the same filesystem is mounted on both.)
>
----
That's unfortunate, as Windows recognizes moves between
the same device and does a rename vs. a copy (i.e. it doesn't
matter if the mounted object from the mount is different, as
long as the rename happens between the same devices).
Seems like the linux kernel should have a check for that case.
(I.e. coreutils isn't likely the right place to try to fix this, since
you'd want the 'rename' optimization to happen no matter what tool
you used.
Anyone know why Linux doesn't do detection by device vs.
by mount point? Both pieces of info have their use, but for rename
seems that 'by device' would be optimal.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Thu, 02 Mar 2017 22:47:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 25930 <at> debbugs.gnu.org (full text, mbox):
On 03/02/2017 10:16 PM, L A Walsh wrote:
> Anyone know why Linux doesn't do detection by device vs.
> by mount point? Both pieces of info have their use, but for rename
> seems that 'by device' would be optimal.
quick guess: because not only the device matters. What if e.g. the
other bind mount is read-only?
Have a nice day,
Berny
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Fri, 03 Mar 2017 02:25:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 25930 <at> debbugs.gnu.org (full text, mbox):
Bernhard Voelker wrote:
> On 03/02/2017 10:16 PM, L A Walsh wrote:
>> Anyone know why Linux doesn't do detection by device vs.
>> by mount point? Both pieces of info have their use, but for rename
>> seems that 'by device' would be optimal.
>
> quick guess: because not only the device matters. What if e.g. the
> other bind mount is read-only?
---
Not "cleanly" supported. mount man page:
Note that the filesystem mount options will remain the same as
those on the original mount point, and cannot be changed by
passing the -o option along with --bind/--rbind. The mount
options can be changed by a separate remount command, for exam-
ple:
mount --bind olddir newdir
mount -o remount,ro newdir
Note that the behavior of the remount operation depends on the
/etc/mtab file. The first command stores the 'bind' flag in the
/etc/mtab file and the second command reads the flag from the
file. If you have a system without the /etc/mtab file or if you
explicitly define source and target for the remount command
(then mount(8) does not read /etc/mtab), then you have to use
the bind flag (or option) for the remount command too. For
example:
mount --bind olddir newdir
mount -o remount,ro,bind olddir newdir
Note that remount,ro,bind will create a read-only mountpoint
(VFS entry), but the original filesystem superblock will still
be writable, meaning that the olddir will be writable, but the
newdir will be read-only.
----
It's a bit bothersome that the behavior of the remount
is dependent on existence of /etc/mtab, contents and mount order.
Interestingly, though, on Windows, I haven't found
any way of mounting a file system "Read-Only", so maybe that's
not something Windows normally has to worry about in moving files.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Sat, 04 Mar 2017 08:02:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 25930 <at> debbugs.gnu.org (full text, mbox):
On 2017-03-02 13:16 -0800, L A Walsh wrote:
> Sven Joachim wrote:
>> ,----
>> | EXDEV oldpath and newpath are not on the same mounted filesystem.
>> | (Linux permits a filesystem to be mounted at multiple
>> | points, but rename() does not work across different mount
>> | points, even if the same filesystem is mounted on both.)
>>
> ----
> That's unfortunate, as Windows recognizes moves between
> the same device and does a rename vs. a copy (i.e. it doesn't
> matter if the mounted object from the mount is different, as
> long as the rename happens between the same devices).
Linux used to do the same in kernel 2.2, but changed the behavior when
bind mounts were introduced in 2.4.0. In other words, it's deliberate.
> Anyone know why Linux doesn't do detection by device vs.
> by mount point? Both pieces of info have their use, but for rename
> seems that 'by device' would be optimal.
It is meant as a security measure, bind mounts give you the option to
confine file renames and hard links to a directory subtree even if that
directory does not live on its own filesystem. See this mail by Al Viro
on that matter: http://yarchive.net/comp/linux/bind_mounts.html.
Cheers,
Sven
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#25930
; Package
coreutils
.
(Mon, 29 Oct 2018 02:51:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 25930 <at> debbugs.gnu.org (full text, mbox):
close 25930
stop
(triaging old bugs)
On 2017-03-04 1:01 a.m., Sven Joachim wrote:
> On 2017-03-02 13:16 -0800, L A Walsh wrote:
>
>> Sven Joachim wrote:
>>> ,----
>>> | EXDEV oldpath and newpath are not on the same mounted filesystem.
>>> | (Linux permits a filesystem to be mounted at multiple
>>> | points, but rename() does not work across different mount
>>> | points, even if the same filesystem is mounted on both.)
>>>
>> ----
>> That's unfortunate, as Windows recognizes moves between
>> the same device and does a rename vs. a copy (i.e. it doesn't
>> matter if the mounted object from the mount is different, as
>> long as the rename happens between the same devices).
>
> Linux used to do the same in kernel 2.2, but changed the behavior when
> bind mounts were introduced in 2.4.0. In other words, it's deliberate.
>
Given the above (kernel limitation), I'm closing this bug.
Discussion can continue by replying to this thread.
-assaf
bug closed, send any further explanations to
25930 <at> debbugs.gnu.org and Ruediger Meier <sweet_f_a <at> gmx.de>
Request was from
Assaf Gordon <assafgordon <at> gmail.com>
to
control <at> debbugs.gnu.org
.
(Mon, 29 Oct 2018 02:51:02 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Mon, 26 Nov 2018 12:24:06 GMT)
Full text and
rfc822 format available.
This bug report was last modified 5 years and 123 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.