GNU bug report logs - #52193
mv broken on non-APFS filesystems on macOS on coreutils >= 9.0

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 52193 in the body.
You can then email your comments to 52193 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: bug-coreutils <at> gnu.org
Cc: Cameron Katri <me <at> cameronkatri.com>
Subject: mv broken on non-APFS filesystems on macOS on coreutils >= 9.0
Date: Mon, 29 Nov 2021 18:50:45 -0600

Hello,

It appears that in coreutils 9.0 and greater, the mv command is broken on non-APFS filesystems on macOS. For example, on Apple tmpfs, it fails whenever moving a file with ENOTSUPP. It works fine on APFS filesystems (which is the primary macOS filesystem), however, so I assume it slipped under the radar till now. Could this possibly be related to the previous cp problem (#51857)?

Thanks,
Sudhip

Message #8 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>, 52193 <at> debbugs.gnu.org
Cc: Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Mon, 29 Nov 2021 18:51:17 -0800

On 11/29/21 16:50, Sudhip Nashi via GNU coreutils Bug Reports wrote:

> It appears that in coreutils 9.0 and greater, the mv command is broken on non-APFS filesystems on macOS. For example, on Apple tmpfs, it fails whenever moving a file with ENOTSUPP.

I don't observe this problem with the latest development version of 
coreutils. Here's how I tried to reproduce the problem on macOS 11.3.1 
20E241:

% touch /tmp/xxxx
% src/mv /tmp/xxxx /tmp/yyyy

and it worked OK.

I just put a copy of the latest coreutils source here:

https://www.cs.ucla.edu/~eggert/coreutils-9.0.36-5e36c.tar.gz

Could you give it a try?

Possibly this is related to the other bug report you mention, since mv 
falls back on cp's algorithm. But if so, I hope that the recent fix to 
cp also fixed mv.

Message #11 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Mon, 29 Nov 2021 22:01:43 -0600

> I don't observe this problem with the latest development version of coreutils. Here's how I tried to reproduce the problem on macOS 11.3.1 20E241:
> 
> % touch /tmp/xxxx
> % src/mv /tmp/xxxx /tmp/yyyy
> 
> and it worked OK.

macOS is different in that /tmp isn’t actually on a tmpfs filesystem, but is rather just a normal directory on the main data volume that is cleaned on boot. One has to manually mount a tmpfs volume themselves with mount_tmpfs. Anyways, I’ll try the tarball linked and see if the issue still persists.

Message #14 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 52193 <at> debbugs.gnu.org,
 Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Sun, 5 Dec 2021 14:21:57 -0600

> macOS is different in that /tmp isn’t actually on a tmpfs filesystem, but is rather just a normal directory on the main data volume that is cleaned on boot. One has to manually mount a tmpfs volume themselves with mount_tmpfs. Anyways, I’ll try the tarball linked and see if the issue still persists.

Sorry for the late response, but it doesn’t look like the latest commit (the tarball you sent me) fixes the issue.

Message #17 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Sun, 5 Dec 2021 12:38:40 -0800

On 12/5/21 12:21, Sudhip Nashi wrote:
> it doesn’t look like the latest commit (the tarball you sent me) fixes the issue.

Can you give a way to reproduce the probglem on a macOS system, assuming 
one has a tmpfs filesystem /A and a non-tmpfs filesystem /B? The 
scenario wasn't entirely clear from the bug report.

Message #20 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Mon, 6 Dec 2021 22:40:16 -0600

> Can you give a way to reproduce the probglem on a macOS system, assuming one has a tmpfs filesystem /A and a non-tmpfs filesystem /B? The scenario wasn't entirely clear from the bug report.

Running the command “mount_tmpfs ~/mntpnt” mounts a tmpfs (and non CoW) filesystem at ~/mntpnt. The issue should be reproducible when moving directories and files within ~/mntpnt, but not between that filesystem and the parent APFS one, and vice versa.

Message #23 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Tue, 7 Dec 2021 10:51:06 -0800

On 12/6/21 20:40, Sudhip Nashi wrote:
> The issue should be reproducible when moving directories and files within ~/mntpnt, but not between that filesystem and the parent APFS one, and vice versa.
> 

Thanks, but as I don't have access to a macOS machine I don't know what 
"the issue" is. What are the symptoms visible to the user? Can you do 
the equivalent of an strace to show us what system calls are being executed?

As near as I can make out, on macOS mv should be doing the equivalent of 
the following:

   sfd = open ("source", O_RDONLY | O_NOFOLLOW);
   if (fclonefileat (sfd, AT_FDCWD, "destination", 0) != 0)
     {
        dfd = open ("destination", O_WRONLY | O_CREAT | O_EXCL, mode);
        next_start = lseek (sfd, 0, SEEK_DATA);
        ...

and evidently something is going wrong at or after the "...". What is it?

Message #26 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Wed, 8 Dec 2021 18:08:08 -0600

> Thanks, but as I don't have access to a macOS machine I don't know what "the issue" is. What are the symptoms visible to the user? Can you do the equivalent of an strace to show us what system calls are being executed?
> 
> As near as I can make out, on macOS mv should be doing the equivalent of the following:
> 
>   sfd = open ("source", O_RDONLY | O_NOFOLLOW);
>   if (fclonefileat (sfd, AT_FDCWD, "destination", 0) != 0)
>     {
>        dfd = open ("destination", O_WRONLY | O_CREAT | O_EXCL, mode);
>        next_start = lseek (sfd, 0, SEEK_DATA);
>        ...
> 
> and evidently something is going wrong at or after the "...". What is it?

Ah, my bad. The issue is that all `mv` operations within the mountpoint fail with an ENOTSUPP error. Here’s a dtruss for an example:

access("/AppleInternal/XBS/.isChrooted\0", 0x0, 0x0)		= -1 Err#2
bsdthread_register(0x1AEC802C8, 0x1AEC802BC, 0x4000)		= 1073742303 0
shm_open(0x1AEB48F55, 0x0, 0x45E4000)		= 3 0
fstat64(0x3, 0x16B81A100, 0x0)		= 0 0
mmap(0x0, 0x4000, 0x1, 0x40001, 0x3, 0x0)		= 0x104718000 0
close(0x3)		= 0 0
ioctl(0x2, 0x4004667A, 0x16B81A1AC)		= 0 0
mprotect(0x104724000, 0x4000, 0x0)		= 0 0
mprotect(0x104730000, 0x4000, 0x0)		= 0 0
mprotect(0x104734000, 0x4000, 0x0)		= 0 0
mprotect(0x104740000, 0x4000, 0x0)		= 0 0
mprotect(0x104744000, 0x4000, 0x0)		= 0 0
mprotect(0x104750000, 0x4000, 0x0)		= 0 0
mprotect(0x10471C000, 0x90, 0x1)		= 0 0
mprotect(0x10471C000, 0x90, 0x3)		= 0 0
mprotect(0x10471C000, 0x90, 0x1)		= 0 0
mprotect(0x104754000, 0x4000, 0x1)		= 0 0
mprotect(0x104758000, 0x90, 0x1)		= 0 0
mprotect(0x104758000, 0x90, 0x3)		= 0 0
mprotect(0x104758000, 0x90, 0x1)		= 0 0
mprotect(0x10471C000, 0x90, 0x3)		= 0 0
mprotect(0x10471C000, 0x90, 0x1)		= 0 0
mprotect(0x104754000, 0x4000, 0x3)		= 0 0
mprotect(0x104754000, 0x4000, 0x1)		= 0 0
objc_bp_assist_cfg_np(0x1AEB103C0, 0x8000000000201048, 0x0)		= -1 Err#5
issetugid(0x0, 0x0, 0x0)		= 0 0
getentropy(0x16B819FC8, 0x20, 0x0)		= 0 0
getentropy(0x16B81A018, 0x40, 0x0)		= 0 0
getpid(0x0, 0x0, 0x0)		= 1135 0
stat64("/AppleInternal\0", 0x16B81A710, 0x0)		= -1 Err#2
csops_audittoken(0x46F, 0x7, 0x16B81A240)		= 0 0
proc_info(0x2, 0x46F, 0xD)		= 64 0
csops_audittoken(0x46F, 0x7, 0x16B81A300)		= 0 0
sysctlbyname(kern.osvariant_status, 0x15, 0x16B81A778, 0x16B81A770, 0x0)		= 0 0
csops(0x46F, 0x0, 0x16B81A79C)		= 0 0
mprotect(0x104610000, 0x100000, 0x1)		= 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_COLLATE\0", 0x0, 0x0)		= 3 0
fcntl_nocancel(0x3, 0x3, 0x0)		= 0 0
getrlimit(0x1008, 0x16B81B148, 0x0)		= 0 0
fstat64(0x3, 0x16B81B0C0, 0x0)		= 0 0
read_nocancel(0x3, "1.1A\n\0", 0x1000)		= 2086 0
close_nocancel(0x3)		= 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_CTYPE\0", 0x0, 0x0)		= 3 0
fcntl_nocancel(0x3, 0x3, 0x0)		= 0 0
fstat64(0x3, 0x16B81B1F0, 0x0)		= 0 0
fstat64(0x3, 0x16B81AFE0, 0x0)		= 0 0
lseek(0x3, 0x0, 0x1)		= 0 0
lseek(0x3, 0x0, 0x0)		= 0 0
read_nocancel(0x3, "RuneMagAUTF-8\0", 0x1000)		= 4096 0
read_nocancel(0x3, "\0", 0x1000)		= 4096 0
read_nocancel(0x3, "\0", 0x1000)		= 4096 0
read_nocancel(0x3, "\0", 0x1000)		= 4096 0
read_nocancel(0x3, "\0", 0x1000)		= 4096 0
read_nocancel(0x3, "\0", 0x1000)		= 4096 0
read_nocancel(0x3, "\0", 0x1000)		= 4096 0
read_nocancel(0x3, "@\004\211\0", 0xF5D0)		= 62928 0
close_nocancel(0x3)		= 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MONETARY\0", 0x0, 0x0)		= 3 0
fstat64(0x3, 0x16B81B210, 0x0)		= 0 0
read_nocancel(0x3, "USD \n$\n.\n,\n3;3\n\n-\n2\n2\n1\n0\n1\n0\n1\n1\n(\0", 0x22)		= 34 0
close_nocancel(0x3)		= 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_NUMERIC\0", 0x0, 0x0)		= 3 0
fstat64(0x3, 0x16B81B210, 0x0)		= 0 0
read_nocancel(0x3, ".\n,\n3;3\n@$\b\0", 0x8)		= 8 0
close_nocancel(0x3)		= 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_TIME\0", 0x0, 0x0)		= 3 0
fstat64(0x3, 0x16B81B220, 0x0)		= 0 0
read_nocancel(0x3, "Jan\nFeb\nMar\nApr\nMay\nJun\nJul\nAug\nSep\nOct\nNov\nDec\nJanuary\nFebruary\nMarch\nApril\nMay\nJune\nJuly\nAugust\nSeptember\nOctober\nNovember\nDecember\nSun\nMon\nTue\nWed\nThu\nFri\nSat\nSunday\nMonday\nTuesday\nWednesday\nThursday\nFriday\nSaturday\n%H:%M:%S\n%m/%d/%Y\n%a %b %e %X %Y\nAM\nP", 0x179)		= 377 0
close_nocancel(0x3)		= 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/LC_MESSAGES\0", 0x0, 0x0)		= 3 0
fstat64(0x3, 0x16B81B220, 0x0)		= 0 0
read_nocancel(0x3, "^[yYsS].*\n^[nN].*\n(\0", 0x12)		= 18 0
close_nocancel(0x3)		= 0 0
geteuid(0x0, 0x0, 0x0)		= 0 0
ioctl(0x0, 0x4004667A, 0x16B81B77C)		= 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16B81BC34, 0x16B81B670)		= -1 Err#2
renameatx_np(0xFFFFFFFFFFFFFFFE, "test1\0", 0xFFFFFFFFFFFFFFFE, "test2\0", 0|0|RENAME_EXCL)		= -1 Err#45
stat64("test2\0", 0x16B81B878, 0x0)		= -1 Err#2
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16B81BC2E, 0x16B81B480)		= 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16B81BC34, 0x16B81B3F0)		= -1 Err#2
fcntl(0x1, 0x3, 0x6B81B320)		= 65538 0
write_nocancel(0x2, "gmv: \0", 0x5)		= 5 0
write_nocancel(0x2, "cannot move 'test1' to 'test2'\0", 0x1E)		= 30 0
write_nocancel(0x2, ": Operation not supported\0", 0x19)		= 25 0
write_nocancel(0x2, "\n\0", 0x1)		= 1 0
lseek(0x0, 0x0, 0x1)		= 34002 0
lseek(0x0, 0x0, 0x1)		= 34002 0
lseek(0x0, 0x84D2, 0x0)		= 34002 0
close_nocancel(0x0)		= 0 0
close_nocancel(0x1)		= 0 0
close_nocancel(0x2)		= 0 0

Message #29 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Wed, 8 Dec 2021 19:17:56 -0800

[Message part 1 (text/plain, inline)]

On 12/8/21 16:08, Sudhip Nashi wrote:

> The issue is that all `mv` operations within the mountpoint fail with an ENOTSUPP error.

Thanks, I think I see the problem now. Please try the attached patch; I 
haven't tested myself as I lack access to macOS. Thanks.

[0001-renameatu-port-to-macOS-tmpfs.patch (text/x-patch, attachment)]

Message #32 received at 52193 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 52193 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Sun, 12 Dec 2021 11:06:37 -0600

> Thanks, I think I see the problem now. Please try the attached patch; I haven't tested myself as I lack access to macOS. Thanks.
> <0001-renameatu-port-to-macOS-tmpfs.patch>

It looks like this patch solves the problem.

Message #37 received at 52193-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: Cameron Katri <me <at> cameronkatri.com>, 52193-done <at> debbugs.gnu.org
Subject: Re: bug#52193: mv broken on non-APFS filesystems on macOS on
 coreutils >= 9.0
Date: Tue, 14 Dec 2021 12:48:17 -0800

[Message part 1 (text/plain, inline)]

On 12/12/21 09:06, Sudhip Nashi wrote:
> 
>> Thanks, I think I see the problem now. Please try the attached patch; I haven't tested myself as I lack access to macOS. Thanks.
>> <0001-renameatu-port-to-macOS-tmpfs.patch>
> 
> It looks like this patch solves the problem.
> 

Thanks, I installed the attached to implement this fix, the first patch 
into Gnulib, the second into coreutils.

[0001-renameatu-port-to-macOS-tmpfs.patch (text/x-patch, attachment)]

[0001-build-update-gnulib-submodule-to-latest.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.