GNU logs - #17676, boring messages

Message sent to bug-coreutils@HIDDEN:

Subject: bug#17676: btrfs subvolumes and bind-mounts make df report incorrect and/or extra results
Resent-From: David Schleimer <dschleimer@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Tue, 03 Jun 2014 15:35:02 +0000
Resent-Message-ID: <handler.17676.B.140180967829594 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 17676 <at> debbugs.gnu.org
From: David Schleimer <dschleimer@HIDDEN>
Thread-Topic: btrfs subvolumes and bind-mounts make df report incorrect
 and/or extra results
Thread-Index: Ac9/P2JUaeigNDLcSXifOx2sy/LiEg==
Date: Tue, 3 Jun 2014 15:21:15 +0000
Message-ID: <B647040404E33F408448F7343768AE7A76F70BC5@HIDDEN>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/mixed;
 boundary="_002_B647040404E33F408448F7343768AE7A76F70BC5PRNMBX011TheFac_"
MIME-Version: 1.0
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

--_002_B647040404E33F408448F7343768AE7A76F70BC5PRNMBX011TheFac_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

When you have btrfs mount where there is a both a subvolume of that mount, =
and the source of a bind-mount under that mount, df will report confusing r=
esults.  It will show results for the bind-mount instead of the main btrfs =
mount when you pass the subvoulme on the command line.  It may (depending o=
n the system) show both the bind-mount and main mount in bare df results.

I've attached a script with the minimal repro instructions.  It needs to be=
 run as root since it (temporarily) mounts filesystems under your tmpdir.

I tested on two machines:
Fedora 20 running kernel 3.14.4-200.fc20.x86_64, against both the system df=
 8.21 and df 8.22 built from source.

Centos 6.4 running a 3.10.39 variant, with significant backports, notably i=
ncluding most btrfs changes from mainline.  Testing against the system df 8=
.4, and df 8.22 again built from source.

On the fedora machine, /etc/mtab is a symlink to /proc/self/mounts.  We see=
 output that refers to the correct block device in all cases.  However, we =
see a report for the bind-mount destination instead of the main btrfs mount=
 when asking specifically for the subvolume path, and see results for both =
the main btrfs filesystem and the bind-mount when we run a bare df.

Transcript of Fedora output:

[root@caitsidhe dschleimer]# ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.00963354 s, 1.1 GB/s SMALL VOLUME: forcing=
 mixed metadata/data groups

WARNING! - Btrfs v3.12 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

Turning ON incompat feature 'mixed-bg': mixed data and metadata block group=
s Turning ON incompat feature 'extref': increased hardlink limit per file t=
o 65536 Created a data/metadata chunk of size 1048576 fs created label (nul=
l) on block
        nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.=
12 Create subvolume 'filesystem/subvolume'



Begin unexpected output
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/loop0         10240    36      6112   1% /tmp/tmp.biUryZcwD9/binddest
Expected report for the main btrfs mount, not the bind-mount
/dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp=
.biUryZcwD9/filesystem
/dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp=
.biUryZcwD9/binddest
Expected only one btrfs filesystem
/home/dschleimer
[root@caitsidhe dschleimer]# PATH=3D/home/dschleimer/Downloads/coreutils-8.=
22/src/:$PATH ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.012752 s, 822 MB/s SMALL VOLUME: forcing m=
ixed metadata/data groups

WARNING! - Btrfs v3.12 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

Turning ON incompat feature 'mixed-bg': mixed data and metadata block group=
s Turning ON incompat feature 'extref': increased hardlink limit per file t=
o 65536 Created a data/metadata chunk of size 1048576 fs created label (nul=
l) on block
        nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.=
12 Create subvolume 'filesystem/subvolume'



Begin unexpected output
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/loop0         10240    36      6112   1% /tmp/tmp.TEBR2D9Vta/binddest
Expected report for the main btrfs mount, not the bind-mount
/dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp=
.TEBR2D9Vta/filesystem
/dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp=
.TEBR2D9Vta/binddest
Expected only one btrfs filesystem
/home/dschleimer

On Centos, /etc/mtab is a plain file.  When asking specifically for the sub=
volume, we see a report for the bind-mount dest which lists the bind-mount =
source as the filesystem.  When we run a bare df, we see only one record wi=
th the file backing the loopback device as the filesystem, but the bind-mou=
nt as the mount-point.

Centos transcript:

[08:03:09 Tue Jun 03 2014] root@HIDDEN /home/dschleimer dschleimer =
 282 $ ./repro_instructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.0101568 s, 1.0 GB/s

WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http=
://btrfs.wiki.kernel.org before using

SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata ch=
unk of size 1048576 fs created label (null) on block
        nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.2=
0-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'



Begin unexpected output
Filesystem           1K-blocks      Used Available Use% Mounted on
/tmp/tmp.V4FNWcBXke/filesystem/subvolume/bindsource
                         10240        36      6112   1% /tmp/tmp.V4FNWcBXke=
/binddest
Expected report for the main btrfs mount, not the bind-mount
df: `/mnt/gvfs': Function not implemented /tmp/tmp.V4FNWcBXke/block
             btrfs       10240        36      6112   1% /tmp/tmp.V4FNWcBXke=
/filesystem
Expected only one btrfs filesystem
/home/dschleimer

[08:09:39 Tue Jun 03 2014] root@HIDDEN /home/dschleimer dschleimer =
 282 $ PATH=3D/data/users/dschleimer/coreutils-8.22/src/:$PATH ./repro_inst=
ructions.sh
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 0.0116229 s, 902 MB/s

WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http=
://btrfs.wiki.kernel.org before using

SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata ch=
unk of size 1048576 fs created label (null) on block
        nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.2=
0-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'



Begin unexpected output
Filesystem                                          1K-blocks  Used Availab=
le Use% Mounted on
/tmp/tmp.o7w6NcN178/filesystem/subvolume/bindsource     10240    36      61=
12   1% /tmp/tmp.o7w6NcN178/binddest
Expected report for the main btrfs mount, not the bind-mount
/tmp/tmp.o7w6NcN178/block                        btrfs      10240         3=
6       6112   1% /tmp/tmp.o7w6NcN178/filesystem
Expected only one btrfs filesystem
/home/dschleimer



--_002_B647040404E33F408448F7343768AE7A76F70BC5PRNMBX011TheFac_
Content-Type: application/octet-stream; name="repro_instructions.sh"
Content-Description: repro_instructions.sh
Content-Disposition: attachment; filename="repro_instructions.sh"; size=606;
	creation-date="Tue, 03 Jun 2014 15:21:04 GMT";
	modification-date="Tue, 03 Jun 2014 15:03:52 GMT"
Content-Transfer-Encoding: base64

IyEvYmluL2Jhc2gKClJFUFJPX0RJUj0kKG1rdGVtcCAtZCkKY2QgJHtSRVBST19ESVJ9CgpkZCBp
Zj0vZGV2L3plcm8gb2Y9YmxvY2sgYnM9MTAyNDAgY291bnQ9MTAyNApta2ZzLmJ0cmZzIGJsb2Nr
Cgpta2RpciBmaWxlc3lzdGVtCm1vdW50IC10IGJ0cmZzIC1vIGxvb3AgYmxvY2sgZmlsZXN5c3Rl
bQoKYnRyZnMgc3Vidm9sdW1lIGNyZWF0ZSBmaWxlc3lzdGVtL3N1YnZvbHVtZQpta2RpciBmaWxl
c3lzdGVtL3N1YnZvbHVtZS97ZGlyZWN0b3J5LGJpbmRzb3VyY2V9Cgpta2RpciBiaW5kZGVzdApt
b3VudCAtLWJpbmQgZmlsZXN5c3RlbS9zdWJ2b2x1bWUvYmluZHNvdXJjZSBiaW5kZGVzdAoKZWNo
bwplY2hvCmVjaG8KZWNobyAiQmVnaW4gdW5leHBlY3RlZCBvdXRwdXQiCmRmIGZpbGVzeXN0ZW0v
c3Vidm9sdW1lL2RpcmVjdG9yeQplY2hvICJFeHBlY3RlZCByZXBvcnQgZm9yIHRoZSBtYWluIGJ0
cmZzIG1vdW50LCBub3QgdGhlIGJpbmQtbW91bnQiCmRmIC1UIHwgZ3JlcCAkUFdECmVjaG8gIkV4
cGVjdGVkIG9ubHkgb25lIGJ0cmZzIGZpbGVzeXN0ZW0iCgp1bW91bnQgYmluZGRlc3QKdW1vdW50
IGZpbGVzeXN0ZW0KY2QgLQpybSAtciAke1JFUFJPX0RJUn0K

--_002_B647040404E33F408448F7343768AE7A76F70BC5PRNMBX011TheFac_--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.503 (Entity 5.503)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: David Schleimer <dschleimer@HIDDEN>
Subject: bug#17676: Acknowledgement (btrfs subvolumes and bind-mounts make
 df report incorrect and/or extra results)
Message-ID: <handler.17676.B.140180967829594.ack <at> debbugs.gnu.org>
References: <B647040404E33F408448F7343768AE7A76F70BC5@HIDDEN>
X-Gnu-PR-Message: ack 17676
X-Gnu-PR-Package: coreutils
Reply-To: 17676 <at> debbugs.gnu.org
Date: Tue, 03 Jun 2014 15:35:03 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-coreutils@HIDDEN

If you wish to submit further information on this problem, please
send it to 17676 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
17676: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D17676
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-coreutils@HIDDEN:

Subject: bug#17676: btrfs subvolumes and bind-mounts make df report incorrect and/or extra results
Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady <P@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Wed, 04 Jun 2014 01:53:02 +0000
Resent-Message-ID: <handler.17676.B17676.140184672315060 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: David Schleimer <dschleimer@HIDDEN>
Cc: 17676 <at> debbugs.gnu.org
Message-ID: <538E7BAF.1090707@HIDDEN>
Date: Wed, 04 Jun 2014 02:51:43 +0100
From: =?UTF-8?Q?P=C3=A1draig?= Brady <P@HIDDEN>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:17.0) Gecko/20130110 Thunderbird/17.0.2
MIME-Version: 1.0
References: <B647040404E33F408448F7343768AE7A76F70BC5@HIDDEN>
In-Reply-To: <B647040404E33F408448F7343768AE7A76F70BC5@HIDDEN>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

On 06/03/2014 04:21 PM, David Schleimer wrote:
> When you have btrfs mount where there is a both a subvolume of that mount, and the source of a bind-mount under that mount, df will report confusing results.  It will show results for the bind-mount instead of the main btrfs mount when you pass the subvoulme on the command line.  It may (depending on the system) show both the bind-mount and main mount in bare df results.
> 
> I've attached a script with the minimal repro instructions.  It needs to be run as root since it (temporarily) mounts filesystems under your tmpdir.
> 
> I tested on two machines:
> Fedora 20 running kernel 3.14.4-200.fc20.x86_64, against both the system df 8.21 and df 8.22 built from source.
> 
> Centos 6.4 running a 3.10.39 variant, with significant backports, notably including most btrfs changes from mainline.  Testing against the system df 8.4, and df 8.22 again built from source.
> 
> On the fedora machine, /etc/mtab is a symlink to /proc/self/mounts.  We see output that refers to the correct block device in all cases.  However, we see a report for the bind-mount destination instead of the main btrfs mount when asking specifically for the subvolume path, and see results for both the main btrfs filesystem and the bind-mount when we run a bare df.
> 
> Transcript of Fedora output:
> 
> [root@caitsidhe dschleimer]# ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.00963354 s, 1.1 GB/s SMALL VOLUME: forcing mixed metadata/data groups
> 
> WARNING! - Btrfs v3.12 IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
> 
> Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 Created a data/metadata chunk of size 1048576 fs created label (null) on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem     1K-blocks  Used Available Use% Mounted on
> /dev/loop0         10240    36      6112   1% /tmp/tmp.biUryZcwD9/binddest
> Expected report for the main btrfs mount, not the bind-mount
> /dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp.biUryZcwD9/filesystem
> /dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp.biUryZcwD9/binddest
> Expected only one btrfs filesystem
> /home/dschleimer
> [root@caitsidhe dschleimer]# PATH=/home/dschleimer/Downloads/coreutils-8.22/src/:$PATH ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.012752 s, 822 MB/s SMALL VOLUME: forcing mixed metadata/data groups
> 
> WARNING! - Btrfs v3.12 IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
> 
> Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 Created a data/metadata chunk of size 1048576 fs created label (null) on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem     1K-blocks  Used Available Use% Mounted on
> /dev/loop0         10240    36      6112   1% /tmp/tmp.TEBR2D9Vta/binddest
> Expected report for the main btrfs mount, not the bind-mount
> /dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp.TEBR2D9Vta/filesystem
> /dev/loop0              btrfs        10240       36      6112   1% /tmp/tmp.TEBR2D9Vta/binddest
> Expected only one btrfs filesystem
> /home/dschleimer
> 
> On Centos, /etc/mtab is a plain file.  When asking specifically for the subvolume, we see a report for the bind-mount dest which lists the bind-mount source as the filesystem.  When we run a bare df, we see only one record with the file backing the loopback device as the filesystem, but the bind-mount as the mount-point.
> 
> Centos transcript:
> 
> [08:03:09 Tue Jun 03 2014] root@HIDDEN /home/dschleimer dschleimer  282 $ ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.0101568 s, 1.0 GB/s
> 
> WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using
> 
> SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /tmp/tmp.V4FNWcBXke/filesystem/subvolume/bindsource
>                          10240        36      6112   1% /tmp/tmp.V4FNWcBXke/binddest
> Expected report for the main btrfs mount, not the bind-mount
> df: `/mnt/gvfs': Function not implemented /tmp/tmp.V4FNWcBXke/block
>              btrfs       10240        36      6112   1% /tmp/tmp.V4FNWcBXke/filesystem
> Expected only one btrfs filesystem
> /home/dschleimer
> 
> [08:09:39 Tue Jun 03 2014] root@HIDDEN /home/dschleimer dschleimer  282 $ PATH=/data/users/dschleimer/coreutils-8.22/src/:$PATH ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.0116229 s, 902 MB/s
> 
> WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using
> 
> SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata chunk of size 1048576 fs created label (null) on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem                                          1K-blocks  Used Available Use% Mounted on
> /tmp/tmp.o7w6NcN178/filesystem/subvolume/bindsource     10240    36      6112   1% /tmp/tmp.o7w6NcN178/binddest
> Expected report for the main btrfs mount, not the bind-mount
> /tmp/tmp.o7w6NcN178/block                        btrfs      10240         36       6112   1% /tmp/tmp.o7w6NcN178/filesystem
> Expected only one btrfs filesystem
> /home/dschleimer


So the crux of the confusion is that the subvolume has a different device ID.
I.E. a separate virtual file system within the main file system,
that shares storage but are otherwise treated like separate file systems
for hard links etc. Here are the device IDs for your test setup:

  # find -printf "%D %p\n"
  34 .
  63 ./binddest
  62 ./filesystem
  63 ./filesystem/subvolume
  63 ./filesystem/subvolume/directory
  63 ./filesystem/subvolume/bindsource
  34 ./block

When you expose that virtual file system to df
with a bind mount it will be identified separately as you've shown.
BTW it's not specific to bind mounts, as you can see the same
thing by mounting the subvolume separately like:
  mkdir submntdest
  mount -o subvolid=256 -o loop block submntdest
It could be argued that since there are explicit mounts to this
subvolume that one (the shortest) of these is the most appropriate to display?

If you don't mount separately, then df notices that ./filesystem/subvolume is
a "separate mount" due to the different device id, and will display that
as the mount dir and "-" as the device which isn't ideal either.

It's more problematic that the btrfs file system is listed
multiple times, especially with --total which will be incorrect.
We might be able to avoid this problem by taking the first
entry for duplicate device names, when those device names exist.
However I'm worried that there are cases where separate file systems
with separate storage (distinguished by mount options) have the
same device listed. Maybe we could suppress these duplicates by
scanning /proc/self/mountinfo and discarding entries that didn't
have '/' in the fourth column, though that's awkward and I'm
not sure how general it is either.

thanks,
P�draig.

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.