GNU bug report logs - #53037
df/total-verify fail with cephfs

Previous Next

To reply to this bug, email your comments to 53037 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dylan Simon <dylan <at> dylex.net>
To: bug-coreutils <at> gnu.org
Subject: df/total-verify fail with cephfs
Date: Wed, 5 Jan 2022 14:27:59 -0500

We have a filesystem that reports statfs f_files = nfiles, f_ffree = -1 
(UINTMAX_MAX).  (See rationale https://github.com/ceph/ceph/pull/36127)
Unfortunately this breaks df -i --total and in particular the df/total-verify
test fails.  Example output:

> df -i --total / /mnt/ceph
Filesystem        Inodes     IUsed    IFree IUse% Mounted on
/dev/sda1       20971520    529832 20441688    3% /
ceph           901006598         -        -     - /mnt/ceph
total          921978118 901536430 20441688   98% -

You can see that the total used is clearly not the computed total.  I think
this is because the individual rows use this logic for used:

  iv->used = UINTMAX_MAX;
  if (known_value (iv->total) && known_value (iv->available_to_root))
      iv->used = iv->total - iv->available_to_root;

while add_to_grand_total does them separately:

  if (known_value (iv->total))
    grand_fsu.fsu_files += iv->total;
  if (known_value (iv->available))
    grand_fsu.fsu_ffree += iv->available;

So the ceph line gets added to total but not available, and as a result, the
used computation of fsu_files - fsu_ffree = root.total + ceph.total -
root.available.  Maybe this is sensible but it's not what the test expects.

I can't think of a way to solve this that makes the test pass, aside from
collecting a grand_used total as well.  Only adding rows with all known values
might make sense but would still break the test (wrong total total instead):

  if (known_value (iv->total) && known_value (iv->available)) {
    grand_fsu.fsu_files += iv->total;
    grand_fsu.fsu_ffree += iv->available;
  }

Message #8 received at 53037 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Dylan Simon <dylan <at> dylex.net>, 53037 <at> debbugs.gnu.org
Subject: Re: bug#53037: df/total-verify fail with cephfs
Date: Wed, 5 Jan 2022 13:05:03 -0800

On 1/5/22 11:27, Dylan Simon wrote:
> Only adding rows with all known values
> might make sense but would still break the test (wrong total total instead):
> 
>    if (known_value (iv->total) && known_value (iv->available)) {
>      grand_fsu.fsu_files += iv->total;
>      grand_fsu.fsu_ffree += iv->available;
>    }

Sorry, I'm not quite following. If you make the above change, what will 
the output look like instead? And how will that break the test?

Message #11 received at 53037 <at> debbugs.gnu.org (full text, mbox):

From: Dylan Simon <dylan <at> dylex.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 53037 <at> debbugs.gnu.org
Subject: Re: bug#53037: df/total-verify fail with cephfs
Date: Wed, 5 Jan 2022 17:11:29 -0500

From Paul Eggert <eggert <at> cs.ucla.edu>, Wed, Jan 05, 2022 at 01:05:03PM -0800:
> On 1/5/22 11:27, Dylan Simon wrote:
> > Only adding rows with all known values
> > might make sense but would still break the test (wrong total total instead):
> > 
> >    if (known_value (iv->total) && known_value (iv->available)) {
> >      grand_fsu.fsu_files += iv->total;
> >      grand_fsu.fsu_ffree += iv->available;
> >    }
> 
> Sorry, I'm not quite following. If you make the above change, what will the
> output look like instead? And how will that break the test?

Then it will look like this (I'm inferring, haven't actually tried it):

Filesystem        Inodes     IUsed    IFree IUse% Mounted on
/dev/sda1       20971520    529832 20441688    3% /
ceph           901006598         -        -     - /mnt/ceph
total           20971520    529832 20441688    3% -

That is, the ceph line will just be excluded completely from the total.  Now
the used and free lines add up correctly, but total Inodes does not.

Message #14 received at 53037 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Dylan Simon <dylan <at> dylex.net>
Cc: 53037 <at> debbugs.gnu.org
Subject: Re: bug#53037: df/total-verify fail with cephfs
Date: Wed, 5 Jan 2022 15:07:42 -0800

[Message part 1 (text/plain, inline)]

On 1/5/22 14:11, Dylan Simon wrote:
> Then it will look like this (I'm inferring, haven't actually tried it):

I'm still not quite following, but does the attached patch address the 
problem?

[df.diff (text/x-patch, attachment)]

Message #17 received at 53037 <at> debbugs.gnu.org (full text, mbox):

From: Dylan Simon <dylan <at> dylex.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 53037 <at> debbugs.gnu.org
Subject: Re: bug#53037: df/total-verify fail with cephfs
Date: Wed, 5 Jan 2022 18:25:38 -0500

From Paul Eggert <eggert <at> cs.ucla.edu>, Wed, Jan 05, 2022 at 03:07:42PM -0800:
> On 1/5/22 14:11, Dylan Simon wrote:
> > Then it will look like this (I'm inferring, haven't actually tried it):
> 
> I'm still not quite following, but does the attached patch address the
> problem?

Hrm, no, with this patch it still fails, but differently (sorry so many
filesystems):

Filesystem        Inodes      IUsed      IFree IUse% Mounted on
devtmpfs        65968964        743   65968221    1% /dev
tmpfs           65988311       1250   65987061    1% /run
/dev/sda1       20971520     529832   20441688    3% /
tmpfs           65988311       4688   65983623    1% /dev/shm
tmpfs           65988311         14   65988297    1% /sys/fs/cgroup
/dev/sda3      212927488      21130  212906358    1% /var
/dev/sdb1      234424320         36  234424284    1% /tmp
ceph           901242617          -          -     - /mnt/ceph
home           260000000  121454853  138545147   47% /mnt/home
bright          26214400   19166782    7047618   74% /cm/shared
tmpfs           65988311          1   65988310    1% /run/user/0
tmpfs           65988311          1   65988310    1% /run/user/1135
tmpfs           65988311          1   65988310    1% /run/user/1000
total          141179331 -934077896 1075257227     - -
2117679175 != 141179331 at check-df line 14, <> line 15.

In case it's helpful, here's the statfs:

statfs("/mnt/ceph", {f_type=FUSE_SUPER_MAGIC, f_bsize=4194304, f_blocks=7243311664, f_bfree=1707119480, f_bavail=1707119480, f_files=901242617, f_ffree=18446744073709551615, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4194304, f_flags=ST_VALID|ST_RELATIME}) = 0

(This is a coreutils 8.32 build by the way but I'm happy to try 9.0 or git if
there's reason to think that will help.)

Message #20 received at 53037 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Dylan Simon <dylan <at> dylex.net>
Cc: 53037 <at> debbugs.gnu.org
Subject: Re: bug#53037: df/total-verify fail with cephfs
Date: Wed, 5 Jan 2022 15:43:49 -0800

On 1/5/22 15:25, Dylan Simon wrote:

> Hrm, no, with this patch it still fails, but differently (sorry so many
> filesystems):

OK, then perhaps someone with a bit more free time will have to look at 
it - unless you can propose a patch that passed "make check".

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.