GNU bug report logs - #6331
solaris: [sshfs] df reports wrong disk space usage

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Package: coreutils; Reported by: Miklos Szeredi <miklos@HIDDEN>; dated Wed, 2 Jun 2010 15:47:02 UTC; Maintainer for coreutils is bug-coreutils@HIDDEN.
Changed bug title to 'solaris: [sshfs] df reports wrong disk space usage' from '[sshfs] df reports wrong disk space usage on solaris' Request was from Assaf Gordon <assafgordon@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 27 Sep 2011 14:12:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 27 10:12:58 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1R8YP4-0008Ew-3i
	for submit <at> debbugs.gnu.org; Tue, 27 Sep 2011 10:12:58 -0400
Received: from mail-bw0-f44.google.com ([209.85.214.44])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1R8YP1-0008Eo-Az
	for 6331 <at> debbugs.gnu.org; Tue, 27 Sep 2011 10:12:56 -0400
Received: by bkaq10 with SMTP id q10so6492428bka.3
	for <6331 <at> debbugs.gnu.org>; Tue, 27 Sep 2011 07:12:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=szeredi.hu; s=google;
	h=from:to:cc:subject:references:date:in-reply-to:message-id
	:user-agent:mime-version:content-type;
	bh=W7WLAXWy4seF8YsDOpzN0CFmY9Xgy7H3A3iJ6BboX4g=;
	b=dxwI+BjW6QwUj2PvElbkV5/S1yb8nPQF/HgjZET6jPzJLZ2PTLbrwE633qD4pulq0L
	9uc76Xs5p/Wmw993r/PshYwyxJY+adO5ETa7NZWyhWKjIMJII6iLpF/AIFASmWDfcF0u
	AEXWJ0H5AUeU/c/earjbQco2R/idU/RG3LPvY=
Received: by 10.204.13.151 with SMTP id c23mr2113094bka.291.1317132724862;
	Tue, 27 Sep 2011 07:12:04 -0700 (PDT)
Received: from tucsk.pomaz.szeredi.hu (apn-94-44-244-141.vodafone.hu.
	[94.44.244.141])
	by mx.google.com with ESMTPS id y8sm24722664bkb.4.2011.09.27.07.11.59
	(version=TLSv1/SSLv3 cipher=OTHER);
	Tue, 27 Sep 2011 07:12:03 -0700 (PDT)
From: Miklos Szeredi <miklos@HIDDEN>
To: Jim Meyering <jim@HIDDEN>
Subject: Re: [sshfs] bug#6331: df reports wrong disk space usage on solaris
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
	<E1OK5yu-00075c-Gi@HIDDEN>
	<E1OOVxw-0006qN-Kg@HIDDEN> <87631jed3w.fsf@HIDDEN>
	<E1OOp1z-0000W5-VX@HIDDEN> <874oh3cmt2.fsf@HIDDEN>
	<E1OOulW-00015P-MZ@HIDDEN>
Date: Tue, 27 Sep 2011 16:11:35 +0200
In-Reply-To: <E1OOulW-00015P-MZ@HIDDEN> (Miklos Szeredi's message
	of "Wed, 16 Jun 2010 17:42:58 +0200")
Message-ID: <87zkhqyt0o.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Score: -3.6 (---)
X-Debbugs-Envelope-To: 6331
Cc: Dave Bechtel <kingneutron@HIDDEN>, bug-gnulib@HIDDEN,
	fuse-sshfs@HIDDEN, 6331 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.6 (---)

Jim,

Any news about this bug?

This problem was again reported against a recent Debian release here:

http://article.gmane.org/gmane.comp.file-systems.fuse.sshfs/1200

Thanks,
Miklos
Information forwarded to bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 16 Jun 2010 15:43:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 16 11:43:17 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OOulo-00079g-N5
	for submit <at> debbugs.gnu.org; Wed, 16 Jun 2010 11:43:16 -0400
Received: from fxip-0047f.externet.hu ([88.209.222.127]
	helo=pomaz-ex.szeredi.hu) by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OOull-00079a-Py
	for 6331 <at> debbugs.gnu.org; Wed, 16 Jun 2010 11:43:14 -0400
Received: from miko by pomaz-ex.szeredi.hu with local (Exim 4.69)
	(envelope-from <miklos@HIDDEN>)
	id 1OOulW-00015P-MZ; Wed, 16 Jun 2010 17:42:58 +0200
To: Jim Meyering <jim@HIDDEN>
In-reply-to: <874oh3cmt2.fsf@HIDDEN> (message from Jim Meyering on Wed, 
	16 Jun 2010 12:08:09 +0200)
Subject: Re: [sshfs] bug#6331: df reports wrong disk space usage on solaris
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
	<E1OK5yu-00075c-Gi@HIDDEN>
	<E1OOVxw-0006qN-Kg@HIDDEN> <87631jed3w.fsf@HIDDEN>
	<E1OOp1z-0000W5-VX@HIDDEN> <874oh3cmt2.fsf@HIDDEN>
Message-Id: <E1OOulW-00015P-MZ@HIDDEN>
From: Miklos Szeredi <miklos@HIDDEN>
Date: Wed, 16 Jun 2010 17:42:58 +0200
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 6331
Cc: 6331 <at> debbugs.gnu.org, bug-gnulib@HIDDEN, fuse-sshfs@HIDDEN,
	miklos@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.6 (--)

On Wed, 16 Jun 2010, Jim Meyering wrote:
> Miklos Szeredi wrote:
> > On Wed, 16 Jun 2010, Jim Meyering wrote:
> >> Miklos Szeredi wrote:
> >> > On Thu, 03 Jun 2010, Miklos Szeredi wrote:
> >> >> Hmm, actually "struct statfs" on linux does have f_frsize, only the
> >> >> manpage doesn't document it.  So correct thing would be to use that,
> >> >> no?
> >> >
> >> > Here's a patch that allows df(1) to correctly calculate the disk usage
> >> > and displays f_frsize in stat(1).
> >>
> >> Hi Miklos,
> >>
> >> [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib]
> >>
> >> Thank you for the patch.
> >> However, I'm not sure it's portable enough.
> >> Do you know when the f_frsize member began to be useful?
> >
> > It was introduced in the linux-2.5 timeframe, so from linux-2.6.0 on
> > it contains useful information.  In 2.4.* and earlier it was part of
> > the f_spare array and was zeroed out by the kernel.
> >
> > Checking f_spare value for zero and using f_bsize instead works fine
> > for 2.4 and earlier kernels.
> 
> That sounds sufficient.
> Can you confirm that this works on a 2.4.x kernel?

Okay, after some unsuccessful attempts to run a current system with a
2.4 kernel (userspace bootup immediately fails with "FATAL: kernel too
old" message) I turned to a virtualized environment.

I compiled the patched coreutils under debian/sarge (linux-2.4.27,
glibc-2.3.6).  This glibc already has f_frsize and configure detects
it.

I then verified that df(1) and stat(1) work correctly on this system.
Then took the binaries to a 2.6 kernel and glibc-2.10.1 and verified
that they indeed take a differing f_frsize and f_bsize into account.

Thanks,
Miklos
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 16 Jun 2010 10:08:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 16 06:08:21 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OOpXg-0003tX-EH
	for submit <at> debbugs.gnu.org; Wed, 16 Jun 2010 06:08:20 -0400
Received: from smtp1-g21.free.fr ([212.27.42.1])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <meyering@HIDDEN>) id 1OOpXe-0003tR-9b
	for 6331 <at> debbugs.gnu.org; Wed, 16 Jun 2010 06:08:19 -0400
Received: from mx.meyering.net (mx.meyering.net [82.230.74.64])
	by smtp1-g21.free.fr (Postfix) with ESMTP id D3A0E940153;
	Wed, 16 Jun 2010 12:08:10 +0200 (CEST)
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id AA9A5D8C9; Wed, 16 Jun 2010 12:08:09 +0200 (CEST)
From: Jim Meyering <jim@HIDDEN>
To: Miklos Szeredi <miklos@HIDDEN>
Subject: Re: bug#6331: [sshfs] df reports wrong disk space usage on solaris
In-Reply-To: <E1OOp1z-0000W5-VX@HIDDEN> (Miklos Szeredi's message
	of "Wed, 16 Jun 2010 11:35:35 +0200")
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
	<E1OK5yu-00075c-Gi@HIDDEN>
	<E1OOVxw-0006qN-Kg@HIDDEN> <87631jed3w.fsf@HIDDEN>
	<E1OOp1z-0000W5-VX@HIDDEN>
Date: Wed, 16 Jun 2010 12:08:09 +0200
Message-ID: <874oh3cmt2.fsf@HIDDEN>
Lines: 54
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Score: -3.4 (---)
X-Debbugs-Envelope-To: 6331
Cc: bug-gnulib@HIDDEN, sandon@HIDDEN, fuse-sshfs@HIDDEN,
	6331 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.4 (---)

Miklos Szeredi wrote:
> On Wed, 16 Jun 2010, Jim Meyering wrote:
>> Miklos Szeredi wrote:
>> > On Thu, 03 Jun 2010, Miklos Szeredi wrote:
>> >> Hmm, actually "struct statfs" on linux does have f_frsize, only the
>> >> manpage doesn't document it.  So correct thing would be to use that,
>> >> no?
>> >
>> > Here's a patch that allows df(1) to correctly calculate the disk usage
>> > and displays f_frsize in stat(1).
>>
>> Hi Miklos,
>>
>> [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib]
>>
>> Thank you for the patch.
>> However, I'm not sure it's portable enough.
>> Do you know when the f_frsize member began to be useful?
>
> It was introduced in the linux-2.5 timeframe, so from linux-2.6.0 on
> it contains useful information.  In 2.4.* and earlier it was part of
> the f_spare array and was zeroed out by the kernel.
>
> Checking f_spare value for zero and using f_bsize instead works fine
> for 2.4 and earlier kernels.

That sounds sufficient.
Can you confirm that this works on a 2.4.x kernel?
If so, I'll apply your changes to gnulib and coreutils.

>> Without knowing that, I cannot judge whether this introduces
>> a portability problem on older glibc, kernels or file systems.
>
> statfs() is (and probably always was) a straight syscall wrapper in
> glibc.  There's also statfs64 (used in case of _FILE_OFFSET_BITS=64)
> which is not necessarily a straight syscall, but has the same spare
> fields at the end of the structure.
>
>> Even merely knowing that is probably not enough.
>>
>> Given the lack of documentation for that member, I suspect
>> that any safe change would involve a run-time check to verify
>> that infrastructure is new enough that the member is usable.
>
> Already talked with the man pages maintainer about lack of
> documentation, and this will be fixed in new versions of the manual.
>
> In the end it's your call, but based on the level of backward
> compatibility built into the statfs structure I'd say such a change is
> perfectly safe.  And there's a real bug that needs to be fixed or
> worked around that boils down to df(1) using the wrong field for
> blocksize on linux.

Thanks.
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 16 Jun 2010 09:35:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 16 05:35:57 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OOp2L-0003fE-4H
	for submit <at> debbugs.gnu.org; Wed, 16 Jun 2010 05:35:57 -0400
Received: from fxip-0047f.externet.hu ([88.209.222.127]
	helo=pomaz-ex.szeredi.hu) by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OOp2H-0003f8-LH
	for 6331 <at> debbugs.gnu.org; Wed, 16 Jun 2010 05:35:54 -0400
Received: from miko by pomaz-ex.szeredi.hu with local (Exim 4.69)
	(envelope-from <miklos@HIDDEN>)
	id 1OOp1z-0000W5-VX; Wed, 16 Jun 2010 11:35:36 +0200
To: Jim Meyering <jim@HIDDEN>
In-reply-to: <87631jed3w.fsf@HIDDEN> (message from Jim Meyering on Wed, 
	16 Jun 2010 07:54:43 +0200)
Subject: Re: bug#6331: [sshfs] df reports wrong disk space usage on solaris
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
	<E1OK5yu-00075c-Gi@HIDDEN>
	<E1OOVxw-0006qN-Kg@HIDDEN> <87631jed3w.fsf@HIDDEN>
Message-Id: <E1OOp1z-0000W5-VX@HIDDEN>
From: Miklos Szeredi <miklos@HIDDEN>
Date: Wed, 16 Jun 2010 11:35:35 +0200
X-Spam-Score: -1.8 (-)
X-Debbugs-Envelope-To: 6331
Cc: bug-gnulib@HIDDEN, 6331 <at> debbugs.gnu.org, sandon@HIDDEN,
	fuse-sshfs@HIDDEN, miklos@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.6 (--)

On Wed, 16 Jun 2010, Jim Meyering wrote:
> Miklos Szeredi wrote:
> > On Thu, 03 Jun 2010, Miklos Szeredi wrote:
> >> Hmm, actually "struct statfs" on linux does have f_frsize, only the
> >> manpage doesn't document it.  So correct thing would be to use that,
> >> no?
> >
> > Here's a patch that allows df(1) to correctly calculate the disk usage
> > and displays f_frsize in stat(1).
> 
> Hi Miklos,
> 
> [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib]
> 
> Thank you for the patch.
> However, I'm not sure it's portable enough.
> Do you know when the f_frsize member began to be useful?

It was introduced in the linux-2.5 timeframe, so from linux-2.6.0 on
it contains useful information.  In 2.4.* and earlier it was part of
the f_spare array and was zeroed out by the kernel.

Checking f_spare value for zero and using f_bsize instead works fine
for 2.4 and earlier kernels.

> Without knowing that, I cannot judge whether this introduces
> a portability problem on older glibc, kernels or file systems.

statfs() is (and probably always was) a straight syscall wrapper in
glibc.  There's also statfs64 (used in case of _FILE_OFFSET_BITS=64)
which is not necessarily a straight syscall, but has the same spare
fields at the end of the structure.

> Even merely knowing that is probably not enough.
> 
> Given the lack of documentation for that member, I suspect
> that any safe change would involve a run-time check to verify
> that infrastructure is new enough that the member is usable.

Already talked with the man pages maintainer about lack of
documentation, and this will be fixed in new versions of the manual.

In the end it's your call, but based on the level of backward
compatibility built into the statfs structure I'd say such a change is
perfectly safe.  And there's a real bug that needs to be fixed or
worked around that boils down to df(1) using the wrong field for
blocksize on linux.

Thanks,
Miklos
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 16 Jun 2010 05:54:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 16 01:54:56 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OOlaR-0001To-Qt
	for submit <at> debbugs.gnu.org; Wed, 16 Jun 2010 01:54:56 -0400
Received: from smtp1-g21.free.fr ([212.27.42.1])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <meyering@HIDDEN>) id 1OOlaP-0001Tj-90
	for 6331 <at> debbugs.gnu.org; Wed, 16 Jun 2010 01:54:54 -0400
Received: from mx.meyering.net (mx.meyering.net [82.230.74.64])
	by smtp1-g21.free.fr (Postfix) with ESMTP id 58D779400EF;
	Wed, 16 Jun 2010 07:54:45 +0200 (CEST)
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id E0495D8B6; Wed, 16 Jun 2010 07:54:43 +0200 (CEST)
From: Jim Meyering <jim@HIDDEN>
To: Miklos Szeredi <miklos@HIDDEN>
Subject: Re: bug#6331: [sshfs] df reports wrong disk space usage on solaris
In-Reply-To: <E1OOVxw-0006qN-Kg@HIDDEN> (Miklos Szeredi's message
	of "Tue, 15 Jun 2010 15:14:08 +0200")
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
	<E1OK5yu-00075c-Gi@HIDDEN>
	<E1OOVxw-0006qN-Kg@HIDDEN>
Date: Wed, 16 Jun 2010 07:54:43 +0200
Message-ID: <87631jed3w.fsf@HIDDEN>
Lines: 79
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Score: -3.3 (---)
X-Debbugs-Envelope-To: 6331
Cc: bug-gnulib@HIDDEN, sandon@HIDDEN, fuse-sshfs@HIDDEN,
	6331 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.3 (---)

Miklos Szeredi wrote:
> On Thu, 03 Jun 2010, Miklos Szeredi wrote:
>> Hmm, actually "struct statfs" on linux does have f_frsize, only the
>> manpage doesn't document it.  So correct thing would be to use that,
>> no?
>
> Here's a patch that allows df(1) to correctly calculate the disk usage
> and displays f_frsize in stat(1).

Hi Miklos,

[Cc'ing bug-gnulib, since all but src/stat.c are from gnulib]

Thank you for the patch.
However, I'm not sure it's portable enough.
Do you know when the f_frsize member began to be useful?
Without knowing that, I cannot judge whether this introduces
a portability problem on older glibc, kernels or file systems.
Even merely knowing that is probably not enough.

Given the lack of documentation for that member, I suspect
that any safe change would involve a run-time check to verify
that infrastructure is new enough that the member is usable.

> ---
>  gnulib/lib/fsusage.c |    7 +++++++
>  m4/stat-prog.m4      |    2 +-
>  src/stat.c           |    6 +++++-
>  3 files changed, 13 insertions(+), 2 deletions(-)
>
> Index: coreutils/m4/stat-prog.m4
> ===================================================================
> --- coreutils.orig/m4/stat-prog.m4	2010-06-15 13:08:56.000000000 +0200
> +++ coreutils/m4/stat-prog.m4	2010-06-15 13:47:51.000000000 +0200
> @@ -71,7 +71,7 @@ AC_INCLUDES_DEFAULT
>        [AC_DEFINE([STRUCT_STATVFS_F_FSID_IS_INTEGER], [1],
>           [Define to 1 if the f_fsid member of struct statvfs is an integer.])])
>    else
> -    AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type],,,
> +    AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type, struct statfs.f_frsize],,,
>        [$statfs_includes])
>      if test $ac_cv_header_OS_h != yes; then
>        AC_COMPILE_IFELSE(
> Index: coreutils/src/stat.c
> ===================================================================
> --- coreutils.orig/src/stat.c	2010-06-15 13:08:56.000000000 +0200
> +++ coreutils/src/stat.c	2010-06-15 13:09:02.000000000 +0200
> @@ -118,7 +118,11 @@ statfs (char const *filename, struct fs_
>  # else
>  #  define STRUCT_STATVFS struct statfs
>  #  define STRUCT_STATXFS_F_FSID_IS_INTEGER STRUCT_STATFS_F_FSID_IS_INTEGER
> -#  define STATFS_FRSIZE(S) 0
> +#  if HAVE_STRUCT_STATFS_F_FRSIZE
> +#   define STATFS_FRSIZE(S) ((S)->f_frsize)
> +#  else
> +#   define STATFS_FRSIZE(S) 0
> +#  endif
>  # endif
>  #endif
>
> Index: coreutils/gnulib/lib/fsusage.c
> ===================================================================
> --- coreutils.orig/gnulib/lib/fsusage.c	2010-06-15 13:46:49.000000000 +0200
> +++ coreutils/gnulib/lib/fsusage.c	2010-06-15 13:46:56.000000000 +0200
> @@ -172,7 +172,14 @@ get_fs_usage (char const *file, char con
>    if (statfs (file, &fsd) < 0)
>      return -1;
>
> +#ifdef HAVE_STRUCT_STATFS_F_FRSIZE
> +  /* f_frsize isn't guaranteed to be supported.  */
> +  fsp->fsu_blocksize = (fsd.f_frsize
> +                        ? PROPAGATE_ALL_ONES (fsd.f_frsize)
> +                        : PROPAGATE_ALL_ONES (fsd.f_bsize));
> +#else
>    fsp->fsu_blocksize = PROPAGATE_ALL_ONES (fsd.f_bsize);
> +#endif
>
>  # ifdef STATFS_TRUNCATES_BLOCK_COUNTS
>
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 15 Jun 2010 13:14:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 15 09:14:27 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OOVyF-00021V-A2
	for submit <at> debbugs.gnu.org; Tue, 15 Jun 2010 09:14:27 -0400
Received: from fxip-0047f.externet.hu ([88.209.222.127]
	helo=pomaz-ex.szeredi.hu) by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OOVyD-00021P-46
	for 6331 <at> debbugs.gnu.org; Tue, 15 Jun 2010 09:14:26 -0400
Received: from miko by pomaz-ex.szeredi.hu with local (Exim 4.69)
	(envelope-from <miklos@HIDDEN>)
	id 1OOVxw-0006qN-Kg; Tue, 15 Jun 2010 15:14:08 +0200
To: jim@HIDDEN
In-reply-to: <E1OK5yu-00075c-Gi@HIDDEN> (message from Miklos
	Szeredi on Thu, 03 Jun 2010 10:40:52 +0200)
Subject: Re: bug#6331: [sshfs] df reports wrong disk space usage on solaris
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
	<E1OK5yu-00075c-Gi@HIDDEN>
Message-Id: <E1OOVxw-0006qN-Kg@HIDDEN>
From: Miklos Szeredi <miklos@HIDDEN>
Date: Tue, 15 Jun 2010 15:14:08 +0200
X-Spam-Score: -1.8 (-)
X-Debbugs-Envelope-To: 6331
Cc: 6331 <at> debbugs.gnu.org, sandon@HIDDEN, fuse-sshfs@HIDDEN,
	miklos@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.7 (--)

On Thu, 03 Jun 2010, Miklos Szeredi wrote:
> Hmm, actually "struct statfs" on linux does have f_frsize, only the
> manpage doesn't document it.  So correct thing would be to use that,
> no?

Here's a patch that allows df(1) to correctly calculate the disk usage
and displays f_frsize in stat(1).

Thanks,
Miklos


---
 gnulib/lib/fsusage.c |    7 +++++++
 m4/stat-prog.m4      |    2 +-
 src/stat.c           |    6 +++++-
 3 files changed, 13 insertions(+), 2 deletions(-)

Index: coreutils/m4/stat-prog.m4
===================================================================
--- coreutils.orig/m4/stat-prog.m4	2010-06-15 13:08:56.000000000 +0200
+++ coreutils/m4/stat-prog.m4	2010-06-15 13:47:51.000000000 +0200
@@ -71,7 +71,7 @@ AC_INCLUDES_DEFAULT
       [AC_DEFINE([STRUCT_STATVFS_F_FSID_IS_INTEGER], [1],
          [Define to 1 if the f_fsid member of struct statvfs is an integer.])])
   else
-    AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type],,,
+    AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type, struct statfs.f_frsize],,,
       [$statfs_includes])
     if test $ac_cv_header_OS_h != yes; then
       AC_COMPILE_IFELSE(
Index: coreutils/src/stat.c
===================================================================
--- coreutils.orig/src/stat.c	2010-06-15 13:08:56.000000000 +0200
+++ coreutils/src/stat.c	2010-06-15 13:09:02.000000000 +0200
@@ -118,7 +118,11 @@ statfs (char const *filename, struct fs_
 # else
 #  define STRUCT_STATVFS struct statfs
 #  define STRUCT_STATXFS_F_FSID_IS_INTEGER STRUCT_STATFS_F_FSID_IS_INTEGER
-#  define STATFS_FRSIZE(S) 0
+#  if HAVE_STRUCT_STATFS_F_FRSIZE
+#   define STATFS_FRSIZE(S) ((S)->f_frsize)
+#  else
+#   define STATFS_FRSIZE(S) 0
+#  endif
 # endif
 #endif
 
Index: coreutils/gnulib/lib/fsusage.c
===================================================================
--- coreutils.orig/gnulib/lib/fsusage.c	2010-06-15 13:46:49.000000000 +0200
+++ coreutils/gnulib/lib/fsusage.c	2010-06-15 13:46:56.000000000 +0200
@@ -172,7 +172,14 @@ get_fs_usage (char const *file, char con
   if (statfs (file, &fsd) < 0)
     return -1;
 
+#ifdef HAVE_STRUCT_STATFS_F_FRSIZE
+  /* f_frsize isn't guaranteed to be supported.  */
+  fsp->fsu_blocksize = (fsd.f_frsize
+                        ? PROPAGATE_ALL_ONES (fsd.f_frsize)
+                        : PROPAGATE_ALL_ONES (fsd.f_bsize));
+#else
   fsp->fsu_blocksize = PROPAGATE_ALL_ONES (fsd.f_bsize);
+#endif
 
 # ifdef STATFS_TRUNCATES_BLOCK_COUNTS
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 3 Jun 2010 08:41:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 03 04:41:09 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OK5zA-0002Ns-Un
	for submit <at> debbugs.gnu.org; Thu, 03 Jun 2010 04:41:09 -0400
Received: from fxip-0047f.externet.hu ([88.209.222.127]
	helo=pomaz-ex.szeredi.hu) by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OK5z8-0002NV-PV
	for 6331 <at> debbugs.gnu.org; Thu, 03 Jun 2010 04:41:07 -0400
Received: from miko by pomaz-ex.szeredi.hu with local (Exim 4.69)
	(envelope-from <miklos@HIDDEN>)
	id 1OK5yu-00075c-Gi; Thu, 03 Jun 2010 10:40:52 +0200
To: Jim Meyering <jim@HIDDEN>
In-reply-to: <87typk7c9w.fsf@HIDDEN> (message from Jim Meyering on Thu, 
	03 Jun 2010 10:26:03 +0200)
Subject: Re: bug#6331: [sshfs] df reports wrong disk space usage on solaris
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN> <87typk7c9w.fsf@HIDDEN>
Message-Id: <E1OK5yu-00075c-Gi@HIDDEN>
From: Miklos Szeredi <miklos@HIDDEN>
Date: Thu, 03 Jun 2010 10:40:52 +0200
X-Spam-Score: -3.6 (---)
X-Debbugs-Envelope-To: 6331
Cc: 6331 <at> debbugs.gnu.org, sandon@HIDDEN, fuse-sshfs@HIDDEN,
	miklos@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.3 (---)

On Thu, 03 Jun 2010, Jim Meyering wrote:
> Miklos Szeredi wrote:
> ...
> > Notice that the real statfs syscall does have an f_frsize field, but
> > for some historical reason it is not exported in the library API.
> >
> > Options to fix this behavior would be:
> >
> >  1) fix df and stat in coreutils to use statvfs
> >  2) make sshfs set f_bsize to f_frsize
> >  3) make sshfs set f_frsize to f_bsize and recalculate stats
> >  4) change both f_frsize and f_bsize to a common value and
> >     recalculate stats (this is what NFS appears to be doing)
> >
> > There is some difficulty with 1) as the libc implementation of statvfs
> > has problems:
> >
> >  "Do not use statvfs on systems with GNU libc on Linux, because that function
> >   stats all preceding entries in /proc/mounts, and that makes df hang if even
> >   one of the corresponding file systems is hard-mounted, but not available.
> >   statvfs in GNU libc on Hurd, BeOS, Haiku operates differently: it only makes
> >   a system call."
> >
> > This might have been fixed in libc since, so it's possible that
> > coreutils developers are willing to using statvfs on linux.
> 
> glibc's statvfs still appears to have this problem, so coreutils' df
> must not use that function.  To demonstrate, I recompiled fsusage.c
> with -DSTAT_STATVFS, relinked df, and ran this:
> 
>     $ strace -e open,stat ./df .
>     ...
>     open("/proc/mounts", O_RDONLY)          = 3
>     stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
>     stat(".", {st_mode=S_IFDIR|0755, st_size=65536, ...}) = 0
>     stat("/", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
>     stat("/boot", {st_mode=S_IFDIR|0555, st_size=3072, ...}) = 0
>     stat("/full", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0
>     stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> 
> It still opens /proc/mounts and stats mount point directories
> until it finds one with a matching dev/inode.  Note that it
> does limit its search, when possible, to file systems that are known
> to be of the same file system type, but that doesn't help when
> the target is NFS mounted.
> 

Hmm, actually "struct statfs" on linux does have f_frsize, only the
manpage doesn't document it.  So correct thing would be to use that,
no?

Thanks,
Miklos
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at 6331 <at> debbugs.gnu.org:


Received: (at 6331) by debbugs.gnu.org; 3 Jun 2010 08:26:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 03 04:26:16 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OK5kl-0002GK-Jr
	for submit <at> debbugs.gnu.org; Thu, 03 Jun 2010 04:26:15 -0400
Received: from smtp1-g21.free.fr ([212.27.42.1])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <meyering@HIDDEN>) id 1OK5ki-0002GC-OA
	for 6331 <at> debbugs.gnu.org; Thu, 03 Jun 2010 04:26:14 -0400
Received: from mx.meyering.net (mx.meyering.net [82.230.74.64])
	by smtp1-g21.free.fr (Postfix) with ESMTP id DDB25940163
	for <6331 <at> debbugs.gnu.org>; Thu,  3 Jun 2010 10:26:04 +0200 (CEST)
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id B36F2A1C; Thu,  3 Jun 2010 10:26:03 +0200 (CEST)
From: Jim Meyering <jim@HIDDEN>
To: Miklos Szeredi <miklos@HIDDEN>
Subject: Re: bug#6331: [sshfs] df reports wrong disk space usage on solaris
In-Reply-To: <E1OJlMq-0004ci-Ve@HIDDEN> (Miklos Szeredi's message
	of "Wed, 02 Jun 2010 12:40:12 +0200")
References: <4C0533FC.5070005@HIDDEN>
	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
	<E1OJlMq-0004ci-Ve@HIDDEN>
Date: Thu, 03 Jun 2010 10:26:03 +0200
Message-ID: <87typk7c9w.fsf@HIDDEN>
Lines: 44
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Score: -3.3 (---)
X-Debbugs-Envelope-To: 6331
Cc: Sandon Van Ness <sandon@HIDDEN>, fuse-sshfs@HIDDEN,
	6331 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.3 (---)

Miklos Szeredi wrote:
...
> Notice that the real statfs syscall does have an f_frsize field, but
> for some historical reason it is not exported in the library API.
>
> Options to fix this behavior would be:
>
>  1) fix df and stat in coreutils to use statvfs
>  2) make sshfs set f_bsize to f_frsize
>  3) make sshfs set f_frsize to f_bsize and recalculate stats
>  4) change both f_frsize and f_bsize to a common value and
>     recalculate stats (this is what NFS appears to be doing)
>
> There is some difficulty with 1) as the libc implementation of statvfs
> has problems:
>
>  "Do not use statvfs on systems with GNU libc on Linux, because that function
>   stats all preceding entries in /proc/mounts, and that makes df hang if even
>   one of the corresponding file systems is hard-mounted, but not available.
>   statvfs in GNU libc on Hurd, BeOS, Haiku operates differently: it only makes
>   a system call."
>
> This might have been fixed in libc since, so it's possible that
> coreutils developers are willing to using statvfs on linux.

glibc's statvfs still appears to have this problem, so coreutils' df
must not use that function.  To demonstrate, I recompiled fsusage.c
with -DSTAT_STATVFS, relinked df, and ran this:

    $ strace -e open,stat ./df .
    ...
    open("/proc/mounts", O_RDONLY)          = 3
    stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    stat(".", {st_mode=S_IFDIR|0755, st_size=65536, ...}) = 0
    stat("/", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
    stat("/boot", {st_mode=S_IFDIR|0555, st_size=3072, ...}) = 0
    stat("/full", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0
    stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0

It still opens /proc/mounts and stats mount point directories
until it finds one with a matching dev/inode.  Note that it
does limit its search, when possible, to file systems that are known
to be of the same file system type, but that doesn't help when
the target is NFS mounted.
Information forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 2 Jun 2010 15:46:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 02 11:46:56 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OJq9f-0003qA-Rx
	for submit <at> debbugs.gnu.org; Wed, 02 Jun 2010 11:46:56 -0400
Received: from mx10.gnu.org ([199.232.76.166])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OJlNU-00016x-35
	for submit <at> debbugs.gnu.org; Wed, 02 Jun 2010 06:40:54 -0400
Received: from lists.gnu.org ([199.232.76.165]:34394)
	by monty-python.gnu.org with esmtps
	(TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60)
	(envelope-from <miklos@HIDDEN>) id 1OJlNO-0004xD-EA
	for submit <at> debbugs.gnu.org; Wed, 02 Jun 2010 06:40:46 -0400
Received: from [140.186.70.92] (port=60536 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1OJlNH-0005ab-AK
	for bug-coreutils@HIDDEN; Wed, 02 Jun 2010 06:40:46 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00
	autolearn=unavailable version=3.3.1
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OJlNF-0006lE-O8
	for bug-coreutils@HIDDEN; Wed, 02 Jun 2010 06:40:39 -0400
Received: from fxip-0047f.externet.hu ([88.209.222.127]:55744
	helo=pomaz-ex.szeredi.hu) by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <miklos@HIDDEN>) id 1OJlNF-0006ih-Gr
	for bug-coreutils@HIDDEN; Wed, 02 Jun 2010 06:40:37 -0400
Received: from miko by pomaz-ex.szeredi.hu with local (Exim 4.69)
	(envelope-from <miklos@HIDDEN>)
	id 1OJlMq-0004ci-Ve; Wed, 02 Jun 2010 12:40:13 +0200
To: Sandon Van Ness <sandon@HIDDEN>
In-reply-to: <4C05B5D1.3000404@HIDDEN> (message from Sandon Van Ness on
	Tue, 01 Jun 2010 18:37:21 -0700)
Subject: Re: [sshfs] df reports wrong disk space usage on solaris
References: <4C0533FC.5070005@HIDDEN>	<E1OJXys-000396-TI@HIDDEN>
	<4C056FC8.3060107@HIDDEN> <4C05B5D1.3000404@HIDDEN>
Message-Id: <E1OJlMq-0004ci-Ve@HIDDEN>
From: Miklos Szeredi <miklos@HIDDEN>
Date: Wed, 02 Jun 2010 12:40:12 +0200
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6,
	seldom 2.4 (older, 4)
X-Spam-Score: -3.3 (---)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Wed, 02 Jun 2010 11:46:53 -0400
Cc: bug-coreutils@HIDDEN, fuse-sshfs@HIDDEN, miklos@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -4.6 (----)

[bug-coreutils@... added to CC]

On Tue, 01 Jun 2010, Sandon Van Ness wrote:
> > NFS:
> >
> > root@sabayonx86-64: 01:27 PM :~# stat -f /osol
> >   File: "/osol"
> >     ID: 0        Namelen: 255     Type: nfs
> > Block size: 32768      Fundamental block size: 32768
> > Blocks: Total: 532021184  Free: 109079823  Available: 109079823
> > Inodes: Total: 6987112837 Free: 6981108616
> >
> > SSHFS:
> > root@sabayonx86-64: 01:27 PM :~# stat -f /sshfs
> >   File: "/sshfs"
> >     ID: 0        Namelen: 255     Type: UNKNOWN (0x65735546)
> > Block size: 131072     Fundamental block size: 131072
> > Blocks: Total: 34049356570 Free: 6980647748 Available: 6980647748
> > Inodes: Total: 6986651970 Free: 6980647748
> >   
> Also here is the stats on the solaris box itself vs ssshfs:
> 
> opensolaris:
> 
> 
> root@opensolaris: 06:38 PM :/usr/local/etc#  stat -f /data
>   File: "/data"
>     ID: 1690006  Namelen: 255     Type: zfs
> Block size: 131072     Fundamental block size: 512
> Blocks: Total: 34049348764 Free: 6056839257 Available: 6056839257
> Inodes: Total: 6064436485 Free: 6056839257
> 
> sshfs:
> root@sabayonx86-64: 06:35 PM :~# stat -f /sshfs/
>   File: "/sshfs/"
>     ID: 0        Namelen: 255     Type: UNKNOWN (0x65735546)
> Block size: 131072     Fundamental block size: 131072
> Blocks: Total: 34049348764 Free: 6056839257 Available: 6056839257
> Inodes: Total: 6064436485 Free: 6056839257
> 
> I noticed the block counts are equal but the block size is 131072 on
> sshfs and 512 on the solaris machine.
> 

I looked more deeply into this issue and it appears that "df" and
"stat" on linux both use the statfs libc function which doesn't have
f_frsize.

To verify this, try doing 

  strace -o /tmp/strace stat -f /sshfs

You should get a line like this in the strace output:

statfs("/sshfs/", {f_type=0x65735546, ..., f_frsize=512}) = 0

Notice that the real statfs syscall does have an f_frsize field, but
for some historical reason it is not exported in the library API.

Options to fix this behavior would be:

 1) fix df and stat in coreutils to use statvfs
 2) make sshfs set f_bsize to f_frsize
 3) make sshfs set f_frsize to f_bsize and recalculate stats
 4) change both f_frsize and f_bsize to a common value and
    recalculate stats (this is what NFS appears to be doing)

There is some difficulty with 1) as the libc implementation of statvfs
has problems:

 "Do not use statvfs on systems with GNU libc on Linux, because that function
  stats all preceding entries in /proc/mounts, and that makes df hang if even
  one of the corresponding file systems is hard-mounted, but not available.
  statvfs in GNU libc on Hurd, BeOS, Haiku operates differently: it only makes
  a system call."

This might have been fixed in libc since, so it's possible that
coreutils developers are willing to using statvfs on linux.

Another option would be to use the direct statfs (and statfs64)
syscall interface on linux, bypassing the unnecessary cruft that libc
put there.  But this adds some complexity.

Thanks,
Miklos
Acknowledgement sent to Miklos Szeredi <miklos@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-coreutils@HIDDEN. Full text available.
Report forwarded to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:
bug#6331; Package coreutils. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.