GNU bug report logs - #9140
[du] broken on OSX 10.7 (Lion) for >4TB file systems

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9140 in the body.
You can then email your comments to 9140 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Herb Wartens <hawartens <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: Coreutils Bug on OSX 10.7 (Lion)
Date: Wed, 20 Jul 2011 23:54:17 -0700

[Message part 1 (text/plain, inline)]

After installing coreutils-8.12 on OSX 10.7 today, I found an issue immediately where df was overflowing the structure returned from statvfs.  On OSX 10.7 the relevant fields of the statvfs structure are defined to be fsblkcnt_t, or only 4 bytes long.  I have put together an initial fix for 10.7.  Hope this helps.

Looking through your bug reports, I believe this is likely a fix for bug 7355.

-Herb

[coreutils_du.diff (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

Message #8 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Herb Wartens <hawartens <at> gmail.com>
Cc: 9140 <at> debbugs.gnu.org
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Thu, 21 Jul 2011 14:39:56 -0700

> On OSX 10.7 the relevant fields of the statvfs structure
> are defined to be fsblkcnt_t, or only 4 bytes long.

Thanks for the bug report.  Could you please give more details about
the problem?  What is the size of your file system?  What are the
types (e.g., are the types signed? and what are their sizes?)
and values of f_frsize, f_bsize, f_blocks, and all the other
members of a struct statvfs?

The patch you sent essentially says "statvfs is broken so badly on
MacOS X 10.7 that nobody should ever use it".  Is that really right?
It sounds extreme.

Message #11 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Herb Wartens <hawartens <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9140 <at> debbugs.gnu.org
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Thu, 21 Jul 2011 16:35:52 -0700

[Message part 1 (text/plain, inline)]

On Thu, Jul 21, 2011 at 2:39 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> > On OSX 10.7 the relevant fields of the statvfs structure
> > are defined to be fsblkcnt_t, or only 4 bytes long.
>
> Thanks for the bug report.  Could you please give more details about
> the problem?  What is the size of your file system?  What are the
> types (e.g., are the types signed? and what are their sizes?)
> and values of f_frsize, f_bsize, f_blocks, and all the other
> members of a struct statvfs?
>
>
Sure.  The filesystem is 5.5TiB (5,858,217,984 bytes).  This is beyond the
capacity an unsigned int can hold.  Here is an example of the bug:
> df /Volumes/VIDEO
Filesystem                         1K-blocks        Used   Available Use%
Mounted on
//user1 <at> 192.168.1.100/VIDEO <http://hawartens <at> 192.168.101.151/VIDEO>
 1563250688 -1474553224 -1257163384  54% /Volumes/VIDEO

Here is what it should look like:
> df /Volumes/VIDEO
Filesystem                         1K-blocks       Used  Available Use%
Mounted on
//user1 <at> 192.168.1.100/VIDEO <http://hawartens <at> 192.168.101.151/VIDEO>
 5858217984 2820414072 3037803912  49% /Volumes/VIDEO


Here are the relevant definitions on OSX:
struct statvfs {
    unsigned long f_bsize;    /* File system block size */
    unsigned long f_frsize;   /* Fundamental file system block size */
    fsblkcnt_t    f_blocks;   /* Blocks on FS in units of f_frsize */
    fsblkcnt_t    f_bfree;    /* Free blocks */
    fsblkcnt_t    f_bavail;   /* Blocks available to non-root */
    fsfilcnt_t    f_files;    /* Total inodes */
    fsfilcnt_t    f_ffree;    /* Free inodes */
    fsfilcnt_t    f_favail;   /* Free inodes for non-root */
    unsigned long f_fsid;     /* Filesystem ID */
    unsigned long f_flag;     /* Bit mask of values */
    unsigned long f_namemax;  /* Max file name length */
};

typedef __darwin_fsblkcnt_t fsblkcnt_t;
typedef __darwin_fsfilcnt_t fsfilcnt_t;

typedef unsigned int __darwin_fsblkcnt_t; /* Used by statvfs and fstatvfs */
typedef unsigned int __darwin_fsfilcnt_t; /* Used by statvfs and fstatvfs */

(gdb) print sizeof(buf.f_bsize)
$2 = 8
(gdb) print sizeof(buf.f_frsize)
$3 = 8
(gdb) print sizeof(buf.f_blocks)
$4 = 4
(gdb) print sizeof(buf.f_bfree)
$5 = 4
(gdb) print sizeof(buf.f_bavail)
$6 = 4
(gdb) print sizeof(buf.f_files)
$7 = 4
(gdb) print sizeof(buf.f_ffree)
$8 = 4
(gdb) print sizeof(buf.f_favail)
$9 = 4
(gdb) print sizeof(buf.f_fsid)
$10 = 8
(gdb) print sizeof(buf.f_flag)
$11 = 8
(gdb) print sizeof(buf.f_namemax)
$12 = 8



> The patch you sent essentially says "statvfs is broken so badly on
> MacOS X 10.7 that nobody should ever use it".  Is that really right?
> It sounds extreme.
>
>
Not saying no one should use it at all.  All I am saying is that when using
really large filesystems, then the statvfs structure is insufficient on OSX.
 Hope this helps.

-Herb

[Message part 2 (text/html, inline)]

Message #14 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bug-gnulib <bug-gnulib <at> gnu.org>
Cc: 9140 <at> debbugs.gnu.org
Subject: [PATCH 2/3] large-inode: New module
Date: Fri, 22 Jul 2011 10:22:37 -0700

Here's another module I'm adding to coreutils.
I plan to change coreutils to use it.
Another possibility, I suppose, would be to fold it into
AC_SYS_LARGEFILE.

Herb, can you please try out the effect of this module by
appending "#define _DARWIN_USE_64_BIT_INODE 1" to lib/config.h
after running 'configure'?  Thanks.

* MODULES.html.sh: Add it.
* modules/large-inode, m4/large-inode.m4: New files.
---
 ChangeLog           |    4 ++++
 MODULES.html.sh     |    1 +
 m4/large-inode.m4   |   24 ++++++++++++++++++++++++
 modules/large-inode |   22 ++++++++++++++++++++++
 4 files changed, 51 insertions(+), 0 deletions(-)
 create mode 100644 m4/large-inode.m4
 create mode 100644 modules/large-inode

diff --git a/ChangeLog b/ChangeLog
index 46d8d04..b95e9d6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
 2011-07-22  Paul Eggert  <eggert <at> cs.ucla.edu>
 
+	large-inode: New module
+	* MODULES.html.sh: Add it.
+	* modules/large-inode, m4/large-inode.m4: New files.
+
 	extensions: Enable extensions on MacOS X 10.5 and later.
 	* m4/extensions.m4 (AC_USE_SYSTEM_EXTENSIONS): Define _DARWIN_C_SOURCE.
 
diff --git a/MODULES.html.sh b/MODULES.html.sh
index bcd8aac..80befa9 100755
--- a/MODULES.html.sh
+++ b/MODULES.html.sh
@@ -2656,6 +2656,7 @@ func_all_modules ()
   func_module file-set
   func_module hash-triple
   func_module i-ring
+  func_module large-inode
   func_module same-inode
   func_end_table
 
diff --git a/m4/large-inode.m4 b/m4/large-inode.m4
new file mode 100644
index 0000000..b41a391
--- /dev/null
+++ b/m4/large-inode.m4
@@ -0,0 +1,24 @@
+# Enable large inode numbers on systems normally without them. -*- Autoconf -*-
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+# This file is free software; the Free Software Foundation
+# gives unlimited permission to copy and/or distribute it,
+# with or without modifications, as long as this notice is preserved.
+
+# written by Paul Eggert
+
+AC_DEFUN([gl_SYS_LARGE_INODE],
+[
+  dnl Many systems enable large inodes if you enable large offsets.
+  AC_REQUIRE([AC_SYS_LARGEFILE])
+
+  dnl Some Mac OS X variants won't access large inode numbers by default.
+  dnl Defining _DARWIN_USE_64_BIT_INODE fixes this.  See
+  dnl <http://developer.apple.com/library/mac/releasenotes/Darwin/SymbolVariantsRelNotes>.
+  dnl
+  dnl The simplest thing is to define this symbol everywhere.
+  dnl That helps on the affected systems, and doesn't hurt anywhere.
+  AC_DEFINE([_DARWIN_USE_64_BIT_INODE], [1],
+    [Define if you want to use large inode numbers
+     when running on Mac OS X 10.5 or later.])
+])
diff --git a/modules/large-inode b/modules/large-inode
new file mode 100644
index 0000000..de067ac
--- /dev/null
+++ b/modules/large-inode
@@ -0,0 +1,22 @@
+Description:
+Use large inode numbers.
+
+Files:
+m4/large-inode.m4
+
+Depends-on:
+
+configure.ac-early:
+AC_REQUIRE([gl_SYS_LARGE_INODE])
+
+configure.ac:
+
+Makefile.am:
+
+Include:
+
+License:
+LGPLv2+
+
+Maintainer:
+all
-- 
1.7.4.4

Message #17 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Herb Wartens <hawartens <at> gmail.com>
Cc: 9140 <at> debbugs.gnu.org, Bug-gnulib <bug-gnulib <at> gnu.org>
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Fri, 22 Jul 2011 10:26:04 -0700

Thanks for the further information about the problem with MacOS X 10.7.
Can you please try this patch?  If you don't have autotools installed,
you can edit 'configure' by hand and install a similar patch there.
Thanks.

I have pushed this into gnulib as it seems pretty simple.  The basic
idea is to avoid statvfs on MacOS X because its block counts can't
go above 2**31.

From 3d2ecb41c4a3e428e8a455c4f5fef49f50144f0f Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert <at> cs.ucla.edu>
Date: Fri, 22 Jul 2011 10:07:18 -0700
Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems

* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
implementations that use only 32 bits to count blocks.
On typical hosts with 1024-byte blocks, this fails with file
systems as small as 4 TiB.  Problem reported by Herb Wartens
<http://debbugs.gnu.org/9140> and this should also fix a similar
problem reported by Tim Spriggs <http://debbugs.gnu.org/7355>.
---
 ChangeLog     |    8 ++++++++
 m4/fsusage.m4 |   18 +++++++++++++-----
 2 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index b95e9d6..b652f25 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,13 @@
 2011-07-22  Paul Eggert  <eggert <at> cs.ucla.edu>
 
+	fsusage: port to MacOS X 10.7 with 4 TiB file systems
+	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
+	implementations that use only 32 bits to count blocks.
+	On typical hosts with 1024-byte blocks, this fails with file
+	systems as small as 4 TiB.  Problem reported by Herb Wartens
+	<http://debbugs.gnu.org/9140> and this should also fix a similar
+	problem reported by Tim Spriggs <http://debbugs.gnu.org/7355>.
+
 	large-inode: New module
 	* MODULES.html.sh: Add it.
 	* modules/large-inode, m4/large-inode.m4: New files.
diff --git a/m4/fsusage.m4 b/m4/fsusage.m4
index 3459e63..41c06bc 100644
--- a/m4/fsusage.m4
+++ b/m4/fsusage.m4
@@ -40,7 +40,7 @@ ac_fsusage_space=no
 # systems.  That system is reported to work fine with STAT_STATFS4 which
 # is what it gets when this test fails.
 if test $ac_fsusage_space = no; then
-  # glibc/{Hurd,kFreeBSD}, MacOS X >= 10.4, FreeBSD >= 5.0, NetBSD >= 3.0,
+  # glibc/{Hurd,kFreeBSD}, FreeBSD >= 5.0, NetBSD >= 3.0,
   # OpenBSD >= 4.4, AIX, HP-UX, IRIX, Solaris, Cygwin, Interix, BeOS.
   AC_CACHE_CHECK([for statvfs function (SVR4)], [fu_cv_sys_stat_statvfs],
                  [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <sys/types.h>
@@ -56,8 +56,16 @@ a system call.
 "Do not use Tru64's statvfs implementation"
 #endif
 
-#include <sys/statvfs.h>]],
-                                    [[struct statvfs fsd; statvfs (0, &fsd);]])],
+#include <limits.h>
+#include <sys/statvfs.h>
+
+/* Reject implementations, such as MacOS X 10.7, where f_blocks is a
+   32-bit quantity; that commonly limits file systems to 4 TiB, a
+   ridiculously small limit these days.  */
+struct statvfs fsd;
+int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT <= 32 ? -1 : 1];
+]],
+                                    [[statvfs (0, &fsd);]])],
                                  [fu_cv_sys_stat_statvfs=yes],
                                  [fu_cv_sys_stat_statvfs=no])])
   if test $fu_cv_sys_stat_statvfs = yes; then
@@ -94,8 +102,8 @@ if test $ac_fsusage_space = no; then
 fi
 
 if test $ac_fsusage_space = no; then
-  # glibc/Linux, MacOS X < 10.4, FreeBSD < 5.0, NetBSD < 3.0, OpenBSD < 4.4.
-  # (glibc/{Hurd,kFreeBSD}, MacOS X >= 10.4, FreeBSD >= 5.0, NetBSD >= 3.0,
+  # glibc/Linux, MacOS X, FreeBSD < 5.0, NetBSD < 3.0, OpenBSD < 4.4.
+  # (glibc/{Hurd,kFreeBSD}, FreeBSD >= 5.0, NetBSD >= 3.0,
   # OpenBSD >= 4.4, AIX, HP-UX, OSF/1, Cygwin already handled above.)
   # (On IRIX you need to include <sys/statfs.h>, not only <sys/mount.h> and
   # <sys/vfs.h>.)
-- 
1.7.4.4

Message #20 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9140 <at> debbugs.gnu.org, Bug-gnulib <bug-gnulib <at> gnu.org>
Subject: Re: [PATCH 2/3] large-inode: New module
Date: Fri, 22 Jul 2011 18:45:45 +0100

On 22/07/11 18:22, Paul Eggert wrote:
> Here's another module I'm adding to coreutils.
> I plan to change coreutils to use it.
> Another possibility, I suppose, would be to fold it into
> AC_SYS_LARGEFILE.
> 
> Herb, can you please try out the effect of this module by
> appending "#define _DARWIN_USE_64_BIT_INODE 1" to lib/config.h
> after running 'configure'?  Thanks.

If it works it would be better in AC_SYS_LARGEFILE (as well) I think.
Though the original reporter of this issue said it didn't work.
Though I now notice I asked him to put it at the top of df.c
rather than somewhere significant to fsusage.c

cheers,
Pádraig.

Message #23 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9140 <at> debbugs.gnu.org, Bug-gnulib <bug-gnulib <at> gnu.org>,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Fri, 22 Jul 2011 20:15:58 +0200

Paul Eggert wrote:
> Thanks for the further information about the problem with MacOS X 10.7.
> Can you please try this patch?  If you don't have autotools installed,
> you can edit 'configure' by hand and install a similar patch there.
> Thanks.
>
> I have pushed this into gnulib as it seems pretty simple.  The basic
> idea is to avoid statvfs on MacOS X because its block counts can't
> go above 2**31.
>
> Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems
>
> * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
> implementations that use only 32 bits to count blocks.
> On typical hosts with 1024-byte blocks, this fails with file
> systems as small as 4 TiB.  Problem reported by Herb Wartens
> <http://debbugs.gnu.org/9140> and this should also fix a similar
> problem reported by Tim Spriggs <http://debbugs.gnu.org/7355>.
...
> +/* Reject implementations, such as MacOS X 10.7, where f_blocks is a
> +   32-bit quantity; that commonly limits file systems to 4 TiB, a
> +   ridiculously small limit these days.  */
> +struct statvfs fsd;
> +int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT <= 32 ? -1 : 1];
> +]],

Nice.  That looks like it will do the job, and since all statvfs
implementations have the f_blocks member, it should be safe.

Thanks.

Message #26 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9140 <at> debbugs.gnu.org, Bug-gnulib <bug-gnulib <at> gnu.org>
Subject: Re: [PATCH 2/3] large-inode: New module
Date: Fri, 22 Jul 2011 15:10:56 -0700

On 07/22/11 10:45, Pádraig Brady wrote:
> If it works it would be better in AC_SYS_LARGEFILE (as well) I think.

OK, thanks, I pushed this into gnulib.  Assuming it works (and I don't
see why it'd hurt) I'll push it into autoconf.

From a7df7c7d3f2f32d0cad31f65505cfee939d91f4a Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert <at> cs.ucla.edu>
Date: Fri, 22 Jul 2011 15:06:36 -0700
Subject: [PATCH] largefile: new module, replacing large-inode
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Pádraig Brady suggested this in	<http://debbugs.gnu.org/9140#20>.
* MODULES.html.sh: Add largefile, remove large-inode.
* modules/largefile, m4/largefile.m4: New files.
* modules/large-inode, m4/large-inode.m4: Remove.
---
 ChangeLog           |    6 +++
 MODULES.html.sh     |    2 +-
 m4/large-inode.m4   |   24 ------------
 m4/largefile.m4     |  104 +++++++++++++++++++++++++++++++++++++++++++++++++++
 modules/large-inode |   22 -----------
 modules/largefile   |   22 +++++++++++
 6 files changed, 133 insertions(+), 47 deletions(-)
 delete mode 100644 m4/large-inode.m4
 create mode 100644 m4/largefile.m4
 delete mode 100644 modules/large-inode
 create mode 100644 modules/largefile

diff --git a/ChangeLog b/ChangeLog
index b652f25..c44292e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
 2011-07-22  Paul Eggert  <eggert <at> cs.ucla.edu>
 
+	largefile: new module, replacing large-inode
+	Pádraig Brady suggested this in	<http://debbugs.gnu.org/9140#20>.
+	* MODULES.html.sh: Add largefile, remove large-inode.
+	* modules/largefile, m4/largefile.m4: New files.
+	* modules/large-inode, m4/large-inode.m4: Remove.
+
 	fsusage: port to MacOS X 10.7 with 4 TiB file systems
 	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
 	implementations that use only 32 bits to count blocks.
diff --git a/MODULES.html.sh b/MODULES.html.sh
index 80befa9..6d84587 100755
--- a/MODULES.html.sh
+++ b/MODULES.html.sh
@@ -2608,6 +2608,7 @@ func_all_modules ()
   func_module filenamecat-lgpl
   func_module fts
   func_module isdir
+  func_module largefile
   func_module lchmod
   func_module lchown
   func_module mkancesdirs
@@ -2656,7 +2657,6 @@ func_all_modules ()
   func_module file-set
   func_module hash-triple
   func_module i-ring
-  func_module large-inode
   func_module same-inode
   func_end_table
 
diff --git a/m4/large-inode.m4 b/m4/large-inode.m4
deleted file mode 100644
index b41a391..0000000
--- a/m4/large-inode.m4
+++ /dev/null
@@ -1,24 +0,0 @@
-# Enable large inode numbers on systems normally without them. -*- Autoconf -*-
-
-# Copyright (C) 2011 Free Software Foundation, Inc.
-# This file is free software; the Free Software Foundation
-# gives unlimited permission to copy and/or distribute it,
-# with or without modifications, as long as this notice is preserved.
-
-# written by Paul Eggert
-
-AC_DEFUN([gl_SYS_LARGE_INODE],
-[
-  dnl Many systems enable large inodes if you enable large offsets.
-  AC_REQUIRE([AC_SYS_LARGEFILE])
-
-  dnl Some Mac OS X variants won't access large inode numbers by default.
-  dnl Defining _DARWIN_USE_64_BIT_INODE fixes this.  See
-  dnl <http://developer.apple.com/library/mac/releasenotes/Darwin/SymbolVariantsRelNotes>.
-  dnl
-  dnl The simplest thing is to define this symbol everywhere.
-  dnl That helps on the affected systems, and doesn't hurt anywhere.
-  AC_DEFINE([_DARWIN_USE_64_BIT_INODE], [1],
-    [Define if you want to use large inode numbers
-     when running on Mac OS X 10.5 or later.])
-])
diff --git a/m4/largefile.m4 b/m4/largefile.m4
new file mode 100644
index 0000000..6986244
--- /dev/null
+++ b/m4/largefile.m4
@@ -0,0 +1,104 @@
+# Enable large files on systems where this is not the default.
+
+# Copyright 1992-1996, 1998-2011 Free Software Foundation, Inc.
+# This file is free software; the Free Software Foundation
+# gives unlimited permission to copy and/or distribute it,
+# with or without modifications, as long as this notice is preserved.
+
+# The following implementation works around a problem in autoconf <= 2.68;
+# AC_SYS_LARGEFILE does not configure for large inodes on Mac OS X 10.5.
+m4_version_prereq([2.69], [] ,[
+
+# _AC_SYS_LARGEFILE_TEST_INCLUDES
+# -------------------------------
+m4_define([_AC_SYS_LARGEFILE_TEST_INCLUDES],
+[@%:@include <sys/types.h>
+ /* Check that off_t can represent 2**63 - 1 correctly.
+    We can't simply define LARGE_OFF_T to be 9223372036854775807,
+    since some C++ compilers masquerading as C compilers
+    incorrectly reject 9223372036854775807.  */
+@%:@define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
+  int off_t_is_large[[(LARGE_OFF_T % 2147483629 == 721
+		       && LARGE_OFF_T % 2147483647 == 1)
+		      ? 1 : -1]];[]dnl
+])
+
+
+# _AC_SYS_LARGEFILE_MACRO_VALUE(C-MACRO, VALUE,
+#				CACHE-VAR,
+#				DESCRIPTION,
+#				PROLOGUE, [FUNCTION-BODY])
+# --------------------------------------------------------
+m4_define([_AC_SYS_LARGEFILE_MACRO_VALUE],
+[AC_CACHE_CHECK([for $1 value needed for large files], [$3],
+[while :; do
+  m4_ifval([$6], [AC_LINK_IFELSE], [AC_COMPILE_IFELSE])(
+    [AC_LANG_PROGRAM([$5], [$6])],
+    [$3=no; break])
+  m4_ifval([$6], [AC_LINK_IFELSE], [AC_COMPILE_IFELSE])(
+    [AC_LANG_PROGRAM([@%:@define $1 $2
+$5], [$6])],
+    [$3=$2; break])
+  $3=unknown
+  break
+done])
+case $$3 in #(
+  no | unknown) ;;
+  *) AC_DEFINE_UNQUOTED([$1], [$$3], [$4]);;
+esac
+rm -rf conftest*[]dnl
+])# _AC_SYS_LARGEFILE_MACRO_VALUE
+
+
+# AC_SYS_LARGEFILE
+# ----------------
+# By default, many hosts won't let programs access large files;
+# one must use special compiler options to get large-file access to work.
+# For more details about this brain damage please see:
+# http://www.unix-systems.org/version2/whatsnew/lfs20mar.html
+AC_DEFUN([AC_SYS_LARGEFILE],
+[AC_ARG_ENABLE(largefile,
+	       [  --disable-largefile     omit support for large files])
+if test "$enable_largefile" != no; then
+
+  AC_CACHE_CHECK([for special C compiler options needed for large files],
+    ac_cv_sys_largefile_CC,
+    [ac_cv_sys_largefile_CC=no
+     if test "$GCC" != yes; then
+       ac_save_CC=$CC
+       while :; do
+	 # IRIX 6.2 and later do not support large files by default,
+	 # so use the C compiler's -n32 option if that helps.
+	 AC_LANG_CONFTEST([AC_LANG_PROGRAM([_AC_SYS_LARGEFILE_TEST_INCLUDES])])
+	 AC_COMPILE_IFELSE([], [break])
+	 CC="$CC -n32"
+	 AC_COMPILE_IFELSE([], [ac_cv_sys_largefile_CC=' -n32'; break])
+	 break
+       done
+       CC=$ac_save_CC
+       rm -f conftest.$ac_ext
+    fi])
+  if test "$ac_cv_sys_largefile_CC" != no; then
+    CC=$CC$ac_cv_sys_largefile_CC
+  fi
+
+  _AC_SYS_LARGEFILE_MACRO_VALUE(_FILE_OFFSET_BITS, 64,
+    ac_cv_sys_file_offset_bits,
+    [Number of bits in a file offset, on hosts where this is settable.],
+    [_AC_SYS_LARGEFILE_TEST_INCLUDES])
+  if test $ac_cv_sys_file_offset_bits = unknown; then
+    _AC_SYS_LARGEFILE_MACRO_VALUE(_LARGE_FILES, 1,
+      ac_cv_sys_large_files,
+      [Define for large files, on AIX-style hosts.],
+      [_AC_SYS_LARGEFILE_TEST_INCLUDES])
+  fi
+
+  AH_VERBATIM([_DARWIN_USE_64_BIT_INODE],
+[/* Enable large inode numbers on Mac OS X.  */
+#ifndef _DARWIN_USE_64_BIT_INODE
+# define _DARWIN_USE_64_BIT_INODE 1
+#endif])
+fi
+])# AC_SYS_LARGEFILE
+
+])# m4_version_prereq 2.69
diff --git a/modules/large-inode b/modules/large-inode
deleted file mode 100644
index de067ac..0000000
--- a/modules/large-inode
+++ /dev/null
@@ -1,22 +0,0 @@
-Description:
-Use large inode numbers.
-
-Files:
-m4/large-inode.m4
-
-Depends-on:
-
-configure.ac-early:
-AC_REQUIRE([gl_SYS_LARGE_INODE])
-
-configure.ac:
-
-Makefile.am:
-
-Include:
-
-License:
-LGPLv2+
-
-Maintainer:
-all
diff --git a/modules/largefile b/modules/largefile
new file mode 100644
index 0000000..d4a2e4b
--- /dev/null
+++ b/modules/largefile
@@ -0,0 +1,22 @@
+Description:
+Use large files.
+
+Files:
+m4/largefile.m4
+
+Depends-on:
+
+configure.ac:
+AC_REQUIRE([AC_SYS_LARGEFILE])
+
+configure.ac:
+
+Makefile.am:
+
+Include:
+
+License:
+LGPLv2+
+
+Maintainer:
+all
-- 
1.7.4.4

Message #29 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Sat, 23 Jul 2011 15:53:57 +0200

Paul Eggert wrote:
> Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems
> 
> * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
> implementations that use only 32 bits to count blocks.

But this change has also effects on other platforms than MacOS X 10.7:

- On glibc/Hurd, the type of f_blocks in 'struct statvfs' is __fsblkcnt64_t or
  __fsblkcnt_t, depending on __USE_FILE_OFFSET64.
  So fsusage.m4 ought to require AC_SYS_LARGEFILE.

- On MacOS X 10.5, the type of f_blocks is fsblkcnt_t, which is
  'unsigned int'. The configure test therefore now fails, STAT_STATFS2_BSIZE
  gets defined to 1, and fsusage.c calls statfs() instead of statvfs().
  And 'struct statfs' contains a f_blocks that has 64 bits if and only if
  __DARWIN_64_BIT_INO_T is defined.
  So fsusage.m4 ought to require AC_SYS_LARGEFILE, and a module dependency
  to 'largefile' is needed, so that it picks up your new AC_SYS_LARGEFILE code.

- On FreeBSD 6.4, NetBSD 3.0..5.0, OpenBSD 4.9, the type of f_blocks is
  fsblkcnt_t, which is already a 64-bit integer.
  So these platforms continue to use statvfs().

- On AIX 5.1..7.1, the type of f_blocks is fsblkcnt_t, which is 'unsigned long',
  hence usually 32-bit. The configure test therefore now fails,
  STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of
  statvfs(). But statfs() has a 'fsblkcnt_t f_blocks' as well, so the switch
  from statvfs() to statfs() buys us nothing.
  What would bring something (on AIX >= 5.2) is to use statvfs64 or statfs64,
  which both have a 64-bit f_blocks field.

- On HP-UX 11.00..11.31, the type of f_blocks is fsblkcnt_t, which is 64-bit
  if and only if _APP32_64BIT_OFF_T is defined. By default it is 32-bit, hence
  the configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1,
  and fsusage.c calls statfs() instead of statvfs(). But statfs() has an
  'int32_t f_blocks', so the switch from statvfs() to statfs() buys us nothing.
  What would bring something is to define _FILE_OFFSET_BITS to 64, which has
  the effect of enabling _APP32_64BIT_OFF_T. Or alternatively, define
  _LARGEFILE64_SOURCE and use statvfs64 instead of statvfs.

- On IRIX 6.5, the type of f_blocks is fsblkcnt_t, which is already a 64-bit
  integer, so these platforms continue to use statvfs(). One could define
  _LARGEFILE64_SOURCE and use statvfs64, but that appears to be not necessary.

- On Solaris 7..10, the type of f_blocks is fsblkcnt_t, which is 64-bit in
  64-bit mode builds (_LP64) or if _FILE_OFFSET_BITS is 64. By default it
  usually is 32-bit, the configure test therefore now fails, STAT_STATFS4
  gets defined to 1, and fsusage.c calls statfs() instead of statvfs().
  But statfs() has a 'long f_blocks', so the switch from statvfs() to statfs()
  buys us nothing.
  What would bring something is to define _FILE_OFFSET_BITS to 64.

- On Cygwin (both 1.5.x and 1.7.5), the type of f_blocks is fsblkcnt_t, which
  is 'unsigned long', that is, 32-bit. The configure test therefore now fails,
  STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of
  statvfs(). But statfs() has a 'long f_blocks', so the switch from statvfs()
  to statfs() buys us nothing.
  I don't see an API that supports 64-bit f_blocks on Cygwin.

- On Interix 3.5, the type of f_blocks is 'unsigned long', which is 32-bit.
  The configure test therefore now fails, and I don't know which API gets used
  instead: Interix has no statfs. So it appears the patch is killing
  portability to Interix.

- On BeOS and Haiku, the type of f_blocks is fsblkcnt_t, which is 'long long',
  i.e. 64-bit, so these platforms continue to use statvfs().

I will therefore propose

1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'.
   This will make statvfs() use a 64-bit f_blocks on
   glibc/Hurd, MacOS X >= 10.4, HP-UX, Solaris.

2) To undo the patch that introduces check_f_blocks_size, because
     - on MacOS X 10.7 it is not needed any more after 1),
     - on AIX and Cywin it causes non-POSIX API to be used for no reason,
     - it kills portability to Interix 3.5.

3) To make use of statvfs64 on AIX.

Bruno
-- 
In memoriam Adam Czerniaków <http://en.wikipedia.org/wiki/Adam_Czerniaków>

Message #32 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
	Herb Wartens <hawartens <at> gmail.com>
Subject: fsusage: add large volume support on glibc/Hurd, HP-UX 11, Solaris,
	MacOS X
Date: Sat, 23 Jul 2011 16:32:37 +0200

> 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'.
>    This will make statvfs() use a 64-bit f_blocks on
>    glibc/Hurd, MacOS X >= 10.4, HP-UX, Solaris.

Here's the first proposed patch. I verified that

- on HP-UX 11.00 and 11.31, Solaris 7 and 10, it causes statvfs() to be used
  again:
    checking for statvfs function (SVR4)... no
  becomes
    checking for statvfs function (SVR4)... yes
  Since the check_f_blocks_size is in place, it proves that this causes
  f_blocks to be 64-bit.

- on MacOS X 10.5, it causes fsusage.o to call statfs$INODE64 instead of
  statfs, thus also supporting large volumes.


2011-07-23  Bruno Haible  <bruno <at> clisp.org>

	fsusage: Enable large volume support on glibc/Hurd, HP-UX, Solaris, MacOS X.
	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Require AC_SYS_LARGEFILE.
	* modules/fsusage (Depends-on): Add largefile.

--- m4/fsusage.m4.orig	Sat Jul 23 16:25:19 2011
+++ m4/fsusage.m4	Sat Jul 23 16:24:57 2011
@@ -1,4 +1,4 @@
-# serial 28
+# serial 29
 # Obtaining file system usage information.
 
 # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc.
@@ -29,6 +29,12 @@
 
 AC_DEFUN([gl_FILE_SYSTEM_USAGE],
 [
+dnl Enable large-file support. This has the effect of changing the size
+dnl of field f_blocks in 'struct statvfs' from 32 bit to 64 bit on
+dnl glibc/Hurd, HP-UX 11, Solaris (32-bit mode). It also changes the size
+dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on
+dnl MacOS X >= 10.5 (32-bit mode).
+AC_REQUIRE([AC_SYS_LARGEFILE])
 
 AC_MSG_NOTICE([checking how to get file system space usage])
 ac_fsusage_space=no
--- modules/fsusage.orig	Sat Jul 23 16:25:19 2011
+++ modules/fsusage	Sat Jul 23 16:06:17 2011
@@ -7,6 +7,7 @@
 m4/fsusage.m4
 
 Depends-on:
+largefile
 stdbool
 stdint
 full-read       [test $gl_cv_fs_space = yes]
-- 
In memoriam Adam Czerniaków <http://en.wikipedia.org/wiki/Adam_Czerniaków>

Message #35 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
	Herb Wartens <hawartens <at> gmail.com>
Subject: fsusage: revert unintended change on AIX, Cygwin, Interix
Date: Sat, 23 Jul 2011 16:53:40 +0200

> 2) To undo the patch that introduces check_f_blocks_size, because
>      - on MacOS X 10.7 it is not needed any more after 1),
>      - on AIX and Cywin it causes non-POSIX API to be used for no reason,
>      - it kills portability to Interix 3.5.

Slight correction: It is still needed after 1), at least on MacOS X 10.5.
But only on MacOS X. So here's the proposed patch 2. Verified on AIX and
Cygwin; I don't have access to an Interix machine.


2011-07-23  Bruno Haible  <bruno <at> clisp.org>

	fsusage: Restore previous behaviour on AIX, Cygwin, Interix.
	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Enforce a 64-bit struct statvfs
	f_blocks field only on MacOS X.

*** m4/fsusage.m4.orig	Sat Jul 23 16:44:07 2011
--- m4/fsusage.m4	Sat Jul 23 16:42:32 2011
***************
*** 62,75 ****
  "Do not use Tru64's statvfs implementation"
  #endif
  
- #include <limits.h>
  #include <sys/statvfs.h>
  
- /* Reject implementations, such as MacOS X 10.7, where f_blocks is a
-    32-bit quantity; that commonly limits file systems to 4 TiB, a
-    ridiculously small limit these days.  */
  struct statvfs fsd;
  int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT <= 32 ? -1 : 1];
  ]],
                                      [[statvfs (0, &fsd);]])],
                                   [fu_cv_sys_stat_statvfs=yes],
--- 62,80 ----
  "Do not use Tru64's statvfs implementation"
  #endif
  
  #include <sys/statvfs.h>
  
  struct statvfs fsd;
+ 
+ #if defined __APPLE__ && defined __MACH__
+ #include <limits.h>
+ /* On MacOS X >= 10.5, f_blocks in 'struct statvfs' is a 32-bit quantity;
+   that commonly limits file systems to 4 TiB.  Whereas f_blocks in
+   'struct statfs' is a 64-bit type, thanks to the large-file support
+   that was enabled above.  In this case, don't use statvfs(); use statfs()
+   instead.  */
  int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT <= 32 ? -1 : 1];
+ #endif
  ]],
                                      [[statvfs (0, &fsd);]])],
                                   [fu_cv_sys_stat_statvfs=yes],
-- 
In memoriam Adam Czerniaków <http://en.wikipedia.org/wiki/Adam_Czerniaków>

Message #38 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Sat, 23 Jul 2011 17:02:46 +0200

Bruno Haible wrote:
> Paul Eggert wrote:
>> Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems
>>
>> * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
>> implementations that use only 32 bits to count blocks.
>
> But this change has also effects on other platforms than MacOS X 10.7:
>
> - On glibc/Hurd, the type of f_blocks in 'struct statvfs' is __fsblkcnt64_t or
...
> - On MacOS X 10.5, the type of f_blocks is fsblkcnt_t, which is
...
> - On FreeBSD 6.4, NetBSD 3.0..5.0, OpenBSD 4.9, the type of f_blocks is
...
> - On AIX 5.1..7.1, the type of f_blocks is fsblkcnt_t, which is 'unsigned long',
...
> - On HP-UX 11.00..11.31, the type of f_blocks is fsblkcnt_t, which is 64-bit
...
> - On IRIX 6.5, the type of f_blocks is fsblkcnt_t, which is already a 64-bit
...
> - On Solaris 7..10, the type of f_blocks is fsblkcnt_t, which is 64-bit in
...
> - On Cygwin (both 1.5.x and 1.7.5), the type of f_blocks is fsblkcnt_t, which
...
> - On Interix 3.5, the type of f_blocks is 'unsigned long', which is 32-bit.
...
> - On BeOS and Haiku, the type of f_blocks is fsblkcnt_t, which is 'long long',
...
>
> I will therefore propose
>
> 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'.
>    This will make statvfs() use a 64-bit f_blocks on
>    glibc/Hurd, MacOS X >= 10.4, HP-UX, Solaris.
>
> 2) To undo the patch that introduces check_f_blocks_size, because
>      - on MacOS X 10.7 it is not needed any more after 1),
>      - on AIX and Cywin it causes non-POSIX API to be used for no reason,
>      - it kills portability to Interix 3.5.
>
> 3) To make use of statvfs64 on AIX.

Wow.  Thanks for all the analysis and testing.

Message #41 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: fsusage: add large volume support on glibc/Hurd, HP-UX 11, Solaris,
	MacOS X
Date: Sat, 23 Jul 2011 17:04:44 +0200

Bruno Haible wrote:
>> 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'.
>>    This will make statvfs() use a 64-bit f_blocks on
>>    glibc/Hurd, MacOS X >= 10.4, HP-UX, Solaris.
>
> Here's the first proposed patch. I verified that
>
> - on HP-UX 11.00 and 11.31, Solaris 7 and 10, it causes statvfs() to be used
>   again:
>     checking for statvfs function (SVR4)... no
>   becomes
>     checking for statvfs function (SVR4)... yes
>   Since the check_f_blocks_size is in place, it proves that this causes
>   f_blocks to be 64-bit.
>
> - on MacOS X 10.5, it causes fsusage.o to call statfs$INODE64 instead of
>   statfs, thus also supporting large volumes.
>
>
> 2011-07-23  Bruno Haible  <bruno <at> clisp.org>
>
> 	fsusage: Enable large volume support on glibc/Hurd, HP-UX, Solaris, MacOS X.
> 	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Require AC_SYS_LARGEFILE.
> 	* modules/fsusage (Depends-on): Add largefile.
>
> --- m4/fsusage.m4.orig	Sat Jul 23 16:25:19 2011
> +++ m4/fsusage.m4	Sat Jul 23 16:24:57 2011
> @@ -1,4 +1,4 @@
> -# serial 28
> +# serial 29
>  # Obtaining file system usage information.
>
>  # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc.
> @@ -29,6 +29,12 @@
>
>  AC_DEFUN([gl_FILE_SYSTEM_USAGE],
>  [
> +dnl Enable large-file support. This has the effect of changing the size
> +dnl of field f_blocks in 'struct statvfs' from 32 bit to 64 bit on
> +dnl glibc/Hurd, HP-UX 11, Solaris (32-bit mode). It also changes the size
> +dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on
> +dnl MacOS X >= 10.5 (32-bit mode).
> +AC_REQUIRE([AC_SYS_LARGEFILE])
>
>  AC_MSG_NOTICE([checking how to get file system space usage])
>  ac_fsusage_space=no
> --- modules/fsusage.orig	Sat Jul 23 16:25:19 2011
> +++ modules/fsusage	Sat Jul 23 16:06:17 2011
> @@ -7,6 +7,7 @@
>  m4/fsusage.m4
>
>  Depends-on:
> +largefile
>  stdbool
>  stdint
>  full-read       [test $gl_cv_fs_space = yes]

Well-documented, tested, and safe-looking.
Thanks!

Message #44 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: fsusage: revert unintended change on AIX, Cygwin, Interix
Date: Sat, 23 Jul 2011 17:06:57 +0200

Bruno Haible wrote:
>> 2) To undo the patch that introduces check_f_blocks_size, because
>>      - on MacOS X 10.7 it is not needed any more after 1),
>>      - on AIX and Cywin it causes non-POSIX API to be used for no reason,
>>      - it kills portability to Interix 3.5.
>
> Slight correction: It is still needed after 1), at least on MacOS X 10.5.
> But only on MacOS X. So here's the proposed patch 2. Verified on AIX and
> Cygwin;

Thanks again.
This looks fine.

> I don't have access to an Interix machine.

No problem.  It seems to have so few coreutils/gnulib users that
it barely qualifies as a reasonable portability target.

> 2011-07-23  Bruno Haible  <bruno <at> clisp.org>
>
> 	fsusage: Restore previous behaviour on AIX, Cygwin, Interix.
> 	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Enforce a 64-bit struct statvfs
> 	f_blocks field only on MacOS X.
...

Message #47 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
	Herb Wartens <hawartens <at> gmail.com>
Subject: fsusage: add large volume support on AIX
Date: Sat, 23 Jul 2011 17:24:31 +0200

> 3) To make use of statvfs64 on AIX.

Here's the proposed patch for it. As expected we get:
  - On AIX 5.2, 6.1, 7.1:
    checking whether to use statvfs64... yes
  - On AIX 5.1, HP-UX, IRIX, Solaris:
    checking whether to use statvfs64... no


2011-07-23  Bruno Haible  <bruno <at> clisp.org>

	fsusage: Enable large volume support on AIX >= 5.2.
	* m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): If 'struct statvfs64' has a larger
	f_blocks field than 'struct statvfs', define STAT_STATVFS64 instead of
	STAT_STATVFS.
	* lib/fsusage.c (get_fs_usage) [STAT_STATVFS64]: Use statvfs64.

--- lib/fsusage.c.orig	Sat Jul 23 17:15:05 2011
+++ lib/fsusage.c	Sat Jul 23 17:13:38 2011
@@ -23,7 +23,7 @@
 #include <limits.h>
 #include <sys/types.h>
 
-#if STAT_STATVFS                /* POSIX 1003.1-2001 (and later) with XSI */
+#if STAT_STATVFS || STAT_STATVFS64 /* POSIX 1003.1-2001 (and later) with XSI */
 # include <sys/statvfs.h>
 #else
 /* Don't include backward-compatibility files unless they're needed.
@@ -106,6 +106,18 @@
                         ? PROPAGATE_ALL_ONES (fsd.f_frsize)
                         : PROPAGATE_ALL_ONES (fsd.f_bsize));
 
+#elif defined STAT_STATVFS64            /* AIX */
+
+  struct statvfs64 fsd;
+
+  if (statvfs64 (file, &fsd) < 0)
+    return -1;
+
+  /* f_frsize isn't guaranteed to be supported.  */
+  fsp->fsu_blocksize = (fsd.f_frsize
+                        ? PROPAGATE_ALL_ONES (fsd.f_frsize)
+                        : PROPAGATE_ALL_ONES (fsd.f_bsize));
+
 #elif defined STAT_STATFS2_FS_DATA      /* Ultrix */
 
   struct fs_data fsd;
@@ -223,7 +235,7 @@
 
 #endif
 
-#if (defined STAT_STATVFS \
+#if (defined STAT_STATVFS || defined STAT_STATVFS64 \
      || (!defined STAT_STATFS2_FS_DATA && !defined STAT_READ_FILSYS))
 
   fsp->fsu_blocks = PROPAGATE_ALL_ONES (fsd.f_blocks);
--- m4/fsusage.m4.orig	Sat Jul 23 17:15:05 2011
+++ m4/fsusage.m4	Sat Jul 23 17:12:20 2011
@@ -1,4 +1,4 @@
-# serial 29
+# serial 30
 # Obtaining file system usage information.
 
 # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc.
@@ -81,8 +81,32 @@
                                  [fu_cv_sys_stat_statvfs=no])])
   if test $fu_cv_sys_stat_statvfs = yes; then
     ac_fsusage_space=yes
-    AC_DEFINE([STAT_STATVFS], [1],
-              [  Define if there is a function named statvfs.  (SVR4)])
+    # AIX >= 5.2 has statvfs64 that has a wider f_blocks field than statvfs.
+    # glibc, HP-UX, IRIX, Solaris have statvfs64 as well, but on these systems
+    # statvfs with large-file support is already equivalent to statvfs64.
+    AC_CACHE_CHECK([whether to use statvfs64],
+      [fu_cv_sys_stat_statvfs64],
+      [AC_LINK_IFELSE(
+         [AC_LANG_PROGRAM(
+            [[#include <sys/types.h>
+              #include <sys/statvfs.h>
+              struct statvfs64 fsd;
+              int check_f_blocks_larger_in_statvfs64
+                [sizeof (((struct statvfs64 *) 0)->f_blocks)
+                 > sizeof (((struct statvfs *) 0)->f_blocks)
+                 ? 1 : -1];
+            ]],
+            [[statvfs64 (0, &fsd);]])],
+         [fu_cv_sys_stat_statvfs64=yes],
+         [fu_cv_sys_stat_statvfs64=no])
+      ])
+    if test $fu_cv_sys_stat_statvfs64 = yes; then
+      AC_DEFINE([STAT_STATVFS64], [1],
+                [  Define if statvfs64 should be preferred over statvfs.])
+    else
+      AC_DEFINE([STAT_STATVFS], [1],
+                [  Define if there is a function named statvfs.  (SVR4)])
+    fi
   fi
 fi
 
-- 
In memoriam Adam Czerniaków <http://en.wikipedia.org/wiki/Adam_Czerniaków>

Message #50 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9140 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: fsusage: add large volume support on AIX
Date: Sat, 23 Jul 2011 08:47:52 -0700

Thanks for all that work in analysis, and porting checking, for
statvfs etc.  Your patches look good to me too.

Message #53 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9140 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: fsusage: add large volume support on AIX
Date: Sun, 24 Jul 2011 12:26:16 +0200

Paul Eggert wrote:
> Your patches look good to me too.

Thanks for the review. I've pushed the 3 patches now.

You can add a coreutils/NEWS entry that says:

'df' now supports volumes larger than 4 TiB on MacOS X 10.5 or newer
and on AIX 5.2 or newer.

Bruno
-- 
In memoriam Ezechiele Ramin <http://en.wikipedia.org/wiki/Ezechiele_Ramin>

Message #56 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: fsusage: add large volume support on AIX
Date: Sun, 24 Jul 2011 23:04:41 +0200

Bruno Haible wrote:
> Paul Eggert wrote:
>> Your patches look good to me too.
>
> Thanks for the review. I've pushed the 3 patches now.
>
> You can add a coreutils/NEWS entry that says:
>
> 'df' now supports volumes larger than 4 TiB on MacOS X 10.5 or newer
> and on AIX 5.2 or newer.

Thanks.  I've done this:

From c29cb31b7ff536d2367dad1dcb19c554a8752233 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Sun, 24 Jul 2011 22:52:34 +0200
Subject: [PATCH] df: support partitions larger than 4 TiB on MacOS >= 10.5 &
 AIX >=5.2

This change derives from improvements to gnulib's fsusage module.
* NEWS (Improvements): df now supports disk partitions larger than
4 TiB on MacOS X 10.5 or newer and on AIX 5.2 or newer.
Alphabetize entries.
* gnulib: Update to latest.
---
 NEWS   |   17 ++++++++++-------
 gnulib |    2 +-
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/NEWS b/NEWS
index 2952dc9..0720719 100644
--- a/NEWS
+++ b/NEWS
@@ -30,6 +30,10 @@ GNU coreutils NEWS                                    -*- outline -*-

 ** New features

+  md5sum accepts the new --strict option.  With --check, it makes the
+  tool exit non-zero for any invalid input line, rather than just warning.
+  This also affects sha1sum, sha224sum, sha384sum and sha512sum.
+
   split accepts a new --filter=CMD option.  With it, split filters output
   through CMD.  CMD may use the $FILE environment variable, which is set to
   the nominal output file name for each invocation of CMD.  For example, to
@@ -38,22 +42,21 @@ GNU coreutils NEWS                                    -*- outline -*-
   Note the use of single quotes, not double quotes.
   That creates files named xaa.xz, xab.xz and xac.xz.

-  md5sum accepts the new --strict option.  With --check, it makes the
-  tool exit non-zero for any invalid input line, rather than just warning.
-  This also affects sha1sum, sha224sum, sha384sum and sha512sum.
-
   timeout accepts a new --foreground option, to support commands not started
   directly from a shell prompt, where the command is interactive or needs to
   receive signals initiated from the terminal.

 ** Improvements

-  shuf outputs small subsets of large permutations much more efficiently.
-  For example `shuf -i1-$((2**32-1)) -n2` no longer exhausts memory.
-
   cp and ls now support HP-UX 11.11's ACLs, thanks to improved support
   in gnulib.

+  df now supports disk partitions larger than 4 TiB on MacOS X 10.5
+  or newer and on AIX 5.2 or newer.
+
+  shuf outputs small subsets of large permutations much more efficiently.
+  For example `shuf -i1-$((2**32-1)) -n2` no longer exhausts memory.
+
   stat -f now recognizes the GPFS, MQUEUE and PSTOREFS file system types.

 ** Build-related
diff --git a/gnulib b/gnulib
index bd399f0..bac9647 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit bd399f07ee4f383fad038efad25a659fcdc0bbb0
+Subproject commit bac964725a412c590498aba68f6b9a8e723ae474
--
1.7.6.609.gbf6a9

Message #59 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Sun, 24 Jul 2011 23:59:42 +0200

Paul Eggert wrote:
> * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs
> implementations that use only 32 bits to count blocks.

Dealing with the statvfs() / statfs() calls is one thing.
But 'df' also relies on the 'mountlist' module, which on MacOS X
uses the function getmntinfo(). This function too exists in
two variants

  getmntinfo
  getmntinfo$INODE64   equivalent to   getmntinfo64

and the first one returns 'struct statfs' entities with 32-bit
f_blocks. Likewise there is

  getfsstat
  getfsstat$INODE64   equivalent to   getfsstat64


Here's an excerpt from the manual page:

NAME
     getmntinfo -- get information about mounted file systems

...
     int
     getmntinfo(struct statfs **mntbufp, int flags);

     int
     getmntinfo64(struct statfs64 **mntbufp, int flags);

...
     The getmntinfo() function passes its flags argument transparently to
     getfsstat(2), while the getmntinfo64() function passes
     its flags argument transparently to getfsstat64().

See [1][2]. So, if the 'mountlist' modules is used outside of coreutils, it
also needs the 'largefile' module, so as to map getmntinfo to
getmntinfo$INODE64. Here's a proposed patch.

[1] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/getmntinfo.3.html
[2] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/getfsstat.2.html


2011-07-24  Bruno Haible  <bruno <at> clisp.org>

	mountlist: Enable large volume support on MacOS X >= 10.5.
	* m4/ls-mntd-fs.m4 (gl_LIST_MOUNTED_FILE_SYSTEMS): Require AC_SYS_LARGEFILE.
	* modules/mountlist (Depends-on): Add largefile.

--- m4/ls-mntd-fs.m4.orig	Sun Jul 24 23:44:01 2011
+++ m4/ls-mntd-fs.m4	Sun Jul 24 22:41:08 2011
@@ -1,4 +1,4 @@
-# serial 29
+# serial 30
 # How to list mounted file systems.
 
 # Copyright (C) 1998-2004, 2006, 2009-2011 Free Software Foundation, Inc.
@@ -26,6 +26,12 @@
 # gl_LIST_MOUNTED_FILE_SYSTEMS([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]])
 AC_DEFUN([gl_LIST_MOUNTED_FILE_SYSTEMS],
   [
+dnl Enable large-file support. This has the effect of changing the size
+dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on
+dnl MacOS X >= 10.5 (32-bit mode), allowing file systems >= 4 TiB to be
+dnl found.
+AC_REQUIRE([AC_SYS_LARGEFILE])
+
 AC_CHECK_FUNCS([listmntent getmntinfo])
 AC_CHECK_HEADERS_ONCE([sys/param.h sys/statvfs.h])
 
--- modules/mountlist.orig	Sun Jul 24 23:44:01 2011
+++ modules/mountlist	Sun Jul 24 22:30:10 2011
@@ -9,6 +9,7 @@
 m4/mountlist.m4
 
 Depends-on:
+largefile
 stdbool
 stdint
 strstr-simple
-- 
In memoriam Ezechiele Ramin <http://en.wikipedia.org/wiki/Ezechiele_Ramin>

Message #62 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9140 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org,
	Herb Wartens <hawartens <at> gmail.com>
Subject: Re: bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Date: Sun, 24 Jul 2011 23:55:02 -0700

On 07/24/11 14:59, Bruno Haible wrote:

> +dnl Enable large-file support. This has the effect of changing the size
> +dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on
> +dnl MacOS X >= 10.5 (32-bit mode), allowing file systems >= 4 TiB to be
> +dnl found.
> +AC_REQUIRE([AC_SYS_LARGEFILE])
> +
>  AC_CHECK_FUNCS([listmntent getmntinfo])
>  AC_CHECK_HEADERS_ONCE([sys/param.h sys/statvfs.h])

Why is this part of the patch needed?  If the mountlist module
depends on the largefile module, AC_SYS_LARGEFILE is already
guaranteed to be invoked.  Please see, for example, my proposed
patch in <http://lists.gnu.org/archive/html/bug-gnulib/2011-07/msg00361.html>
and updated just now, which changes mountlist to depend on largefile,
but does not add the AC_REQUIRE (and in fact removes a couple of
AC_REQUIRE calls that are no longer needed, in mkstemp.m4 and
tempname.m4).

Message #65 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9140 <at> debbugs.gnu.org, Bug-gnulib <bug-gnulib <at> gnu.org>,
	Pádraig Brady <P <at> draigBrady.com>
Subject: Re: [PATCH 2/3] large-inode: New module
Date: Mon, 08 Aug 2011 09:36:29 -0600

On 07/22/2011 04:10 PM, Paul Eggert wrote:
> On 07/22/11 10:45, Pádraig Brady wrote:
>> If it works it would be better in AC_SYS_LARGEFILE (as well) I think.
>
> OK, thanks, I pushed this into gnulib.  Assuming it works (and I don't
> see why it'd hurt) I'll push it into autoconf.

Ouch.  This commit broke libvirt 0.9.4, because it unintentionally broke 
large-file support altogether.  It will probably break any other 
packages that have picked up the latest gnulib in the last few days.

> Pádraig Brady suggested this in	<http://debbugs.gnu.org/9140#20>.
> * MODULES.html.sh: Add largefile, remove large-inode.
> * modules/largefile, m4/largefile.m4: New files.
> * modules/large-inode, m4/large-inode.m4: Remove.

> -
> -configure.ac-early:
> -AC_REQUIRE([gl_SYS_LARGE_INODE])
> -
> -configure.ac:
> -

> +
> +configure.ac:
> +AC_REQUIRE([AC_SYS_LARGEFILE])
> +
> +configure.ac:
> +

The first configure.ac should have been configure.ac-early; as a result 
of the duplicate listing, gnulib-tool completely omits AC_SYS_LARGEFILE 
from any configure.ac that was using the broken largefile module.  Test 
by looking at './configure --help | grep large' pre- and post-patch. 
I'm pushing this:

diff --git i/ChangeLog w/ChangeLog
index 72e7611..3fd0c47 100644
--- i/ChangeLog
+++ w/ChangeLog
@@ -1,3 +1,8 @@
+2011-08-08  Eric Blake  <eblake <at> redhat.com>
+
+	largefile: fix typo that regressed large file support
+	* modules/largefile (configure.ac-early): Fix section name.
+
 2011-08-06  Karl Berry  <karl <at> gnu.org>

 	* MODULES.html.sh (func_all_files): _Noreturn is no longer
diff --git i/modules/largefile w/modules/largefile
index d4a2e4b..ca10d48 100644
--- i/modules/largefile
+++ w/modules/largefile
@@ -6,7 +6,7 @@ m4/largefile.m4

 Depends-on:

-configure.ac:
+configure.ac-early:
 AC_REQUIRE([AC_SYS_LARGEFILE])

 configure.ac:


-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

Message #68 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9140 <at> debbugs.gnu.org, Bug-gnulib <bug-gnulib <at> gnu.org>,
	Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#9140: [PATCH 2/3] large-inode: New module
Date: Mon, 08 Aug 2011 09:38:26 -0600

On 07/22/2011 04:10 PM, Paul Eggert wrote:
> On 07/22/11 10:45, Pádraig Brady wrote:
>> If it works it would be better in AC_SYS_LARGEFILE (as well) I think.
>
> OK, thanks, I pushed this into gnulib.  Assuming it works (and I don't
> see why it'd hurt) I'll push it into autoconf.

No problem with me to pushing the AC_SYS_LARGEFILE improvements into 
autoconf.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

Message #71 received at 9140 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: 9140 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
	Eric Blake <eblake <at> redhat.com>
Subject: Re: [PATCH 2/3] large-inode: New module
Date: Mon, 8 Aug 2011 22:38:24 +0200

Eric Blake wrote:
> It will probably break any other 
> packages that have picked up the latest gnulib in the last few days.

Indeed. It broke libiconv-1.14. But fortunately it's not a regression.
Large file support in the 'iconv' program would have been an undocumented
improvement. Now it's delayed to libiconv-1.15...

Thanks for having discovered and fixed this, Eric!!

Bruno
-- 
In memoriam Edward Pimental <http://en.wikipedia.org/wiki/Edward_Pimental>

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.