GNU logs - #35488, boring messages

Message sent to bug-coreutils@HIDDEN:

Subject: bug#35488: Feature du --files-only request
Resent-From: David Ellenberger <davidmarioellenberger@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Mon, 29 Apr 2019 15:06:02 +0000
Resent-Message-ID: <handler.35488.B.155655034120215 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 35488 <at> debbugs.gnu.org
MIME-Version: 1.0
From: David Ellenberger <davidmarioellenberger@HIDDEN>
Date: Mon, 29 Apr 2019 13:36:31 +0200
Message-ID: <CADH3eD1EqJMhxowYYZT9gYZ-cjgJnbHa-uqUJb_est3G_zAjEQ@HIDDEN>
Content-Type: multipart/alternative; boundary="000000000000d61e1e0587a9b251"
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

--000000000000d61e1e0587a9b251
Content-Type: text/plain; charset="UTF-8"

Dear maintainers

As you're probably aware, du --apparent-size calculates/reports 1 file
system block for a [empty] folder (typically 4096 bytes). I find that a bit
inconsistent with what the option suggests. The manual entry to
--apparent-size doesn't help the understanding.

From a practical point of view, when we admins copy a folder&files
structure from one to another file system where block size doesn't
correspond, we cannot use du to get a count comparison and have to resort
to something like:
$ ls -anR | grep -v '^d' | awk '{total += $5} END {print total, "Bytes"}'

Windows explorer shows zero bytes for an empty folder or folder containing
multiple empty folders. This way, comparing two copied folders&files's
content by size works out well regardless of file system and its block
sizes it uses.

I understand that admins have become accustomed to see 4096 in directories
as it's consistent with the ls command and the technicality behind it.

In my daily admin tasks I never had to count sizes of empty folders. The
overhead of provisioning and enable the file system to work is something we
typically accept and do not require to re-calculate nor even to understand
in all details. Anyway, the FS provisioning and logical blocks perspective
is a complete different things for which we have the df command and other
tools.

I'd therefore suggest a new option --files-only (which calculates only the
size of files and skips over anything else that has a directory attribute
flag, device, symbolic link etc..).

Like that we would finally be able to count file sizes consistently align
with the du manual entry which says 'DESCRIPTION: Summarize disk usage of
each FILE, recursively for directories.'

Thanks for reading,

David

--000000000000d61e1e0587a9b251
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr">Dear maintainers<div><br=
></div><div>As you&#39;re probably aware, du=C2=A0<font color=3D"#000000">-=
-apparent-size calculates/reports 1 file system block for a [empty] folder =
(typically 4096 bytes). I find that a bit inconsistent with what the option=
 suggests. The manual entry </font><span style=3D"color:rgb(0,0,0)">to=C2=
=A0</span><span style=3D"color:rgb(0,0,0)">--apparent-size</span><span styl=
e=3D"color:rgb(0,0,0)">=C2=A0doesn&#39;t help the understanding.</span></di=
v><div><span style=3D"color:rgb(0,0,0)"><br></span></div><div><font color=
=3D"#000000">From a practical point of view, when we admins copy a folder&a=
mp;files structure from one to another file system where block size doesn&#=
39;t correspond, we cannot use du to get a count comparison and have to res=
ort to something like:</font></div><div><font color=3D"#000000">$ ls -anR |=
 grep -v &#39;^d&#39; | awk &#39;{total +=3D $5} END {print total, &quot;By=
tes&quot;}&#39;</font></div><div><font color=3D"#000000"><br></font></div><=
div><font color=3D"#000000">Windows explorer shows zero bytes for an empty =
folder or folder containing multiple empty folders. This way, comparing two=
 copied folders&amp;files&#39;s content by size works out well regardless o=
f file system and its block sizes it uses.</font></div><div><font color=3D"=
#000000"><br></font></div><div>I understand that admins have become accusto=
med to see 4096 in directories as it&#39;s consistent with the ls command a=
nd the technicality behind it.</div><div><br></div><div>In my daily admin t=
asks I never had to count sizes of empty folders. The overhead of provision=
ing and enable the file system to work is something we typically accept and=
 do not require to re-calculate nor even to understand in all details. Anyw=
ay, the FS provisioning and logical blocks perspective is a complete differ=
ent things for which we have the df command and other tools.</div><div><br>=
</div><div><div>I&#39;d therefore suggest a new option --files-only (which =
calculates only the size of files and skips over anything else that has a d=
irectory attribute flag, device, symbolic link etc..).</div><div><br></div>=
<div>Like that we would finally be able to count file sizes consistently al=
ign with the du manual entry which says &#39;DESCRIPTION: Summarize disk us=
age of each FILE, recursively for directories.&#39;</div></div><div><br></d=
iv><div>Thanks for reading,</div><div><font color=3D"#000000"><br></font></=
div><div><font color=3D"#000000">David</font></div></div></div></div>

--000000000000d61e1e0587a9b251--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: David Ellenberger <davidmarioellenberger@HIDDEN>
Subject: bug#35488: Acknowledgement (Feature du --files-only request)
Message-ID: <handler.35488.B.155655034120215.ack <at> debbugs.gnu.org>
References: <CADH3eD1EqJMhxowYYZT9gYZ-cjgJnbHa-uqUJb_est3G_zAjEQ@HIDDEN>
X-Gnu-PR-Message: ack 35488
X-Gnu-PR-Package: coreutils
Reply-To: 35488 <at> debbugs.gnu.org
Date: Mon, 29 Apr 2019 15:06:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-coreutils@HIDDEN

If you wish to submit further information on this problem, please
send it to 35488 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
35488: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D35488
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-coreutils@HIDDEN:

Subject: bug#35488: Feature du --files-only request
Resent-From: L A Walsh <coreutils@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Thu, 02 May 2019 09:41:02 +0000
Resent-Message-ID: <handler.35488.B.15567900609028 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: David Ellenberger <davidmarioellenberger@HIDDEN>
Cc: 35488 <at> debbugs.gnu.org
Message-ID: <5CCABAEB.6040700@HIDDEN>
Date: Thu, 02 May 2019 02:39:55 -0700
From: L A Walsh <coreutils@HIDDEN>
User-Agent: Thunderbird
MIME-Version: 1.0
References: <CADH3eD1EqJMhxowYYZT9gYZ-cjgJnbHa-uqUJb_est3G_zAjEQ@HIDDEN>
In-Reply-To: <CADH3eD1EqJMhxowYYZT9gYZ-cjgJnbHa-uqUJb_est3G_zAjEQ@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

On 4/29/2019 4:36 AM, David Ellenberger wrote:
> Dear maintainers
>
> I understand that admins have become accustomed to see 4096 in directories
> as it's consistent with the ls command and the technicality behind it.
>   
---
    Except that it isn't 4096 on all file systems.  For empty
directories, I usually see 0 bytes allocated.  And for very large
directories, it may be megabytes of space in the directory.
The directory takes real space on linux/unix and different depending
on what filesystem you use.  On windows in NTFS, directories, I think
can be virtual since the meta info and names are in a file control
block.  In that single case, the directories may really take zero
space in the file system, but that's a quirk of how NTFS works in
windows.  If the real intent is to measure used disk space, including
directories seems advisable as they can take real space on most
file systems that is counted against a user quota if it exists.

    Confusingly, depending on the file-allocation block size on the
source and target (they are often 4k, but don't have to be) and on
the amount of ***fragmentation*** in the free space, of the source
and target.  If free space is heavily fragmented, a directory may
need to be spread out into several areas, making it larger than
necessary if free space wasn't so fragmented.  That's why you often
hear people say you should keep about 15-25% of your disk space
free -- thats so large contiguous areas won't entirely disappear and
storage will be more efficient.

    Anyway, just my opinion, but not sure if du should exclude
directories entirely, but I wouldn't be against separate subtotal
lines for directories and files -- that would make it even more helpful!

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.