GNU logs - #7399, boring messages

Message sent to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:

Subject: bug#7399: du may count one file multiple times if it visible through multiple mounts
Resent-From: Evgeny Kapun <abacabadabacaba@HIDDEN>
Original-Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Resent-To: owner <at> debbugs.gnu.org
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Sun, 14 Nov 2010 14:00:04 +0000
Resent-Message-ID: <handler.7399.B.128974315921200 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 7399 <at> debbugs.gnu.org
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:date:from:user-agent:mime-version:to:subject
	:x-enigmail-version:content-type:content-transfer-encoding;
	b=cx0hMHWIayVsNyUrIdQ/nneMIU4oGTmpL0OAElRuO+NsCQaH5PmGqzMGlbdD84P4Ld
	AkUXkRvsttR/Q10HgVqBuV2VadYdHHr470GynbS0EoKYGM+DdoOdwh3Jm5NcHxUhNXOA
	kmBJfRJY6RObOS5tlBz+9/x1xUcLahCCRMkkw=
Message-ID: <4CDFE134.3000208@HIDDEN>
Date: Sun, 14 Nov 2010 16:16:36 +0300
From: Evgeny Kapun <abacabadabacaba@HIDDEN>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org

Some kernels, such as Linux, permit mounting one filesystem multiple
times. This can make multiple paths refer to the same file, although
neither hard nor symbolic links are involved. In this case, du sometimes
incorrectly counts that file many times:

	$ mkdir dir1 dir2
	# mount --bind dir1 dir2
	$ dd if=/dev/urandom of=dir1/file bs=1k count=1000
	1000+0 records in
	1000+0 records out
	1024000 bytes (1.0 MB) copied, 0.179289 s, 5.7 MB/s
	$ du
	1008	./dir1
	1008	./dir2
	2020	.

As you may see, the file is counted twice, once as dir1/file and then as
dir2/file. However, if du is run with repeated argument, it's behavior
is different:

	$ du . .
	1008	./dir1
	1012	.

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Evgeny Kapun <abacabadabacaba@HIDDEN>
Subject: bug#7399: Acknowledgement (du may count one file multiple times
 if it visible through multiple mounts)
Message-ID: <handler.7399.B.128974315921200.ack <at> debbugs.gnu.org>
References: <4CDFE134.3000208@HIDDEN>
X-Gnu-PR-Message: ack 7399
X-Gnu-PR-Package: coreutils
Reply-To: 7399 <at> debbugs.gnu.org
Date: Sun, 14 Nov 2010 14:00:04 +0000

Thank you for filing a new bug report with GNU.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-coreutils@HIDDEN

If you wish to submit further information on this problem, please
send it to 7399 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
7399: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D7399
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:

Subject: bug#7399: du may count one file multiple times if it visible through multiple mounts
Resent-From: Paul Eggert <eggert@HIDDEN>
Original-Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Resent-To: owner <at> debbugs.gnu.org
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Mon, 15 Nov 2010 07:12:02 +0000
Resent-Message-ID: <handler.7399.B7399.128980510918682 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Evgeny Kapun <abacabadabacaba@HIDDEN>
Cc: 7399 <at> debbugs.gnu.org
Message-ID: <4CE0DE5A.8090409@HIDDEN>
Date: Sun, 14 Nov 2010 23:16:42 -0800
From: Paul Eggert <eggert@HIDDEN>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6
MIME-Version: 1.0
References: <4CDFE134.3000208@HIDDEN>
In-Reply-To: <4CDFE134.3000208@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org

On 11/14/2010 05:16 AM, Evgeny Kapun wrote:
> Some kernels, such as Linux, permit mounting one filesystem multiple
> times. This can make multiple paths refer to the same file, although
> neither hard nor symbolic links are involved.

GNU du (as well as a lot of other programs I expect) doesn't work well
in such environments, which do not conform to POSIX requirements for
file system link counts.  GNU du could easily be fixed to handle these
environments, but at a substantial runtime cost in the normal case,
because it'd have to hash every file it runs across, not just files
with link counts > 1 or that result from multiple arguments.

One possible workaround is to add an option, --hash-all-files say, which causes
du to hash every file it runs across, and thus not double-count files
in such cases.

(Another possible workaround is to tell users "don't do that".  :-)

Message sent to owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:

Subject: bug#7399: du may count one file multiple times if it visible through multiple mounts
Resent-From: Jim Meyering <jim@HIDDEN>
Original-Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Resent-To: owner <at> debbugs.gnu.org
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Mon, 15 Nov 2010 08:02:02 +0000
Resent-Message-ID: <handler.7399.B7399.128980807319970 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Paul Eggert <eggert@HIDDEN>
Cc: Evgeny Kapun <abacabadabacaba@HIDDEN>, 7399 <at> debbugs.gnu.org
From: Jim Meyering <jim@HIDDEN>
In-Reply-To: <4CE0DE5A.8090409@HIDDEN> (Paul Eggert's message of "Sun, 14
	Nov 2010 23:16:42 -0800")
References: <4CDFE134.3000208@HIDDEN> <4CE0DE5A.8090409@HIDDEN>
Date: Mon, 15 Nov 2010 09:06:05 +0100
Message-ID: <87zktbypsi.fsf@HIDDEN>
Lines: 27
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org

Paul Eggert wrote:

> On 11/14/2010 05:16 AM, Evgeny Kapun wrote:
>> Some kernels, such as Linux, permit mounting one filesystem multiple
>> times. This can make multiple paths refer to the same file, although
>> neither hard nor symbolic links are involved.
>
> GNU du (as well as a lot of other programs I expect) doesn't work well
> in such environments, which do not conform to POSIX requirements for
> file system link counts.  GNU du could easily be fixed to handle these
> environments, but at a substantial runtime cost in the normal case,
> because it'd have to hash every file it runs across, not just files
> with link counts > 1 or that result from multiple arguments.
>
> One possible workaround is to add an option, --hash-all-files say, which causes
> du to hash every file it runs across, and thus not double-count files
> in such cases.

du.c already has an internal hash_all variable, and it so happens you
can set it by using du's --files0-from= option.  This should do the trick:

  find dir -print0 | du --files0-from=-

Obviously that's a bit of a kludge.
We shouldn't require a separate find process (and disabling
du's internal traversal code) just to turn this on, so adding
that option might make sense.

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 11 Jan 2019 09:31:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 11 04:31:41 2019
Received: from localhost ([127.0.0.1]:54611 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ghtAB-0005Qn-DE
	for submit <at> debbugs.gnu.org; Fri, 11 Jan 2019 04:31:40 -0500
Received: from mail-pf1-f181.google.com ([209.85.210.181]:40468)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <assafgordon@HIDDEN>) id 1ghtA8-0005KA-Qr
 for control <at> debbugs.gnu.org; Fri, 11 Jan 2019 04:31:37 -0500
Received: by mail-pf1-f181.google.com with SMTP id i12so6705709pfo.7
 for <control <at> debbugs.gnu.org>; Fri, 11 Jan 2019 01:31:36 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=to:references:from:message-id:date:user-agent:mime-version
 :in-reply-to:content-language:content-transfer-encoding;
 bh=RN/B68cVmYg3ooXoFxlT691nU8yF9WNqyBroH588FXg=;
 b=lslrz9CeDMC4SMoSbwUiOQ+woe9p4eQL1UeqbPZnraRcMVoVgOOK6qOI/9YKIsHN49
 5j0+TH7BYVuSHvDkya7uuD+ih0BMzcmsSnFREUm27aYK1/Oyc4F5E6mvMksX7nyShcUs
 u6mU5gmgJqwoNnoabH4sPhAz2NGKw5qXVrIeq/5ktgdfUD1W0taRR4MCB8NZjhG/6Dnq
 9R0F1pVu4nAbMKGW+mJtGswah6M09uUHoezLFMysNeK4B4iZMhHaMbDbBRSuTcHq6aTt
 YQtXqxGtY5MQE89111ma99hUq8PUHrqjeiztCly++VoEg5wrI70Sew1QsMKwXsTk76fQ
 yu6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:to:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=RN/B68cVmYg3ooXoFxlT691nU8yF9WNqyBroH588FXg=;
 b=rviaQtdG4gXgXZRFa7fBJlIS7PqS7F4W/GAXc0rgn+LpwNWUQdcqhEvILQSNVxO7yi
 3ud81aQLrxuCd10a8oR5anB+pUjcDMz5vkRxjrTyzWRE0aFiqOlhA40VEyI7sPlHZVQ+
 kAxkMS1yElrUKS8L8eGggeeCUKJlToHR1rJNCKErlgdGtUpgH9p0jaYqflnudFkj4gYT
 4J5GMM1JIaWfJse5N6nPOjfCOTEYm7f9D+2InHcKrZnTMNVcEbKeGzXUUG87KV5oEbsf
 TFpGfPyw26Q+lYr2ly3K7N9vurxoO9HtVJkUfpn5s335r5pTcgwBed9VBosBV0C+E7+Q
 fxgg==
X-Gm-Message-State: AJcUukdnWrNuiX9VpwfqC/t+cHmvMmp7YZpKYFnRSaW4A1YZBpN32EsF
 Ai0IN96DxPkxl8CYlgUQMnd8SuVm
X-Google-Smtp-Source: ALg8bN6yBub0ZGfRoBIeYYwdyCaIEbMgEYZzm/GDbhNbYmDqnXznVcrRpqXRilFpBeOsyC3fPYDHlA==
X-Received: by 2002:a62:190e:: with SMTP id 14mr13751310pfz.70.1547199090245; 
 Fri, 11 Jan 2019 01:31:30 -0800 (PST)
Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38])
 by smtp.googlemail.com with ESMTPSA id
 w185sm115373318pfb.135.2019.01.11.01.31.28
 for <control <at> debbugs.gnu.org>
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 11 Jan 2019 01:31:28 -0800 (PST)
To: control <at> debbugs.gnu.org
References: <4CDFE134.3000208@HIDDEN> <4CE0DE5A.8090409@HIDDEN>
 <87zktbypsi.fsf@HIDDEN>
From: Assaf Gordon <assafgordon@HIDDEN>
Message-ID: <da123c0d-8fe5-d306-2edb-302a83352583@HIDDEN>
Date: Fri, 11 Jan 2019 02:31:27 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <87zktbypsi.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: severity 7399 wishlist retitle 7399 du: add
 "--hash-all-files" option stop 
 Content analysis details:   (2.0 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail
 provider (assafgordon[at]gmail.com)
 -0.0 SPF_PASS               SPF: sender matches SPF record
 -0.0 RCVD_IN_DNSWL_NONE     RBL: Sender listed at http://www.dnswl.org/,
 no trust [209.85.210.181 listed in list.dnswl.org]
 1.8 MISSING_SUBJECT        Missing Subject: header
 0.2 NO_SUBJECT             Extra score for no subject
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

severity 7399 wishlist
retitle 7399 du: add "--hash-all-files" option
stop

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 11 Jan 2019 09:31:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 11 04:31:41 2019
Received: from localhost ([127.0.0.1]:54611 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ghtAB-0005Qn-DE
	for submit <at> debbugs.gnu.org; Fri, 11 Jan 2019 04:31:40 -0500
Received: from mail-pf1-f181.google.com ([209.85.210.181]:40468)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <assafgordon@HIDDEN>) id 1ghtA8-0005KA-Qr
 for control <at> debbugs.gnu.org; Fri, 11 Jan 2019 04:31:37 -0500
Received: by mail-pf1-f181.google.com with SMTP id i12so6705709pfo.7
 for <control <at> debbugs.gnu.org>; Fri, 11 Jan 2019 01:31:36 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=to:references:from:message-id:date:user-agent:mime-version
 :in-reply-to:content-language:content-transfer-encoding;
 bh=RN/B68cVmYg3ooXoFxlT691nU8yF9WNqyBroH588FXg=;
 b=lslrz9CeDMC4SMoSbwUiOQ+woe9p4eQL1UeqbPZnraRcMVoVgOOK6qOI/9YKIsHN49
 5j0+TH7BYVuSHvDkya7uuD+ih0BMzcmsSnFREUm27aYK1/Oyc4F5E6mvMksX7nyShcUs
 u6mU5gmgJqwoNnoabH4sPhAz2NGKw5qXVrIeq/5ktgdfUD1W0taRR4MCB8NZjhG/6Dnq
 9R0F1pVu4nAbMKGW+mJtGswah6M09uUHoezLFMysNeK4B4iZMhHaMbDbBRSuTcHq6aTt
 YQtXqxGtY5MQE89111ma99hUq8PUHrqjeiztCly++VoEg5wrI70Sew1QsMKwXsTk76fQ
 yu6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:to:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=RN/B68cVmYg3ooXoFxlT691nU8yF9WNqyBroH588FXg=;
 b=rviaQtdG4gXgXZRFa7fBJlIS7PqS7F4W/GAXc0rgn+LpwNWUQdcqhEvILQSNVxO7yi
 3ud81aQLrxuCd10a8oR5anB+pUjcDMz5vkRxjrTyzWRE0aFiqOlhA40VEyI7sPlHZVQ+
 kAxkMS1yElrUKS8L8eGggeeCUKJlToHR1rJNCKErlgdGtUpgH9p0jaYqflnudFkj4gYT
 4J5GMM1JIaWfJse5N6nPOjfCOTEYm7f9D+2InHcKrZnTMNVcEbKeGzXUUG87KV5oEbsf
 TFpGfPyw26Q+lYr2ly3K7N9vurxoO9HtVJkUfpn5s335r5pTcgwBed9VBosBV0C+E7+Q
 fxgg==
X-Gm-Message-State: AJcUukdnWrNuiX9VpwfqC/t+cHmvMmp7YZpKYFnRSaW4A1YZBpN32EsF
 Ai0IN96DxPkxl8CYlgUQMnd8SuVm
X-Google-Smtp-Source: ALg8bN6yBub0ZGfRoBIeYYwdyCaIEbMgEYZzm/GDbhNbYmDqnXznVcrRpqXRilFpBeOsyC3fPYDHlA==
X-Received: by 2002:a62:190e:: with SMTP id 14mr13751310pfz.70.1547199090245; 
 Fri, 11 Jan 2019 01:31:30 -0800 (PST)
Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38])
 by smtp.googlemail.com with ESMTPSA id
 w185sm115373318pfb.135.2019.01.11.01.31.28
 for <control <at> debbugs.gnu.org>
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 11 Jan 2019 01:31:28 -0800 (PST)
To: control <at> debbugs.gnu.org
References: <4CDFE134.3000208@HIDDEN> <4CE0DE5A.8090409@HIDDEN>
 <87zktbypsi.fsf@HIDDEN>
From: Assaf Gordon <assafgordon@HIDDEN>
Message-ID: <da123c0d-8fe5-d306-2edb-302a83352583@HIDDEN>
Date: Fri, 11 Jan 2019 02:31:27 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <87zktbypsi.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: severity 7399 wishlist retitle 7399 du: add
 "--hash-all-files" option stop 
 Content analysis details:   (2.0 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail
 provider (assafgordon[at]gmail.com)
 -0.0 SPF_PASS               SPF: sender matches SPF record
 -0.0 RCVD_IN_DNSWL_NONE     RBL: Sender listed at http://www.dnswl.org/,
 no trust [209.85.210.181 listed in list.dnswl.org]
 1.8 MISSING_SUBJECT        Missing Subject: header
 0.2 NO_SUBJECT             Extra score for no subject
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

severity 7399 wishlist
retitle 7399 du: add "--hash-all-files" option
stop

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.