era eriksson <era@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.era eriksson <era@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Received: (at submit) by debbugs.gnu.org; 13 Nov 2010 21:21:45 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 13 16:21:45 2010 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1PHNXd-0005uB-5I for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:21:45 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <eblake@HIDDEN>) id 1PHNXb-0005u6-Dd for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:21:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eblake@HIDDEN>) id 1PHNcF-0000Bs-9D for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:26:35 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:34014) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eblake@HIDDEN>) id 1PHNcF-0000Bo-78 for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:26:31 -0500 Received: from [140.186.70.92] (port=55801 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PHNcB-00012j-17 for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 16:26:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eblake@HIDDEN>) id 1PHNcA-0000AS-4U for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 16:26:26 -0500 Received: from mx1.redhat.com ([209.132.183.28]:44848) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eblake@HIDDEN>) id 1PHNc3-00008e-NI; Sat, 13 Nov 2010 16:26:19 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id oADLQHs0004049 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sat, 13 Nov 2010 16:26:17 -0500 Received: from [10.3.113.9] ([10.3.113.9]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id oADLQGfn014121; Sat, 13 Nov 2010 16:26:16 -0500 Message-ID: <4CDF0277.3040509@HIDDEN> Date: Sat, 13 Nov 2010 14:26:15 -0700 From: Eric Blake <eblake@HIDDEN> Organization: Red Hat User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101103 Fedora/1.0-0.33.b2pre.fc14 Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.6 MIME-Version: 1.0 To: Bruno Haible <bruno@HIDDEN> Subject: Re: rename() over NFS References: <1286899722-20690-1-git-send-email-gary@HIDDEN> <201011131258.57634.bruno@HIDDEN> <8762w1779e.fsf@HIDDEN> <201011131513.17364.bruno@HIDDEN> In-Reply-To: <201011131513.17364.bruno@HIDDEN> X-Enigmail-Version: 1.1.2 OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enigEE1C28B12D687DEFDAEA0578" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Spam-Score: -7.8 (-------) X-Debbugs-Envelope-To: submit Cc: bug-coreutils@HIDDEN, Paul Eggert <eggert@HIDDEN>, bug-gnulib@HIDDEN, Jim Meyering <jim@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Sender: debbugs-submit-bounces <at> debbugs.gnu.org Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org X-Spam-Score: -7.8 (-------) This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigEE1C28B12D687DEFDAEA0578 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 11/13/2010 07:13 AM, Bruno Haible wrote: > So, assuming Eric also agrees with approach (a), here's a proposal for > fixing the testsuite: >=20 >=20 > 2010-11-13 Bruno Haible <bruno@HIDDEN> >=20 > rename, renameat: Avoid test failures at NFS mounted locations. > * tests/test-rename.h (dentry_exists, assert_nonexistent): New > functions. > (test_rename): Use assert_nonexistent. > * tests/test-rename.c: Include <dirent.h>. > * tests/test-renameat.c: Likewise. > Reported by Gary V. Vaughan <gary@HIDDEN>. For now, I'm okay with weakening the testsuite to ignore the NFS bug; please apply this patch, but let's also document the bug. --=20 Eric Blake eblake@HIDDEN +1-801-349-2682 Libvirt virtualization library http://libvirt.org --------------enigEE1C28B12D687DEFDAEA0578 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBCAAGBQJM3wJ4AAoJEKeha0olJ0NqM2YIAI0Y/Q9wVJxvOWt5NQjxGhwB gdXtBMRYx0KRBsg765b5EnVYom3y2xh3UMBZO0q33nAMOLP1Wj4gwvoQ/jTrScxm FK/NS2PAUsSS1hJrPCpqZnpbMP5jJ1sZTRW0LHKIt8prNC+T2ba4AyQerFVGGGKQ vF87o1H2fBFREh5MJVvAylxRetVcNXUvMHimLNyAmCdDRJr6n/8F1qd4rSWcDxul qoI0Y992TbhDEs+2xFHKbcUw/Sye/yssOBbJ6xLuoZ+1bZLMxakwgE4S3M6P+Rj0 FswOgX9Pf6XVIEkL7V0xzUIipXn87ESMs/s3HMo2mdWtvNtGYuxOax4tgjksTpw= =uTGM -----END PGP SIGNATURE----- --------------enigEE1C28B12D687DEFDAEA0578--
owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:bug#7389; Package coreutils.
Full text available.
Received: (at submit) by debbugs.gnu.org; 13 Nov 2010 21:19:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 13 16:19:38 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.69)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1PHNVa-0005sk-2M
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:19:38 -0500
Received: from eggs.gnu.org ([140.186.70.92])
by debbugs.gnu.org with esmtp (Exim 4.69)
(envelope-from <eblake@HIDDEN>) id 1PHNVW-0005sf-98
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:19:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <eblake@HIDDEN>) id 1PHNaD-0008BT-43
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:24:26 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org
X-Spam-Level:
X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI,
T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1
Received: from lists.gnu.org ([199.232.76.165]:32812)
by eggs.gnu.org with esmtp (Exim 4.71)
(envelope-from <eblake@HIDDEN>) id 1PHNaD-0008BP-1b
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 16:24:25 -0500
Received: from [140.186.70.92] (port=47032 helo=eggs.gnu.org)
by lists.gnu.org with esmtp (Exim 4.43) id 1PHNaB-0008Re-RH
for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 16:24:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <eblake@HIDDEN>) id 1PHNaA-0008At-Sl
for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 16:24:23 -0500
Received: from mx1.redhat.com ([209.132.183.28]:35968)
by eggs.gnu.org with esmtp (Exim 4.71)
(envelope-from <eblake@HIDDEN>)
id 1PHNa4-0008AW-Am; Sat, 13 Nov 2010 16:24:16 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id oADLOCIe004654
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
Sat, 13 Nov 2010 16:24:12 -0500
Received: from [10.3.113.9] ([10.3.113.9])
by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
id oADLOBhu013511; Sat, 13 Nov 2010 16:24:11 -0500
Message-ID: <4CDF01FB.3050504@HIDDEN>
Date: Sat, 13 Nov 2010 14:24:11 -0700
From: Eric Blake <eblake@HIDDEN>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
rv:1.9.2.12) Gecko/20101103 Fedora/1.0-0.33.b2pre.fc14
Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.6
MIME-Version: 1.0
To: Jim Meyering <jim@HIDDEN>
Subject: Re: rename() over NFS
References: <1286899722-20690-1-git-send-email-gary@HIDDEN> <201011131258.57634.bruno@HIDDEN>
<8762w1779e.fsf@HIDDEN> <201011131513.17364.bruno@HIDDEN>
<87mxpd5m93.fsf@HIDDEN>
In-Reply-To: <87mxpd5m93.fsf@HIDDEN>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature";
boundary="------------enig54EBFEF2F8F7F4395E1F34F3"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
recognized.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Spam-Score: -7.8 (-------)
X-Debbugs-Envelope-To: submit
Cc: bug-gnulib@HIDDEN, Paul Eggert <eggert@HIDDEN>,
Bruno Haible <bruno@HIDDEN>, bug-coreutils@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -7.8 (-------)
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig54EBFEF2F8F7F4395E1F34F3
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
On 11/13/2010 07:30 AM, Jim Meyering wrote:
> Bruno Haible wrote:
>> Jim Meyering wrote:
>>> if you see ways to improve things without impacting performance
>>> or maintainability, I'm all for it.
>>
>> Maintainability wouldn't be impacted, because the fix would be to use
>> lib/rename.c, with
>> #define RENAME_DEST_EXISTS_BUG 1
>> #define RENAME_HARD_LINK_BUG 1
>>
>> But performance would be impacted, since 2 lstat() calls would happen
>> before every rename().
>=20
> That sounds like it would be excessive.
> Currently, moving 100 regular files into a subdir
> incurs 100% of its time in 100 rename and 200 stat calls:
>=20
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 79.69 0.000051 1 100 rename
> 20.31 0.000013 0 200 100 lstat
> 0.00 0.000000 0 8 read
>=20
> Adding a rename wrapper and doubling the number of lstat calls
> per file would represent a significant slow-down.
The two lstat() calls occur before coreutils calls rename(). And it
appears that the NFS rename() bug that we are concerned about is only in
the case of renaming one directory onto another empty one; which is
generally only possible via 'mv -T a b' (where b is the empty directory
to be overwritten by directory a).
So maybe it's still possible to do a relatively-maintainable patch to
coreutils that detects when we are about to call rename() onto an
existing directory (basically, when 'mv -T' is in effect), and manually
call rmdir("b") before renaming "a", which should work around the NFS
bug, all without needing a rename() wrapper. At any rate, the gnulib
documentation would need a patch describing the bug, and stating that
gnulib does not work around it.
But I'm not sure if changing mv to avoid this corner-case POSIX
violation on NFS not renaming directories properly is made any better by
inserting the rmdir() call, since that also feels like it is a POSIX
violation for not making an atomic rename attempt. Maybe all the more
we can do is document this, and ping the kernel folks to see if they can
fix their NFS implementation bug.
--=20
Eric Blake eblake@HIDDEN +1-801-349-2682
Libvirt virtualization library http://libvirt.org
--------------enig54EBFEF2F8F7F4395E1F34F3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
iQEcBAEBCAAGBQJM3wH7AAoJEKeha0olJ0NqwqIH/iQvTMqej995BMQQfjjVD0Hi
Gxd5kTnV/+v89YksakYwU4PjJDAHo3MC5c1B1ZkKPWxq0zTZszOLEKaF/D49UYr7
RbfS6eopOuvIQkx3FzfXEYDVo862fbtNHGzrFGgeGNYBRtY/EEohDj1wSHJQxXLD
dsmINqfMFKE5Wz0yGZBwmMJ9Nm8YZN4yhwg8QwYsnUM8ZZ0V95cw1MQgIIFDOMrF
qx9gM04jlofYbrAgGehMCW0h67Mu9JM5VxAMl00Dp01dVDwjH/Xe89xqA1TybsBh
yLW7h3pzri/eaHTv9sxHufaj0Ds0YBdia087txPLgOduyKJz6jmulIoGGRdST8s=
=fdB0
-----END PGP SIGNATURE-----
--------------enig54EBFEF2F8F7F4395E1F34F3--
owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:bug#7389; Package coreutils.
Full text available.Received: (at submit) by debbugs.gnu.org; 13 Nov 2010 14:26:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 13 09:26:06 2010 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1PHH3O-0001iu-6p for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:26:06 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <jim@HIDDEN>) id 1PHH3K-0001iX-Kd for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:26:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHH80-0001Uf-O3 for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:30:54 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:36248) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHH80-0001Ua-MG for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:30:52 -0500 Received: from [140.186.70.92] (port=60888 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PHH7z-0000Dr-K9 for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 09:30:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHH7y-0001U5-Km for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 09:30:51 -0500 Received: from mx.meyering.net ([82.230.74.64]:34720) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHH7y-0001Tu-El; Sat, 13 Nov 2010 09:30:50 -0500 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id F38AE6003A; Sat, 13 Nov 2010 15:30:48 +0100 (CET) From: Jim Meyering <jim@HIDDEN> To: Bruno Haible <bruno@HIDDEN> Subject: Re: rename() over NFS In-Reply-To: <201011131513.17364.bruno@HIDDEN> (Bruno Haible's message of "Sat, 13 Nov 2010 15:13:15 +0100") References: <1286899722-20690-1-git-send-email-gary@HIDDEN> <201011131258.57634.bruno@HIDDEN> <8762w1779e.fsf@HIDDEN> <201011131513.17364.bruno@HIDDEN> Date: Sat, 13 Nov 2010 15:30:48 +0100 Message-ID: <87mxpd5m93.fsf@HIDDEN> Lines: 25 MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Spam-Score: -5.6 (-----) X-Debbugs-Envelope-To: submit Cc: bug-gnulib@HIDDEN, Paul Eggert <eggert@HIDDEN>, bug-coreutils@HIDDEN, Eric Blake <eblake@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Sender: debbugs-submit-bounces <at> debbugs.gnu.org Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org X-Spam-Score: -5.6 (-----) Bruno Haible wrote: > Jim Meyering wrote: >> if you see ways to improve things without impacting performance >> or maintainability, I'm all for it. > > Maintainability wouldn't be impacted, because the fix would be to use > lib/rename.c, with > #define RENAME_DEST_EXISTS_BUG 1 > #define RENAME_HARD_LINK_BUG 1 > > But performance would be impacted, since 2 lstat() calls would happen > before every rename(). That sounds like it would be excessive. Currently, moving 100 regular files into a subdir incurs 100% of its time in 100 rename and 200 stat calls: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 79.69 0.000051 1 100 rename 20.31 0.000013 0 200 100 lstat 0.00 0.000000 0 8 read Adding a rename wrapper and doubling the number of lstat calls per file would represent a significant slow-down.
owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:bug#7389; Package coreutils.
Full text available.
Received: (at submit) by debbugs.gnu.org; 13 Nov 2010 14:08:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 13 09:08:58 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.69)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1PHGmn-0001bf-Si
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:08:58 -0500
Received: from eggs.gnu.org ([140.186.70.92])
by debbugs.gnu.org with esmtp (Exim 4.69)
(envelope-from <bruno@HIDDEN>) id 1PHGmi-0001ba-Q8
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:08:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <bruno@HIDDEN>) id 1PHGrO-0006Yo-Pd
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:13:44 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org
X-Spam-Level:
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE,
T_DKIM_INVALID autolearn=unavailable version=3.3.1
Received: from lists.gnu.org ([199.232.76.165]:51603)
by eggs.gnu.org with esmtp (Exim 4.71)
(envelope-from <bruno@HIDDEN>) id 1PHGrO-0006Yj-Mw
for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 09:13:42 -0500
Received: from [140.186.70.92] (port=51981 helo=eggs.gnu.org)
by lists.gnu.org with esmtp (Exim 4.43) id 1PHGrM-0002mr-WF
for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 09:13:42 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <bruno@HIDDEN>) id 1PHGrH-0006X5-C5
for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 09:13:40 -0500
Received: from mo-p00-ob.rzone.de ([81.169.146.162]:58226)
by eggs.gnu.org with esmtp (Exim 4.71)
(envelope-from <bruno@HIDDEN>)
id 1PHGrG-0006Ws-OM; Sat, 13 Nov 2010 09:13:35 -0500
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1289657612; l=3715;
s=domk; d=haible.de;
h=Content-Transfer-Encoding:Content-Type:MIME-Version:In-Reply-To:
References:Cc:Date:Subject:To:From:X-RZG-CLASS-ID:X-RZG-AUTH;
bh=PXrsZeyjbaLRDEHn0G6ybnx87nc=;
b=pvHB6b0tz12hMMide+gWLUMKsq6J8888jIu/MhVz24VCD+TG907Xauv/BSriFYHKDFx
rORT3lbw6BpbO16p/Ly9UC/DNITHiZfS5xdjsyPAdGH8IlfC82avgWJFR9Mzmbr5mcdum
6Fs9aFPQ8S13JczydcW0l9r2CAOQXp2jLWQ=
X-RZG-AUTH: :Ln4Re0+Ic/6oZXR1YgKryK8brksyK8dozXDwHXjf9hj/zDJRaPAi/IaFpUs=
X-RZG-CLASS-ID: mo00
Received: from linuix.haible.de
(dslb-088-069-164-180.pools.arcor-ip.net [88.69.164.180])
by post.strato.de (jimi mo54) (RZmta 24.4) with ESMTP id N04904mADDOTI4
; Sat, 13 Nov 2010 15:13:17 +0100 (MET)
From: Bruno Haible <bruno@HIDDEN>
To: Jim Meyering <jim@HIDDEN>
Subject: Re: rename() over NFS
Date: Sat, 13 Nov 2010 15:13:15 +0100
User-Agent: KMail/1.9.9
References: <1286899722-20690-1-git-send-email-gary@HIDDEN>
<201011131258.57634.bruno@HIDDEN> <8762w1779e.fsf@HIDDEN>
In-Reply-To: <8762w1779e.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain;
charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <201011131513.17364.bruno@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Spam-Score: -5.8 (-----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnulib@HIDDEN, Paul Eggert <eggert@HIDDEN>, bug-coreutils@HIDDEN,
Eric Blake <eblake@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -5.9 (-----)
Jim Meyering wrote:
> if you see ways to improve things without impacting performance
> or maintainability, I'm all for it.
Maintainability wouldn't be impacted, because the fix would be to use
lib/rename.c, with
#define RENAME_DEST_EXISTS_BUG 1
#define RENAME_HARD_LINK_BUG 1
But performance would be impacted, since 2 lstat() calls would happen
before every rename().
The bug may not have a big impact, since coreutils 'mv' handles a move of a
directory to a directory differently anyway, and few other programs use
rename() on directories.
So, assuming Eric also agrees with approach (a), here's a proposal for
fixing the testsuite:
2010-11-13 Bruno Haible <bruno@HIDDEN>
rename, renameat: Avoid test failures at NFS mounted locations.
* tests/test-rename.h (dentry_exists, assert_nonexistent): New
functions.
(test_rename): Use assert_nonexistent.
* tests/test-rename.c: Include <dirent.h>.
* tests/test-renameat.c: Likewise.
Reported by Gary V. Vaughan <gary@HIDDEN>.
--- tests/test-rename.h.orig Sat Nov 13 14:35:50 2010
+++ tests/test-rename.h Sat Nov 13 14:35:23 2010
@@ -20,6 +20,49 @@
appropriate headers are already included. If PRINT, warn before
skipping symlink tests with status 77. */
+/* Tests whether a file, given by a file name without slashes, exists in
+ the current directory, by scanning the directory entries. */
+static bool
+dentry_exists (const char *filename)
+{
+ bool exists = false;
+ DIR *dir = opendir (".");
+
+ ASSERT (dir != NULL);
+ for (;;)
+ {
+ struct dirent *d = readdir (dir);
+ if (d == NULL)
+ break;
+ if (strcmp (d->d_name, filename) == 0)
+ {
+ exists = true;
+ break;
+ }
+ }
+ ASSERT (closedir (dir) == 0);
+ return exists;
+}
+
+/* Asserts that a specific file, given by a file name without slashes, does
+ not exist in the current directory. */
+static void
+assert_nonexistent (const char *filename)
+{
+ struct stat st;
+
+ /* The usual way to test the presence of a file is via stat() or lstat(). */
+ errno = 0;
+ if (stat (filename, &st) == -1)
+ ASSERT (errno == ENOENT);
+ else
+ /* But after renaming a directory over an empty directory on an NFS-mounted
+ file system, on Linux 2.6.18, for a period of 30 seconds the old
+ directory name is "present" according to stat() but "nonexistent"
+ according to dentry_exists(). */
+ ASSERT (!dentry_exists (filename));
+}
+
static int
test_rename (int (*func) (char const *, char const *), bool print)
{
@@ -226,9 +269,7 @@
}
{ /* Full onto empty. */
ASSERT (func (BASE "dir", BASE "dir2") == 0);
- errno = 0;
- ASSERT (stat (BASE "dir", &st) == -1);
- ASSERT (errno == ENOENT);
+ assert_nonexistent (BASE "dir");
ASSERT (stat (BASE "dir2/file", &st) == 0);
/* Files present here:
{BASE}file
@@ -263,9 +304,7 @@
*/
{
ASSERT (func (BASE "dir", BASE "dir2/") == 0);
- errno = 0;
- ASSERT (stat (BASE "dir", &st) == -1);
- ASSERT (errno == ENOENT);
+ assert_nonexistent (BASE "dir");
ASSERT (stat (BASE "dir2/file", &st) == 0);
}
/* Files present here:
--- tests/test-rename.c.orig Sat Nov 13 14:35:50 2010
+++ tests/test-rename.c Sat Nov 13 14:15:34 2010
@@ -21,6 +21,7 @@
#include "signature.h"
SIGNATURE_CHECK (rename, int, (char const *, char const *));
+#include <dirent.h>
#include <errno.h>
#include <fcntl.h>
#include <stdbool.h>
--- tests/test-renameat.c.orig Sat Nov 13 14:35:50 2010
+++ tests/test-renameat.c Sat Nov 13 14:15:30 2010
@@ -23,6 +23,7 @@
#include "signature.h"
SIGNATURE_CHECK (renameat, int, (int, char const *, int, char const *));
+#include <dirent.h>
#include <fcntl.h>
#include <errno.h>
#include <stdbool.h>
owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:bug#7389; Package coreutils.
Full text available.Received: (at submit) by debbugs.gnu.org; 13 Nov 2010 12:07:08 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 13 07:07:08 2010 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1PHEst-00008C-79 for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 07:07:07 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <jim@HIDDEN>) id 1PHEsn-00007p-SA for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 07:07:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHExU-0004vq-8X for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 07:11:53 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:37865) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHExU-0004vm-6W for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 07:11:52 -0500 Received: from [140.186.70.92] (port=57826 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PHExS-0005JW-1Q for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 07:11:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHExN-0004u3-JF for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 07:11:49 -0500 Received: from mx.meyering.net ([82.230.74.64]:39886) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <jim@HIDDEN>) id 1PHExN-0004tN-CZ; Sat, 13 Nov 2010 07:11:45 -0500 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id D3D3C60037; Sat, 13 Nov 2010 13:11:41 +0100 (CET) From: Jim Meyering <jim@HIDDEN> To: Bruno Haible <bruno@HIDDEN> Subject: Re: rename() over NFS In-Reply-To: <201011131258.57634.bruno@HIDDEN> (Bruno Haible's message of "Sat, 13 Nov 2010 12:58:56 +0100") References: <1286899722-20690-1-git-send-email-gary@HIDDEN> <201011130113.54559.bruno@HIDDEN> <4CDE3486.5030608@HIDDEN> <201011131258.57634.bruno@HIDDEN> Date: Sat, 13 Nov 2010 13:11:41 +0100 Message-ID: <8762w1779e.fsf@HIDDEN> Lines: 38 MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Spam-Score: -5.6 (-----) X-Debbugs-Envelope-To: submit Cc: bug-gnulib@HIDDEN, Paul Eggert <eggert@HIDDEN>, bug-coreutils@HIDDEN, Eric Blake <eblake@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Sender: debbugs-submit-bounces <at> debbugs.gnu.org Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org X-Spam-Score: -5.6 (-----) Bruno Haible wrote: > [bug-coreutils readers: This is a reply to > <http://lists.gnu.org/archive/html/bug-gnulib/2010-11/msg00155.html>]. > > Paul Eggert wrote: >> > What should we do? >> > a) Patch the test so that it uses a readdir() loop to detect the absence of >> > the file even when stat() pretends it's still present. Or >> > b) Use an rpl_rename override that will make the unit test work. >> >> It's long been well-known that NFS is not POSIX-compliant. >> If we start down the path of putting wrappers around NFS to make it >> POSIX-compliant, >> we'll have a lot of work to do and a job that will probably never end, >> and we'll make coreutils slow down for the common case even though >> in practice the problems are rare and users don't care about them. >> >> So, my vote is for (a). Or maybe even (c): skip the test if it's >> running atop NFS. >> >> I've been observing similar test failures for months, by the way, but >> haven't bothered to report them, because, hey! it's NFS! of course it's >> going to screw up in those cases! > > But NFS is widely used, in small networks of 2 to 50 Linux/Unix machines. > Users are running shell scripts in such situations, with lots of coreutils > commands. Hi Bruno, Rename is fundamental enough and such NFS-specific problems rare enough that I would prefer not to impose a rename wrapper on everyone. I agree with Paul that this is a very steep slippery slope. If we start down it in earnest, we may end up making our tools less maintainable in the long run, and for only very marginal benefit. However, if you see ways to improve things without impacting performance or maintainability, I'm all for it.
owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:bug#7389; Package coreutils.
Full text available.Received: (at submit) by debbugs.gnu.org; 13 Nov 2010 11:54:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 13 06:54:39 2010 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1PHEgo-0008Ui-0B for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 06:54:39 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from <bruno@HIDDEN>) id 1PHEgi-0008Ud-Ow for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 06:54:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <bruno@HIDDEN>) id 1PHElP-0002Kr-2z for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 06:59:24 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:58518) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <bruno@HIDDEN>) id 1PHElP-0002Ki-0y for submit <at> debbugs.gnu.org; Sat, 13 Nov 2010 06:59:23 -0500 Received: from [140.186.70.92] (port=35106 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PHElN-0008QH-3k for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 06:59:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <bruno@HIDDEN>) id 1PHElI-0002JP-Nw for bug-coreutils@HIDDEN; Sat, 13 Nov 2010 06:59:20 -0500 Received: from mo-p00-ob.rzone.de ([81.169.146.161]:59145) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <bruno@HIDDEN>) id 1PHElI-0002J9-71; Sat, 13 Nov 2010 06:59:16 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1289649554; l=1197; s=domk; d=haible.de; h=Content-Transfer-Encoding:Content-Type:MIME-Version:In-Reply-To: References:Cc:Date:Subject:To:From:X-RZG-CLASS-ID:X-RZG-AUTH; bh=B1zYuyR/Xry+eZmTf0EhueWVtVw=; b=otVGpd/NXAFG65orXya6pkW2RUSJvQKh6l1+Sffj0P5gkPEV8lKnNXJBtENNpQnCGnx BCQjuXiEEVeGU9/bCYx8Cx+HeCE13zdSQm9oBw8HmbGLl5IeICQsGHE4xCBBCr/ItqSbu 5ICcJN4Tu1cIk+PzaEazfvMYszVSaBU5rYY= X-RZG-AUTH: :Ln4Re0+Ic/6oZXR1YgKryK8brksyK8dozXDwHXjf9hj/zDJRaPAi/IaFpUs= X-RZG-CLASS-ID: mo00 Received: from linuix.haible.de (dslb-088-069-164-180.pools.arcor-ip.net [88.69.164.180]) by post.strato.de (fruni mo44) (RZmta 24.4) with ESMTP id j06141mAD9pD7Q ; Sat, 13 Nov 2010 12:58:59 +0100 (MET) From: Bruno Haible <bruno@HIDDEN> To: Paul Eggert <eggert@HIDDEN>, bug-coreutils@HIDDEN Subject: Re: rename() over NFS Date: Sat, 13 Nov 2010 12:58:56 +0100 User-Agent: KMail/1.9.9 References: <1286899722-20690-1-git-send-email-gary@HIDDEN> <201011130113.54559.bruno@HIDDEN> <4CDE3486.5030608@HIDDEN> In-Reply-To: <4CDE3486.5030608@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201011131258.57634.bruno@HIDDEN> X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Spam-Score: -5.7 (-----) X-Debbugs-Envelope-To: submit Cc: bug-gnulib@HIDDEN, Eric Blake <eblake@HIDDEN>, "Gary V. Vaughan" <gary@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Sender: debbugs-submit-bounces <at> debbugs.gnu.org Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org X-Spam-Score: -5.8 (-----) [bug-coreutils readers: This is a reply to <http://lists.gnu.org/archive/html/bug-gnulib/2010-11/msg00155.html>]. Paul Eggert wrote: > > What should we do? > > a) Patch the test so that it uses a readdir() loop to detect the absence of > > the file even when stat() pretends it's still present. Or > > b) Use an rpl_rename override that will make the unit test work. > > It's long been well-known that NFS is not POSIX-compliant. > If we start down the path of putting wrappers around NFS to make it POSIX-compliant, > we'll have a lot of work to do and a job that will probably never end, > and we'll make coreutils slow down for the common case even though > in practice the problems are rare and users don't care about them. > > So, my vote is for (a). Or maybe even (c): skip the test if it's > running atop NFS. > > I've been observing similar test failures for months, by the way, but > haven't bothered to report them, because, hey! it's NFS! of course it's > going to screw up in those cases! But NFS is widely used, in small networks of 2 to 50 Linux/Unix machines. Users are running shell scripts in such situations, with lots of coreutils commands. Bruno
Bruno Haible <bruno@HIDDEN>:bug-coreutils@HIDDEN.
Full text available.owner <at> debbugs.gnu.org, bug-coreutils@HIDDEN:bug#7389; Package coreutils.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.