GNU bug report logs -
#29092
Bug related to 64-bit inodes
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29092 in the body.
You can then email your comments to 29092 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Tue, 31 Oct 2017 19:20:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Curt McDowell <csm <at> purestorage.com>:
New bug report received and forwarded. Copy sent to
bug-idutils <at> gnu.org.
(Tue, 31 Oct 2017 19:20:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi,
I'm running idutils 4.6 on Ubuntu 14.04 to index a large source base that
is on an NFS-mounted filesystem that uses 64-bit inodes. mkid incorrectly
issues many warnings such as the following:
/home/csm/src/idutils-4.6/src/mkid: warning:
`/df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py' and
`/df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp' are the same
file, but yield different scans!
It turns out these inodes are equal in the lower 32 bits, but are not equal.
% ls -li /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
/df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
33776997256654722 -rwxr-xr-x 1 csm staff 3290 May 22 22:03
/df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
63050394834562946 -rw-r--r-- 1 csm staff 28973 May 22 22:03
/df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
The inode numbers in hex are 780000030FEF82 and E00000030FEF82,
respectively.
I believe the bug is that the inode hash functions should account for the
size of the di_ino field in case it is 8 bytes wide, rather than assuming
they are always 4 bytes:
/****************************************************************************/
/* Hash stuff for `struct dev_ino'. */
static unsigned long
dev_ino_hash_1 (void const *key)
{
unsigned long result = 0;
INTEGER_HASH_1 (((struct dev_ino const *) key)->di_dev, result);
INTEGER_HASH_1 (((struct dev_ino const *) key)->di_ino, result);
return result;
}
static unsigned long
dev_ino_hash_2 (void const *key)
{
unsigned long result = 0;
INTEGER_HASH_2 (((struct dev_ino const *) key)->di_dev, result);
INTEGER_HASH_2 (((struct dev_ino const *) key)->di_ino, result);
return result;
}
As an unrelated issue, in order to get idutils 4.6 to compile on Ubuntu
14.0.4 I had to edit lib/stdio.h and change "#if 1" to "#if 0" for the
section that deals with the "gets" function (libc 2.19-0ubuntu6.13).
Regards,
--
Curt McDowell / Pure Storage / FlashBlade
csm <at> purestorage.com
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Tue, 07 Nov 2017 21:38:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 29092 <at> debbugs.gnu.org (full text, mbox):
On Tue, Oct 31, 2017 at 12:03 PM, Curt McDowell <csm <at> purestorage.com> wrote:
> Hi,
>
> I'm running idutils 4.6 on Ubuntu 14.04 to index a large source base that is
> on an NFS-mounted filesystem that uses 64-bit inodes. mkid incorrectly
> issues many warnings such as the following:
>
> /home/csm/src/idutils-4.6/src/mkid: warning:
> `/df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py' and
> `/df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp' are the same
> file, but yield different scans!
>
> It turns out these inodes are equal in the lower 32 bits, but are not equal.
>
> % ls -li /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
> /df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
> 33776997256654722 -rwxr-xr-x 1 csm staff 3290 May 22 22:03
> /df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
> 63050394834562946 -rw-r--r-- 1 csm staff 28973 May 22 22:03
> /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
>
> The inode numbers in hex are 780000030FEF82 and E00000030FEF82,
> respectively.
>
> I believe the bug is that the inode hash functions should account for the
> size of the di_ino field in case it is 8 bytes wide, rather than assuming
> they are always 4 bytes:
>
> /****************************************************************************/
> /* Hash stuff for `struct dev_ino'. */
>
> static unsigned long
> dev_ino_hash_1 (void const *key)
> {
> unsigned long result = 0;
> INTEGER_HASH_1 (((struct dev_ino const *) key)->di_dev, result);
> INTEGER_HASH_1 (((struct dev_ino const *) key)->di_ino, result);
> return result;
> }
>
> static unsigned long
> dev_ino_hash_2 (void const *key)
> {
> unsigned long result = 0;
> INTEGER_HASH_2 (((struct dev_ino const *) key)->di_dev, result);
> INTEGER_HASH_2 (((struct dev_ino const *) key)->di_ino, result);
> return result;
> }
>
> As an unrelated issue, in order to get idutils 4.6 to compile on Ubuntu
> 14.0.4 I had to edit lib/stdio.h and change "#if 1" to "#if 0" for the
> section that deals with the "gets" function (libc 2.19-0ubuntu6.13).
Thank you for the report.
That is definitely a bug -- and it is over two decades old!
I will fix this shortly, and will soon make a test release.
Information forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Sun, 17 Dec 2017 01:00:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 29092 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Tue, Nov 7, 2017 at 1:36 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Tue, Oct 31, 2017 at 12:03 PM, Curt McDowell <csm <at> purestorage.com> wrote:
>> Hi,
>>
>> I'm running idutils 4.6 on Ubuntu 14.04 to index a large source base that is
>> on an NFS-mounted filesystem that uses 64-bit inodes. mkid incorrectly
>> issues many warnings such as the following:
>>
>> /home/csm/src/idutils-4.6/src/mkid: warning:
>> `/df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py' and
>> `/df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp' are the same
>> file, but yield different scans!
>>
>> It turns out these inodes are equal in the lower 32 bits, but are not equal.
>>
>> % ls -li /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
>> /df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
>> 33776997256654722 -rwxr-xr-x 1 csm staff 3290 May 22 22:03
>> /df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
>> 63050394834562946 -rw-r--r-- 1 csm staff 28973 May 22 22:03
>> /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
>>
>> The inode numbers in hex are 780000030FEF82 and E00000030FEF82,
>> respectively.
>>
>> I believe the bug is that the inode hash functions should account for the
>> size of the di_ino field in case it is 8 bytes wide, rather than assuming
>> they are always 4 bytes:
>>
>> /****************************************************************************/
>> /* Hash stuff for `struct dev_ino'. */
>>
>> static unsigned long
>> dev_ino_hash_1 (void const *key)
>> {
>> unsigned long result = 0;
>> INTEGER_HASH_1 (((struct dev_ino const *) key)->di_dev, result);
>> INTEGER_HASH_1 (((struct dev_ino const *) key)->di_ino, result);
>> return result;
>> }
>>
>> static unsigned long
>> dev_ino_hash_2 (void const *key)
>> {
>> unsigned long result = 0;
>> INTEGER_HASH_2 (((struct dev_ino const *) key)->di_dev, result);
>> INTEGER_HASH_2 (((struct dev_ino const *) key)->di_ino, result);
>> return result;
>> }
>>
>> As an unrelated issue, in order to get idutils 4.6 to compile on Ubuntu
>> 14.0.4 I had to edit lib/stdio.h and change "#if 1" to "#if 0" for the
>> section that deals with the "gets" function (libc 2.19-0ubuntu6.13).
>
> Thank you for the report.
> That is definitely a bug -- and it is over two decades old!
> I will fix this shortly, and will soon make a test release.
It wasn't as prompt as I would have liked, but here's a patch.
I'll make a snapshot today and post separately to the bug-idutils list.
[idu-inode-collision.diff (text/plain, attachment)]
Information forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Mon, 18 Dec 2017 23:32:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 29092 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Jim,
Thanks for getting to this. I applied the patch and was surprised to see it
still has the same bug! There is another comparison function that needs
fixing. I verified that 64-bit inodes work properly with this additional
change:
static int
dev_ino_hash_compare (void const *x, void const *y)
{
int result;
result = memcmp(&((struct dev_ino const *) x)->di_ino,
&((struct dev_ino const *) y)->di_ino, sizeof (ino_t));
if (result)
return result;
result = memcmp(&((struct dev_ino const *) x)->di_dev,
&((struct dev_ino const *) y)->di_dev, sizeof (ino_t));
return result;
}
On Sat, Dec 16, 2017 at 4:58 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Tue, Nov 7, 2017 at 1:36 PM, Jim Meyering <jim <at> meyering.net> wrote:
> > On Tue, Oct 31, 2017 at 12:03 PM, Curt McDowell <csm <at> purestorage.com>
> wrote:
> >> Hi,
> >>
> >> I'm running idutils 4.6 on Ubuntu 14.04 to index a large source base
> that is
> >> on an NFS-mounted filesystem that uses 64-bit inodes. mkid incorrectly
> >> issues many warnings such as the following:
> >>
> >> /home/csm/src/idutils-4.6/src/mkid: warning:
> >> `/df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py' and
> >> `/df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp' are the same
> >> file, but yield different scans!
> >>
> >> It turns out these inodes are equal in the lower 32 bits, but are not
> equal.
> >>
> >> % ls -li /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
> >> /df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
> >> 33776997256654722 -rwxr-xr-x 1 csm staff 3290 May 22 22:03
> >> /df-csm/ir-csm7/hardware/perf/CTRL_systemC/src/demux.hpp
> >> 63050394834562946 -rw-r--r-- 1 csm staff 28973 May 22 22:03
> >> /df-csm/ir-csm7/platform/dot/ir/component/si5338/si5338.py
> >>
> >> The inode numbers in hex are 780000030FEF82 and E00000030FEF82,
> >> respectively.
> >>
> >> I believe the bug is that the inode hash functions should account for
> the
> >> size of the di_ino field in case it is 8 bytes wide, rather than
> assuming
> >> they are always 4 bytes:
> >>
> >> /***********************************************************
> *****************/
> >> /* Hash stuff for `struct dev_ino'. */
> >>
> >> static unsigned long
> >> dev_ino_hash_1 (void const *key)
> >> {
> >> unsigned long result = 0;
> >> INTEGER_HASH_1 (((struct dev_ino const *) key)->di_dev, result);
> >> INTEGER_HASH_1 (((struct dev_ino const *) key)->di_ino, result);
> >> return result;
> >> }
> >>
> >> static unsigned long
> >> dev_ino_hash_2 (void const *key)
> >> {
> >> unsigned long result = 0;
> >> INTEGER_HASH_2 (((struct dev_ino const *) key)->di_dev, result);
> >> INTEGER_HASH_2 (((struct dev_ino const *) key)->di_ino, result);
> >> return result;
> >> }
> >>
> >> As an unrelated issue, in order to get idutils 4.6 to compile on Ubuntu
> >> 14.0.4 I had to edit lib/stdio.h and change "#if 1" to "#if 0" for the
> >> section that deals with the "gets" function (libc 2.19-0ubuntu6.13).
> >
> > Thank you for the report.
> > That is definitely a bug -- and it is over two decades old!
> > I will fix this shortly, and will soon make a test release.
>
> It wasn't as prompt as I would have liked, but here's a patch.
> I'll make a snapshot today and post separately to the bug-idutils list.
>
--
Curt McDowell / Pure Storage / FlashBlade
csm <at> purestorage.com
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Tue, 19 Dec 2017 04:38:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 29092 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Mon, Dec 18, 2017 at 3:31 PM, Curt McDowell <csm <at> purestorage.com> wrote:
> Jim,
>
> Thanks for getting to this. I applied the patch and was surprised to see it
> still has the same bug! There is another comparison function that needs
> fixing. I verified that 64-bit inodes work properly with this additional
> change:
>
> static int
> dev_ino_hash_compare (void const *x, void const *y)
> {
> int result;
> result = memcmp(&((struct dev_ino const *) x)->di_ino,
> &((struct dev_ino const *) y)->di_ino, sizeof (ino_t));
> if (result)
> return result;
> result = memcmp(&((struct dev_ino const *) x)->di_dev,
> &((struct dev_ino const *) y)->di_dev, sizeof (ino_t));
> return result;
> }
Thank you for the quick testing and patch.
I've simplified that but left it in your name. However, since it's
still in your name, yet modified, I'll wait for your ACK before
pushing (check both author-name+email and wording of the commit log as
well as the actual patch, since we treat master-pushed commits as
immutable):
[idu-inode-collision-2.diff (text/plain, attachment)]
Information forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Sat, 06 Jan 2018 04:29:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 29092 <at> debbugs.gnu.org (full text, mbox):
On Mon, Dec 18, 2017 at 8:36 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Mon, Dec 18, 2017 at 3:31 PM, Curt McDowell <csm <at> purestorage.com> wrote:
>> Jim,
>>
>> Thanks for getting to this. I applied the patch and was surprised to see it
>> still has the same bug! There is another comparison function that needs
>> fixing. I verified that 64-bit inodes work properly with this additional
>> change:
>>
>> static int
>> dev_ino_hash_compare (void const *x, void const *y)
>> {
>> int result;
>> result = memcmp(&((struct dev_ino const *) x)->di_ino,
>> &((struct dev_ino const *) y)->di_ino, sizeof (ino_t));
>> if (result)
>> return result;
>> result = memcmp(&((struct dev_ino const *) x)->di_dev,
>> &((struct dev_ino const *) y)->di_dev, sizeof (ino_t));
>> return result;
>> }
>
> Thank you for the quick testing and patch.
>
> I've simplified that but left it in your name. However, since it's
> still in your name, yet modified, I'll wait for your ACK before
> pushing (check both author-name+email and wording of the commit log as
> well as the actual patch, since we treat master-pushed commits as
> immutable):
Ping?
Information forwarded
to
bug-idutils <at> gnu.org:
bug#29092; Package
idutils.
(Sun, 07 Jan 2018 00:57:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 29092 <at> debbugs.gnu.org (full text, mbox):
On Fri, Jan 5, 2018 at 8:27 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Mon, Dec 18, 2017 at 8:36 PM, Jim Meyering <jim <at> meyering.net> wrote:
>> On Mon, Dec 18, 2017 at 3:31 PM, Curt McDowell <csm <at> purestorage.com> wrote:
>>> Jim,
>>>
>>> Thanks for getting to this. I applied the patch and was surprised to see it
>>> still has the same bug! There is another comparison function that needs
>>> fixing. I verified that 64-bit inodes work properly with this additional
>>> change:
>>>
>>> static int
>>> dev_ino_hash_compare (void const *x, void const *y)
>>> {
>>> int result;
>>> result = memcmp(&((struct dev_ino const *) x)->di_ino,
>>> &((struct dev_ino const *) y)->di_ino, sizeof (ino_t));
>>> if (result)
>>> return result;
>>> result = memcmp(&((struct dev_ino const *) x)->di_dev,
>>> &((struct dev_ino const *) y)->di_dev, sizeof (ino_t));
>>> return result;
>>> }
>>
>> Thank you for the quick testing and patch.
>>
>> I've simplified that but left it in your name. However, since it's
>> still in your name, yet modified, I'll wait for your ACK before
>> pushing (check both author-name+email and wording of the commit log as
>> well as the actual patch, since we treat master-pushed commits as
>> immutable):
>
> Ping?
Pushed, in spite of no reply.
Reply sent
to
Jim Meyering <jim <at> meyering.net>:
You have taken responsibility.
(Sun, 07 Jan 2018 01:04:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Curt McDowell <csm <at> purestorage.com>:
bug acknowledged by developer.
(Sun, 07 Jan 2018 01:04:02 GMT)
Full text and
rfc822 format available.
Message #28 received at 29092-done <at> debbugs.gnu.org (full text, mbox):
On Sat, Jan 6, 2018 at 4:56 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Fri, Jan 5, 2018 at 8:27 PM, Jim Meyering <jim <at> meyering.net> wrote:
>> On Mon, Dec 18, 2017 at 8:36 PM, Jim Meyering <jim <at> meyering.net> wrote:
>>> On Mon, Dec 18, 2017 at 3:31 PM, Curt McDowell <csm <at> purestorage.com> wrote:
>>>> Jim,
>>>>
>>>> Thanks for getting to this. I applied the patch and was surprised to see it
>>>> still has the same bug! There is another comparison function that needs
>>>> fixing. I verified that 64-bit inodes work properly with this additional
>>>> change:
>>>>
>>>> static int
>>>> dev_ino_hash_compare (void const *x, void const *y)
>>>> {
>>>> int result;
>>>> result = memcmp(&((struct dev_ino const *) x)->di_ino,
>>>> &((struct dev_ino const *) y)->di_ino, sizeof (ino_t));
>>>> if (result)
>>>> return result;
>>>> result = memcmp(&((struct dev_ino const *) x)->di_dev,
>>>> &((struct dev_ino const *) y)->di_dev, sizeof (ino_t));
>>>> return result;
>>>> }
>>>
>>> Thank you for the quick testing and patch.
>>>
>>> I've simplified that but left it in your name. However, since it's
>>> still in your name, yet modified, I'll wait for your ACK before
>>> pushing (check both author-name+email and wording of the commit log as
>>> well as the actual patch, since we treat master-pushed commits as
>>> immutable):
>>
>> Ping?
>
> Pushed, in spite of no reply.
Incidentally, I've just updated gnulib and copyright notices, too.
Marking this as "done".
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 04 Feb 2018 12:24:06 GMT)
Full text and
rfc822 format available.
This bug report was last modified 6 years and 76 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.