GNU bug report logs - #71191
od does unnecessary reads instead of seeking

Previous Next

To reply to this bug, email your comments to 71191 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Joseph C. Sible" <josephcsible <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: od does unnecessary reads instead of seeking
Date: Sat, 25 May 2024 03:05:06 -0400

Consider running the following command, for looking at the pagemap
bits of a given memory page:

od -t x8 -N 8 -j 34359738368 /proc/PID/pagemap

That file supports seeking, but od will unnecessarily read and discard
32GB worth of data instead of doing so.

Looking at the skip function in od.c, it looks like this happens
because the file has a bogus st_size of 0 (as is typical for files in
/proc) despite usable_st_size returning true for it, which results in
the calls to fseeko never even being tried.

Message #8 received at 71191 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: "Joseph C. Sible" <josephcsible <at> gmail.com>, 71191 <at> debbugs.gnu.org
Subject: Re: bug#71191: od does unnecessary reads instead of seeking
Date: Sat, 25 May 2024 13:40:35 +0100

On 25/05/2024 08:05, Joseph C. Sible wrote:
> Consider running the following command, for looking at the pagemap
> bits of a given memory page:
> 
> od -t x8 -N 8 -j 34359738368 /proc/PID/pagemap
> 
> That file supports seeking, but od will unnecessarily read and discard
> 32GB worth of data instead of doing so.
> 
> Looking at the skip function in od.c, it looks like this happens
> because the file has a bogus st_size of 0 (as is typical for files in
> /proc) despite usable_st_size returning true for it, which results in
> the calls to fseeko never even being tried.

This is related to https://bugs.gnu.org/36291
Note the workaround with dd mentioned there ¹.
In that report I detailed four cases of /proc /sys and /dev files
that od needs to handle, and suggested we try the lseek() for
the non regular file case (where we don't worry about seeking past EOF).

As an aside we can actually _seek_ in /proc and /sys files
¹ dd used to report an error when seeking in such files,
when in fact seeking is actually supported. The warning was removed
(for all empty files) in https://github.com/coreutils/coreutils/commit/ac6b8d822
(as part of https://bugs.gnu.org/70231).
But this case is more complicated by the fact that od supports combining files,
so you need to avoid seeking beyond EOF.
I.e. `od --skip-bytes 1 empty nonempty` would give different results
if we blindly tried an lseek() if st_stize==0

I wonder could you efficiently seek to offset, avoiding seeking past EOF,
while ignoring st_size with something like:
  if (lseek(N-1))
    if read(1)
      skip -= N
    else
      lseek(-(N-1))
I'm not sure how general that would be.

cheers,
Pádraig

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.