GNU bug report logs - #70411
[bug] install(1) fails to read /dev/stdin on Darwin

Previous Next

To reply to this bug, email your comments to 70411 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alejandro Colomar <alx <at> kernel.org>
To: bug-coreutils <at> gnu.org
Cc: Sergei Trofimovich <slyich <at> gmail.com>, Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 02:19:26 +0200

[Message part 1 (text/plain, inline)]

Hi!

I don't own a Darwin system, so I can't help much reproduce.  However,
I've received a bug report to the Linux man-pages, that our build
system (GNUmakefile-based), which ends up calling

	... | install /dev/stdin $@

doesn't work on Darwin.  Here's the original bug report:
<https://github.com/NixOS/nixpkgs/pull/300797>.

Here are the reported error messages:

...
INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
install: skipping file '/dev/stdin', as it was replaced while being copied
make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3] Error 1
make: *** Waiting for unfinished jobs....
install: skipping file '/dev/stdin', as it was replaced while being copied
make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3] Error 1
install: skipping file '/dev/stdin', as it was replaced while being copied
install: skipping file '/dev/stdin', as it was replaced while being copied
install: skipping file '/dev/stdin', as it was replaced while being copied
...

I don't see why install(1) should fail to read /dev/stdin under any
POSIX system.

Reported-by: Mario Rodas <https://github.com/marsam>
Reported-by: Sergei Trofimovich <slyich <at> gmail.com>
Reported-by: Alejandro Colomar <alx <at> kernel.org>
Cc: Eli Schwartz <eschwartz93 <at> gmail.com>

Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Alejandro Colomar <alx <at> kernel.org>, 70411 <at> debbugs.gnu.org
Cc: Sergei Trofimovich <slyich <at> gmail.com>, Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 12:33:44 +0100

On 16/04/2024 01:19, Alejandro Colomar wrote:
> Hi!
> 
> I don't own a Darwin system, so I can't help much reproduce.  However,
> I've received a bug report to the Linux man-pages, that our build
> system (GNUmakefile-based), which ends up calling
> 
> 	... | install /dev/stdin $@
> 
> doesn't work on Darwin.  Here's the original bug report:
> <https://github.com/NixOS/nixpkgs/pull/300797>.
> 
> Here are the reported error messages:
> 
> ...
> INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
> INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
> install: skipping file '/dev/stdin', as it was replaced while being copied
> make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3] Error 1
> make: *** Waiting for unfinished jobs....
> install: skipping file '/dev/stdin', as it was replaced while being copied
> make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3] Error 1
> install: skipping file '/dev/stdin', as it was replaced while being copied
> install: skipping file '/dev/stdin', as it was replaced while being copied
> install: skipping file '/dev/stdin', as it was replaced while being copied
> ...
> 
> I don't see why install(1) should fail to read /dev/stdin under any
> POSIX system

What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
The issue seems to be that /dev/stdin returns a varying inode which install(1) doesn't like currently

cheers,
Pádraig

Message #11 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Alejandro Colomar <alx <at> kernel.org>, 70411 <at> debbugs.gnu.org
Cc: Sergei Trofimovich <slyich <at> gmail.com>, Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 15:25:22 +0100

On 16/04/2024 12:33, Pádraig Brady wrote:
> On 16/04/2024 01:19, Alejandro Colomar wrote:
>> Hi!
>>
>> I don't own a Darwin system, so I can't help much reproduce.  However,
>> I've received a bug report to the Linux man-pages, that our build
>> system (GNUmakefile-based), which ends up calling
>>
>> 	... | install /dev/stdin $@
>>
>> doesn't work on Darwin.  Here's the original bug report:
>> <https://github.com/NixOS/nixpkgs/pull/300797>.
>>
>> Here are the reported error messages:
>>
>> ...
>> INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
>> INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
>> install: skipping file '/dev/stdin', as it was replaced while being copied
>> make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3] Error 1
>> make: *** Waiting for unfinished jobs....
>> install: skipping file '/dev/stdin', as it was replaced while being copied
>> make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3] Error 1
>> install: skipping file '/dev/stdin', as it was replaced while being copied
>> install: skipping file '/dev/stdin', as it was replaced while being copied
>> install: skipping file '/dev/stdin', as it was replaced while being copied
>> ...
>>
>> I don't see why install(1) should fail to read /dev/stdin under any
>> POSIX system
> 
> What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
> The issue seems to be that /dev/stdin returns a varying inode which install(1) doesn't like currently

Note that check originally came from:
https://github.com/coreutils/coreutils/commit/dea4262fa

I suppose we could relax the check as follows, for files of apparent size 0
which would cater for this, and others that may also have unstable inodes.

cheers,
Pádraig.

diff --git a/src/copy.c b/src/copy.c
index 2145d89d5..fb5f0a1a0 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -1263,8 +1263,11 @@ copy_reg (char const *src_name, char const *dst_name,
     }

   /* Compare the source dev/ino from the open file to the incoming,
-     saved ones obtained via a previous call to stat.  */
-  if (! psame_inode (src_sb, &src_open_sb))
+     saved ones obtained via a previous call to stat.  Restrict
+     the check to files with an apparent size, to support "files"
+     with unstable inodes, like /dev/stdin on macOS.  */
+  if (! psame_inode (src_sb, &src_open_sb)
+      && (src_sb->st_size || src_open_sb.st_size))
     {
       error (0, 0,
              _("skipping file %s, as it was replaced while being copied"),

Message #14 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Alejandro Colomar <alx <at> kernel.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Sergei Trofimovich <slyich <at> gmail.com>, 70411 <at> debbugs.gnu.org,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 16:47:39 +0200

[Message part 1 (text/plain, inline)]

Hi Pádraig,

On Tue, Apr 16, 2024 at 03:25:22PM +0100, Pádraig Brady wrote:
> > What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
> > The issue seems to be that /dev/stdin returns a varying inode which install(1) doesn't like currently

I don't know.  The reporter didn't tell.  I see you also asked on the
Github original report.

> Note that check originally came from:
> https://github.com/coreutils/coreutils/commit/dea4262fa
> 
> I suppose we could relax the check as follows, for files of apparent size 0
> which would cater for this, and others that may also have unstable inodes.

Hmmmm.  Since you couldn't reprodude it in a recent Darwin, maybe it's
just a bug in an old Darwin.  And since noone else seems to have met
this Darwin's bug, maybe we can just ignore it.  (And if it were a
regression in a more recent Darwin, hopefully they should fix their
kernel.)

I'm not happy relaxing a security check, without making sure that there
are no implications at all.

I vote for claiming only limited support to such a Darwin system.  I
already workarounded it in the Linux man-pages, by not piping to
install(1) in a common task; and nobody else seems to be affected.

Unless you feel confident that it's perfectly fine to do it.  But I have
no sympathy for workarounding Darwin bugs here.

Have a lovely day!
Alex

> 
> cheers,
> Pádraig.
> 
> diff --git a/src/copy.c b/src/copy.c
> index 2145d89d5..fb5f0a1a0 100644
> --- a/src/copy.c
> +++ b/src/copy.c
> @@ -1263,8 +1263,11 @@ copy_reg (char const *src_name, char const *dst_name,
>      }
> 
>    /* Compare the source dev/ino from the open file to the incoming,
> -     saved ones obtained via a previous call to stat.  */
> -  if (! psame_inode (src_sb, &src_open_sb))
> +     saved ones obtained via a previous call to stat.  Restrict
> +     the check to files with an apparent size, to support "files"
> +     with unstable inodes, like /dev/stdin on macOS.  */
> +  if (! psame_inode (src_sb, &src_open_sb)
> +      && (src_sb->st_size || src_open_sb.st_size))
>      {
>        error (0, 0,
>               _("skipping file %s, as it was replaced while being copied"),
> 
> 

-- 
<https://www.alejandro-colomar.es/>

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Alejandro Colomar <alx <at> kernel.org>
Cc: Marc Chantreux <mc <at> unistra.fr>, Sergei Trofimovich <slyich <at> gmail.com>,
 70411 <at> debbugs.gnu.org, Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 20:44:06 +0100

On 16/04/2024 15:47, Alejandro Colomar wrote:
> Hi Pádraig,
> 
> On Tue, Apr 16, 2024 at 03:25:22PM +0100, Pádraig Brady wrote:
>>> What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
>>> The issue seems to be that /dev/stdin returns a varying inode which install(1) doesn't like currently
> 
> I don't know.  The reporter didn't tell.  I see you also asked on the
> Github original report.
> 
>> Note that check originally came from:
>> https://github.com/coreutils/coreutils/commit/dea4262fa
>>
>> I suppose we could relax the check as follows, for files of apparent size 0
>> which would cater for this, and others that may also have unstable inodes.
> 
> Hmmmm.  Since you couldn't reprodude it in a recent Darwin, maybe it's
> just a bug in an old Darwin.  And since noone else seems to have met
> this Darwin's bug, maybe we can just ignore it.  (And if it were a
> regression in a more recent Darwin, hopefully they should fix their
> kernel.)
> 
> I'm not happy relaxing a security check, without making sure that there
> are no implications at all.
> 
> I vote for claiming only limited support to such a Darwin system.  I
> already workarounded it in the Linux man-pages, by not piping to
> install(1) in a common task; and nobody else seems to be affected.
> 
> Unless you feel confident that it's perfectly fine to do it.  But I have
> no sympathy for workarounding Darwin bugs here.

I agree if it's older Darwin only, we can ignore.
The version I tested on is 3 years old now though,
so I'm not sure whether the issue is on newer or older.

Note we had similar issue on Solaris,
where we used an fstat() wrapper to adjust things:
https://bugs.gnu.org/35713

A related suggestion was from Marc Chantreux (CC'd)
to support '-' to imply stdin, which would be more portable.
There is some merit to that suggestion too.

cheers,
Pádraig

Message #20 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Alejandro Colomar <alx <at> kernel.org>, Pádraig Brady
 <P <at> draigbrady.com>
Cc: Sergei Trofimovich <slyich <at> gmail.com>, 70411 <at> debbugs.gnu.org,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 12:56:07 -0700

On 4/16/24 07:47, Alejandro Colomar wrote:
> Since you couldn't reprodude it in a recent Darwin, maybe it's
> just a bug in an old Darwin.

It'd have to be pretty old. As near as I can see from 
xnu/bsd/kern/sys_pipe.c, the st_ino field was zero (i.e., not random) 
even in xnu-792 dated 2005.

I'd be leery of changing this part of coreutils/src/copy.c without 
knowing more about exactly why bug occurs on (older?) Darwin, as 
weakening the check would make GNU 'install' easier to attack.

Message #23 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Alejandro Colomar <alx <at> kernel.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Marc Chantreux <mc <at> unistra.fr>, Sergei Trofimovich <slyich <at> gmail.com>,
 70411 <at> debbugs.gnu.org, Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 22:02:15 +0200

[Message part 1 (text/plain, inline)]

Hi,

On Tue, Apr 16, 2024 at 08:44:06PM +0100, Pádraig Brady wrote:
> I agree if it's older Darwin only, we can ignore.
> The version I tested on is 3 years old now though,
> so I'm not sure whether the issue is on newer or older.
> 
> Note we had similar issue on Solaris,
> where we used an fstat() wrapper to adjust things:
> https://bugs.gnu.org/35713
> 
> A related suggestion was from Marc Chantreux (CC'd)
> to support '-' to imply stdin, which would be more portable.
> There is some merit to that suggestion too.

I think /dev/stdin is a good name for stdin.  It should work everywhere,
and if not the system is broken, in which case, I'd rather fix the
system than adding a slightly different special case for - in every
program.  Of course there are systems one can't fix, like Solaris...

Commands that special-case - I always need to remember, and check the
docs.  But it's a personal opinion, and I don't feel too strongly about
it.

Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es/>

[signature.asc (application/pgp-signature, inline)]

Message #26 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Alejandro Colomar <alx <at> kernel.org>
Cc: Eli Schwartz <eschwartz93 <at> gmail.com>, Sergei Trofimovich <slyich <at> gmail.com>,
 70411 <at> debbugs.gnu.org, Marc Chantreux <mc <at> unistra.fr>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 13:07:59 -0700

On 4/16/24 12:44, Pádraig Brady wrote:
> A related suggestion was from Marc Chantreux (CC'd)
> to support '-' to imply stdin, which would be more portable.
> There is some merit to that suggestion too.

I see that merit too, as when 'install' reads from stdin it needn't do 
the inode check. However, POSIX requires 'cp - X' to copy a file named 
'-' to X; cp cannot copy from stdin. Although POSIX doesn't specify 
'install', there is some value to having 'install' behave like 'cp'.

I suppose we could have install special-case "/dev/stdin", on platforms 
where /dev/stdin is known to mean standard input. Seems like a bit much 
to work around a Darwin kernel bug, though.

Message #29 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Sergei Trofimovich <slyich <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 70411 <at> debbugs.gnu.org, Alejandro Colomar <alx <at> kernel.org>,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Tue, 16 Apr 2024 22:37:58 +0100

On Tue, 16 Apr 2024 12:33:44 +0100
Pádraig Brady <P <at> draigBrady.com> wrote:

> On 16/04/2024 01:19, Alejandro Colomar wrote:
> > Hi!
> > 
> > I don't own a Darwin system, so I can't help much reproduce.  However,
> > I've received a bug report to the Linux man-pages, that our build
> > system (GNUmakefile-based), which ends up calling
> > 
> > 	... | install /dev/stdin $@
> > 
> > doesn't work on Darwin.  Here's the original bug report:
> > <https://github.com/NixOS/nixpkgs/pull/300797>.
> > 
> > Here are the reported error messages:
> > 
> > ...
> > INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
> > INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3] Error 1
> > make: *** Waiting for unfinished jobs....
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3] Error 1
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > ...
> > 
> > I don't see why install(1) should fail to read /dev/stdin under any
> > POSIX system  
> 
> What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
> The issue seems to be that /dev/stdin returns a varying inode which install(1) doesn't like currently

The system I have locally is this one:

    $ uname -a
    Darwin tests-iMac.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64 x86_64

-- 

  Sergei

Message #32 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Sergei Trofimovich <slyich <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 70411 <at> debbugs.gnu.org, Alejandro Colomar <alx <at> kernel.org>,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Thu, 18 Apr 2024 22:52:32 +0100

On Tue, 16 Apr 2024 22:37:58 +0100
Sergei Trofimovich <slyich <at> gmail.com> wrote:

> On Tue, 16 Apr 2024 12:33:44 +0100
> Pádraig Brady <P <at> draigBrady.com> wrote:
> 
> > On 16/04/2024 01:19, Alejandro Colomar wrote:  
> > > Hi!
> > > 
> > > I don't own a Darwin system, so I can't help much reproduce.  However,
> > > I've received a bug report to the Linux man-pages, that our build
> > > system (GNUmakefile-based), which ends up calling
> > > 
> > > 	... | install /dev/stdin $@
> > > 
> > > doesn't work on Darwin.  Here's the original bug report:
> > > <https://github.com/NixOS/nixpkgs/pull/300797>.
> > > 
> > > Here are the reported error messages:
> > > 
> > > ...
> > > INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
> > > INSTALL         /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3] Error 1
> > > make: *** Waiting for unfinished jobs....
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > make: *** [share/mk/install/man.mk:54: /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3] Error 1
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > ...
> > > 
> > > I don't see why install(1) should fail to read /dev/stdin under any
> > > POSIX system    
> > 
> > What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
> > The issue seems to be that /dev/stdin returns a varying inode which install(1) doesn't like currently  
> 
> The system I have locally is this one:
> 
>     $ uname -a
>     Darwin tests-iMac.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64 x86_64

I debugged it a bit more locally and it looks like it's a known
limitation of fstat()/stat() mismatch on FIFOs on Darwin. Example
program:

$ cat simple.c
#include <sys/stat.h>
#include <assert.h>
#include <sys/fcntl.h>
#include <unistd.h>
#include <stdio.h>

int main() {
      struct stat s;
      int fd;
      int r;

      fd = open("/dev/fd/0", O_RDONLY);
      assert(fd > 0);
      r = fstat(fd, &s);
      assert(r == 0);
      fprintf(stderr, "1: ino=%llu\n", s.st_ino);
      close(fd);

      fd = open("/dev/fd/0", O_RDONLY);
      assert(fd > 0);
      r = fstat(fd, &s);
      assert(r == 0);
      fprintf(stderr, "2: ino=%llu\n", s.st_ino);
      close(fd);

      r = stat("/dev/fd/0", &s);
      assert(r == 0);
      fprintf(stderr, "3: ino=%llu\n", s.st_ino);
      close(fd);
}

Running:

$ clang simple.c -o simple && echo 42 | ./simple
1: ino=3009428657538693161
2: ino=3009428657538693161
3: ino=1568241705

Note how stat() and fstat() don't agree on inode.

Apparently it's documented in
https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
as

  BUGS
     Applying fstat to a socket (and thus to a pipe) returns a zero'd buffer,
     except for the blocksize field, and a unique device and inode number.

Perhaps coreutils should avoid inode checks on FIFOs and sockets. They
are already volatile by nature.

-- 

  Sergei

Message #35 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Alejandro Colomar <alx <at> kernel.org>
To: Sergei Trofimovich <slyich <at> gmail.com>
Cc: Pádraig Brady <P <at> draigbrady.com>,
 Eli Schwartz <eschwartz93 <at> gmail.com>, 70411 <at> debbugs.gnu.org
Subject: Re: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Fri, 19 Apr 2024 00:04:19 +0200

[Message part 1 (text/plain, inline)]

Hi Sergei,

On Thu, Apr 18, 2024 at 10:52:32PM +0100, Sergei Trofimovich wrote:
> > The system I have locally is this one:
> > 
> >     $ uname -a
> >     Darwin tests-iMac.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64 x86_64
> 
> I debugged it a bit more locally and it looks like it's a known
> limitation of fstat()/stat() mismatch on FIFOs on Darwin. Example
> program:

Thanks!

> $ cat simple.c
> #include <sys/stat.h>
> #include <assert.h>
> #include <sys/fcntl.h>
> #include <unistd.h>
> #include <stdio.h>
> 
> int main() {
>       struct stat s;
>       int fd;
>       int r;
> 
>       fd = open("/dev/fd/0", O_RDONLY);
>       assert(fd > 0);
>       r = fstat(fd, &s);
>       assert(r == 0);
>       fprintf(stderr, "1: ino=%llu\n", s.st_ino);
>       close(fd);
> 
>       fd = open("/dev/fd/0", O_RDONLY);
>       assert(fd > 0);
>       r = fstat(fd, &s);
>       assert(r == 0);
>       fprintf(stderr, "2: ino=%llu\n", s.st_ino);
>       close(fd);
> 
>       r = stat("/dev/fd/0", &s);
>       assert(r == 0);
>       fprintf(stderr, "3: ino=%llu\n", s.st_ino);
>       close(fd);
> }
> 
> Running:
> 
> $ clang simple.c -o simple && echo 42 | ./simple
> 1: ino=3009428657538693161
> 2: ino=3009428657538693161
> 3: ino=1568241705
> 
> Note how stat() and fstat() don't agree on inode.
> 
> Apparently it's documented in
> https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
> as
> 
>   BUGS
>      Applying fstat to a socket (and thus to a pipe) returns a zero'd buffer,
>      except for the blocksize field, and a unique device and inode number.
> 
> Perhaps coreutils should avoid inode checks on FIFOs and sockets. They
> are already volatile by nature.

I disagree.  I think coreutils should not try to work on broken systems;
especially when the checks are done for security reasons.  Let the
system die.

Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es/>

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sergei Trofimovich <slyich <at> gmail.com>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: Alejandro Colomar <alx <at> kernel.org>, 70411 <at> debbugs.gnu.org,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Thu, 18 Apr 2024 15:51:57 -0700

On 4/18/24 14:52, Sergei Trofimovich wrote:

> https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
> as
> 
>    BUGS
>       Applying fstat to a socket (and thus to a pipe) returns a zero'd buffer,
>       except for the blocksize field, and a unique device and inode number.

How is a macOS program supposed to tell whether a file has been changed 
between stat and open? Is it impossible?

Message #41 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sergei Trofimovich <slyich <at> gmail.com>
Cc: Gnulib bugs <bug-gnulib <at> gnu.org>,
 Pádraig Brady <P <at> draigBrady.com>,
 Alejandro Colomar <alx <at> kernel.org>, 70411 <at> debbugs.gnu.org,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Fri, 19 Apr 2024 00:33:52 -0700

[Message part 1 (text/plain, inline)]

On 2024-04-18 14:52, Sergei Trofimovich wrote:
> $ clang simple.c -o simple && echo 42 | ./simple
> 1: ino=3009428657538693161
> 2: ino=3009428657538693161
> 3: ino=1568241705
> 
> Note how stat() and fstat() don't agree on inode.
> 
> Apparently it's documented in
> https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
> as
> 
>    BUGS
>       Applying fstat to a socket (and thus to a pipe) returns a zero'd buffer,
>       except for the blocksize field, and a unique device and inode number.

The BUGS note simply means that a pipe has a unique inode number, which 
is what we want. So that's not indicating any problem.


Oh, I see the problem now.  For a socket or pipe, macOS fstat returns 
the full 64-bit inode number, whereas macOS stat returns only the low 
order 32 bits.  In your example, 3009428657538693161 % (2**32) == 
1568241705.

This is a kernel bug in macOS. Can you report it or otherwise arrange to 
have the kernel bug fixed? I expect that you have better connections 
with Apple than I do. A proposed patch (relative to xnu-10063.101.15) is 
attached; I have not tested it as I don't use macOS. Thanks.

Also, I am documenting this macOS bug in Gnulib by installing the second 
attached patch to Gnulib, and am cc'ing this email to bug-gnulib.

[0001-Fix-bug-with-stat-truncating-pipe-socket-st_ino.patch (text/x-patch, attachment)]

[0001-fstatat-stat-document-macOS-st_ino-pipe-bug.patch (text/x-patch, attachment)]

Message #44 received at 70411 <at> debbugs.gnu.org (full text, mbox):

From: Sergei Trofimovich <slyich <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Gnulib bugs <bug-gnulib <at> gnu.org>,
 Pádraig Brady <P <at> draigBrady.com>,
 Alejandro Colomar <alx <at> kernel.org>, 70411 <at> debbugs.gnu.org,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin
Date: Fri, 19 Apr 2024 22:33:00 +0100

On Fri, 19 Apr 2024 00:33:52 -0700
Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> On 2024-04-18 14:52, Sergei Trofimovich wrote:
> > $ clang simple.c -o simple && echo 42 | ./simple
> > 1: ino=3009428657538693161
> > 2: ino=3009428657538693161
> > 3: ino=1568241705
> > 
> > Note how stat() and fstat() don't agree on inode.
> > 
> > Apparently it's documented in
> > https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
> > as
> > 
> >    BUGS
> >       Applying fstat to a socket (and thus to a pipe) returns a zero'd buffer,
> >       except for the blocksize field, and a unique device and inode number.  
> 
> The BUGS note simply means that a pipe has a unique inode number, which 
> is what we want. So that's not indicating any problem.
> 
> 
> Oh, I see the problem now.  For a socket or pipe, macOS fstat returns 
> the full 64-bit inode number, whereas macOS stat returns only the low 
> order 32 bits.  In your example, 3009428657538693161 % (2**32) == 
> 1568241705.
> 
> This is a kernel bug in macOS. Can you report it or otherwise arrange to 
> have the kernel bug fixed? I expect that you have better connections 
> with Apple than I do. A proposed patch (relative to xnu-10063.101.15) is 
> attached; I have not tested it as I don't use macOS. Thanks.

I reported it via https://www.apple.com/feedback/macos.html

> Also, I am documenting this macOS bug in Gnulib by installing the second 
> attached patch to Gnulib, and am cc'ing this email to bug-gnulib.

Thank you, Paul!

-- 

  Sergei

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.