GNU bug report logs - #53801
[PATCH] tests: have printf(1) use octal numbers for POSIX conformance

Previous Next

To reply to this bug, email your comments to 53801 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kerin Millar <kfm <at> plushkava.net>
To: bug-parted <at> gnu.org
Subject: [PATCH] tests: have printf(1) use octal numbers for POSIX conformance
Date: Sat, 5 Feb 2022 10:44:29 +0000

[Message part 1 (text/plain, inline)]

Hello,

The attached patch rectifies some test failures that were reported at https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, POSIX does not require for the printf utility to support hexadecimal notation.

-- 
Kerin Millar

[0001-tests-have-printf-1-use-octal-numbers-for-POSIX-conf.patch (application/octet-stream, attachment)]

Message #8 received at 53801 <at> debbugs.gnu.org (full text, mbox):

From: "Brian C. Lane" <bcl <at> redhat.com>
To: Kerin Millar <kfm <at> plushkava.net>
Cc: 53801 <at> debbugs.gnu.org
Subject: Re: bug#53801: [PATCH] tests: have printf(1) use octal numbers for
 POSIX conformance
Date: Mon, 7 Feb 2022 08:52:29 -0800

On Sat, Feb 05, 2022 at 10:44:29AM +0000, Kerin Millar wrote:
> Hello,
> 
> The attached patch rectifies some test failures that were reported at https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, POSIX does not require for the printf utility to support hexadecimal notation.
> 
> -- 
> Kerin Millar

Thanks for the patch. I'll have to ponder this for a bit. So the issue
is showing up in Gentoo because it uses dash, right? The issue I have
with the change is that it then obscures what's being written. Everyone
knows what 0x55aa is :) I'd prefer a solution that continues to use hex
so that future users don't have to figure out what's going on.

Brian

-- 
Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart

Message #11 received at 53801 <at> debbugs.gnu.org (full text, mbox):

From: "Brian C. Lane" <bcl <at> redhat.com>
To: Kerin Millar <kfm <at> plushkava.net>
Cc: 53801 <at> debbugs.gnu.org
Subject: Re: bug#53801: [PATCH] tests: have printf(1) use octal numbers for
 POSIX conformance
Date: Mon, 7 Feb 2022 09:10:12 -0800

On Mon, Feb 07, 2022 at 08:52:33AM -0800, Brian C. Lane wrote:
> On Sat, Feb 05, 2022 at 10:44:29AM +0000, Kerin Millar wrote:
> > Hello,
> > 
> > The attached patch rectifies some test failures that were reported at https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, POSIX does not require for the printf utility to support hexadecimal notation.
> > 
> > -- 
> > Kerin Millar
> 
> Thanks for the patch. I'll have to ponder this for a bit. So the issue
> is showing up in Gentoo because it uses dash, right? The issue I have
> with the change is that it then obscures what's being written. Everyone
> knows what 0x55aa is :) I'd prefer a solution that continues to use hex
> so that future users don't have to figure out what's going on.

Actually, on Fedora at least, printf comes from coreutils so the shell
shouldn't matter at all. The tests are single quoted to prevent shell
expansion, so maybe dash is using a builtin printf that's different?

Brian

-- 
Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart

Message #14 received at 53801 <at> debbugs.gnu.org (full text, mbox):

From: Kerin Millar <kfm <at> plushkava.net>
To: "Brian C. Lane" <bcl <at> redhat.com>
Cc: 53801 <at> debbugs.gnu.org
Subject: Re: bug#53801: [PATCH] tests: have printf(1) use octal numbers for
 POSIX conformance
Date: Mon, 7 Feb 2022 18:11:19 +0000

Hi Brian,

On Mon, 7 Feb 2022 08:52:29 -0800
"Brian C. Lane" <bcl <at> redhat.com> wrote:

> On Sat, Feb 05, 2022 at 10:44:29AM +0000, Kerin Millar wrote:
> > Hello,
> > 
> > The attached patch rectifies some test failures that were reported at https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, POSIX does not require for the printf utility to support hexadecimal notation.
> > 
> > -- 
> > Kerin Millar
> 
> Thanks for the patch. I'll have to ponder this for a bit. So the issue
> is showing up in Gentoo because it uses dash, right? The issue I have
> with the change is that it then obscures what's being written. Everyone
> knows what 0x55aa is :) I'd prefer a solution that continues to use hex
> so that future users don't have to figure out what's going on.

In fact, Gentoo provides bash as a default sh(1) implementation, by which I mean that /bin/sh begins life as being a symlink to bash. Still, that's neither here nor there. The policy of the distro is that the user may select another sh(1) provider, as long as it is POSIX-conforming. Dash is merely one such example of a shell that implements the Shell Command Language specification correctly and in full. Should an issue arise as a consequence of /bin/sh being any conforming implementation other than bash, it is rightly treated as a bug. Often, patches are proposed and the more responsible contributors try to get them upstreamed so that everyone benefits. Of course, there exists other distributions where /bin/sh is some other implementation by default.

Essentially, this is one of those cases where one cannot have one's cake and eat it. By beginning a script with a shebang that reads #!/bin/sh, it is implied - in no uncertain terms - that the entirety of the script should run correctly in any conforming implementation of the Shell Command Language. In almost all modern implementations, the printf utility is implemented as a builtin and their authors are in no way obligated to implement anything above and beyond the syntax described by https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html. While bash is - at heart - a valid implementation of the Shell Command Language, it supports a great many extensions and does not suppress them, even when running in its "posix" mode. In fact, this is not the first time this issue has come up; see "t0100-print.sh" and "t0251-gpt-unicode.sh".

I understand that most developers will find hex notation easier to read, which is why my patch also adjusts the surrounding commentary to that end. In my view, the alternatives are worse:-

a) changing the shebangs to #!/bin/bash, rendering bash a hard requirement for running the affected tests (merely for a printf)
b) combining arithmetic expansion with printf

Regarding (b), below is an example.

# Write the two magic bytes
magic=$(printf '\\%03o' $(( 0x55 )) $(( 0x0A )))
printf "$magic" | ...

Not only is this more expensive, one might then need to be cautious as to the potential matter of unintentionally injecting the results of expansions into the format string, owing to the double-quoted context. I certainly wouldn't consider it to be any easier to digest than the following.

# Write the two magic bytes, 0x55 0xAA.
printf '\125\252' | ...

Incidentally, octal is trivial to convert to peruse as hex using only standard utilities.

$ printf '\125\252' | od -An -t x1
 55 aa

Regards,

-- 
Kerin Millar

Message #17 received at 53801 <at> debbugs.gnu.org (full text, mbox):

From: Kerin Millar <kfm <at> plushkava.net>
To: "Brian C. Lane" <bcl <at> redhat.com>
Cc: 53801 <at> debbugs.gnu.org
Subject: Re: bug#53801: [PATCH] tests: have printf(1) use octal numbers for
 POSIX conformance
Date: Mon, 7 Feb 2022 18:21:31 +0000

On Mon, 7 Feb 2022 09:10:12 -0800
"Brian C. Lane" <bcl <at> redhat.com> wrote:

> On Mon, Feb 07, 2022 at 08:52:33AM -0800, Brian C. Lane wrote:
> > On Sat, Feb 05, 2022 at 10:44:29AM +0000, Kerin Millar wrote:
> > > Hello,
> > > 
> > > The attached patch rectifies some test failures that were reported at https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, POSIX does not require for the printf utility to support hexadecimal notation.
> > > 
> > > -- 
> > > Kerin Millar
> > 
> > Thanks for the patch. I'll have to ponder this for a bit. So the issue
> > is showing up in Gentoo because it uses dash, right? The issue I have
> > with the change is that it then obscures what's being written. Everyone
> > knows what 0x55aa is :) I'd prefer a solution that continues to use hex
> > so that future users don't have to figure out what's going on.
> 
> Actually, on Fedora at least, printf comes from coreutils so the shell
> shouldn't matter at all. The tests are single quoted to prevent shell
> expansion, so maybe dash is using a builtin printf that's different?

$ bash -c 'type printf'
printf is a shell builtin

Yes, it's also a builtin in dash (it is rare these days for printf to ever be invoked in its capacity as a binary). As explained by my prior message, no implementation of the POSIX shell and utilities (XCU) is under any obligation to support GNU-specific features. Therein lies the rub. Octal sequences are guaranteed to work. The problem is a great many Linux developers only ever test things in bash, even where their shebangs purport to target sh.

-- 
Kerin Millar

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.