GNU bug report logs - #7574
[PATCH]: echo,printf,stat: Allow only up to 8 bit octal input for backslash-escaped chars

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 7574 in the body.
You can then email your comments to 7574 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ondrej Vasik <ovasik <at> redhat.com>
To: CoreutilsBugs <bug-coreutils <at> gnu.org>
Subject: [PATCH]: echo,printf,stat: Allow only up to 8 bit octal input for
	backslash-escaped chars
Date: Mon, 06 Dec 2010 17:34:03 +0100

[Message part 1 (text/plain, inline)]

Hi,
as reported in RHBZ#660033
( https://bugzilla.redhat.com/show_bug.cgi?id=660033 ), echo, printf and
stat allows 3 octal digits without limitation to 8-bit.
Documentation(manpages, info) refers to "byte with octal value" or
"8-bit octal value". Therefore 9-bit octal values should not be allowed.
Especially in echo, only unsigned char is used for storing this octal
number, so 9-bit values overflow.
I see two ways of fixing : a) change documentation (informing only 1-3
octal digits input, no "8-bit" or "byte" words)
                           b) accept only up to 8-bit octals

Because of the unsigned char overflow, I prefer the 8-bit limit - and I
did so in attached patch. As I don't expect this will be noticed by
anyone (probably most of users already limit these octals to 8-bit
independently), I didn't added NEWS entry. Test testing printf '\0610'
output is added. Previously it was interpreted as 392 and this was
passed to putchar(), after the patch it is interpreted as '\061' + '0'
=> 10 .
I have also added missing \NNN GNU extension to --help output of echo.

Greetings,
          Ondrej Vasik

[escaped-ninebit-octal-char.patch (text/x-patch, attachment)]

Message #8 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: ovasik <at> redhat.com
Cc: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,	stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Mon, 06 Dec 2010 11:32:35 -0700

[Message part 1 (text/plain, inline)]

On 12/06/2010 09:34 AM, Ondrej Vasik wrote:
> Hi,
> as reported in RHBZ#660033
> ( https://bugzilla.redhat.com/show_bug.cgi?id=660033 ), echo, printf and
> stat allows 3 octal digits without limitation to 8-bit.

First, let's look at POSIX, which has slightly different wording in its
requirements for 'echo' and 'printf' (for stat, we can do whatever we want).

echo: Support is only required for an "8-bit value" \0num, that is, a
leading 0 is required, and num may be 0-3 (but no more) digits.  \1 is
implementation-defined (we happen to define it as shorthand for \01).

printf: In the format argument, support is required for \ddd, where ddd
may be 1-3 (but no more) digits; no leading 0 required, and leading zero
counts against your 3 digits.  When matching up to the %b escape in the
format argument, support is required for \0ddd, where ddd may be 0-3
(but no more digits), with the same rules about leading 0 as in echo.
Both cases mention "converted to a byte with the numeric value", and
POSIX requires bytes to be 8-bit values.

Given the similarity between printf %b and echo, I think we can safely
assume that "byte with the numeric value" of printf should be treated as
"8-bit value" of echo.  Therefore, in either case, I agree with your
assessment that \0610 should necessarily be treated as '\061' followed
by the character '0', since the octal value 0610 is not 8-bit, and that
printf \300 should be treated as '\30' followed by '0'.

> Because of the unsigned char overflow, I prefer the 8-bit limit - and I
> did so in attached patch. As I don't expect this will be noticed by
> anyone (probably most of users already limit these octals to 8-bit
> independently), I didn't added NEWS entry.

It still deserves a NEWS entry, since this is a bug fix.

> Test testing printf '\0610'
> output is added. Previously it was interpreted as 392 and this was
> passed to putchar(), after the patch it is interpreted as '\061' + '0'
> => 10 .

Sounds correct to me, although I haven't closely reviewed the patch yet.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #11 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: ovasik <at> redhat.com
Cc: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,	stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Mon, 06 Dec 2010 12:43:02 -0800

As a minor point, both uses like this:

'4' <= *p && *p <= '7'

can be replaced with something like this:

'4' <= *p

and similarly, this:

'0' <= c && c <= '3'

can be replaced with this:

c < '4'

The replacements are not only shorter, but easier
to understand, because otherwise the reader is left
wondering why that unnecessary comparison to '7'
(or '0') is in there.

Message #14 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Ondrej Vasik <ovasik <at> redhat.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf, stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Tue, 07 Dec 2010 09:41:59 +0100

[Message part 1 (text/plain, inline)]

Paul Eggert wrote:
> As a minor point, both uses like this:
> 
> '4' <= *p && *p <= '7'
> 
> can be replaced with something like this:
> 
> '4' <= *p
> 
...
> 
> The replacements are not only shorter, but easier
> to understand, because otherwise the reader is left
> wondering why that unnecessary comparison to '7'
> (or '0') is in there.

Yep, you are right - as the octal value of *p/*b is checked later, this
shorter way is better readable and shorter.
Added the NEWS entry in addition to address Eric's email.

Greetings,
         Ondrej Vasik

[escaped-ninebit-octal-char.patch (text/x-patch, attachment)]

Message #17 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: ovasik <at> redhat.com
Cc: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,	stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Tue, 07 Dec 2010 09:48:03 +0000

There is an extraneous \n in the NEWS.

Also I'd rephrase:

+(if @var{ooo} is 1 to 3 octal digits byte value) specifying a character
+to print, and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is
+1 to 2 hex digits) specifying a character to print.

as:

(if @var{ooo} is 1 to 3 octal digits) specifying an 8 bit character
to print, and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is
1 to 2 hex digits) specifying a character to print.

cheers,
Pádraig.

Message #20 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Ondrej Vasik <ovasik <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf, stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Tue, 07 Dec 2010 12:14:12 +0100

[Message part 1 (text/plain, inline)]

Pádraig Brady píše v Út 07. 12. 2010 v 09:48 +0000:
> There is an extraneous \n in the NEWS.
> 
> Also I'd rephrase:
> 
> +(if @var{ooo} is 1 to 3 octal digits byte value) specifying a character
> +to print, and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is
> +1 to 2 hex digits) specifying a character to print.
> 
> as:
> 
> (if @var{ooo} is 1 to 3 octal digits) specifying an 8 bit character
> to print, and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is
> 1 to 2 hex digits) specifying a character to print.

Ah, sorry... redundant newline removed and info rephrased.

Greetings,
         Ondrej

[escaped-ninebit-octal-char.patch (text/x-patch, attachment)]

Message #23 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Ondrej Vasik <ovasik <at> redhat.com>
To: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,	stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Fri, 14 Jan 2011 14:32:05 +0100

As the same bash request for change in builtin echo and printf
(http://lists.gnu.org/archive/html/bug-bash/2010-12/msg00030.html and
https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=15087 ) was rejected, I think we should do the same here to keep echo and printf implementations as close as possible.

Anyway, it would be better to be consistent in all utilities -e.g.
tr.c:502 now behaves the way proposed in the patch in this bugzilla -
and at least document that the ninth bit is ignored in the \@var{ooo}
section of info documentation.

Message #28 received at 7574-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Ondřej Vašík <ondrej.vasik <at> gmail.com>
Cc: 7574-done <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,
	stat: Allow only up to 8 bit octal input for backslash-escaped chars
Date: Fri, 14 Jan 2011 15:47:13 +0100

Ondrej Vasik wrote:
> As the same bash request for change in builtin echo and printf
> (http://lists.gnu.org/archive/html/bug-bash/2010-12/msg00030.html and
> https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=15087
> ) was rejected, I think we should do the same here to keep echo and
> printf implementations as close as possible.

Keeping echo implementations in sync (between coreutils and shells)
is desirable, but as you imply below, keeping tools consistent, e.g.,
in how they handle an argument specified like \400 is important, too.

Note that GNU tr interprets \400 like this:

    $ tr '\400' x
    tr: warning: the ambiguous octal escape \400 is being
            interpreted as the 2-byte sequence \040, 0

That does not match how printf interprets '\400':

    $ printf '\400' | od -An -c
      \0

> Anyway, it would be better to be consistent in all utilities -e.g.
> tr.c:502 now behaves the way proposed in the patch in this bugzilla -
> and at least document that the ninth bit is ignored in the \@var{ooo}
> section of info documentation.

Good idea.
Patch below.

I'm marking this issue as "done", but feel free to open-and-retitle or
simply to file a new bug if you want to pursue this.

From 63fdb7c671cd0c05ba1b1710d0922016ce687362 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering <at> redhat.com>
Date: Fri, 14 Jan 2011 15:45:58 +0100
Subject: [PATCH] doc: document how printf treats e.g., \400
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* doc/coreutils.texi (printf invocation): Document that any
ninth bit in \OOO is ignored.  Suggested by Ondřej Vašík in
http://debbugs.gnu.org/7574
---
 doc/coreutils.texi |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 85d5201..a51af26 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -11195,9 +11195,12 @@ printf invocation
 @kindex \@var{ooo}
 @kindex \x <at> var{hh}
 @command{printf} interprets @samp{\@var{ooo}} in @var{format} as an octal number
-(if @var{ooo} is 1 to 3 octal digits) specifying a character to print,
+(if @var{ooo} is 1 to 3 octal digits) specifying a byte to print,
 and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is 1 to 2 hex
 digits) specifying a character to print.
+Note however that when @samp{\@var{ooo}} specifies a number larger than 255,
+the ninth bit is ignored.  For example, @samp{printf '\400'} is equivalent
+to @samp{printf '\0'}.

 @kindex \uhhhh
 @kindex \Uhhhhhhhh
--
1.7.3.5

Message #31 received at 7574 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: ovasik <at> redhat.com
Cc: 7574 <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,	stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Fri, 14 Jan 2011 08:07:56 -0700

[Message part 1 (text/plain, inline)]

On 01/14/2011 06:32 AM, Ondrej Vasik wrote:
> As the same bash request for change in builtin echo and printf
> (http://lists.gnu.org/archive/html/bug-bash/2010-12/msg00030.html and
> https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=15087 ) was rejected, I think we should do the same here to keep echo and printf implementations as close as possible.
> 
> Anyway, it would be better to be consistent in all utilities -e.g.
> tr.c:502 now behaves the way proposed in the patch in this bugzilla -
> and at least document that the ninth bit is ignored in the \@var{ooo}
> section of info documentation.

The POSIX decision was to document that \400 has unspecified behavior
(meaning the ninth bit can be ignored, an error issued, the string
treated as \40 followed by a literal 0, or any other action).  It
doesn't bother me if we differ slightly from bash, as the behavior is
already undefined and people should not be relying on it.

http://austingroupbugs.net/view.php?id=249

        If more than two hexadecimal digits immediately follow \x or
        if the octal value specified by \XXX will not fit in a byte, the
        results are unspecified.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #32 received at 7574-done <at> debbugs.gnu.org (full text, mbox):

From: Ondrej Vasik <ovasik <at> redhat.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: Ondřej Vašík <ondrej.vasik <at> gmail.com>,
	7574-done <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf, stat: Allow only up to 8 bit
	octal input for backslash-escaped chars
Date: Fri, 14 Jan 2011 16:45:46 +0100

[Message part 1 (text/plain, inline)]

Jim Meyering wrote:
> Ondrej Vasik wrote:
> > Anyway, it would be better to be consistent in all utilities -e.g.
> > tr.c:502 now behaves the way proposed in the patch in this bugzilla -
> > and at least document that the ninth bit is ignored in the \@var{ooo}
> > section of info documentation.
> 
> Good idea.
> Patch below.
> 
> I'm marking this issue as "done", but feel free to open-and-retitle or
> simply to file a new bug if you want to pursue this.
> 
> From 63fdb7c671cd0c05ba1b1710d0922016ce687362 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering <at> redhat.com>
> Date: Fri, 14 Jan 2011 15:45:58 +0100
> Subject: [PATCH] doc: document how printf treats e.g., \400
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> * doc/coreutils.texi (printf invocation): Document that any
> ninth bit in \OOO is ignored.  Suggested by Ondřej Vašík in
> http://debbugs.gnu.org/7574
> ---
>  doc/coreutils.texi |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
> 
> diff --git a/doc/coreutils.texi b/doc/coreutils.texi
> index 85d5201..a51af26 100644
> --- a/doc/coreutils.texi
> +++ b/doc/coreutils.texi
> @@ -11195,9 +11195,12 @@ printf invocation
>  @kindex \@var{ooo}
>  @kindex \x <at> var{hh}
>  @command{printf} interprets @samp{\@var{ooo}} in @var{format} as an octal number
> -(if @var{ooo} is 1 to 3 octal digits) specifying a character to print,
> +(if @var{ooo} is 1 to 3 octal digits) specifying a byte to print,
>  and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is 1 to 2 hex
>  digits) specifying a character to print.
> +Note however that when @samp{\@var{ooo}} specifies a number larger than 255,
> +the ninth bit is ignored.  For example, @samp{printf '\400'} is equivalent
> +to @samp{printf '\0'}.
> 
>  @kindex \uhhhh
>  @kindex \Uhhhhhhhh
> --
> 1.7.3.5

There were more instances of the \OOO in texi documentation, they should
be all covered by attached patch (including 8-bit character mentioned in
tr info). 
As discussed via IRC, sending adjusted patch.

Ondrej

[coreutils-octalinfo.patch (text/x-patch, attachment)]

Message #33 received at 7574-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: ovasik <at> redhat.com
Cc: Ondřej Vašík <ondrej.vasik <at> gmail.com>,
	7574-done <at> debbugs.gnu.org
Subject: Re: bug#7574: [PATCH]: echo, printf,
	stat: Allow only up to 8 bit octal input for backslash-escaped chars
Date: Sat, 15 Jan 2011 11:55:43 +0100

Ondrej Vasik wrote:
> Jim Meyering wrote:
>> Ondrej Vasik wrote:
>> > Anyway, it would be better to be consistent in all utilities -e.g.
>> > tr.c:502 now behaves the way proposed in the patch in this bugzilla -
>> > and at least document that the ninth bit is ignored in the \@var{ooo}
>> > section of info documentation.
>>
>> Good idea.
>> Patch below.
>>
>> I'm marking this issue as "done", but feel free to open-and-retitle or
>> simply to file a new bug if you want to pursue this.
...
> There were more instances of the \OOO in texi documentation, they should
> be all covered by attached patch (including 8-bit character mentioned in
> tr info).
> As discussed via IRC, sending adjusted patch.

Thanks.
I've added this sentence to the "tr" section:

  Note that @samp{'\400'} is interpreted as the two-byte
  sequence, @samp{'\040'} @samp{0}.

then, I converted some passive "is ignored" uses to an active voice, e.g.,

-@samp{\@var{ooo}} is nine-bit value, the ninth bit is ignored.
+@samp{\@var{ooo}} is nine-bit value, ignore the ninth bit.

and mentioned each affected section in the log message:

From 442b068ac1573ae00fdf029ddb77b35fdcdf8fc0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ond=C5=99ej=20Va=C5=A1=C3=ADk?= <ovasik <at> redhat.com>
Date: Fri, 14 Jan 2011 16:38:57 +0100
Subject: [PATCH] doc: specify how tr, echo, printf treat octal numbers

* doc/coreutils.texi (tr's Character sets): Document how a 9-bit
octal value is interpreted.  tr does not ignore the ninth bit.
(echo invocation, printf invocation): Document that any ninth
bit in \OOO is ignored. (http://debbugs.gnu.org/7574)
---
 doc/coreutils.texi |   19 +++++++++++++------
 1 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 85d5201..9c3e2ed 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -5850,8 +5850,9 @@ Character sets
 @item \v
 Control-K.
 @item \@var{ooo}
-The character with the value given by @var{ooo}, which is 1 to 3
-octal digits,
+The 8-bit character with the value given by @var{ooo}, which is 1 to 3
+octal digits.  Note that @samp{\400} is interpreted as the two-byte
+sequence, @samp{\040} @samp{0}.
 @item \\
 A backslash.
 @end table
@@ -11080,10 +11081,12 @@ echo invocation
 backslash
 @item \0 <at> var{nnn}
 the eight-bit value that is the octal number @var{nnn}
-(zero to three octal digits)
+(zero to three octal digits), if @var{nnn} is
+a nine-bit value, the ninth bit is ignored
 @item \@var{nnn}
 the eight-bit value that is the octal number @var{nnn}
-(one to three octal digits)
+(one to three octal digits), if @var{nnn} is
+a nine-bit value, the ninth bit is ignored
 @item \x <at> var{hh}
 the eight-bit value that is the hexadecimal number @var{hh}
 (one or two hexadecimal digits)
@@ -11164,7 +11167,8 @@ printf invocation
 @command{printf} has an additional directive, @samp{%b}, which prints its
 argument string with @samp{\} escapes interpreted in the same way as in
 the @var{format} string, except that octal escapes are of the form
-@samp{\0 <at> var{ooo}} where @var{ooo} is 0 to 3 octal digits.
+@samp{\0 <at> var{ooo}} where @var{ooo} is 0 to 3 octal digits.  If
+@samp{\@var{ooo}} is nine-bit value, ignore the ninth bit.
 If a precision is also given, it limits the number of bytes printed
 from the converted string.

@@ -11195,9 +11199,12 @@ printf invocation
 @kindex \@var{ooo}
 @kindex \x <at> var{hh}
 @command{printf} interprets @samp{\@var{ooo}} in @var{format} as an octal number
-(if @var{ooo} is 1 to 3 octal digits) specifying a character to print,
+(if @var{ooo} is 1 to 3 octal digits) specifying a byte to print,
 and @samp{\x <at> var{hh}} as a hexadecimal number (if @var{hh} is 1 to 2 hex
 digits) specifying a character to print.
+Note however that when @samp{\@var{ooo}} specifies a number larger than 255,
+@command{printf} ignores the ninth bit.
+For example, @samp{printf '\400'} is equivalent to @samp{printf '\0'}.

 @kindex \uhhhh
 @kindex \Uhhhhhhhh
--
1.7.3.5

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.