GNU bug report logs - #22254
[PATCH] sed: do not elide an invalid byte in a substitution RHS

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22254 in the body.
You can then email your comments to 22254 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: bug-sed <at> gnu.org
Subject: [PATCH] sed: do not elide an invalid byte in a substitution RHS
Date: Sun, 27 Dec 2015 22:02:24 -0800

[Message part 1 (text/plain, inline)]

Just after pushing the fix for that heap-clobbering bug,
http://debbugs.gnu.org/22127, I noticed there was another minor
problem: sed would silently elide any invalid byte specified in a
substitution replacement string.

Here's a patch that's not quite complete (no NEWS entry, and I haven't
yet tried to determine when it was introduced).

[0001-sed-do-not-elide-an-invalid-byte-in-a-substitution-R.patch (text/x-patch, attachment)]

Message #8 received at 22254 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: 22254 <at> debbugs.gnu.org
Subject: Re: bug#22254: [PATCH] sed: do not elide an invalid byte in a
 substitution RHS
Date: Sun, 27 Dec 2015 22:24:13 -0800

On Sun, Dec 27, 2015 at 10:02 PM, Jim Meyering <jim <at> meyering.net> wrote:
> Just after pushing the fix for that heap-clobbering bug,
> http://debbugs.gnu.org/22127, I noticed there was another minor
> problem: sed would silently elide any invalid byte specified in a
> substitution replacement string.
>
> Here's a patch that's not quite complete (no NEWS entry, and I haven't
> yet tried to determine when it was introduced).

This one has a checkered past. I keep a decent sampling of old
versions of sed around.
Running one of the two tests that would be affected shows how
treatment of an invalid byte in a multi-byte locale has differed over
the years:

$ for i in $(env ls -dv /p/p/sed-*); do printf '%-14s: ' $i; echo a
|LC_ALL=ja_JP $i/bin/sed 's/a/b\U\xb2c/'|od -tx1|head -1; done|sed
s/.....//
sed-1.18 : 0000000 62 55 78 62 32 63 0a
sed-2.05 : 0000000 62 55 78 62 32 63 0a
sed-3.01 : 0000000 62 55 78 62 32 63 0a
sed-3.02 : 0000000 62 55 78 62 32 63 0a
sed-4.0a : 0000000 62 00 43 0a
sed-4.0b : 0000000 62 00 43 0a
sed-4.0c : 0000000 62 00 43 0a
sed-4.0d : 0000000 62 00 43 0a
sed-4.0e : 0000000 62 00 43 0a
sed-4.0.6: 0000000 62 b2 43 0a
sed-4.0.7: 0000000 62 b2 43 0a
sed-4.0.8: 0000000 62 b2 43 0a
sed-4.0.9: 0000000 62 b2 43 0a
sed-4.1  : 0000000 62 00 43 0a
sed-4.1c : 0000000 62 00 43 0a
sed-4.1d : 0000000 62 00 43 0a
sed-4.1e : 0000000 62 00 43 0a
sed-4.1f : 0000000 62 00 43 0a
sed-4.1.1: 0000000 62 00 43 0a
sed-4.1.2: 0000000 62 00 43 0a
sed-4.1.3: 0000000 62 00 43 0a
sed-4.1.4: 0000000 62 00 43 0a
sed-4.1.5: 0000000 62 00 43 0a
sed-4.2  : 0000000 62 00 43 0a
sed-4.2.1: 0000000 62 00 43 0a
sed-4.2.2: 0000000 62 00 43 0a

Now, it does what 4.0.6..4.0.9 did.

Message #11 received at 22254 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: 22254 <at> debbugs.gnu.org
Subject: Re: bug#22254: [PATCH] sed: do not elide an invalid byte in a
 substitution RHS
Date: Sun, 27 Dec 2015 22:42:53 -0800

[Message part 1 (text/plain, inline)]

On Sun, Dec 27, 2015 at 10:24 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Sun, Dec 27, 2015 at 10:02 PM, Jim Meyering <jim <at> meyering.net> wrote:
>> Just after pushing the fix for that heap-clobbering bug,
>> http://debbugs.gnu.org/22127, I noticed there was another minor
>> problem: sed would silently elide any invalid byte specified in a
>> substitution replacement string.
>>
>> Here's a patch that's not quite complete (no NEWS entry, and I haven't
>> yet tried to determine when it was introduced).
>
> This one has a checkered past. I keep a decent sampling of old
> versions of sed around.
> Running one of the two tests that would be affected shows how
> treatment of an invalid byte in a multi-byte locale has differed over
> the years:
>
> $ for i in $(env ls -dv /p/p/sed-*); do printf '%-14s: ' $i; echo a
> |LC_ALL=ja_JP $i/bin/sed 's/a/b\U\xb2c/'|od -tx1|head -1; done|sed
> s/.....//
> sed-1.18 : 0000000 62 55 78 62 32 63 0a
> sed-2.05 : 0000000 62 55 78 62 32 63 0a
> sed-3.01 : 0000000 62 55 78 62 32 63 0a
> sed-3.02 : 0000000 62 55 78 62 32 63 0a
> sed-4.0a : 0000000 62 00 43 0a
> sed-4.0b : 0000000 62 00 43 0a
> sed-4.0c : 0000000 62 00 43 0a
> sed-4.0d : 0000000 62 00 43 0a
> sed-4.0e : 0000000 62 00 43 0a
> sed-4.0.6: 0000000 62 b2 43 0a
> sed-4.0.7: 0000000 62 b2 43 0a
> sed-4.0.8: 0000000 62 b2 43 0a
> sed-4.0.9: 0000000 62 b2 43 0a
> sed-4.1  : 0000000 62 00 43 0a
> sed-4.1c : 0000000 62 00 43 0a
> sed-4.1d : 0000000 62 00 43 0a
> sed-4.1e : 0000000 62 00 43 0a
> sed-4.1f : 0000000 62 00 43 0a
> sed-4.1.1: 0000000 62 00 43 0a
> sed-4.1.2: 0000000 62 00 43 0a
> sed-4.1.3: 0000000 62 00 43 0a
> sed-4.1.4: 0000000 62 00 43 0a
> sed-4.1.5: 0000000 62 00 43 0a
> sed-4.2  : 0000000 62 00 43 0a
> sed-4.2.1: 0000000 62 00 43 0a
> sed-4.2.2: 0000000 62 00 43 0a
>
> Now, it does what 4.0.6..4.0.9 did.

Here's a complete patch:

[0001-sed-do-not-elide-an-invalid-byte-in-a-substitution-R.patch (text/x-patch, attachment)]

Message #16 received at 22254-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: 22254-done <at> debbugs.gnu.org
Subject: Re: bug#22254: [PATCH] sed: do not elide an invalid byte in a
 substitution RHS
Date: Mon, 28 Dec 2015 22:18:22 -0800

On Sun, Dec 27, 2015 at 10:42 PM, Jim Meyering <jim <at> meyering.net> wrote:
> On Sun, Dec 27, 2015 at 10:24 PM, Jim Meyering <jim <at> meyering.net> wrote:
>> On Sun, Dec 27, 2015 at 10:02 PM, Jim Meyering <jim <at> meyering.net> wrote:
>>> Just after pushing the fix for that heap-clobbering bug,
>>> http://debbugs.gnu.org/22127, I noticed there was another minor
>>> problem: sed would silently elide any invalid byte specified in a
>>> substitution replacement string.
>>>
>>> Here's a patch that's not quite complete (no NEWS entry, and I haven't
>>> yet tried to determine when it was introduced).
>>
>> This one has a checkered past. I keep a decent sampling of old
>> versions of sed around.
>> Running one of the two tests that would be affected shows how
>> treatment of an invalid byte in a multi-byte locale has differed over
>> the years:
>>
>> $ for i in $(env ls -dv /p/p/sed-*); do printf '%-14s: ' $i; echo a
>> |LC_ALL=ja_JP $i/bin/sed 's/a/b\U\xb2c/'|od -tx1|head -1; done|sed
>> s/.....//
>> sed-1.18 : 0000000 62 55 78 62 32 63 0a
>> sed-2.05 : 0000000 62 55 78 62 32 63 0a
>> sed-3.01 : 0000000 62 55 78 62 32 63 0a
>> sed-3.02 : 0000000 62 55 78 62 32 63 0a
>> sed-4.0a : 0000000 62 00 43 0a
...
>> sed-4.0e : 0000000 62 00 43 0a
>> sed-4.0.6: 0000000 62 b2 43 0a
>> sed-4.0.7: 0000000 62 b2 43 0a
>> sed-4.0.8: 0000000 62 b2 43 0a
>> sed-4.0.9: 0000000 62 b2 43 0a
>> sed-4.1  : 0000000 62 00 43 0a
...
>> sed-4.2.2: 0000000 62 00 43 0a
>>
>> Now, it does what 4.0.6..4.0.9 did.
>
> Here's a complete patch:

Pushed.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.