GNU bug report logs - #18991
[PATCH] tests: fix encoding with `tr' to support multibyte in test

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18991 in the body.
You can then email your comments to 18991 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: bug-grep <at> gnu.org
Subject: [PATCH] tests: fix encoding with `tr' to support multibyte in test
Date: Sat, 08 Nov 2014 17:07:40 +0900

[Message part 1 (text/plain, inline)]

It seems that `tr' in GNU coreutils does not recoginize multibyte
character, but other imprementation, e.g. HP-UX, Solaris, recoginizes it.

As a result, [ echo AB | LC_ALL=ja_JP.eucJP tr AB '\244\263' ] is
transformed as [ echo AB | LC_ALL=ja_JP.eucJP tr A '\244\263' ], so that
'\244\263' is recognized as a single multibyte character.  We do not
expect that.

[0001-grep-fix-encoding-with-tr-to-support-multibyte-in-te.patch (text/plain, attachment)]

Message #10 received at 18991-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Cc: 18991-done <at> debbugs.gnu.org
Subject: Re: bug#18991: [PATCH] tests: fix encoding with `tr' to support
 multibyte in test
Date: Sat, 8 Nov 2014 19:00:55 -0800

[Message part 1 (text/plain, inline)]

On Sat, Nov 8, 2014 at 12:07 AM, Norihiro Tanaka <noritnk <at> kcn.ne.jp> wrote:
> It seems that `tr' in GNU coreutils does not recoginize multibyte
> character, but other imprementation, e.g. HP-UX, Solaris, recoginizes it.
>
> As a result, [ echo AB | LC_ALL=ja_JP.eucJP tr AB '\244\263' ] is
> transformed as [ echo AB | LC_ALL=ja_JP.eucJP tr A '\244\263' ], so that
> '\244\263' is recognized as a single multibyte character.  We do not
> expect that.

Thank you for the report and patch.
However, it is not maintainable to modify every use of "tr" in
the tests.  Instead, I've addressed this by making all of the
tests use tr through a wrapper that always sets LC_ALL=C:

[0001-tests-avoid-a-multibyte-tr-portability-problem.patch (application/octet-stream, attachment)]

Message #13 received at 18991-done <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: Jim Meyering <jim <at> meyering.net>
Cc: 18991-done <at> debbugs.gnu.org
Subject: Re: bug#18991: [PATCH] tests: fix encoding with `tr' to support
 multibyte in test
Date: Sun, 09 Nov 2014 14:02:21 +0900

On Sat, 8 Nov 2014 19:00:55 -0800
Jim Meyering <jim <at> meyering.net> wrote:
> Thank you for the report and patch.
> However, it is not maintainable to modify every use of "tr" in
> the tests.  Instead, I've addressed this by making all of the
> tests use tr through a wrapper that always sets LC_ALL=C:

Thanks, I also like it.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.