GNU bug report logs - #6934
multibyte: expand miscalculate width of ideographs

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 28 Aug 2010 01:45:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Aug 27 21:45:39 2010
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1OpAUE-0003JX-4X
	for submit <at> debbugs.gnu.org; Fri, 27 Aug 2010 21:45:38 -0400
Received: from mx10.gnu.org ([199.232.76.166])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <zhangweiwu@HIDDEN>) id 1OpAJx-0003Ez-SN
	for submit <at> debbugs.gnu.org; Fri, 27 Aug 2010 21:35:02 -0400
Received: from lists.gnu.org ([199.232.76.165]:40173)
	by monty-python.gnu.org with esmtps
	(TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60)
	(envelope-from <zhangweiwu@HIDDEN>) id 1OpALO-0005N7-UP
	for submit <at> debbugs.gnu.org; Fri, 27 Aug 2010 21:36:31 -0400
Received: from [140.186.70.92] (port=57572 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1OpALJ-0008TJ-Pe
	for bug-coreutils@HIDDEN; Fri, 27 Aug 2010 21:36:30 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD
	autolearn=unavailable version=3.3.1
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <zhangweiwu@HIDDEN>) id 1OpALE-0005TZ-Gv
	for bug-coreutils@HIDDEN; Fri, 27 Aug 2010 21:36:25 -0400
Received: from bossdog.realss.com ([211.157.114.130]:47818)
	by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <zhangweiwu@HIDDEN>) id 1OpALD-0005T3-RM
	for bug-coreutils@HIDDEN; Fri, 27 Aug 2010 21:36:20 -0400
Received: from localhost (localhost [127.0.0.1])
	by bossdog.realss.com (Postfix) with ESMTP id CFC932481A9
	for <bug-coreutils@HIDDEN>; Sat, 28 Aug 2010 09:37:51 +0800 (HKT)
Received: from bossdog.realss.com ([127.0.0.1])
	by localhost (bossdog.realss.com [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id dybDpBBps9xY for <bug-coreutils@HIDDEN>;
	Sat, 28 Aug 2010 09:37:51 +0800 (HKT)
Received: from [192.168.0.2] (unknown [123.116.121.254])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by bossdog.realss.com (Postfix) with ESMTPSA id 9030A24819E
	for <bug-coreutils@HIDDEN>; Sat, 28 Aug 2010 09:37:51 +0800 (HKT)
Message-ID: <4C78680C.2050708@HIDDEN>
Date: Sat, 28 Aug 2010 09:36:12 +0800
From: Zhang Weiwu <zhangweiwu@HIDDEN>
Organization: Real Softservice
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.9) Gecko/20100423 Lightning/1.0b1 Mnenhy/0.8.2
	Thunderbird/3.0.4
MIME-Version: 1.0
To: bug-coreutils@HIDDEN
Subject: "expand" miscalculate width of ideographs
X-Enigmail-Version: 1.0.1
X-Face: >#.%7:EO6),a-)\4'^9bU\`a#~[ub/o)\9Bl;
	$Pk2owT'eBE[8`o-A/Z={ziW!H#rYz%xc=bzEf%p;
	*-V^iPYVn(3Frok2JUOm.T#HC6TS&q=Oi_Mgw:$7zS1?w(LTm#,+"9v>7u,T|GD{^8BmL+n}-Zf*9,Y/%~PiJM*D#m^{9~\r'>@lj3C0O1b~U[JdX9kXQ
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6,
	seldom 2.4 (older, 4)
X-Spam-Score: -4.2 (----)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Fri, 27 Aug 2010 21:45:37 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -5.4 (-----)

reproduce:

$ printf "1This is a test\t3This is test too\t5And more test\n2=E6=B5=8B=E8=
=AF=95=E4=B8=80=E6=98=AF=E8=BF=99=E6=A0=B7\t4=E6=B5=8B=E8=AF=95=E4=BA=8C=E5=
=A6=82=E6=AD=A4\t6=E6=B5=8B=E8=AF=95=E4=B8=89=EF=BC=8C=E5=97=AF=E3=80=82\=
n" | expand -t 12
1This is a test         3This is test too       5And more test
2=E6=B5=8B=E8=AF=95=E4=B8=80=E6=98=AF=E8=BF=99=E6=A0=B7     4=E6=B5=8B=E8=
=AF=95=E4=BA=8C=E5=A6=82=E6=AD=A4        6=E6=B5=8B=E8=AF=95=E4=B8=89=EF=BC=
=8C=E5=97=AF=E3=80=82

expected:
$ printf "1This is a test\t3This is test too\t5And more test\n2=E6=B5=8B=E8=
=AF=95=E4=B8=80=E6=98=AF=E8=BF=99=E6=A0=B7\t4=E6=B5=8B=E8=AF=95=E4=BA=8C=E5=
=A6=82=E6=AD=A4\t6=E6=B5=8B=E8=AF=95=E4=B8=89=EF=BC=8C=E5=97=AF=E3=80=82\=
n" | expand -t 12
1This is a test         3This is test too       5And more test
2=E6=B5=8B=E8=AF=95=E4=B8=80=E6=98=AF=E8=BF=99=E6=A0=B7           4=E6=B5=
=8B=E8=AF=95=E4=BA=8C=E5=A6=82=E6=AD=A4             6=E6=B5=8B=E8=AF=95=E4=
=B8=89=EF=BC=8C=E5=97=AF=E3=80=82

Note that your email software must be able to show ideograph correctly in=
 order to see the problem.=20

Note that other coreutils do not suffer from this problem. Take ls for ex=
ample:

$ touch "1This is a test" "2=E6=B5=8B=E8=AF=95=E4=B8=80=E6=98=AF=E8=BF=99=
=E6=A0=B7" "3This is test too" "4=E6=B5=8B=E8=AF=95=E4=BA=8C=E5=A6=82=E6=AD=
=A4" "5And more test" "6=E6=B5=8B=E8=AF=95=E4=B8=89=EF=BC=8C=E5=97=AF=E3=80=
=82"
2$ ls
1This is a test  3This is test too  5And more test
2=E6=B5=8B=E8=AF=95=E4=B8=80=E6=98=AF=E8=BF=99=E6=A0=B7    4=E6=B5=8B=E8=AF=
=95=E4=BA=8C=E5=A6=82=E6=AD=A4        6=E6=B5=8B=E8=AF=95=E4=B8=89=EF=BC=8C=
=E5=97=AF=E3=80=82

So you see ls(1) output have ideograph width correctly calculated.

So I can guess that simply taking the width calculation routine of ls(1)
and use it in expand would solve the problem.