GNU bug report logs - #20140
24.4; M17n shaper output rejected

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Richard Wordingham <richard.wordingham@HIDDEN>; dated Wed, 18 Mar 2015 22:21:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 5 Apr 2015 19:48:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 05 15:48:34 2015
Received: from localhost ([127.0.0.1]:45687 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YeqX3-0001XM-7I
	for submit <at> debbugs.gnu.org; Sun, 05 Apr 2015 15:48:33 -0400
Received: from know-smtprelay-omc-9.server.virginmedia.net
 ([80.0.253.73]:36228) by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YeqX1-0001X2-Aw
 for 20140 <at> debbugs.gnu.org; Sun, 05 Apr 2015 15:48:32 -0400
Received: from JRWUBU2 ([81.103.224.4]) by know-smtprelay-9-imp with bizsmtp
 id CKoR1q00M06JmVd01KoRl8; Sun, 05 Apr 2015 20:48:25 +0100
X-Originating-IP: [81.103.224.4]
X-Spam: 0
X-Authority: v=2.1 cv=dJgomYpb c=1 sm=1 tr=0 a=pLuj3OkTrmEUIJBpyvkqVg==:117
 a=pLuj3OkTrmEUIJBpyvkqVg==:17 a=NLZqzBF-AAAA:8 a=mDV3o1hIAAAA:8
 a=k8Yw-NZnoM8DthYqnh4A:9 a=CjuIK1q_8ugA:10 a=pdTaYS8AUncam0ZpjhcA:9
 a=oAEvv6KhJA_pUhrM:18 a=HXjIzolwW10A:10
Date: Sun, 5 Apr 2015 20:48:24 +0100
From: Richard Wordingham <richard.wordingham@HIDDEN>
To: handa@HIDDEN (K. Handa)
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
Message-ID: <20150405204824.35d870b1@JRWUBU2>
In-Reply-To: <87mw31887h.fsf@HIDDEN>
References: <20150321175818.1b125eba@JRWUBU2>
	<87mw31887h.fsf@HIDDEN>
X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.10; i686-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="MP_/1FbX5iSymwVXmjko80irIQH"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

--MP_/1FbX5iSymwVXmjko80irIQH
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Wed, 25 Mar 2015 23:25:54 +0900
handa@HIDDEN (K. Handa) wrote:

> Could you try these patches and test the usability of
> forward-char-intrusive and backward-char-intrusive?

The results weren't quite what I'd hoped for, but the results are
usable.  Thank you.

The text I principally tried the commands out on was Tai Tham text <LOW
PA, SIGN II, TONE-1, SAKOT, NA, OA BELOW, TONE-2>.  I used corrections
for the bugs that had been affecting its rendering.

It seems that the commands prevent shaping across the cursor, but do
not inhibit shaping within the former cluster.  I was only doing shaping
on complete orthographic syllables, so entering a cluster chiefly had
the effect of losing all positioning of marks and making the text
unreadable.  However, the behaviour may make a good teaching aid!

I then tried the command on Thai, and there the commands worked well.
I therefore added to LANA-OTF.flt, for marks not in complete syllables,
the command:

    ("(M)" [ (1 = ) ] ) ; For stepping through.

I attach the results of not stepping through (labelled 0), and
stepping through by 1 to 7 characters (labelled 1 to 7).  The result
is not so good at 3 steps - I think because the extra rendering
command does not handle SAKOT.  Aggressive use of dotted circles might
improve the display.  I don't know why, at the end, there is a delay in
TONE-2 rising to its proper height.

C-S-f is not a good key sequence for me.  C-S is one of my X-keyboard
switching combinations - I chose it for compatibility with the Xming X
server.

Richard.
--MP_/1FbX5iSymwVXmjko80irIQH
Content-Type: image/png
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=emacs_entry.png

iVBORw0KGgoAAAANSUhEUgAAAEYAAAIPCAYAAADO5qgJAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A
/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB98EBRIzI6RMMfQAAAAZdEVYdENv
bW1lbnQAQ3JlYXRlZCB3aXRoIEdJTVBXgQ4XAAAV60lEQVR42u2deVhTZ9rG7xNCFiBIRKoibh13
i06t2tEWtGr9bItTbevSRT+tVevU7ap2HMd26tWp4+dgdWqtX/20Wre6dBv3DQR3LcVprQW1rrUi
AgYJS0KAvN8fAUSakJNATt5z8jzXxR8ESODO+z7v8zzc53cEc8FdBorfhIokcB5qCHVrYzAYJP2F
BEEAY4xWDG0lEoaE8SrspmNYMncpTt61A8yC7KvXkVfKADBYb6QCADKsgShMQTrW/WsDvjczoOgo
pjzcB7O3fYFFY3shus2zAICQINpKgO0W1o19Azu0Y7D1cjYAoE0wD8e11CEEQx+qh1oAoG6OwfNW
Y/arY/BYcw1XOUYwm82M6hg/rhjb5c34+4fHkW+vFCCsByaODcORy30weWhLBIOvkCzH2H7dgxUf
fYWzWTnIyb6Ig+u348Kl/dicko1yLlsCKbdJWFe8MPUdjOwpYOVT8+VfxzDzGSwbHQujIEDQtcbA
ObuQ5eHbLKi00BUlY8bTE/F1th0avQ5qtRZ6LZ81pvvkG8aQOrkjBqWOxu7Uf+APeWswMm42Sv6Z
iZTJbdwvOVaK/Fs5KKpw/8sEhTVFi8ZaLpIvzGYzq+uDmZPYqEbBLG7DLVbBGGOshJ2eGs2Cfv8h
u1LG3EdhCnupERylrZsP49jDDADjIdyu4/LcH5FR9AB6tC/H8cRpmLslC9F92gBXT+Oa2NJd3Rgj
dt6AyWSCyWTCjZ0j0Nj4AnbcMFU+dgO7RkX6oaiqR46xW+7CwvRoFHQbqZtWY+3287CFhkCw5MNi
F/8yWkMEjEYjjEYjIgxaqFQaGCKM1Y8ZOMs1bn8blT4CesEKM7rgne8tyN78NAwlJWD6COi9/FuC
QmLQqVMMQjnu7d2uXnVULLqE3caZC2bYe+qhghXXTl0H2o5CG513L6rvtRBHj8l97BDWC6+PiMSJ
9xcjJceGooz1mL/uDvpM/iNaqqHYcC+MEI5+H+zC4m578FxTLQwPvw/LpG3Y8lobKFgXaiK5nMdU
5CRh4VtLkWZmMtxKvhQmPw3rl29FRgl/wqglUgAZaRnIr9UWlF2/iiI7uAxphCk6jlmDhmJfsbOz
u79jmheoydd2aSPeW8sw7t0xaKeh5HtPmJv7sXLlPvx6Jwkjm3fGvB9KgdKbOLLiTxg+LRWFgZx8
WeltnP32LLLMuUhfMxXxrVsjYWUB+g7vDH1A5piag6phyQCA8JRCzFmVib0J7REayDmGCryA6ZVI
GD6Ci3kvrRgShoSRrTDujEMrF6wm45Az49DKa3oyDjmaqPuNQ+mrXibjEBmHZBhkHPJ3jiHjUF37
VlnGITvykubh2d4PIkIQIKgHY1eBF6IozjhkCIXp1Gf4/MID6FC8GAkzNfg69wASGontCj0zDjU3
asDF3MqtcahGFOxNYLrgJ9nOux44cDw0DhUxJg/jUMNkMvkZh9RSpTKHcSgMABBs0EKlsjuMQ2EA
oOHOOOSXN8lhHLJzbRwSUlJS6GJRVyumQ4cOpAQNqkgY3wtjv3MAs4aNxYrzpdXVsClpFoaN+RjV
D9GKobGDFy30FWx6ewGORk1G4uzeMAhykYHBdusYNqzYgAM/ZMOCkAYWhtlw91YuslCCChmtDlby
E1bPTcQ+UwR6PDsWsaqfGlgYbSe8sWEH3pDZtrFk/huHTICu92RMGdUdoXhcpDCu+A+CB3vFfgf7
Zo/HikuOT8OfXITV0zpD11Bf9zrsKMkzwQYA3y7CxJc8yDGq4BBoVXaU2+8VyRVldqg0IRDd4qgi
MWTJDgzx1dfrcf7ojREIBhD8yETMHtYKwaJPJX0MujcrQvqBNPxaUoZSUwaSk3+Bpt1DiKopre0y
Ns4eh0n/cxqFMmo0QrokIK4RUJK+HbtPn8eVKxdEbiV1SwybOw15y1Zj+uhElCMEMY+OxrwpPRBW
czexMhTcNiFbZZFV8hXCHsbrC2fBsGoLknZvQjqrbCKpV6ICj4RpkMr34sWLpETtvEOuTdpKJAwJ
I5uxQwPFO9OnOnVFTHzsOj5OTEOfOTPQJ6LyPWUWZF/LgTq6FZpoAeuNw1i3/hLiZr2GLjqFrRhX
rghbTZtaVfjIrsalMFWuiGVrluDFtiKcMz6wq0nmdvAkXLoiatrUqpOBw652LOsmjq+agSfb6BvE
FMCd28FjTIqr5xd0aBId5TVmgTu3AzzFpLh8/jj2xZ1AdzuoIzF6fy4sFgssFgvups1B23qO9hTi
dhCg1uqg0znU0Hadju3HxyOmEffC1MolPnY7CPpoxPaQYYEnB0yKOGHsZSi1VcBqs4MxO2xWK6y6
IGi0wXwUQsyGO5fO4awxpMbuNaBN57YID/KhMIXJo9Bs8Dcoqfz8+WZ6IGwEkrK2YSAPxvEKM/a+
1gd773swDl/cOYIXGtM8pjpKf3wbXXofwaKb3gsjq+66+Ng0xD2zBBk1HBa2Sxvx9rwNuGSrIa7a
gCZNG0NXjxJYVsKUF/yM/5z9BSU1CCLO0AiaB19B4p+j8enfUpU1dqjVIeLq1x9jc2YJLJnnUZyX
j08XLcB+XRM8Nn4CeuJ+NIJ5zVTEb12L75uOwjtLOytZmHLcOXsYSYcLUJaTC1gtOJ2ShPPa1mj6
wnj08hEaQVbJt2D3ELR4vRNSL/4LPX1MyaDRpiKO6woriktV0IdofP6OygtlF6RDaIhUbS8FCUPC
KFEYwqTQiiFhSJj6BGFSXAlDmBSxUwXCpFQ2Q4RJkXUQJsXfOYYwKXXtW0VhUmxXsWVWAro3D4Ug
CBAiOuGZt3fh1zIPRVEcJiXoAha+tR3Nhj+P/rHNUJr2IV5+LhGqxPM4MaOd+9wQCJgUxhhj9ly2
5XGB6YbuZQWESamxAAp/wqHLwYh9sgNETxkVj0mpuI2dM8diS6v5ODLhQQ9+WMmYlIo8HHxrEMZ9
NwLfpMxB93oMpeWASREnTEUuDsweiNGHnsTWQ4swILJ+f5Ey7q9UcRt7ZvbHyP39sGH33xEXVgar
1QqrzQ4lh1thmCkZC5ZnoCBzORJahkGv1zs+4tbhloK1oSaSx3kMz/dXIkZVXacSXXfN2VYiYZQq
DDGqKLxsIt0FMapcPT8xqpwHMaqqHidGFTGqnA9QiFHl4uwiRhUFFXgkjPeVLzGqnOQdYlTRViJh
SBjZjB3qmkh4YDVbMnepJBwqLlaMJ1YzqThU3Gwlj9lT9ykrrR+YS6uZVByq+tUxElvNnDKqfMWh
qit4s5rBGaPKRxwqZVjNfMCharjj2q9Ws4bnUDWMMJxZzRqCQ1V/YQLQaqYWs332zByAl5IHYNOB
KqtZGaDSQKeRsAzyAYeqXsJUW82QgYSWy+99ofcaZJ0cj+ZSaeMDDpUi5zENwaGSXXctFYdKdsJI
xaHiYuzgpiGr5lABQHHeSUk4VDIQ5h6HCgBgvSYJh0p2yTc0ZoYkHCoabSplxRQVl0rCoZJBjrk/
QkP8f3E6bSUShoRRpjDEqKIVQ8KQMPUJYlS5EoYYVWInDsSoqmyGiFEl6yBGlb9zDDGq6tq3imJU
lV3DhvF/QFuj2mEcCmmJvq/+L9ILPAM7KI9RFVKOs6n/gS2mPVpFhaD06k7MfWY8jr6SjguLe7i/
kEqmjCr3WynIiG4DB1R/Wm5vi2iDGsFi/29dfBJTuzyBz0XcQtE49jBurItHqHySbyl+WtALRp0a
wVFPYHv8Rhx89/fiL7uTIaNKpDBadJyxBxk/Z+LU5jfR6MuZmPPvLA8u1qoyDhlhNBoRYdBCpdI4
jEOVj/HGqBL926jDotC8ZXs8Ovp9/N/rOnzzj2247uU56zAOxSiAUXV/voZKrYLdaobVS+KQIoxD
5Vm7sXh9PvoM7YfYGB3yTq3Cmx9dQauJT6OtBooNt4tZCAJykxZgZK9WiIx4AB1HrAFe/RwH3n8E
euXqQk0kl/MYYlTJdexA111ztpVIGKUKQ4wqivq0BHUEMapcPT8xqlxMJ4hRVfk4MaqIUeV8gEKM
KhdnFzGqKKjAI2G8r3yJUeUk79Bok7YSCUPCKEmY+3y+JEwNYWr6fEkY2koiCoTg3zKoqI6RWeUr
RYj2+foBBujXrSTa58vJhRd8AgI5uPCCP5+vH2CA3iVfiX2+0RFMehigV8lXYp+v6WO7i+/3HXap
HjlGYp+vxDDAeggjtc/3HgxQp9MhvOt0bD++CgMbcSeMf32+DhhgRxglPLLJ5+utMH7x+UoMA/RK
mGqf73tjkGMBYPgdBvra5ysxDFC2TaSvYYDczWOKj03jAgbInTDlBT9zAQP069ihxp9eDQO0ZJ5H
cV6+32GAnAhzDwZYlpMLWC1+hwFyl3wLdg9Bi9c7+R0GyF2O4Tn8e1xXWFFcqvI7DJCTHFOzadIh
NITPFUNbiYQhYUgYEoaEIWFImNpBgEBXwhAgUOwUggCBld0ZAQJlHQQI9HeOIUBgXftWUYDAGsHM
RzG9jQDBMBLJhR6KojhAYFXyZYU4MTser53IRua5OCRlbcNAMXlZsYBAx1+HotPvYUryU1g25xSe
muTBKygaEFj8Hd6ftBPxy/6CR7z5H4YMAYEifpdipC+cgG0PL0Xa4+FQ7fJO//rfWZQzYaw/LsaE
z9ph0XeDEakCChrgRRvizqI+T77EqKpjxdB11zx21ySMAoUhRhWFj5pIYlS5en5iVDkPYlRVPU6M
KmJUOQ1iVLk6u4hRRUEFHgnjfeVLjConeYfHG4HzcD9a2kokDAkjT2GIUeVKGGJU0Vby8DwmRpWs
6xgurWbZV68To8qZ1YwYVS5VlAOjqkZIZTWTB6NKYqtZi8ZamG7+4ndGFcxmM6vrwxF2VnhyNuvW
fS47+PkTTB02giVVfcldFKawlxrBcRWFmw/j2MMMgIvvj2Nf3GGSBZ9WMw4YVZxaze4xqgBA23U6
th8fj5hGHAnDg9XMwajibMUU/XQQP9w8jpHN7//WQeHRmHr6Mj7q7TmUQRGMqiajj4GNvvd5wY6B
aPJyJPaJPZW8CTkwqvwSxKgS10QGBKPKGZeqdgQko8oZl6p2BBCjqm4uVf+m6lpdRMAwqurmUvVv
ev93BySjyh2Xiv4TyXlIf1y74VIF3Mz3XqPEL5eKthIJQ8IoXxgeEi+tGBKGhJGvMMSociUMMarE
TiSIUVXZDBGjStZBjCp/5xhiVNW1bxXFqGJ3sC1OcNzYrvrjISy+VOaZKIpjVIXZsC0+BnPjjuHM
37pCW6lnsFaDIDHDaWUzqgBBrYVOp6sUxoNQNKMKNlxeOgDRhnA80KEfJiw7CZMnlwIoklElhKD7
mxuxJ7ob2hltuLxvMabMGIDndJlImtRG5B+jQEYVoEfH4S+iY+Vn7Tt8gk+/3YPBaw4h57VXEe3F
30OMKrlXvnTdNY/dNQmjQGGIUUXhoyaSGFWunp8YVc6DGFVVjxOjihhVztslYlS5OLuIUUVBBR4J
433lS4wqJ3mH2A60lUgYEkZJwhCjypUwxKiireTheUyMKlnXMfKxmjELsq/lSMauko/VTGJPMJ+M
KpfqSucJlo/VrNITLBW7ijurWYvGWnklX6msZjKrY8hq5mSpkNXMRQSm1Uztze4L1gSBVVTA7mWO
VASjym46jU07TXgo/hE82LgC1w9+gBmbzOi56Ak0DYJiw/2KKc/FsQ8nYOK4HJQCCIrqhmHv7Mby
qe24u2KEmkilz2MqcpKw8K2lSKv13tRmVAWeMPlpWL98KzJK7hemNqMKpTdxZMWfMHxaKgq5yTEN
owAy0jKQX6stKLt+FUV2Z12Ea0aVXqp3TRy2tp5xdycbEuoCV6vvzzZmV1R/KwDWosbXw2NHsQU7
LrIiO5M0KPnymGN4DhKGhCFhSBgShoQhRhWtGBKGhKlfEKPKlTDEqBIZxKiqap+JUSXrIEaVv3MM
Marq2reKYlQBQPltJC94Ht2aqCEIAnTR8Xj3jMUzURTHqDJokLmoLx7+ZxTe3bQU43pGoPjyj7jZ
4gn0ixGRGWTKqHL/X4LiE2xycwMbsvU2q/Bm3O7hze2KGB/hdh2X56TjdH402mf+GY8200OtjUTn
P87HwdseXA4qQ+OQW2HsxXdQbL2AT/c9iCU/mmG9tQf/nbcIz43/Atl28S/jMA4ZYTQaEWHQQqXS
OIxDlY/xZhxy+9sIOgO00KL/X6cjLioY6saPYtJbj8N27CucK/buRR3GoRhiVMm28qXrrmlQRcJI
Igwxqih81EQSo8rV8xOjynkQo6rqcWJUEaPKaRCjytXZRYwqCirwSBjvK19iVDnJO+QMp61EwpAw
ShKGGFWuhCFGFW0lD89jYlTJuo7hy2rGHJ6bvFImCYeKi60kympWdBQAuPD9csmo4sH3y5fVTN0c
ACTjUNUz+UprNeOFUcWd1QwAWc3IakZWM3EhB6uZ29UbHNUV7cOCYGvA40EOjCr375mhFyYO1ePQ
ghU4ZSpHRcEZrFlyHJrHnsdDoVBsuBdGaIz/+ngH5kdvwlMPBEMdEY9Pwmbhy7UvoJmCe3NqInmc
x7hiVAX8oMoVo0oWp1IDKeARoypwhCk6jlmDhmKfs7pH35/LaZ5kydd2aSPeW8sw7t0xaKeh5HtP
mJv7ueDbcZl8a/Lt0tdMRXzr1khYWYC+wyXk23GVY2oOqoYlAwDCUwoxZ1Um9ia0R2gg5xgq8AKm
VyJh+AhiVNGKIWFImPoEMapcCUOMKrFNFDGqKktbYlTJOohR5e8cQ4yquvatkhhVtgsL0VmofXM7
AUKnBTjvwW04lMeoCguBrbQM1cN820V80PdRbH7le5z5S0e4PUsUy6iqFSVpb7JW+r5s1S/lgc2o
uv/dv4ujS9fh7oBZeLaFB+WpIm9uV7Ocz96DxO0qDNsxCFEqz1KZAm9uVxVluLIlEYcjx+Bkn/B6
vagcbm5HjKq6Vgxdd02DKhJGEmGIUUXhoyaSGFWunp8YVc6DGFVVjxOjihhVToMYVa7OLmJUUVCB
R8J4X/kSo8pJ3pHLv2iFyppJKisabSUShoSRpzDEqHIlDDGqaCt5eO4So0rWdQxfVrOKLGT/UgZ1
dKvfMKoUu5VEWc2uvM2N75c/RhUnvl++rGahsRg8b7VMGFUSW82ijRouki93VjNXUfU9ZDXjYVAl
upwnq5mzCCyrmXhhrOewatk5dJiyBbH1vKxeGYyqyig4vhRrbvXGzJfbQQPlh7gVY7+NfYlfwTpo
vWdJV8ZBTSSP8xhiVLkShhhVxKhyHsSoqqNLF8moCrjkS4yqOicQxKhyPagiRhUVeIoMEsbfOabe
iVtidhWtGBKGhJGnMMSociUMMarENlHEqKosYYlRRQWeqB1DjCoXwhCjqo59qyRGFQBYL23GtPiW
0AsCBHUkYkcuxgmTZ1Ns5TGqdNn4oHdnvNdiJU5vG4/fFaXir3GD8FnfQ7iytj/cpmbFMqoKD7EX
w4NY3La8KusQS5sZw9Q9PmGivEOKZVSFxOKlhChkfPYlzpkrUJp1BJv2mNFjdD80FVuhytA49P+D
0DskC7r57gAAAABJRU5ErkJggg==

--MP_/1FbX5iSymwVXmjko80irIQH--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 25 Mar 2015 21:45:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 25 17:45:18 2015
Received: from localhost ([127.0.0.1]:36664 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Yat6z-0002JF-DC
	for submit <at> debbugs.gnu.org; Wed, 25 Mar 2015 17:45:18 -0400
Received: from know-smtprelay-omc-9.server.virginmedia.net
 ([80.0.253.73]:56507) by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <richard.wordingham@HIDDEN>) id 1Yat6x-0002Io-Bl
 for 20140 <at> debbugs.gnu.org; Wed, 25 Mar 2015 17:45:16 -0400
Received: from JRWUBU2 ([81.103.224.4]) by know-smtprelay-9-imp with bizsmtp
 id 7xl91q00n06JmVd01xl9he; Wed, 25 Mar 2015 21:45:09 +0000
X-Originating-IP: [81.103.224.4]
X-Spam: 0
X-Authority: v=2.1 cv=dJgomYpb c=1 sm=1 tr=0 a=pLuj3OkTrmEUIJBpyvkqVg==:117
 a=pLuj3OkTrmEUIJBpyvkqVg==:17 a=kj9zAlcOel0A:10 a=NLZqzBF-AAAA:8
 a=mDV3o1hIAAAA:8 a=LsZXWq4u80ueHZGQUpMA:9 a=CjuIK1q_8ugA:10
Date: Wed, 25 Mar 2015 21:45:07 +0000
From: Richard Wordingham <richard.wordingham@HIDDEN>
To: 20140 <at> debbugs.gnu.org
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
Message-ID: <20150325214507.3dd335de@JRWUBU2>
In-Reply-To: <87mw31887h.fsf@HIDDEN>
References: <20150321175818.1b125eba@JRWUBU2>
	<87mw31887h.fsf@HIDDEN>
X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.10; i686-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 20140
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

On Wed, 25 Mar 2015 23:25:54 +0900
handa@HIDDEN (K. Handa) wrote:

> Hi, thank you for the detailed explanation.
> 
> In article <20150321175818.1b125eba@JRWUBU2>, Richard Wordingham
> <richard.wordingham@HIDDEN> writes:
> 
> > What I ought to want is SIL's split cursor scheme, which indicated
> > the next ('point') and previous characters, even in bidirectional
> > text. Unfortunately, that's not compatible with m17n, which seems
> > to assume that cursor position will be a single number.  

> > The Emacs
> > functions forward-char-intrusive and backward-char-intrusive
> > provided a pleasant, more intuitive, alternative, and I am sad to
> > hear they are gone. Perhaps I'll have to start using
> > toggle-auto-composition.
> 
> Those Emacs functions are just my idea for improving Emacs
> for CTL users, and have never been included in the official
> Emacs verison.

I think I must have confused them with the behaviour of Emacs 22.1
on Windows XP.  I didn't do anything to enable the visual decomposition
of the clusters - it just happened when moving with the arrow keys.
Indeed, it is conceivable that the characters weren't decomposed, but
were simply being rendered by Windows without any need for composition.

I haven't had time to try out the experimental code yet.

Richard.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 25 Mar 2015 14:26:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 25 10:26:00 2015
Received: from localhost ([127.0.0.1]:36486 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YamFr-0008K8-8R
	for submit <at> debbugs.gnu.org; Wed, 25 Mar 2015 10:25:59 -0400
Received: from fencepost.gnu.org ([208.118.235.10]:54581 ident=Debian-exim)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <handa@HIDDEN>) id 1YamFo-0008Jy-1J
 for 20140 <at> debbugs.gnu.org; Wed, 25 Mar 2015 10:25:57 -0400
Received: from fl1-122-134-88-3.iba.mesh.ad.jp ([122.134.88.3]:56196
 helo=tinhau)
 by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <handa@HIDDEN>)
 id 1YamFn-0007fb-8G; Wed, 25 Mar 2015 10:25:55 -0400
Received: from handa by tinhau with local (Exim 4.80)
 (envelope-from <handa@HIDDEN>)
 id 1YamFm-0004U5-Jj; Wed, 25 Mar 2015 23:25:54 +0900
From: handa@HIDDEN (K. Handa)
To: Richard Wordingham <richard.wordingham@HIDDEN>
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
In-Reply-To: <20150321175818.1b125eba@JRWUBU2> (message from Richard
 Wordingham on Sat, 21 Mar 2015 17:58:18 +0000)
Date: Wed, 25 Mar 2015 23:25:54 +0900
Message-ID: <87mw31887h.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Hi, thank you for the detailed explanation.

In article <20150321175818.1b125eba@JRWUBU2>, Richard Wordingham <richard.w=
ordingham@HIDDEN> writes:

> What I ought to want is SIL's split cursor scheme, which indicated the
> next ('point') and previous characters, even in bidirectional text.
> Unfortunately, that's not compatible with m17n, which seems to assume
> that cursor position will be a single number.  The Emacs functions
> forward-char-intrusive and backward-char-intrusive provided a pleasant,
> more intuitive, alternative, and I am sad to hear they are gone.
> Perhaps I'll have to start using toggle-auto-composition.

Those Emacs functions are just my idea for improving Emacs
for CTL users, and have never been included in the official
Emacs verison.  I check the code and found two problems:

(1) When the command sets disable-point-adjustment to t,
command_loop_1 should force updating the display if point is
within a grapheme cluster.  So we need this patch:

diff --git a/src/keyboard.c b/src/keyboard.c
index bf65df1..13125c1 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -1636,6 +1636,16 @@ command_loop_1 (void)
 	    adjust_point_for_property (last_point_position,
 				       MODIFF !=3D prev_modiff);
 	}
+      else if (current_buffer =3D=3D prev_buffer
+	       && last_point_position !=3D PT)
+	{
+	  if (PT > BEGV && PT < ZV
+	      && (composition_adjust_point (last_point_position, PT) !=3D PT))
+	    /* Now point is within a grapheme cluster.  We must update
+	       the display so that this cluster is discomosed on the
+	       screen and the cursor is correctly placed at point.  */
+	    windows_or_buffers_changed =3D 22;
+	}
=20
       /* Install chars successfully executed in kbd macro.  */
=20
(2) We should break a grapheme cluster at point.  So we need
this patch.

diff --git a/src/xdisp.c b/src/xdisp.c
index a17f5a9..0c56395 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -3408,6 +3408,9 @@ compute_stop_pos (struct it *it)
       pos =3D next_overlay_change (charpos);
       if (pos < it->stop_charpos)
 	it->stop_charpos =3D pos;
+      /* If point is in front of the current stop pos, stop there.  */
+      if (charpos < PT && PT < it->stop_charpos)
+	it->stop_charpos =3D PT;
=20
       /* Set up variables for computing the stop position from text
          property changes.  */
@@ -8166,7 +8169,12 @@ next_element_from_buffer (struct it *it)
 	  && IT_CHARPOS (*it) >=3D it->redisplay_end_trigger_charpos)
 	run_redisplay_end_trigger_hook (it);
=20
-      stop =3D it->bidi_it.scan_dir < 0 ? -1 : it->end_charpos;
+      /* Set stop position considering the bidi direction and point.  */
+      if (it->bidi_it.scan_dir < 0)
+	stop =3D (PT < IT_CHARPOS (*it)) ? PT : -1;
+      else
+	stop =3D ((IT_CHARPOS (*it) < PT && PT < it->end_charpos)
+		? PT : it->end_charpos);
       if (CHAR_COMPOSED_P (it, IT_CHARPOS (*it), IT_BYTEPOS (*it),
 			   stop)
 	  && next_element_from_composition (it))

Could you try these patches and test the usability of
forward-char-intrusive and backward-char-intrusive?

> > Please try to move cursor over this Devanagri text "=E0=A4=B9=E0=A4=BF=
=E0=A4=82=E0=A4=A6=E0=A5=80" on
> > Emacs, gedit, and, for instance, firefox.  They all treat
> > that text as 2 grapheme clusters "=E0=A4=B9=E0=A4=BF=E0=A4=82" and "=E0=
=A4=A6=E0=A5=80".  The first
> > one corresponds to character the sequence U+935 U+93F, and
> > U+93F (vowel I) is displayed before U+935 (base cosonant).

> Note that those clusters are only 3 and 2 characters long.  Retyping
> them is tolerable.  Now consider the Sanskrit Devanagari text =E0=A4=B8=
=E0=A5=8D=E0=A4=A4=E0=A5=8D=E0=A4=B0=E0=A5=80,
> which contains two consonant-combining viramas.  Emacs moves across it
> in 1 step, but Claws e-mail (GTK-based, I believe) and LibreOffice
> (HarfBuzz-based, at least for linux) both take 3 steps to move across
> it.  Claws and LibreOffice use different algorithms to position the
> cursor.  That of LibreOffice seems more reasonable, but that of
> Claws works better!  The reason is that Unicode did not declare virama
> as forming grapheme clusters.

Ah, hmmm, that a problem of DEVA-OTF.flt and DEV2-OTF.flt of
the m17n library.  I'll try to fix them.

> It seems to have solved all of them.  When I reported the bug, I was
> having problems with my font because libotf was silently ignoring half
> the lookups in my font.

Could you please send me (not on this list) an appropriate
bug/problem report if libotf should be fixed?

> I though I might have problems with U+1A58 TAI THAM SIGN MAI KANG LAI,
> which in Lao visually groups (usually) with the following base
> consonant and in Tai Khuen groups with the preceding base consonant. My
> clustering in Emacs follows the Tai Khuen scheme.  (I compose two
> orthographic clusters together in Emacs, but declare two grapheme
> clusters in the FLT processing.)  However, my font follows a major
> Northern Thai dictionary and places it on the following base consonant
> if there is nothing above it, but otherwise places it on the preceding
> base consonant.  However, my implementation is too dirty to cause
> problems - the second cluster is not reported as deriving from the
> mai kang lai character.

> I wonder, though, what will happen if I manage to implement the
> Universal Shaping Engine's (USE) rphf feature. The author of a Lao-style
> Tai Tham font wanted this feature in HarfBuzz.  The desired effect seems
> easy to achieve in m17n-flt, but placing it under font control is more
> difficult.  I'm studying MLM2-OTF.flt to see how to do it.

I've just started to study the Universal Shaping Engine.  It
seems that we can implement it by a proper FLT file.

> > > However, it then makes editing of the 'clusters' more
> > > difficult.  Note that there are examples above with 5
> > > characters in a cluster, and this is by no means the
> > > limit.
> >=20
> > But, it seems that the current behavior is accepted, at
> > least, by Indic people.

> Who do you mean by 'Indic people'?

I just mean that I have not heard any complaints about that
"too long cluster problem" of Emacs.  No one is using Emacs
for Indic scripts?

> New Tai Lue is an interesting case.  Microsoft delayed support for this
> simple Indic script for so long that most apparently Unicode-encoded
> New Tai Lue text was actually encoded in visual order.  With Unicode
> 8.0, New Tai Lue is changing from phonetic order to visual order, and
> it will no longer need any clusters at all!=20=20

Wow, I didn't know that.

> Emacs 23.3 (which is what is in long-term support Ubuntu
> 12.04) offers no support for New Tai Lue, so I am not sure
> that there is yet a New Tai Lue view on composition in
> Emacs.

We may be able to provide supports for new scripts in elpa.

---
K. Handa
handa@HIDDEN




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 21 Mar 2015 18:26:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 21 14:26:41 2015
Received: from localhost ([127.0.0.1]:60024 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YZO6b-00051d-Io
	for submit <at> debbugs.gnu.org; Sat, 21 Mar 2015 14:26:41 -0400
Received: from mtaout20.012.net.il ([80.179.55.166]:60788)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <eliz@HIDDEN>) id 1YZO6X-00051N-Ua
 for 20140 <at> debbugs.gnu.org; Sat, 21 Mar 2015 14:26:39 -0400
Received: from conversion-daemon.a-mtaout20.012.net.il by
 a-mtaout20.012.net.il (HyperSendmail v2007.08) id
 <0NLK00900QZOHE00@HIDDEN> for 20140 <at> debbugs.gnu.org;
 Sat, 21 Mar 2015 20:26:31 +0200 (IST)
Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il
 (HyperSendmail v2007.08) with ESMTPA id
 <0NLK009R9R861Q70@HIDDEN>;
 Sat, 21 Mar 2015 20:26:31 +0200 (IST)
Date: Sat, 21 Mar 2015 20:26:20 +0200
From: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
In-reply-to: <20150321175818.1b125eba@JRWUBU2>
X-012-Sender: halo1@HIDDEN
To: Richard Wordingham <richard.wordingham@HIDDEN>
Message-id: <83384ytdf7.fsf@HIDDEN>
References: <20150318222040.4066e6e9@JRWUBU2> <87pp8292cy.fsf@HIDDEN>
 <20150321175818.1b125eba@JRWUBU2>
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org, handa@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: Eli Zaretskii <eliz@HIDDEN>
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

> Date: Sat, 21 Mar 2015 17:58:18 +0000
> From: Richard Wordingham <richard.wordingham@HIDDEN>
> Cc: 20140 <at> debbugs.gnu.org
> 
> Another implementation could choose to move in visual order.

Emacs 24.4 does have visual-order cursor movement.  Customize the
variable visual-order-cursor-movement to get that.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 21 Mar 2015 17:58:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 21 13:58:34 2015
Received: from localhost ([127.0.0.1]:60020 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YZNfN-0004KQ-Kb
	for submit <at> debbugs.gnu.org; Sat, 21 Mar 2015 13:58:34 -0400
Received: from know-smtprelay-omc-9.server.virginmedia.net
 ([80.0.253.73]:44830) by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YZNfK-0004K9-Hq
 for 20140 <at> debbugs.gnu.org; Sat, 21 Mar 2015 13:58:31 -0400
Received: from JRWUBU2 ([81.103.224.4]) by know-smtprelay-9-imp with bizsmtp
 id 6HyP1q02R06JmVd01HyQjP; Sat, 21 Mar 2015 17:58:24 +0000
X-Originating-IP: [81.103.224.4]
X-Spam: 0
X-Authority: v=2.1 cv=dJgomYpb c=1 sm=1 tr=0 a=pLuj3OkTrmEUIJBpyvkqVg==:117
 a=pLuj3OkTrmEUIJBpyvkqVg==:17 a=IkcTkHD0fZMA:10 a=NLZqzBF-AAAA:8
 a=mDV3o1hIAAAA:8 a=ct7dnu-KoUftn3s5otsA:9 a=UsawN17YW7ie1F18:21
 a=WAKGHCdlqEPa-ZRm:21 a=QEXdDO2ut3YA:10
Date: Sat, 21 Mar 2015 17:58:18 +0000
From: Richard Wordingham <richard.wordingham@HIDDEN>
To: handa@HIDDEN (K. Handa)
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
Message-ID: <20150321175818.1b125eba@JRWUBU2>
In-Reply-To: <87pp8292cy.fsf@HIDDEN>
References: <20150318222040.4066e6e9@JRWUBU2>
	<87pp8292cy.fsf@HIDDEN>
X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.10; i686-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

On Sat, 21 Mar 2015 17:33:17 +0900
handa@HIDDEN (K. Handa) wrote:

> In article <20150318222040.4066e6e9@JRWUBU2>, Richard Wordingham
> <richard.wordingham@HIDDEN> writes: [...]
> > I extract and analyse what was rendered as shaped ('accepted') and
> > what was not ('rejected'), quoting the monitoring output.  I
> > suspect the problem is the strict testing of the from and to fields
> > in Lisp function font-shape-gstring, which is defined in file
> > font.c.
> [...]
> > The shaping of the following, with vowels or MEDIAL RA that should
> > be rendered before the consonant, was rejected:
>=20
> > mflt_run( 1A3E 1A6E 1A6C 1A65) produced ( 1A6E>872:1:1 1A3E>810:0:3
> 1A6C>869:0:3 1A65>862:0:3)=20
>=20
> If U+1A6E is displayed before U+1A3E, and they are in
> different grapheme cluster, when you move point forward one
> step by one, the cursor must move back and forth as below
> (cursor is indicated by dashes):
>=20
>  display: SPC 1A6E 1A3E+1A6C+1A65 SPC
>  step 1:  ---   =20
>  step 2:           --------------
>  step 3:      ----
>  step 4:                          ---
>=20
> Is that what you want?

It gives me more control for editing in Emacs.  Another implementation
could choose to move in visual order. The policing function could
choose to merge the 'out of order' clusters
- that is what new HarfBuzz does, though I think that should only be
done if the client requests it.

What I ought to want is SIL's split cursor scheme, which indicated the
next ('point') and previous characters, even in bidirectional text.
Unfortunately, that's not compatible with m17n, which seems to assume
that cursor position will be a single number.  The Emacs functions
forward-char-intrusive and backward-char-intrusive provided a pleasant,
more intuitive, alternative, and I am sad to hear they are gone.
Perhaps I'll have to start using toggle-auto-composition.

The one consolation in Emacs is that delete-forward-char
deletes a single character, rather than a whole cluster.  That
greatly reduces the disadvantage of having clusters.  Also,
search still works by characters rather than by clusters.  If I want to
search for a character in LibreOffice, I have to go into the
special regular expression find and replace menu.  That is unpleasant.

> At least, the support for all Indic scripts (they have
> characters in logical order as your example of Tai Tham
> text) treats re-ordered glyphs as one grapheme cluster.
> That is not only Emacs but also gtk (pango) applications.

That's a nasty fault with HarfBuzz.

> Please try to move cursor over this Devanagri text "=E0=A4=B9=E0=A4=BF=E0=
=A4=82=E0=A4=A6=E0=A5=80" on
> Emacs, gedit, and, for instance, firefox.  They all treat
> that text as 2 grapheme clusters "=E0=A4=B9=E0=A4=BF=E0=A4=82" and "=E0=
=A4=A6=E0=A5=80".  The first
> one corresponds to character the sequence U+935 U+93F, and
> U+93F (vowel I) is displayed before U+935 (base cosonant).

Note that those clusters are only 3 and 2 characters long.  Retyping
them is tolerable.  Now consider the Sanskrit Devanagari text =E0=A4=B8=E0=
=A5=8D=E0=A4=A4=E0=A5=8D=E0=A4=B0=E0=A5=80,
which contains two consonant-combining viramas.  Emacs moves across it
in 1 step, but Claws e-mail (GTK-based, I believe) and LibreOffice
(HarfBuzz-based, at least for linux) both take 3 steps to move across
it.  Claws and LibreOffice use different algorithms to position the
cursor.  That of LibreOffice seems more reasonable, but that of
Claws works better!  The reason is that Unicode did not declare virama
as forming grapheme clusters.

> [...]
>=20
> > There does appear to be a work around, which is to have m17n declare
> > the orthographic syllables it receives to be 'grapheme clusters'.
>=20
> I think that's the right solution; i.e. make all combined
> and out-of-ordered glyphs as one cluster.
>=20
> > It solves at least some of the problems above.
>=20
> Which one is not solved by it?

It seems to have solved all of them.  When I reported the bug, I was
having problems with my font because libotf was silently ignoring half
the lookups in my font.

I though I might have problems with U+1A58 TAI THAM SIGN MAI KANG LAI,
which in Lao visually groups (usually) with the following base
consonant and in Tai Khuen groups with the preceding base consonant. My
clustering in Emacs follows the Tai Khuen scheme.  (I compose two
orthographic clusters together in Emacs, but declare two grapheme
clusters in the FLT processing.)  However, my font follows a major
Northern Thai dictionary and places it on the following base consonant
if there is nothing above it, but otherwise places it on the preceding
base consonant.  However, my implementation is too dirty to cause
problems - the second cluster is not reported as deriving from the
mai kang lai character.

I wonder, though, what will happen if I manage to implement the
Universal Shaping Engine's (USE) rphf feature. The author of a Lao-style
Tai Tham font wanted this feature in HarfBuzz.  The desired effect seems
easy to achieve in m17n-flt, but placing it under font control is more
difficult.  I'm studying MLM2-OTF.flt to see how to do it.

> > However, it then makes editing of the 'clusters' more
> > difficult.  Note that there are examples above with 5
> > characters in a cluster, and this is by no means the
> > limit.
>=20
> But, it seems that the current behavior is accepted, at
> least, by Indic people.

Who do you mean by 'Indic people'?  I can see at least three groups:

1) Indian speakers of Indic languages who use Indic scripts, thus
including users of Hindi, Gujarati and Bengali.

See my comments above.

2) Indian users of Indic scripts, thus also including speakers of
Malayalam and Tamil.

In Tamil, a phonetically CVCCV word will normally naturally split into
clusters as CV.C+virama.CV.  I must admit I am surprised that they have
accepted CV.CCV - or do Tamils not use Emacs for Tamil?

Tamils are notorious for regarding their writing system as a syllabary
rather than as an abugida.

I haven't studied the Malayalam script - that does seem a fairly
complicated Indian script, as one might expect when Dravidians use a
script tailored to Middle Indic and stretched to cover Old Indic.

3) Users of Indic scripts, thus also including the Burmese, Thai,
Cambodians and Lao as well as the users of the Tai Tham script.

Rebellion is rampant.  The original Unicode encoding of Thai
followed the phonetic order (allegedly - it was probably the
collation order instead).  This was rapidly thrown out as
incompatible with the current, working encoding.  Unicode responded
with the derogatory property of 'logical order exception'.

Around Unicode 5.1, the preposed vowels of Thai and Lao were suddenly
included in grapheme clusters with the base consonant. As the
consequences started to appear in applications, there were howls of
rage from Thais, and the characters were restored to their original
status as fully independent characters.

It doesn't seem so long ago that the Cambodian government imposed
Unicode on Cambodia.  You'd have thought that access to applications
would have made Unicode the obvious choice.

New Tai Lue is an interesting case.  Microsoft delayed support for this
simple Indic script for so long that most apparently Unicode-encoded
New Tai Lue text was actually encoded in visual order.  With Unicode
8.0, New Tai Lue is changing from phonetic order to visual order, and
it will no longer need any clusters at all!  Emacs 23.3 (which is what
is in long-term support Ubuntu 12.04) offers no support for New Tai
Lue, so I am not sure that there is yet a New Tai Lue view on
composition in Emacs.

Richard.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 21 Mar 2015 17:25:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 21 13:25:14 2015
Received: from localhost ([127.0.0.1]:60006 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YZN98-0003Yt-Ii
	for submit <at> debbugs.gnu.org; Sat, 21 Mar 2015 13:25:14 -0400
Received: from b2bfep15.mx.upcmail.net ([62.179.121.60]:40950)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <xdsl0000474440@HIDDEN>) id 1YZN95-0003Ye-Tl
 for 20140 <at> debbugs.gnu.org; Sat, 21 Mar 2015 13:25:12 -0400
Received: from edge12.upcmail.net ([192.168.13.82]) by b2bfep15.mx.upcmail.net
 (InterMail vM.8.01.05.05 201-2260-151-110-20120111) with ESMTP
 id <20150321172505.GDMY14231.b2bfep15-int.chello.at@HIDDEN>
 for <20140 <at> debbugs.gnu.org>; Sat, 21 Mar 2015 18:25:05 +0100
Received: from iznogoud.viz ([91.119.226.187]) by edge12.upcmail.net with edge
 id 6HR41q00D43DRrP0CHR46Q; Sat, 21 Mar 2015 18:25:05 +0100
X-SourceIP: 91.119.226.187
Received: from wolfgang by iznogoud.viz with local (Exim 4.85 (FreeBSD))
 (envelope-from <wolfgang@HIDDEN>)
 id 1YZN8x-0000Q4-Sn; Sat, 21 Mar 2015 18:25:03 +0100
From: Wolfgang Jenkner <wjenkner@HIDDEN>
To: handa@HIDDEN (K. Handa)
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
Date: Sat, 21 Mar 2015 18:20:26 +0100
References: <20150318222040.4066e6e9@JRWUBU2> <87pp8292cy.fsf@HIDDEN>
Message-ID: <85zj76qn4g.fsf@HIDDEN>
User-Agent: Gnus/5.130012 (Ma Gnus v0.12) Emacs/25.0.50 (berkeley-unix)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org, Richard Wordingham <richard.wordingham@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

On Sat, Mar 21 2015, K. Handa wrote:

> By the way, I long ago proposed these commands which enables
> you to move point into a grapheme cluster (by suppressing
> composing of a cluster temporarily).  It worked in old Emacs (I
> don't remember the version), but not in the latest Emacs.
>
> (defun forward-char-intrusive ()
>   (interactive)
>   (setq disable-point-adjustment t)
>   (forward-char 1))

It actually works in trunk emacs, I think.  If we start with point at
the beginning of the word (I use the itrans transcription for clarity)

-!-hiMdI

then calling the function once /appears/ to leave point here

hiM-!-dI

but C-x = shows that it is really here

h-!-iMdI

And so on.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 21 Mar 2015 08:33:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 21 04:33:34 2015
Received: from localhost ([127.0.0.1]:59559 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YZEqc-0005l2-8l
	for submit <at> debbugs.gnu.org; Sat, 21 Mar 2015 04:33:34 -0400
Received: from fencepost.gnu.org ([208.118.235.10]:55573 ident=Debian-exim)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <handa@HIDDEN>) id 1YZEqZ-0005kt-Kt
 for 20140 <at> debbugs.gnu.org; Sat, 21 Mar 2015 04:33:32 -0400
Received: from fl1-122-134-88-3.iba.mesh.ad.jp ([122.134.88.3]:59731
 helo=tinhau)
 by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <handa@HIDDEN>) id 1YZEqY-0005Gj-Tx
 for 20140 <at> debbugs.gnu.org; Sat, 21 Mar 2015 04:33:31 -0400
Received: from handa by tinhau with local (Exim 4.80)
 (envelope-from <handa@HIDDEN>)
 id 1YZEqL-0007d2-Ea; Sat, 21 Mar 2015 17:33:17 +0900
From: handa@HIDDEN (K. Handa)
To: Richard Wordingham <richard.wordingham@HIDDEN>
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
In-Reply-To: <20150318222040.4066e6e9@JRWUBU2> (message from Richard
 Wordingham on Wed, 18 Mar 2015 22:20:40 +0000)
Date: Sat, 21 Mar 2015 17:33:17 +0900
Message-ID: <87pp8292cy.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

In article <20150318222040.4066e6e9@JRWUBU2>, Richard Wordingham <richard.w=
ordingham@HIDDEN> writes:
[...]
> I extract and analyse what was rendered as shaped ('accepted') and what
> was not ('rejected'), quoting the monitoring output.  I suspect the
> problem is the strict testing of the from and to fields in Lisp function
> font-shape-gstring, which is defined in file font.c.
[...]
> The shaping of the following, with vowels or MEDIAL RA that should be
> rendered before the consonant, was rejected:

> mflt_run( 1A3E 1A6E 1A6C 1A65) produced ( 1A6E>872:1:1 1A3E>810:0:3
1A6C>869:0:3 1A65>862:0:3)=20

If U+1A6E is displayed before U+1A3E, and they are in
different grapheme cluster, when you move point forward one
step by one, the cursor must move back and forth as below
(cursor is indicated by dashes):

 display: SPC 1A6E 1A3E+1A6C+1A65 SPC
 step 1:  ---=20=20=20=20
 step 2:           --------------
 step 3:      ----
 step 4:                          ---

Is that what you want?

At least, the support for all Indic scripts (they have
characters in logical order as your example of Tai Tham
text) treats re-ordered glyphs as one grapheme cluster.
That is not only Emacs but also gtk (pango) applications.
Please try to move cursor over this Devanagri text "=E0=A4=B9=E0=A4=BF=E0=
=A4=82=E0=A4=A6=E0=A5=80" on
Emacs, gedit, and, for instance, firefox.  They all treat
that text as 2 grapheme clusters "=E0=A4=B9=E0=A4=BF=E0=A4=82" and "=E0=A4=
=A6=E0=A5=80".  The first
one corresponds to character the sequence U+935 U+93F, and
U+93F (vowel I) is displayed before U+935 (base cosonant).

[...]

> There does appear to be a work around, which is to have m17n declare
> the orthographic syllables it receives to be 'grapheme clusters'.

I think that's the right solution; i.e. make all combined
and out-of-ordered glyphs as one cluster.

> It solves at least some of the problems above.

Which one is not solved by it?

> However, it then makes editing of the 'clusters' more
> difficult.  Note that there are examples above with 5
> characters in a cluster, and this is by no means the
> limit.

But, it seems that the current behavior is accepted, at
least, by Indic people.

By the way, I long ago proposed these commands which enables
you to move point into a grapheme cluster (by suppressing
composing of a cluster temporarily).  It worked in old Emacs (I
don't remember the version), but not in the latest Emacs.

(defun forward-char-intrusive ()
  (interactive)
  (setq disable-point-adjustment t)
  (forward-char 1))

(defun backward-char-intrusive ()
  (interactive)
  (setq disable-point-adjustment t)
  (forward-char -1))

(global-set-key (kbd "C-S-f") 'forward-char-intrusive)
(global-set-key (kbd "C-S-b") 'backward-char-intrusive)

---
K. Handa
handa@HIDDEN




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at 20140 <at> debbugs.gnu.org:


Received: (at 20140) by debbugs.gnu.org; 19 Mar 2015 03:43:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 18 23:43:45 2015
Received: from localhost ([127.0.0.1]:52160 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YYRN3-00061I-D3
	for submit <at> debbugs.gnu.org; Wed, 18 Mar 2015 23:43:45 -0400
Received: from mtaout22.012.net.il ([80.179.55.172]:42630)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <eliz@HIDDEN>) id 1YYRN0-000619-OS
 for 20140 <at> debbugs.gnu.org; Wed, 18 Mar 2015 23:43:44 -0400
Received: from conversion-daemon.a-mtaout22.012.net.il by
 a-mtaout22.012.net.il (HyperSendmail v2007.08) id
 <0NLF00J00W6LZ500@HIDDEN> for 20140 <at> debbugs.gnu.org;
 Thu, 19 Mar 2015 05:43:40 +0200 (IST)
Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il
 (HyperSendmail v2007.08) with ESMTPA id
 <0NLF00J3YX0SZ430@HIDDEN>;
 Thu, 19 Mar 2015 05:43:40 +0200 (IST)
Date: Thu, 19 Mar 2015 05:43:34 +0200
From: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#20140: 24.4; M17n shaper output rejected
In-reply-to: <20150318222040.4066e6e9@JRWUBU2>
X-012-Sender: halo1@HIDDEN
To: Richard Wordingham <richard.wordingham@HIDDEN>,
 Kenichi Handa <handa@HIDDEN>
Message-id: <83oanpwt21.fsf@HIDDEN>
References: <20150318222040.4066e6e9@JRWUBU2>
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 20140
Cc: 20140 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: Eli Zaretskii <eliz@HIDDEN>
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

> Date: Wed, 18 Mar 2015 22:20:40 +0000
> From: Richard Wordingham <richard.wordingham@HIDDEN>
> 
> I am running Emacs 24.4 in a Ubuntu 12.04 Precise Pangolin
> installation, for which the version of libm17n-0 is 1.6.3-1.  I am
> attempting to induce Emacs to render the Tai Tham script.  There
> appears to be a bug/feature in Emacs which makes this unnecessarily
> difficult.

Thanks for the report.  I hope Handa-san (CC'ed) could look into it.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 18 Mar 2015 22:20:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 18 18:20:55 2015
Received: from localhost ([127.0.0.1]:52055 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1YYMKc-0006e5-TD
	for submit <at> debbugs.gnu.org; Wed, 18 Mar 2015 18:20:55 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41301)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YYMKa-0006dw-5d
 for submit <at> debbugs.gnu.org; Wed, 18 Mar 2015 18:20:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YYMKY-00046U-FW
 for submit <at> debbugs.gnu.org; Wed, 18 Mar 2015 18:20:51 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:50987)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YYMKY-00046Q-Cx
 for submit <at> debbugs.gnu.org; Wed, 18 Mar 2015 18:20:50 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:43022)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YYMKW-00070s-Ok
 for bug-gnu-emacs@HIDDEN; Wed, 18 Mar 2015 18:20:50 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YYMKT-000467-BB
 for bug-gnu-emacs@HIDDEN; Wed, 18 Mar 2015 18:20:48 -0400
Received: from know-smtprelay-omc-10.server.virginmedia.net
 ([80.0.253.74]:46876) by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <richard.wordingham@HIDDEN>) id 1YYMKS-00045w-TU
 for bug-gnu-emacs@HIDDEN; Wed, 18 Mar 2015 18:20:45 -0400
Received: from JRWUBU2 ([81.103.224.4]) by know-smtprelay-10-imp with bizsmtp
 id 5ALj1q02L06JmVd01ALjNr; Wed, 18 Mar 2015 22:20:43 +0000
X-Originating-IP: [81.103.224.4]
X-Spam: 0
X-Authority: v=2.1 cv=dY0O3Bne c=1 sm=1 tr=0 a=pLuj3OkTrmEUIJBpyvkqVg==:117
 a=pLuj3OkTrmEUIJBpyvkqVg==:17 a=IkcTkHD0fZMA:10 a=NLZqzBF-AAAA:8
 a=mDV3o1hIAAAA:8 a=ibnQV_NrJ8uHORisgRgA:9 a=QEXdDO2ut3YA:10
Date: Wed, 18 Mar 2015 22:20:40 +0000
From: Richard Wordingham <richard.wordingham@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 24.4; M17n shaper output rejected
Message-ID: <20150318222040.4066e6e9@JRWUBU2>
X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.10; i686-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.3 (----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.3 (----)

I am running Emacs 24.4 in a Ubuntu 12.04 Precise Pangolin
installation, for which the version of libm17n-0 is 1.6.3-1.  I am
attempting to induce Emacs to render the Tai Tham script.  There
appears to be a bug/feature in Emacs which makes this unnecessarily
difficult.

To achieve Tai Tham rendering, I added the following in new, loaded file
tai-tham.el:

(defvar tai-tham-composable-pattern
  (let ((table
	 ;; C is letters, independent vowels, digits, punctuation and
symbols. '(("C" .
"[\u1A20-\u1A54\u1A80-\u1A89\u1A90-\u1A99\u1AA0-\u1AAD]") ("M" .
"[\u1A55-\u1A5E\u1A61-\u1A7C\u1A7F]"); Mark ("S" . "[\u1A75-\u1A7C]") ;
Marks commuting with sakot ("H" . "\u1A60") ; sakot
	   ("N" . "\u1A58"))) ; mai kang lai - also included in M.
;; Which orthographic syllable mai kang lai belongs to can depend on
the font! (regexp "C\\(M\\|HS*C?\\)*\\(NC\\(M\\|HS*C?\\)*\\)*N?"))
    (let ((case-fold-search nil))
      (dolist (elt table)
	(setq regexp (replace-regexp-in-string (car elt) (cdr elt)
					       regexp t t))))
    regexp))

(let ((elt (list (vector tai-tham-composable-pattern 0
'font-shape-gstring) (vector "." 0 'font-shape-gstring)
		 )))
  (set-char-table-range composition-function-table '(#x1A20 . #x1AAD)
  elt))

I added the following (cut-down) file LANA-OFT.flt to the m17n database:

(font layouter lana-otf nil
      (font (nil nil unicode-bmp :otf=3Dlana)))
(category
 ;; H: SAKOT
 ;; N: Other character with non-zero canonical combining class
 ;; Z: Character with ccc=3D0 or other with ccc=3D9=20
 (0x0000 0x1A5F ?Z)bug-gnu-emacs@HIDDEN
 (0x1A60        ?H)
 (0x1A61 0x1A74 ?Z)
 (0x1A75 0x1A7C ?N)
 (0x1A7D 0xFFFF ?Z)
)

(generator
  (0
    (cond
      ("(H)(N+)" (2 =3D *) (1 =3D))
      ("." =3D)
    ) *
  )
)

(category
 ;; C: Consonant and non-mark (lenient processing)
 ;; H: SAKOT
 ;; P: Preposed vowelbug-gnu-emacs@HIDDEN
 ;; R: Medial RA (preposed dependent consonant)
 ;; M: Mark
 (0x1A20 0x1A54 ?C)
 (0x1A55 0x1A55 ?R)
 (0x1A56 0x1A5E ?M)
 (0x1A5F        ?C) ; Unassigned
 (0x1A60        ?H)
 (0x1A61 0x1A6D ?M)
 (0x1A6E 0x1A72 ?P)
 (0x1A73 0x1A7C ?M)
 (0x1A7D 0x1A7E ?C) ; Unassigned
 (0x1A7F        ?M)
 (0x1A80 0x1A89 ?C)
 (0x1A8A 0x1A8F ?C) ; Unassigned
 (0x1A90 0x1A99 ?C)
 (0x1A9A 0x1A9F ?C) ; Unassigned
 (0x1AA0 0x1AAC ?C) ; Punctuation
 (0x1AAD        ?C) ; Can take a vowel!
 (0x1AAE 0x1AAF ?C) ; Unassigned
)

(generator
  (0
    (cond
      ("(C)(R|P)" (2 =3D) (1 =3D) )
      ("." =3D)
    )*
  )
)

(generator (0 otf:lana))

However, much Tai Tham text failed to render properly.  To determine
what was wrong, I added some monitoring code to ftfont.c:

*** ftfont.c.orig	2014-03-21 05:34:40.000000000 +0000
--- ftfont.c	2015-03-18 19:47:30.032718995 +0000
***************
*** 2516,2522 ****
--- 2516,2553 ----
      flt =3D mflt_get (msymbol ("combining"));
    for (i =3D 0; i < 3; i++)
      {
+       int k;
+       fprintf(stdout, "mflt_run(");
+       if (gstring.glyphs[0].encoded) {
+ 	for (k =3D 0; k < len; k++) {
+ 	  fprintf(stdout, " %d", gstring.glyphs[k].code);
+ 	}
+       } else {
+ 	for (k =3D 0; k < len; k++) {
+ 	  fprintf(stdout, " %4.4X", gstring.glyphs[k].c);
+ 	}
+       }
        int result =3D mflt_run (&gstring, 0, len, &flt_font_ft.flt_font,
flt);
+       if (-1 =3D=3D result) {
+ 	fprintf(stdout, ") failed.\n");
+       } else if (result >=3D 0) {
+ 	fprintf(stdout, ") produced (");
+ 	for (k =3D 0; k < result; k++) {
+ #if 0
+ 	  fprintf(stdout, " %d", gstring.glyphs[k].code);
+ #else
+ 	  fprintf(stdout, " %4.4X>%d:%d:%d",
+ 		  gstring.glyphs[k].c, gstring.glyphs[k].code,
+ 		  gstring.glyphs[k].from, gstring.glyphs[k].to);
+ #endif
+ 	}
+ 	fprintf(stdout, ")\n");
+ 	if (result !=3D gstring.used) {
+ 	  fprintf(stdout, "Anomalously, gstring.used =3D %d\n",
+ 		  (int) gstring.used);
+ 	}
+ 	fflush(0);
+       }
        if (result !=3D -2)
  	break;
        if (INT_MAX / 2 < gstring.allocated)

The sample Tai Tham text was:
;; =E1=A9=88=E1=A9=A3=E1=A9=B4=E1=A9=81=E1=A9=A2=E1=A9=A0=E1=A8=B7=E1=A8=BD=
=E1=A9=A3=E1=A9=88=E1=A9=A3=E1=A9=83=E1=A9=B6=E1=A9=A3=E1=A9=A0=E1=A8=B6=E1=
=A8=B6=E1=A9=A3 / =E1=A8=A3=E1=A9=A3=E1=A9=B4=E1=A8=BE=E1=A9=AE=E1=A9=AC=E1=
=A9=A5=E1=A8=A6 - =E1=A9=88=E1=A9=A2=E1=A8=AC=E1=A9=A0=E1=A8=AC=E1=A9=A3 =
=E1=A8=A0=E1=A9=A0=E1=A9=B5=E1=A8=B7 =E1=A9=83=E1=A9=A0=E1=A9=B6=E1=A8=AF =
=E1=A8=AE=E1=A9=A0 =E1=A8=B3=E1=A9=AB=E1=A9=A0=E1=A9=B5=E1=A8=B6
=E1=A8=A0=E1=A9=A2=E1=A9=A0=E1=A9=B5=E1=A8=B7=E1=A8=A0=E1=A9=AB=E1=A9=B6=E1=
=A9=A0=E1=A8=AF=E1=A8=BF=E1=A9=A5=E1=A9=A0=E1=A8=B7=E1=A8=B6=E1=A9=A6=E1=A9=
=B5=E1=A9=A0=E1=A8=B7
;; =E1=A8=A3=E1=A9=95   =E1=A8=B2=E1=A9=B1

I extract and analyse what was rendered as shaped ('accepted') and what
was not ('rejected'), quoting the monitoring output.  I suspect the
problem is the strict testing of the from and to fields in Lisp function
font-shape-gstring, which is defined in file font.c.

The shaping of the following was accepted:
mflt_run( 1A48 1A63 1A74) produced ( 1A48>820:0:0 1A63>858:1:1 1A74>878:2:2)

mflt_run( 1A41 1A62 1A60 1A37) produced ( 1A41>813:0:1 1A62>853:0:1
0000>953:2:3)

mflt_run( 1A3D 1A63) produced ( 1A3D>808:0:0 1A63>858:1:1)

mflt_run( 1A48 1A63) produced ( 1A48>820:0:0 1A63>858:1:1)

mflt_run( 1A43 1A76 1A63 1A60 1A36) produced ( 1A43>815:0:1
1A76>890:0:1 1A63>858:2:4 0000>952:2:4)=20

mflt_run( 1A36 1A63) produced ( 1A36>800:0:0 1A63>858:1:1)

mflt_run( 1A23 1A63 1A74) produced ( 1A23>777:0:0 0000>859:1:2)

mflt_run( 1A26) produced ( 1A26>780:0:0)

mflt_run( 1A48 1A62) produced ( 1A48>820:0:1 1A62>853:0:1)

mflt_run( 1A2C 1A60 1A2C 1A63) produced ( 0000>789:0:2 1A63>858:3:3)

mflt_run( 1A43 1A60 1A76 1A2F) produced ( 1A43>815:0:3 1A76>890:0:3
0000>941:0:3)=20

mflt_run( 1A2E 1A60) produced ( 1A2E>792:0:1 1A60>851:0:1)

mflt_run( 1A33 1A6B 1A60 1A75 1A36) produced ( 1A33>797:0:4
1A6B>868:0:4 1A75>889:0:4 0000>952:0:4)=20

mflt_run( 1A20 1A6B 1A76 1A60 1A2F) produced ( 1A20>774:0:4
1A6B>868:0:4 1A76>890:0:4 0000>941:0:4)

mflt_run( 1A3F 1A65 1A60 1A37) produced ( 1A3F>811:0:1 1A65>862:0:1
0000>953:2:3)

The shaping of the following, with vowels or MEDIAL RA that should be
rendered before the consonant, was rejected:

mflt_run( 1A3E 1A6E 1A6C 1A65) produced ( 1A6E>872:1:1 1A3E>810:0:3
1A6C>869:0:3 1A65>862:0:3)=20

mflt_run( 1A23 1A55) produced ( 1A55>835:1:1 1A23>777:0:0)

mflt_run( 1A32 1A71) produced ( 1A71>875:1:1 1A32>796:0:0)

The problem is that the first glyph does not derive from the first
character.

The shaping of the following was rejected:

mflt_run( 1A20 1A60 1A75 1A37) produced ( 1A20>774:0:2 1A75>889:0:2
0000>953:1:3)

In this case, character 2 is stacked below character 0,
and characters 1 and 3 combine to form a spacing glyph.

mflt_run( 1A20 1A62 1A60 1A75 1A37) produced ( 1A20>774:0:1
1A62>853:0:3 1A75>889:0:3 0000>953:2:4)

Character 1 is mounted on character 0, and character 3 on character 1.
Characters 2 and 4 combine to form a spacing glyph. =20

mflt_run( 1A36 1A66 1A75 1A60 1A37) produced ( 1A36>800:0:1
1A66>863:0:2 1A75>889:0:2 0000>953:3:4)

Character 1 is mounted on character 0. and character 2 on character 1.
Characters 3 and 4 form a spacing glyph.

There does appear to be a work around, which is to have m17n declare
the orthographic syllables it receives to be 'grapheme clusters'.  It
solves at least some of the problems above.  However, it then makes
editing of the 'clusters' more difficult.  Note that there are examples
above with 5 characters in a cluster, and this is by no means the limit.

Richard.




Acknowledgement sent to Richard Wordingham <richard.wordingham@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#20140; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.