GNU bug report logs - #28312
25.2; Arabic script changes when scrolling

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28312 in the body.
You can then email your comments to 28312 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nick Helm <nick <at> tenpoint.co.nz>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 13:03:02 +1200

With bidi enabled, Arabic text changes unexpectedly when scrolling or
moving point with the mouse.

Recipe:
  Emacs -Q
  C-h h ;visit HELLO file
  ;use the mouse to scroll slowly

Observe the text on the line that starts with the word Arabic. As the
buffer scrolls, some of the characters in the Arabic representations of
the words "Arabic" and "Hello" change between at least two glyphs.

Here's a quick screen cap to show what I mean:
https://ln.sync.com/dl/22d37f320/fwujc85a-dhb63yfw-cv5vhzj5-5qsis2h8
(this is the mac-port, but it occurs with a standard build as well.)

The problem goes away when I change the file local variable:
  ;;; bidi-display-reordering: nil 

My system defaults to Geeza Pro to display Arabic script, but the
problem also occurs with other fonts, such as Arial Unicode MS. I don't
see the problem with other RTL scripts in the HELLO file, such as
Hebrew.

In case it matters, when I place point over one of the affected
characters and do C-u C-x = before and after it changes, I get the
following in both cases (ie Emacs thinks it is displaying the same
char).

-----------------
             position: 884 of 3322 (27%), column: 41
            character: ي‎ (displayed as ي‎) (codepoint 1610, #o3112, #x64a)
    preferred charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.)
code point in charset: 0x2E2A
               script: arabic
               syntax: w 	which means: word
             category: .:Base, R:Right-to-left (strong), b:Arabic
             to input: type "C-x 8 RET 64a" or "C-x 8 RET ARABIC LETTER YEH"
          buffer code: #xD9 #x8A
            file code: ESC #x24 #x2C #x31 #x2E #x2A (encoded by coding system iso-2022-7bit-unix)
              display: composed to form "ي" (see below)

Composed using this font:
  mac-ct:-*-Geeza Pro-normal-normal-normal-*-10-*-*-*-p-0-iso10646-1
by these glyphs:
  [2 2 0 364 3 -1 5 4 3 nil]

Character code properties: customize what to show
  name: ARABIC LETTER YEH
  old-name: ARABIC LETTER YA
  general-category: Lo (Letter, Other)
  decomposition: (1610) ('ي')

There are text properties here:
  charset              mule-unicode-0100-24ff
-----------------




In GNU Emacs 25.2.1 (x86_64-apple-darwin16.7.0, Carbon Version 157 AppKit 1504.83)
 of 2017-08-28 built on oberon
Windowing system distributor 'Apple Inc.', version 10.12.6
Configured using:
 'configure --with-mac
 --enable-locallisppath=/usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/site-lisp
 --infodir=/usr/local/kemp-emacs/emacs-25.2-mac-6.6/info/emacs
 --prefix=/usr/local/kemp-emacs/emacs-25.2-mac-6.6
 --enable-mac-app=/usr/local/kemp-emacs/emacs-25.2-mac-6.6'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS

Important settings:
  value of $LANG: en_NZ.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  savehist-mode: t
  global-eldoc-mode: t
  mac-mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  visual-line-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Emacs start-up time: 0.3 seconds [2 times]

Load-path shadows:
/Users/nick/.emacs.d/lisp/org/ox hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox
/Users/nick/.emacs.d/lisp/org/ox-texinfo hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-texinfo
/Users/nick/.emacs.d/lisp/org/ox-publish hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-publish
/Users/nick/.emacs.d/lisp/org/ox-org hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-org
/Users/nick/.emacs.d/lisp/org/ox-odt hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-odt
/Users/nick/.emacs.d/lisp/org/ox-md hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-md
/Users/nick/.emacs.d/lisp/org/ox-man hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-man
/Users/nick/.emacs.d/lisp/org/ox-latex hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-latex
/Users/nick/.emacs.d/lisp/org/ox-icalendar hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-icalendar
/Users/nick/.emacs.d/lisp/org/ox-html hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-html
/Users/nick/.emacs.d/lisp/org/ox-beamer hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-beamer
/Users/nick/.emacs.d/lisp/org/ox-ascii hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ox-ascii
/Users/nick/.emacs.d/lisp/org/org hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org
/Users/nick/.emacs.d/lisp/org/org-w3m hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-w3m
/Users/nick/.emacs.d/lisp/org/org-version hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-version
/Users/nick/.emacs.d/lisp/org/org-timer hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-timer
/Users/nick/.emacs.d/lisp/org/org-table hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-table
/Users/nick/.emacs.d/lisp/org/org-src hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-src
/Users/nick/.emacs.d/lisp/org/org-rmail hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-rmail
/Users/nick/.emacs.d/lisp/org/org-protocol hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-protocol
/Users/nick/.emacs.d/lisp/org/org-plot hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-plot
/Users/nick/.emacs.d/lisp/org/org-pcomplete hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-pcomplete
/Users/nick/.emacs.d/lisp/org/org-mouse hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-mouse
/Users/nick/.emacs.d/lisp/org/org-mobile hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-mobile
/Users/nick/.emacs.d/lisp/org/org-mhe hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-mhe
/Users/nick/.emacs.d/lisp/org/org-macs hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-macs
/Users/nick/.emacs.d/lisp/org/org-macro hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-macro
/Users/nick/.emacs.d/lisp/org/org-loaddefs hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-loaddefs
/Users/nick/.emacs.d/lisp/org/org-list hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-list
/Users/nick/.emacs.d/lisp/org/org-irc hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-irc
/Users/nick/.emacs.d/lisp/org/org-install hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-install
/Users/nick/.emacs.d/lisp/org/org-inlinetask hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-inlinetask
/Users/nick/.emacs.d/lisp/org/org-info hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-info
/Users/nick/.emacs.d/lisp/org/org-indent hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-indent
/Users/nick/.emacs.d/lisp/org/org-id hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-id
/Users/nick/.emacs.d/lisp/org/org-habit hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-habit
/Users/nick/.emacs.d/lisp/org/org-gnus hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-gnus
/Users/nick/.emacs.d/lisp/org/org-footnote hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-footnote
/Users/nick/.emacs.d/lisp/org/org-feed hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-feed
/Users/nick/.emacs.d/lisp/org/org-faces hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-faces
/Users/nick/.emacs.d/lisp/org/org-eshell hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-eshell
/Users/nick/.emacs.d/lisp/org/org-entities hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-entities
/Users/nick/.emacs.d/lisp/org/org-element hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-element
/Users/nick/.emacs.d/lisp/org/org-docview hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-docview
/Users/nick/.emacs.d/lisp/org/org-datetree hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-datetree
/Users/nick/.emacs.d/lisp/org/org-ctags hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-ctags
/Users/nick/.emacs.d/lisp/org/org-crypt hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-crypt
/Users/nick/.emacs.d/lisp/org/org-compat hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-compat
/Users/nick/.emacs.d/lisp/org/org-colview hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-colview
/Users/nick/.emacs.d/lisp/org/org-clock hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-clock
/Users/nick/.emacs.d/lisp/org/org-capture hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-capture
/Users/nick/.emacs.d/lisp/org/org-bibtex hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-bibtex
/Users/nick/.emacs.d/lisp/org/org-bbdb hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-bbdb
/Users/nick/.emacs.d/lisp/org/org-attach hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-attach
/Users/nick/.emacs.d/lisp/org/org-archive hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-archive
/Users/nick/.emacs.d/lisp/org/org-agenda hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/org-agenda
/Users/nick/.emacs.d/lisp/org/ob hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob
/Users/nick/.emacs.d/lisp/org/ob-tangle hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-tangle
/Users/nick/.emacs.d/lisp/org/ob-table hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-table
/Users/nick/.emacs.d/lisp/org/ob-sqlite hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-sqlite
/Users/nick/.emacs.d/lisp/org/ob-sql hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-sql
/Users/nick/.emacs.d/lisp/org/ob-shen hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-shen
/Users/nick/.emacs.d/lisp/org/ob-screen hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-screen
/Users/nick/.emacs.d/lisp/org/ob-scheme hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-scheme
/Users/nick/.emacs.d/lisp/org/ob-scala hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-scala
/Users/nick/.emacs.d/lisp/org/ob-sass hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-sass
/Users/nick/.emacs.d/lisp/org/ob-ruby hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-ruby
/Users/nick/.emacs.d/lisp/org/ob-ref hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-ref
/Users/nick/.emacs.d/lisp/org/ob-R hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-R
/Users/nick/.emacs.d/lisp/org/ob-python hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-python
/Users/nick/.emacs.d/lisp/org/ob-plantuml hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-plantuml
/Users/nick/.emacs.d/lisp/org/ob-picolisp hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-picolisp
/Users/nick/.emacs.d/lisp/org/ob-perl hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-perl
/Users/nick/.emacs.d/lisp/org/ob-org hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-org
/Users/nick/.emacs.d/lisp/org/ob-octave hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-octave
/Users/nick/.emacs.d/lisp/org/ob-ocaml hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-ocaml
/Users/nick/.emacs.d/lisp/org/ob-mscgen hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-mscgen
/Users/nick/.emacs.d/lisp/org/ob-maxima hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-maxima
/Users/nick/.emacs.d/lisp/org/ob-matlab hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-matlab
/Users/nick/.emacs.d/lisp/org/ob-makefile hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-makefile
/Users/nick/.emacs.d/lisp/org/ob-lob hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-lob
/Users/nick/.emacs.d/lisp/org/ob-lisp hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-lisp
/Users/nick/.emacs.d/lisp/org/ob-lilypond hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-lilypond
/Users/nick/.emacs.d/lisp/org/ob-ledger hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-ledger
/Users/nick/.emacs.d/lisp/org/ob-latex hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-latex
/Users/nick/.emacs.d/lisp/org/ob-keys hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-keys
/Users/nick/.emacs.d/lisp/org/ob-js hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-js
/Users/nick/.emacs.d/lisp/org/ob-java hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-java
/Users/nick/.emacs.d/lisp/org/ob-io hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-io
/Users/nick/.emacs.d/lisp/org/ob-haskell hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-haskell
/Users/nick/.emacs.d/lisp/org/ob-gnuplot hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-gnuplot
/Users/nick/.emacs.d/lisp/org/ob-fortran hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-fortran
/Users/nick/.emacs.d/lisp/org/ob-exp hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-exp
/Users/nick/.emacs.d/lisp/org/ob-eval hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-eval
/Users/nick/.emacs.d/lisp/org/ob-emacs-lisp hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-emacs-lisp
/Users/nick/.emacs.d/lisp/org/ob-dot hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-dot
/Users/nick/.emacs.d/lisp/org/ob-ditaa hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-ditaa
/Users/nick/.emacs.d/lisp/org/ob-css hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-css
/Users/nick/.emacs.d/lisp/org/ob-core hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-core
/Users/nick/.emacs.d/lisp/org/ob-comint hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-comint
/Users/nick/.emacs.d/lisp/org/ob-clojure hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-clojure
/Users/nick/.emacs.d/lisp/org/ob-calc hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-calc
/Users/nick/.emacs.d/lisp/org/ob-C hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-C
/Users/nick/.emacs.d/lisp/org/ob-awk hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-awk
/Users/nick/.emacs.d/lisp/org/ob-asymptote hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/org/ob-asymptote
/Users/nick/.emacs.d/lisp/gnus/auth-source hides /usr/local/kemp-emacs/emacs-25.2-mac-6.6/share/emacs/25.2/lisp/gnus/auth-source

Features:
(shadow sort mail-extr gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime
smime dig mailcap gnus-sum gnus-group gnus-undo gnus-start gnus-cloud
nnimap nnmail mail-source tls gnutls utf7 netrc nnoo parse-time
gnus-spec gnus-int gnus-range gnus-win gnus gnus-ems nnheader wid-edit
emacsbug message format-spec rfc822 mml mml-sec password-cache epg
epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns help-mode mail-prsvr mail-utils jka-compr info time
dired-x easymenu dired savehist multiple-cursors
mc-hide-unmatched-lines-mode mc-separate-operations
rectangular-region-mode mc-mark-pop mc-mark-more thingatpt
mc-cycle-cursors mc-edit-lines multiple-cursors-core advice rect
nh-macdict easy-mmode iso-transl gv flyspell ispell linum plain-theme
edmacro kmacro cl-loaddefs pcase cl-lib mule-util time-date tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
mac-win term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
kqueue mac multi-tty make-network-process emacs)

Memory information:
((conses 16 158045 2804)
 (symbols 48 27577 0)
 (miscs 40 66 98)
 (strings 32 43155 8088)
 (string-bytes 1 1485587)
 (vectors 16 19115)
 (vector-slots 8 522339 1270)
 (floats 8 330 3)
 (intervals 56 225 0)
 (buffers 976 17))

Message #8 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Nick Helm <nick <at> tenpoint.co.nz>
Cc: 28312 <at> debbugs.gnu.org
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 09:45:08 +0300

> From: Nick Helm <nick <at> tenpoint.co.nz>
> Date: Fri, 01 Sep 2017 13:03:02 +1200
> 
> With bidi enabled, Arabic text changes unexpectedly when scrolling or
> moving point with the mouse.
> 
> Recipe:
>   Emacs -Q
>   C-h h ;visit HELLO file
>   ;use the mouse to scroll slowly
> 
> Observe the text on the line that starts with the word Arabic. As the
> buffer scrolls, some of the characters in the Arabic representations of
> the words "Arabic" and "Hello" change between at least two glyphs.

I don't see this here, but I'm not on Darwin.  Can anyone reproduce
this on GNU/Linux?

Does the character also change its shape if you move cursor across
that line, especially when the cursor moves across the character?

> Here's a quick screen cap to show what I mean:
> https://ln.sync.com/dl/22d37f320/fwujc85a-dhb63yfw-cv5vhzj5-5qsis2h8

I cannot watch this movie with the software I have installed.

> The problem goes away when I change the file local variable:
>   ;;; bidi-display-reordering: nil 

Not much sense in doing that for displaying Arabic!

> My system defaults to Geeza Pro to display Arabic script, but the
> problem also occurs with other fonts, such as Arial Unicode MS. I don't
> see the problem with other RTL scripts in the HELLO file, such as
> Hebrew.

Arabic needs shaping support from the underlying font back-end, so
(unless this will be reproduced on GNU/Linux) it sounds like something
is wrong with either the back-end Emacs uses on Darwin, or its version
you have installed, or with the support code in Emacs which is needed
for displaying complex RTL scripts such as Arabic.

FWIW, I tried Arial Unicode here, and saw no problem with that,
either.

Message #11 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 28312 <at> debbugs.gnu.org, Nick Helm <nick <at> tenpoint.co.nz>
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 16:36:13 +0900

>>>>> On Fri, 01 Sep 2017 09:45:08 +0300, Eli Zaretskii <eliz <at> gnu.org> said:

>> With bidi enabled, Arabic text changes unexpectedly when scrolling
>> or moving point with the mouse.
>> 
>> Recipe: Emacs -Q C-h h ;visit HELLO file ;use the mouse to scroll
>> slowly
>> 
>> Observe the text on the line that starts with the word Arabic. As
>> the buffer scrolls, some of the characters in the Arabic
>> representations of the words "Arabic" and "Hello" change between at
>> least two glyphs.

> I don't see this here, but I'm not on Darwin.  Can anyone reproduce
> this on GNU/Linux?

I could reproduce a similar effect also on Darwin, but a different
font backend driver (xft with libotf 0.9.3, m17n lib/db 1.7.0).

1. emacs -Q &
2. M-x describe-language-environment RET Arabic RET
3. Press and release the mouse button around the Arabic sample text.
   (say, "Arabic" to the left)

The appearance of the sample text changes if I press the button, and
it changes back again when I release it.

The font used for displaying Arabic characters is "Arial Unicode MS".

				     YAMAMOTO Mitsuharu
				mituharu <at> math.s.chiba-u.ac.jp

Message #14 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Alexis <flexibeast <at> gmail.com>
To: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 28312 <at> debbugs.gnu.org,
 Nick Helm <nick <at> tenpoint.co.nz>
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 18:01:22 +1000

YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp> writes:

> I could reproduce a similar effect also on Darwin, but a 
> different
> font backend driver (xft with libotf 0.9.3, m17n lib/db 1.7.0).
>
> 1. emacs -Q &
> 2. M-x describe-language-environment RET Arabic RET
> 3. Press and release the mouse button around the Arabic sample 
> text.
>    (say, "Arabic" to the left)
>
> The appearance of the sample text changes if I press the button, 
> and
> it changes back again when I release it.

Confirmed on locally compiled Emacs 25.2.1 on Debian Stretch 
x86_64;
xft with libotf 0.9.13-3+b1 and m17n 1.7.0-3+b1.

> The font used for displaying Arabic characters is "Arial Unicode 
> MS".

Here it's "DejaVu Sans Mono".


Alexis.

Message #17 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 28312 <at> debbugs.gnu.org,
 Nick Helm <nick <at> tenpoint.co.nz>
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 10:16:06 +0200

On Sep 01 2017, YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp> wrote:

> I could reproduce a similar effect also on Darwin, but a different
> font backend driver (xft with libotf 0.9.3, m17n lib/db 1.7.0).
>
> 1. emacs -Q &
> 2. M-x describe-language-environment RET Arabic RET
> 3. Press and release the mouse button around the Arabic sample text.
>    (say, "Arabic" to the left)
>
> The appearance of the sample text changes if I press the button, and
> it changes back again when I release it.

I see such an effect if I start a (highlighted) region anywhere before
the arabic sample text.  The appearance changes while changing the
extend of the region.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #20 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Nick Helm <nick <at> tenpoint.co.nz>
To: <28312 <at> debbugs.gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 20:16:55 +1200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Observe the text on the line that starts with the word Arabic. As the
>> buffer scrolls, some of the characters in the Arabic representations of
>> the words "Arabic" and "Hello" change between at least two glyphs.
>
> Does the character also change its shape if you move cursor across
> that line, especially when the cursor moves across the character?

If I use the keyboard to move the cursor, no. The cursor changes shape
to reflect the width of the glyph, and the direction of cursor movement
correctly changes when over RTL text, as expected, but the chars
themselves do not change.

If I use the mouse to place the cursor in the line, yes. Glyphs near the
cursor change shape and stay changed until I release <mouse-1>.

>> Here's a quick screen cap to show what I mean:
>> https://ln.sync.com/dl/22d37f320/fwujc85a-dhb63yfw-cv5vhzj5-5qsis2h8
>
> I cannot watch this movie with the software I have installed.

Gah, sorry. Try this:
https://ln.sync.com/dl/c0466f180/ju7j9ka3-ebq4uzfz-2wp4ehnu-3szvp5jd

> Arabic needs shaping support from the underlying font back-end, so
> (unless this will be reproduced on GNU/Linux) it sounds like something
> is wrong with either the back-end Emacs uses on Darwin, or its version
> you have installed, or with the support code in Emacs which is needed
> for displaying complex RTL scripts such as Arabic.

When I open HELLO using LibreOffice, which also supports RTL on mac, the
problem doesn't seem to occur using Geeza Pro or Arial Unicode MS. Not
sure if that is relevant.

Message #23 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>,
 Kenichi Handa <handa <at> gnu.org>
Cc: 28312 <at> debbugs.gnu.org, nick <at> tenpoint.co.nz
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Fri, 01 Sep 2017 11:35:38 +0300

> Date: Fri, 01 Sep 2017 16:36:13 +0900
> From: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
> Cc: Nick Helm <nick <at> tenpoint.co.nz>,
> 	28312 <at> debbugs.gnu.org
> 
> 1. emacs -Q &
> 2. M-x describe-language-environment RET Arabic RET
> 3. Press and release the mouse button around the Arabic sample text.
>    (say, "Arabic" to the left)
> 
> The appearance of the sample text changes if I press the button, and
> it changes back again when I release it.

Thanks, I see this, too.  And it's a very old problem, I see it in
Emacs 24.3 already.

Also, the mouse is not necessary, I can see the problem if I scroll
the *Help* window created by "C-h L" either via the scroll bar or even
with "C-u 1 C-v".

I'm guessing that somehow we don't redraw enough characters to give
the font shaper a chance to shape them correctly.  CC'ing Handa-san,
in the hope that he could have some insights on this.

Message #26 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: mituharu <at> math.s.chiba-u.ac.jp, handa <at> gnu.org, nick <at> tenpoint.co.nz
Cc: 28312 <at> debbugs.gnu.org
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Thu, 06 Jun 2019 17:04:13 +0300

tags 28312 patch
thanks

> Date: Fri, 01 Sep 2017 11:35:38 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 28312 <at> debbugs.gnu.org, nick <at> tenpoint.co.nz
> 
> I'm guessing that somehow we don't redraw enough characters to give
> the font shaper a chance to shape them correctly.  CC'ing Handa-san,
> in the hope that he could have some insights on this.

The guess was correct, but finding why this happens and under what
circumstances proved an elusive problem.

I eventually found the culprit: it turned out we sometimes stop
examining buffer text at arbitrary positions, and don't make sure
these positions are never in the middle of a composable character
sequence.  I implemented a solution, the patch is below.
Unfortunately, it makes redisplay slower in buffers with lots of
composable characters, so I hesitate to install it.

Would people please try the patch and see if Emacs is still reasonably
responsive in buffers with Arabic text, especially when marking or
extending the region under transient-mark-mode?  If I get enough
positive feedback, I will install this.  TIA.

diff --git a/src/composite.c b/src/composite.c
index 7d7ed3f..aba47a9 100644
--- a/src/composite.c
+++ b/src/composite.c
@@ -1427,7 +1427,7 @@ struct position_record
    representing the composition, and return true.  Otherwise, *GSTRING to
    Qnil, and return false.  */
 
-static bool
+bool
 find_automatic_composition (ptrdiff_t pos, ptrdiff_t limit,
 			    ptrdiff_t *start, ptrdiff_t *end,
 			    Lisp_Object *gstring, Lisp_Object string)
diff --git a/src/composite.h b/src/composite.h
index 8675163..b4c6a93 100644
--- a/src/composite.h
+++ b/src/composite.h
@@ -324,7 +324,9 @@ extern bool composition_reseat_it (struct composition_it *, ptrdiff_t,
 				   struct face *, Lisp_Object);
 extern int composition_update_it (struct composition_it *,
                                   ptrdiff_t, ptrdiff_t, Lisp_Object);
-
+extern bool find_automatic_composition (ptrdiff_t, ptrdiff_t,
+					ptrdiff_t *, ptrdiff_t *,
+					Lisp_Object *, Lisp_Object);
 extern ptrdiff_t composition_adjust_point (ptrdiff_t, ptrdiff_t);
 
 INLINE_HEADER_END
diff --git a/src/xdisp.c b/src/xdisp.c
index 5f43815..e4fb48b 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -3618,7 +3618,25 @@ compute_stop_pos (struct it *it)
       /* Set up variables for computing the stop position from text
          property changes.  */
       XSETBUFFER (object, current_buffer);
-      limit = make_fixnum (IT_CHARPOS (*it) + TEXT_PROP_DISTANCE_LIMIT);
+      /* Find the position to limit the search for properties.  It
+	 must not be in the middle of a composable sequence, because
+	 having a stop_charpos there risks breaking the composition
+	 into two or more separate parts, to be submitted separately
+	 to the shaping engine, and that will produce incorrect
+	 shaping with some scripts (e.g., Arabic).  */
+      pos = charpos + TEXT_PROP_DISTANCE_LIMIT;
+      if (!NILP (Vauto_composition_mode))
+	{
+	  ptrdiff_t cpos = charpos, posmax = min (pos, ZV);
+	  ptrdiff_t start = posmax, end = posmax;
+	  Lisp_Object ignored;
+	  while (find_automatic_composition (cpos, posmax, &start, &end,
+					     &ignored, Qnil)
+		 && end < posmax)
+	    cpos = end;
+	  pos = max (end, posmax);
+	}
+      limit = make_fixnum (pos);
     }
 
   /* Get the interval containing IT's position.  Value is a null

Message #31 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Nick Helm <nick <at> tenpoint.co.nz>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: "handa <at> gnu.org" <handa <at> gnu.org>,
 "28312 <at> debbugs.gnu.org" <28312 <at> debbugs.gnu.org>,
 "mituharu <at> math.s.chiba-u.ac.jp" <mituharu <at> math.s.chiba-u.ac.jp>
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Sun, 9 Jun 2019 23:56:22 +0000

Eli Zaretskii <eliz <at> gnu.org> writes:

> I eventually found the culprit: it turned out we sometimes stop
> examining buffer text at arbitrary positions, and don't make sure
> these positions are never in the middle of a composable character
> sequence.  I implemented a solution, the patch is below.
> Unfortunately, it makes redisplay slower in buffers with lots of
> composable characters, so I hesitate to install it.
>
> Would people please try the patch and see if Emacs is still reasonably
> responsive in buffers with Arabic text, especially when marking or
> extending the region under transient-mark-mode?  If I get enough
> positive feedback, I will install this.  TIA.

I built yesterday's master and tested with and without the patch.

With the patch applied, I can confirm that the Arabic text in HELLO no
longer changes when scrolling, either with the keyboard or mouse.
However, I noticed that it still seems to change when extending the
region over Arabic text. That is, place point on the "c" of Arabic,
C-SPC, and C-f multiple times to extend the region and notice the
graphemes change. The mouse click and release method described up-thread
also shows the same thing. 

Next, I opened a file written entirely in Arabic (~3000 words). Without
the patch, scrolling works as expected in all cases I tried. With the
patch applied, scrolling also works as expected unless pixel-scroll-mode
is active and scrolling is done with the mouse. With that mode active,
mouse scrolling introduces lag that takes 3–4 seconds to recover after
each scroll action. I would describe the lag as significant. I did not
notice the text changing when scrolling or clicking in this file.

I didn't notice any problems with setting the mark or extending the
region in any tests, other than in the HELLO file described above. 

Thank you for working on this.

Nick

Message #34 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Nick Helm <nick <at> tenpoint.co.nz>
Cc: handa <at> gnu.org, 28312 <at> debbugs.gnu.org, mituharu <at> math.s.chiba-u.ac.jp
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Mon, 10 Jun 2019 20:27:02 +0300

> From: Nick Helm <nick <at> tenpoint.co.nz>
> CC: "mituharu <at> math.s.chiba-u.ac.jp" <mituharu <at> math.s.chiba-u.ac.jp>,
> 	"handa <at> gnu.org" <handa <at> gnu.org>, "28312 <at> debbugs.gnu.org"
> 	<28312 <at> debbugs.gnu.org>
> Date: Sun, 9 Jun 2019 23:56:22 +0000
> 
> > Would people please try the patch and see if Emacs is still reasonably
> > responsive in buffers with Arabic text, especially when marking or
> > extending the region under transient-mark-mode?  If I get enough
> > positive feedback, I will install this.  TIA.
> 
> I built yesterday's master and tested with and without the patch.
> 
> With the patch applied, I can confirm that the Arabic text in HELLO no
> longer changes when scrolling, either with the keyboard or mouse.

Thanks for testing.

> However, I noticed that it still seems to change when extending the
> region over Arabic text. That is, place point on the "c" of Arabic,
> C-SPC, and C-f multiple times to extend the region and notice the
> graphemes change. The mouse click and release method described up-thread
> also shows the same thing. 

This is a separate problem, one whose solution would basically need
redesigning how we handle faces in redisplay.  The problem is that we
currently process buffer text for display in chunks that have the same
face, so if the region starts or ends in the middle of an Arabic word,
we submit this word to the shaping engine in 2 chunks, thus breaking
character composition.  This is very basic in the display engine's
design, so don't hold your breath on that one.

> Next, I opened a file written entirely in Arabic (~3000 words). Without
> the patch, scrolling works as expected in all cases I tried. With the
> patch applied, scrolling also works as expected unless pixel-scroll-mode
> is active and scrolling is done with the mouse. With that mode active,
> mouse scrolling introduces lag that takes 3–4 seconds to recover after
> each scroll action. I would describe the lag as significant. I did not
> notice the text changing when scrolling or clicking in this file.

I'm working on a less expensive solution, stay tuned.

Message #37 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: nick <at> tenpoint.co.nz
Cc: 28312 <at> debbugs.gnu.org
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Tue, 11 Jun 2019 17:49:15 +0300

> Date: Mon, 10 Jun 2019 20:27:02 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 28312 <at> debbugs.gnu.org
> 
> > Next, I opened a file written entirely in Arabic (~3000 words). Without
> > the patch, scrolling works as expected in all cases I tried. With the
> > patch applied, scrolling also works as expected unless pixel-scroll-mode
> > is active and scrolling is done with the mouse. With that mode active,
> > mouse scrolling introduces lag that takes 3–4 seconds to recover after
> > each scroll action. I would describe the lag as significant. I did not
> > notice the text changing when scrolling or clicking in this file.
> 
> I'm working on a less expensive solution, stay tuned.

I installed a solution that should be much less expensive, please take
a look.  If you still see lag in scrolling, try enabling
garbage-collection-messages to see whether GC is the culprit; if it
is, then raise gc-cons-threshold while testing, to exclude this
factor's influence.

Thanks.

Message #40 received at 28312 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 28312 <at> debbugs.gnu.org, nick <at> tenpoint.co.nz
Subject: Re: bug#28312: 25.2; Arabic script changes when scrolling
Date: Mon, 10 Aug 2020 16:13:30 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> I installed a solution that should be much less expensive, please take
> a look.  If you still see lag in scrolling, try enabling
> garbage-collection-messages to see whether GC is the culprit; if it
> is, then raise gc-cons-threshold while testing, to exclude this
> factor's influence.

It looks like this bug was fixed by this change, so I'm closing this bug
report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.