GNU bug report logs - #36171
25.1; Emacs displays Hebrew text incorrectly

Previous Next

Package: emacs;

Reported by: Raoul Comninos <revrari <at> mweb.co.za>

Date: Tue, 11 Jun 2019 15:36:01 UTC

Severity: normal

Found in version 25.1

Done: Eli Zaretskii <eliz <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36171 in the body.
You can then email your comments to 36171 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Tue, 11 Jun 2019 15:36:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Raoul Comninos <revrari <at> mweb.co.za>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Tue, 11 Jun 2019 15:36:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.1; Emacs displays Hebrew text incorrectly
Date: Tue, 11 Jun 2019 17:11:37 +0200
Emacs does not correctly display Hebrew text with vowels. The vowels
appear next to the consonants rather than beneath them. When they are
beneath the consonants they are not in the correct position. There are a
few other anomalies too, such as, the dagesh appears misplaced followed
by a circle. If I export my documents with Hebrew text in org mode to html
or odf then the Hebrew IS displayed correctly. I have tried changing the
font, installing "unicode-fonts" and a host of other solutions. I have
tried using different versions of Emacs. I use utf-8 as my default
encoding system. If I open the same text document in LibreOffice, again,
the text is displayed correctly. If I strip the Hebrew text of vowels
then the consonants appear correctly in Emacs. If I use
auto-composition-mode it makes no difference. The Hebrew text remains
incorrectly displayed

In GNU Emacs 25.1.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.11)
of 2017-09-15, modified by Debian built on trouble
Windowing system distributor 'The X.Org Foundation', version 11.0.11902000
System Description: Debian GNU/Linux 9.9 (stretch)

Configured using:
'configure --build x86_64-linux-gnu --prefix=/usr
--sharedstatedir=/var/lib --libexecdir=/usr/lib
--localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.1/site-lisp:/usr/share/emacs/site-lisp 

--with-sound=alsa --build x86_64-linux-gnu --prefix=/usr
--sharedstatedir=/var/lib --libexecdir=/usr/lib
--localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.1/site-lisp:/usr/share/emacs/site-lisp 

--with-sound=alsa --with-x=yes --with-x-toolkit=gtk3
--with-toolkit-scroll-bars 'CFLAGS=-g -O2
-fdebug-prefix-map=/build/emacs25-wN2qS3/emacs25-25.1+1=. 
-fstack-protector-strong
-Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
-D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
value of $LANG: en_ZA.UTF-8
locale-coding-system: utf-8-unix

Major mode: Org

Minor modes in effect:
flyspell-mode: t
desktop-save-mode: t
pdf-occur-global-minor-mode: t
override-global-mode: t
dired-omit-mode: t
savehist-mode: t
global-undo-tree-mode: t
undo-tree-mode: t
winner-mode: t
recentf-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
column-number-mode: t
line-number-mode: t
global-visual-line-mode: t
visual-line-mode: t
transient-mark-mode: t
abbrev-mode: t

Recent messages:
Loading /var/cache/dictionaries-common/emacsen-ispell-dicts.el 
(source)...done
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...done
Loading /etc/emacs/site-start.d/50emacs-intl-fonts.el (source)...done
Loading recentf...done
Loading /home/red/.emacs.d/recentf...done
Cleaning up the recentf list...done (0 removed)
Starting new Ispell process aspell with default dictionary...
Wrote /home/red/.emacs.d/.emacs.desktop.lock
Desktop: 1 frame, 1 buffer restored.
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
/usr/share/emacs/25.1/site-lisp/debian-startup hides 
/usr/share/emacs/site-lisp/debian-startup
/home/red/.emacs.d/elpa/org-20190527/org-table hides 
/usr/share/emacs/25.1/lisp/org/org-table
/home/red/.emacs.d/elpa/org-20190527/ox-publish hides 
/usr/share/emacs/25.1/lisp/org/ox-publish
/home/red/.emacs.d/elpa/org-20190527/ob-ocaml hides 
/usr/share/emacs/25.1/lisp/org/ob-ocaml
/home/red/.emacs.d/elpa/org-20190527/ob-asymptote hides 
/usr/share/emacs/25.1/lisp/org/ob-asymptote
/home/red/.emacs.d/elpa/org-20190527/ox hides 
/usr/share/emacs/25.1/lisp/org/ox
/home/red/.emacs.d/elpa/org-20190527/org-attach hides 
/usr/share/emacs/25.1/lisp/org/org-attach
/home/red/.emacs.d/elpa/org-20190527/ob-tangle hides 
/usr/share/emacs/25.1/lisp/org/ob-tangle
/home/red/.emacs.d/elpa/org-20190527/ob-perl hides 
/usr/share/emacs/25.1/lisp/org/ob-perl
/home/red/.emacs.d/elpa/org-20190527/org-w3m hides 
/usr/share/emacs/25.1/lisp/org/org-w3m
/home/red/.emacs.d/elpa/org-20190527/ob-latex hides 
/usr/share/emacs/25.1/lisp/org/ob-latex
/home/red/.emacs.d/elpa/org-20190527/org-habit hides 
/usr/share/emacs/25.1/lisp/org/org-habit
/home/red/.emacs.d/elpa/org-20190527/ob-ruby hides 
/usr/share/emacs/25.1/lisp/org/ob-ruby
/home/red/.emacs.d/elpa/org-20190527/ob-maxima hides 
/usr/share/emacs/25.1/lisp/org/ob-maxima
/home/red/.emacs.d/elpa/org-20190527/ob-ledger hides 
/usr/share/emacs/25.1/lisp/org/ob-ledger
/home/red/.emacs.d/elpa/org-20190527/org-macs hides 
/usr/share/emacs/25.1/lisp/org/org-macs
/home/red/.emacs.d/elpa/org-20190527/ob-awk hides 
/usr/share/emacs/25.1/lisp/org/ob-awk
/home/red/.emacs.d/elpa/org-20190527/org-gnus hides 
/usr/share/emacs/25.1/lisp/org/org-gnus
/home/red/.emacs.d/elpa/org-20190527/ob-ref hides 
/usr/share/emacs/25.1/lisp/org/ob-ref
/home/red/.emacs.d/elpa/org-20190527/ob-haskell hides 
/usr/share/emacs/25.1/lisp/org/ob-haskell
/home/red/.emacs.d/elpa/org-20190527/org-inlinetask hides 
/usr/share/emacs/25.1/lisp/org/org-inlinetask
/home/red/.emacs.d/elpa/org-20190527/org-timer hides 
/usr/share/emacs/25.1/lisp/org/org-timer
/home/red/.emacs.d/elpa/org-20190527/org-archive hides 
/usr/share/emacs/25.1/lisp/org/org-archive
/home/red/.emacs.d/elpa/org-20190527/org-list hides 
/usr/share/emacs/25.1/lisp/org/org-list
/home/red/.emacs.d/elpa/org-20190527/org-datetree hides 
/usr/share/emacs/25.1/lisp/org/org-datetree
/home/red/.emacs.d/elpa/org-20190527/org-mouse hides 
/usr/share/emacs/25.1/lisp/org/org-mouse
/home/red/.emacs.d/elpa/org-20190527/org-bbdb hides 
/usr/share/emacs/25.1/lisp/org/org-bbdb
/home/red/.emacs.d/elpa/org-20190527/ob-calc hides 
/usr/share/emacs/25.1/lisp/org/ob-calc
/home/red/.emacs.d/elpa/org-20190527/ob-eval hides 
/usr/share/emacs/25.1/lisp/org/ob-eval
/home/red/.emacs.d/elpa/org-20190527/ob-makefile hides 
/usr/share/emacs/25.1/lisp/org/ob-makefile
/home/red/.emacs.d/elpa/org-20190527/ox-man hides 
/usr/share/emacs/25.1/lisp/org/ox-man
/home/red/.emacs.d/elpa/org-20190527/org-id hides 
/usr/share/emacs/25.1/lisp/org/org-id
/home/red/.emacs.d/elpa/org-20190527/org-macro hides 
/usr/share/emacs/25.1/lisp/org/org-macro
/home/red/.emacs.d/elpa/org-20190527/ob-gnuplot hides 
/usr/share/emacs/25.1/lisp/org/ob-gnuplot
/home/red/.emacs.d/elpa/org-20190527/org-rmail hides 
/usr/share/emacs/25.1/lisp/org/org-rmail
/home/red/.emacs.d/elpa/org-20190527/ox-md hides 
/usr/share/emacs/25.1/lisp/org/ox-md
/home/red/.emacs.d/elpa/org-20190527/ob-scheme hides 
/usr/share/emacs/25.1/lisp/org/ob-scheme
/home/red/.emacs.d/elpa/org-20190527/org-feed hides 
/usr/share/emacs/25.1/lisp/org/org-feed
/home/red/.emacs.d/elpa/org-20190527/org-indent hides 
/usr/share/emacs/25.1/lisp/org/org-indent
/home/red/.emacs.d/elpa/org-20190527/ob-sqlite hides 
/usr/share/emacs/25.1/lisp/org/ob-sqlite
/home/red/.emacs.d/elpa/org-20190527/ob-org hides 
/usr/share/emacs/25.1/lisp/org/ob-org
/home/red/.emacs.d/elpa/org-20190527/ox-latex hides 
/usr/share/emacs/25.1/lisp/org/ox-latex
/home/red/.emacs.d/elpa/org-20190527/ob-ditaa hides 
/usr/share/emacs/25.1/lisp/org/ob-ditaa
/home/red/.emacs.d/elpa/org-20190527/ob-core hides 
/usr/share/emacs/25.1/lisp/org/ob-core
/home/red/.emacs.d/elpa/org-20190527/org-capture hides 
/usr/share/emacs/25.1/lisp/org/org-capture
/home/red/.emacs.d/elpa/org-20190527/ob-io hides 
/usr/share/emacs/25.1/lisp/org/ob-io
/home/red/.emacs.d/elpa/org-20190527/org-crypt hides 
/usr/share/emacs/25.1/lisp/org/org-crypt
/home/red/.emacs.d/elpa/org-20190527/org-mobile hides 
/usr/share/emacs/25.1/lisp/org/org-mobile
/home/red/.emacs.d/elpa/org-20190527/org-mhe hides 
/usr/share/emacs/25.1/lisp/org/org-mhe
/home/red/.emacs.d/elpa/org-20190527/ox-html hides 
/usr/share/emacs/25.1/lisp/org/ox-html
/home/red/.emacs.d/elpa/org-20190527/ox-texinfo hides 
/usr/share/emacs/25.1/lisp/org/ox-texinfo
/home/red/.emacs.d/elpa/org-20190527/ob-table hides 
/usr/share/emacs/25.1/lisp/org/ob-table
/home/red/.emacs.d/elpa/org-20190527/ob-plantuml hides 
/usr/share/emacs/25.1/lisp/org/ob-plantuml
/home/red/.emacs.d/elpa/org-20190527/org-src hides 
/usr/share/emacs/25.1/lisp/org/org-src
/home/red/.emacs.d/elpa/org-20190527/ox-org hides 
/usr/share/emacs/25.1/lisp/org/ox-org
/home/red/.emacs.d/elpa/org-20190527/org-plot hides 
/usr/share/emacs/25.1/lisp/org/org-plot
/home/red/.emacs.d/elpa/org-20190527/ob-sql hides 
/usr/share/emacs/25.1/lisp/org/ob-sql
/home/red/.emacs.d/elpa/org-20190527/ob-octave hides 
/usr/share/emacs/25.1/lisp/org/ob-octave
/home/red/.emacs.d/elpa/org-20190527/ob-sass hides 
/usr/share/emacs/25.1/lisp/org/ob-sass
/home/red/.emacs.d/elpa/org-20190527/ox-icalendar hides 
/usr/share/emacs/25.1/lisp/org/ox-icalendar
/home/red/.emacs.d/elpa/org-20190527/org-bibtex hides 
/usr/share/emacs/25.1/lisp/org/org-bibtex
/home/red/.emacs.d/elpa/org-20190527/org-entities hides 
/usr/share/emacs/25.1/lisp/org/org-entities
/home/red/.emacs.d/elpa/org-20190527/org-irc hides 
/usr/share/emacs/25.1/lisp/org/org-irc
/home/red/.emacs.d/elpa/org-20190527/ob-mscgen hides 
/usr/share/emacs/25.1/lisp/org/ob-mscgen
/home/red/.emacs.d/elpa/org-20190527/org-install hides 
/usr/share/emacs/25.1/lisp/org/org-install
/home/red/.emacs.d/elpa/org-20190527/ob-emacs-lisp hides 
/usr/share/emacs/25.1/lisp/org/ob-emacs-lisp
/home/red/.emacs.d/elpa/org-20190527/ob-java hides 
/usr/share/emacs/25.1/lisp/org/ob-java
/home/red/.emacs.d/elpa/org-20190527/ob-js hides 
/usr/share/emacs/25.1/lisp/org/ob-js
/home/red/.emacs.d/elpa/org-20190527/org-docview hides 
/usr/share/emacs/25.1/lisp/org/org-docview
/home/red/.emacs.d/elpa/org-20190527/ob-exp hides 
/usr/share/emacs/25.1/lisp/org/ob-exp
/home/red/.emacs.d/elpa/org-20190527/org-ctags hides 
/usr/share/emacs/25.1/lisp/org/org-ctags
/home/red/.emacs.d/elpa/org-20190527/org hides 
/usr/share/emacs/25.1/lisp/org/org
/home/red/.emacs.d/elpa/org-20190527/ob-lisp hides 
/usr/share/emacs/25.1/lisp/org/ob-lisp
/home/red/.emacs.d/elpa/org-20190527/ob-lilypond hides 
/usr/share/emacs/25.1/lisp/org/ob-lilypond
/home/red/.emacs.d/elpa/org-20190527/ob-comint hides 
/usr/share/emacs/25.1/lisp/org/ob-comint
/home/red/.emacs.d/elpa/org-20190527/ox-odt hides 
/usr/share/emacs/25.1/lisp/org/ox-odt
/home/red/.emacs.d/elpa/org-20190527/org-info hides 
/usr/share/emacs/25.1/lisp/org/org-info
/home/red/.emacs.d/elpa/org-20190527/ob-lob hides 
/usr/share/emacs/25.1/lisp/org/ob-lob
/home/red/.emacs.d/elpa/org-20190527/ob-matlab hides 
/usr/share/emacs/25.1/lisp/org/ob-matlab
/home/red/.emacs.d/elpa/org-20190527/ob-css hides 
/usr/share/emacs/25.1/lisp/org/ob-css
/home/red/.emacs.d/elpa/org-20190527/ob-picolisp hides 
/usr/share/emacs/25.1/lisp/org/ob-picolisp
/home/red/.emacs.d/elpa/org-20190527/org-agenda hides 
/usr/share/emacs/25.1/lisp/org/org-agenda
/home/red/.emacs.d/elpa/org-20190527/org-eshell hides 
/usr/share/emacs/25.1/lisp/org/org-eshell
/home/red/.emacs.d/elpa/org-20190527/ox-ascii hides 
/usr/share/emacs/25.1/lisp/org/ox-ascii
/home/red/.emacs.d/elpa/org-20190527/ob hides 
/usr/share/emacs/25.1/lisp/org/ob
/home/red/.emacs.d/elpa/org-20190527/ob-screen hides 
/usr/share/emacs/25.1/lisp/org/ob-screen
/home/red/.emacs.d/elpa/org-20190527/ob-python hides 
/usr/share/emacs/25.1/lisp/org/ob-python
/home/red/.emacs.d/elpa/org-20190527/ob-dot hides 
/usr/share/emacs/25.1/lisp/org/ob-dot
/home/red/.emacs.d/elpa/org-20190527/org-colview hides 
/usr/share/emacs/25.1/lisp/org/org-colview
/home/red/.emacs.d/elpa/org-20190527/org-faces hides 
/usr/share/emacs/25.1/lisp/org/org-faces
/home/red/.emacs.d/elpa/org-20190527/org-footnote hides 
/usr/share/emacs/25.1/lisp/org/org-footnote
/home/red/.emacs.d/elpa/org-20190527/org-protocol hides 
/usr/share/emacs/25.1/lisp/org/org-protocol
/home/red/.emacs.d/elpa/org-20190527/ob-C hides 
/usr/share/emacs/25.1/lisp/org/ob-C
/home/red/.emacs.d/elpa/org-20190527/org-pcomplete hides 
/usr/share/emacs/25.1/lisp/org/org-pcomplete
/home/red/.emacs.d/elpa/org-20190527/org-clock hides 
/usr/share/emacs/25.1/lisp/org/org-clock
/home/red/.emacs.d/elpa/org-20190527/org-element hides 
/usr/share/emacs/25.1/lisp/org/org-element
/home/red/.emacs.d/elpa/org-20190527/org-version hides 
/usr/share/emacs/25.1/lisp/org/org-version
/home/red/.emacs.d/elpa/org-20190527/ob-clojure hides 
/usr/share/emacs/25.1/lisp/org/ob-clojure
/home/red/.emacs.d/elpa/org-20190527/ob-R hides 
/usr/share/emacs/25.1/lisp/org/ob-R
/home/red/.emacs.d/elpa/org-20190527/ox-beamer hides 
/usr/share/emacs/25.1/lisp/org/ox-beamer
/home/red/.emacs.d/elpa/org-20190527/org-loaddefs hides 
/usr/share/emacs/25.1/lisp/org/org-loaddefs
/home/red/.emacs.d/elpa/org-20190527/org-compat hides 
/usr/share/emacs/25.1/lisp/org/org-compat
/home/red/.emacs.d/elpa/org-20190527/ob-keys hides 
/usr/share/emacs/25.1/lisp/org/ob-keys
/home/red/.emacs.d/elpa/org-20190527/ob-fortran hides 
/usr/share/emacs/25.1/lisp/org/ob-fortran
/home/red/.emacs.d/elpa/org-20190527/ob-shen hides 
/usr/share/emacs/25.1/lisp/org/ob-shen

Features:
(shadow sort mail-extr emacsbug sendmail flyspell ispell org-eww
org-rmail org-mhe org-irc org-info org-gnus nnir org-docview doc-view
org-bibtex bibtex org-bbdb org-w3m dired-single iso-transl desktop
frameset transpose-frame org-pdfview pdf-occur ibuf-ext ibuffer tablist
tablist-filter semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw eieio eieio-core mode-local cedet
pdf-isearch let-alist pdf-misc imenu pdf-tools compile cus-edit pdf-view
bookmark pp jka-compr pdf-cache pdf-info tq pdf-util image-mode
use-package-bind-key bind-key use-package-ensure use-package-core
gnus-demon gnus-delay gnus-draft gnus-agent gnus-srvr gnus-score
score-mode nnvirtual nntp gnus-cache gnus-msg gnus-art mm-uu mml2015
mm-view mml-smime smime dig mailcap gnus-sum nndraft nnmh gnus-group
gnus-undo gnus-start gnus-cloud nnimap tls gnutls utf7 netrc nnoo
parse-time gnus-spec gnus-win nnmail gnus-int gnus-range mail-source
message dired-x dired rfc822 mml mml-sec password-cache epg mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev gmm-utils mailheader gnus gnus-ems nnheader gnus-util mm-util
help-fns mail-prsvr mail-utils calfw-org org-capture org-agenda
org-element avl-tree org ob ob-tangle ob-ref ob-lob ob-table ob-exp
org-macro org-footnote org-src ob-comint ob-keys org-pcomplete pcomplete
comint ansi-color org-list org-faces org-entities noutline outline
easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-compat org-macs
org-loaddefs find-func calfw format-spec holidays hol-loaddefs cal-menu
calendar cal-loaddefs savehist subr-x advice undo-tree diff winner ring
remember edmacro kmacro leuven-theme recentf tree-widget wid-edit
cus-start cus-load finder-inf cl-seq cl-macs cl info package epg-config
seq byte-opt gv bytecomp byte-compile cl-extra help-mode easymenu cconv
cl-loaddefs pcase cl-lib time-date mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 434732 18001)
(symbols 48 49645 0)
(miscs 40 1972 3191)
(strings 32 131759 10159)
(string-bytes 1 4250030)
(vectors 16 43640)
(vector-slots 8 796717 10429)
(floats 8 582 60)
(intervals 56 1314 0)
(buffers 976 20))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Tue, 11 Jun 2019 16:30:02 GMT) Full text and rfc822 format available.

Message #8 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Tue, 11 Jun 2019 19:29:18 +0300
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Tue, 11 Jun 2019 17:11:37 +0200
> 
> Emacs does not correctly display Hebrew text with vowels. The vowels
> appear next to the consonants rather than beneath them. When they are
> beneath the consonants they are not in the correct position. There are a
> few other anomalies too, such as, the dagesh appears misplaced followed
> by a circle. If I export my documents with Hebrew text in org mode to html
> or odf then the Hebrew IS displayed correctly. I have tried changing the
> font, installing "unicode-fonts" and a host of other solutions. I have
> tried using different versions of Emacs. I use utf-8 as my default
> encoding system. If I open the same text document in LibreOffice, again,
> the text is displayed correctly. If I strip the Hebrew text of vowels
> then the consonants appear correctly in Emacs. If I use
> auto-composition-mode it makes no difference. The Hebrew text remains
> incorrectly displayed

This description most probably means that the font Emacs uses for
Hebrew characters either doesn't have glyphs for the diacritics, or
the font is somehow unsuitable for displaying Hebrew due to bad
metrics.

FWIW, Hebrew display works just fine for me since Emacs 24 and to this
day, including all the versions in-between.  So this certainly isn't
an inherent problem with Emacs.

Therefore, I suggest to investigate this one step at a time:

  . Does the problem happen in "emacs -Q"?  If not, some of your
    customizations cause the problem, and you need to find out which
    one(s).

  . If the problem does happen in "emacs -Q", please move the cursor
    to one instance of such incorrect display and type "C-u C-x =",
    then post here the full contents of the *Help* buffer that Emacs
    pops up in response.

Please note that auto-composition-mode _must_ be turned on for the
Hebrew diacritics to be displayed correctly.  So in all of the
experiments you will do during discussion of this problem, please
always have auto-composition-mode turned on (as it is by default).

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Wed, 12 Jun 2019 16:24:01 GMT) Full text and rfc822 format available.

Message #11 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Wed, 12 Jun 2019 19:23:39 +0300
[Message part 1 (text/plain, inline)]
[Please use Reply to All to reply to the bug tracker as well, so this
discussion gets recorded by the tracker.]

> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Tue, 11 Jun 2019 20:59:24 +0200
> 
> Thank you for your response. Yes, the problem still occurs in Emacs -Q.
> 
> I did as you asked and here are the full contents of the help buffers of 
> _two_ instances:
> 
>               position: 71 of 9244927 (0%), column: 7
>              character: ְ (displayed as ְ) (codepoint 1456, #o2660, #x5b0)
>      preferred charset: unicode (Unicode (ISO10646))
> code point in charset: 0x05B0
>                 script: hebrew
>                 syntax: w     which means: word
>               category: ^:Combining
>               to input: type "C-x 8 RET 5b0" or "C-x 8 RET HEBREW POINT SHEVA"
>            buffer code: #xD6 #xB0
>              file code: #xD6 #xB0 (encoded by coding system utf-8-unix)
>                display: by this font (glyph code)
>      xft:-unknown-Ezra SIL-normal-normal-normal-*-22-*-*-*-*-0-iso10646-1 (#xFE)
> 
> Character code properties: customize what to show
>    name: HEBREW POINT SHEVA
>    general-category: Mn (Mark, Nonspacing)
>    decomposition: (1456) ('ְ')
> 
> There are text properties here:
>    fontified            t
> 
> --------------------------------------------------------------------------------------------------------------
> 
>               position: 733239 of 9244928 (8%), column: 5
>              character: ׁ (displayed as ׁ) (codepoint 1473, #o2701, #x5c1)
>                charset: unicode-bmp (Unicode Basic Multilingual Plane (U+0000..U+FFFF))
> code point in charset: 0x05C1
>                 script: hebrew
>                 syntax: w     which means: word
>               category: ^:Combining
>               to input: type "C-x 8 RET 5c1" or "C-x 8 RET HEBREW POINT SHIN DOT"
>            buffer code: #xD7 #x81
>              file code: #xD7 #x81 (encoded by coding system utf-8-unix)
>                display: composed to form "ׁ" (see below)
> 
> Composed using this font:
>    xft:-unknown-Ezra SIL-normal-normal-normal-*-24-*-*-*-*-0-iso10646-1
> by these glyphs:
>    [2 2 9676 334 18 2 17 15 -1 nil]
>    [2 2 1473 271 0 13 16 20 -18 nil]
> 
> Character code properties: customize what to show
>    name: HEBREW POINT SHIN DOT
>    general-category: Mn (Mark, Nonspacing)
>    decomposition: (1473) ('ׁ')
> 
> There are text properties here:
>    fontified            t
> 
> -------------------------------------------------------------------------------------------------------

These two characters are diacriticals.  I'd also ask to please do the
same with cursor on the base character (consonant), where a problem
happens.  For example, on the LAMED from the first screenshot.

Also, can you try with a more standard font, like DejaVu Sans Mono?

And finally, do you know what version of the libm17n-flt do you have
installed?  This library is used for shaping text of scripts such as
Arabic and Hebrew.

> For your interest I also attach a few screenshots as examples.

I attach them below, since you didn't CC the bug address.

[Screenshot at 2019-06-11 20-50-27.png (image/png, attachment)]
[Screenshot at 2019-06-11 20-53-15.png (image/png, attachment)]
[Screenshot at 2019-06-11 20-56-24.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Wed, 12 Jun 2019 18:21:01 GMT) Full text and rfc822 format available.

Message #14 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Wed, 12 Jun 2019 20:19:52 +0200
I have libm17n-0 and libm17n-dev installed.

I changed to dejavu-sans-mono. (No change to the Hebrew font display.)

Below follows the information you asked for:

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

             position: 935 of 9244927 (0%), column: 7
            character: לּ‎ (displayed as לּ‎) (codepoint 64316, #o175474, 
#xfb3c)
              charset: unicode-bmp (Unicode Basic Multilingual Plane 
(U+0000..U+FFFF))
code point in charset: 0xFB3C
               script: hebrew
               syntax: w     which means: word
             category: .:Base, R:Right-to-left (strong)
             to input: type "C-x 8 RET fb3c" or "C-x 8 RET HEBREW 
LETTER LAMED WITH DAGESH"
          buffer code: #xEF #xAC #xBC
            file code: #xEF #xAC #xBC (encoded by coding system utf-8-unix)
              display: by this font (glyph code)
    xft:-unknown-DejaVu 
Sans-normal-normal-normal-*-29-*-*-*-*-0-iso10646-1 (#x13DB)

Character code properties: customize what to show
  name: HEBREW LETTER LAMED WITH DAGESH
  general-category: Lo (Letter, Other)
  decomposition: (1500 1468) ('ל' 'ּ')

There are text properties here:
  fontified

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

On 2019/06/12 18:23, Eli Zaretskii wrote:
> [Please use Reply to All to reply to the bug tracker as well, so this
> discussion gets recorded by the tracker.]
>
>> From: Raoul Comninos <revrari <at> mweb.co.za>
>> Date: Tue, 11 Jun 2019 20:59:24 +0200
>>
>> Thank you for your response. Yes, the problem still occurs in Emacs -Q.
>>
>> I did as you asked and here are the full contents of the help buffers of
>> _two_ instances:
>>
>>                position: 71 of 9244927 (0%), column: 7
>>               character: ְ (displayed as ְ) (codepoint 1456, #o2660, #x5b0)
>>       preferred charset: unicode (Unicode (ISO10646))
>> code point in charset: 0x05B0
>>                  script: hebrew
>>                  syntax: w     which means: word
>>                category: ^:Combining
>>                to input: type "C-x 8 RET 5b0" or "C-x 8 RET HEBREW POINT SHEVA"
>>             buffer code: #xD6 #xB0
>>               file code: #xD6 #xB0 (encoded by coding system utf-8-unix)
>>                 display: by this font (glyph code)
>>       xft:-unknown-Ezra SIL-normal-normal-normal-*-22-*-*-*-*-0-iso10646-1 (#xFE)
>>
>> Character code properties: customize what to show
>>     name: HEBREW POINT SHEVA
>>     general-category: Mn (Mark, Nonspacing)
>>     decomposition: (1456) ('ְ')
>>
>> There are text properties here:
>>     fontified            t
>>
>> --------------------------------------------------------------------------------------------------------------
>>
>>                position: 733239 of 9244928 (8%), column: 5
>>               character: ׁ (displayed as ׁ) (codepoint 1473, #o2701, #x5c1)
>>                 charset: unicode-bmp (Unicode Basic Multilingual Plane (U+0000..U+FFFF))
>> code point in charset: 0x05C1
>>                  script: hebrew
>>                  syntax: w     which means: word
>>                category: ^:Combining
>>                to input: type "C-x 8 RET 5c1" or "C-x 8 RET HEBREW POINT SHIN DOT"
>>             buffer code: #xD7 #x81
>>               file code: #xD7 #x81 (encoded by coding system utf-8-unix)
>>                 display: composed to form "ׁ" (see below)
>>
>> Composed using this font:
>>     xft:-unknown-Ezra SIL-normal-normal-normal-*-24-*-*-*-*-0-iso10646-1
>> by these glyphs:
>>     [2 2 9676 334 18 2 17 15 -1 nil]
>>     [2 2 1473 271 0 13 16 20 -18 nil]
>>
>> Character code properties: customize what to show
>>     name: HEBREW POINT SHIN DOT
>>     general-category: Mn (Mark, Nonspacing)
>>     decomposition: (1473) ('ׁ')
>>
>> There are text properties here:
>>     fontified            t
>>
>> -------------------------------------------------------------------------------------------------------
> These two characters are diacriticals.  I'd also ask to please do the
> same with cursor on the base character (consonant), where a problem
> happens.  For example, on the LAMED from the first screenshot.
>
> Also, can you try with a more standard font, like DejaVu Sans Mono?
>
> And finally, do you know what version of the libm17n-flt do you have
> installed?  This library is used for shaping text of scripts such as
> Arabic and Hebrew.
>
>> For your interest I also attach a few screenshots as examples.
> I attach them below, since you didn't CC the bug address.
>




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Wed, 12 Jun 2019 18:49:02 GMT) Full text and rfc822 format available.

Message #17 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Wed, 12 Jun 2019 21:48:07 +0300
> Cc: 36171 <at> debbugs.gnu.org
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Wed, 12 Jun 2019 20:19:52 +0200
> 
> I have libm17n-0 and libm17n-dev installed.
> 
> I changed to dejavu-sans-mono. (No change to the Hebrew font display.)
> 
> Below follows the information you asked for:
> 
> ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
> 
>               position: 935 of 9244927 (0%), column: 7
>              character: לּ‎ (displayed as לּ‎) (codepoint 64316, #o175474, 
> #xfb3c)
>                charset: unicode-bmp (Unicode Basic Multilingual Plane 
> (U+0000..U+FFFF))
> code point in charset: 0xFB3C
>                 script: hebrew
>                 syntax: w     which means: word
>               category: .:Base, R:Right-to-left (strong)
>               to input: type "C-x 8 RET fb3c" or "C-x 8 RET HEBREW 
> LETTER LAMED WITH DAGESH"
>            buffer code: #xEF #xAC #xBC
>              file code: #xEF #xAC #xBC (encoded by coding system utf-8-unix)
>                display: by this font (glyph code)
>      xft:-unknown-DejaVu 
> Sans-normal-normal-normal-*-29-*-*-*-*-0-iso10646-1 (#x13DB)
> 
> Character code properties: customize what to show
>    name: HEBREW LETTER LAMED WITH DAGESH
>    general-category: Lo (Letter, Other)
>    decomposition: (1500 1468) ('ל' 'ּ')

So you are saying that this character was not displayed correctly,
i.e. DAGESH was not inside LAMED?  If not, what is the problem with
displaying this character and the characters around it?

Anyway, it is very strange that only you see these problems.  Would it
be possible for you to build the latest master branch of the Emacs
development repository?  There we support text shaping with HarfBuzz
(which you will need to install separately before building Emacs), so
any potential problems with libm17n-flt you have installed should not
affect the display.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Wed, 12 Jun 2019 19:12:02 GMT) Full text and rfc822 format available.

Message #20 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: revrari <at> mweb.co.za
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Wed, 12 Jun 2019 22:11:07 +0300
It could be a problem with the order of typing the diacriticals when
there are more than one for a single base letter.  For example, for
BET WITH DAGESH AND SHEVA, try typing BET, then SHEVA, and only after
that DAGESH.  And similarly in the other cases where you have DAGESH
and another diacritical.

Does that help?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Wed, 12 Jun 2019 19:28:03 GMT) Full text and rfc822 format available.

Message #23 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Wed, 12 Jun 2019 21:27:10 +0200
The problem is not with the lamed. It does have the dagesh. The vowel 
which should be below the lamed is not directly underneath it but 
misplaced. To be clear, the problem seems to be with how Emacs reads 
these particular documents. If I type the Hebrew myself directly into 
Emacs there are no problems with the display. These documents were 
originally created in other word processors: Word, Nota Bene , and 
Google Docs. They were converted then to plain text (utf-8) which is the 
format they are now in: plain text.

I am currently trying the latest Emacs from git. I have also installed 
gir1.2-harfbuzz-0.0. Is that the correct program?

On 2019/06/12 20:48, Eli Zaretskii wrote:
>> Cc: 36171 <at> debbugs.gnu.org
>> From: Raoul Comninos <revrari <at> mweb.co.za>
>> Date: Wed, 12 Jun 2019 20:19:52 +0200
>>
>> I have libm17n-0 and libm17n-dev installed.
>>
>> I changed to dejavu-sans-mono. (No change to the Hebrew font display.)
>>
>> Below follows the information you asked for:
>>
>> ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
>>
>>                position: 935 of 9244927 (0%), column: 7
>>               character: לּ‎ (displayed as לּ‎) (codepoint 64316, #o175474,
>> #xfb3c)
>>                 charset: unicode-bmp (Unicode Basic Multilingual Plane
>> (U+0000..U+FFFF))
>> code point in charset: 0xFB3C
>>                  script: hebrew
>>                  syntax: w     which means: word
>>                category: .:Base, R:Right-to-left (strong)
>>                to input: type "C-x 8 RET fb3c" or "C-x 8 RET HEBREW
>> LETTER LAMED WITH DAGESH"
>>             buffer code: #xEF #xAC #xBC
>>               file code: #xEF #xAC #xBC (encoded by coding system utf-8-unix)
>>                 display: by this font (glyph code)
>>       xft:-unknown-DejaVu
>> Sans-normal-normal-normal-*-29-*-*-*-*-0-iso10646-1 (#x13DB)
>>
>> Character code properties: customize what to show
>>     name: HEBREW LETTER LAMED WITH DAGESH
>>     general-category: Lo (Letter, Other)
>>     decomposition: (1500 1468) ('ל' 'ּ')
> So you are saying that this character was not displayed correctly,
> i.e. DAGESH was not inside LAMED?  If not, what is the problem with
> displaying this character and the characters around it?
>
> Anyway, it is very strange that only you see these problems.  Would it
> be possible for you to build the latest master branch of the Emacs
> development repository?  There we support text shaping with HarfBuzz
> (which you will need to install separately before building Emacs), so
> any potential problems with libm17n-flt you have installed should not
> affect the display.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 03:33:02 GMT) Full text and rfc822 format available.

Message #26 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 06:32:06 +0300
> Cc: 36171 <at> debbugs.gnu.org
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Wed, 12 Jun 2019 21:27:10 +0200
> 
> To be clear, the problem seems to be with how Emacs reads 
> these particular documents. If I type the Hebrew myself directly into 
> Emacs there are no problems with the display. These documents were 
> originally created in other word processors: Word, Nota Bene , and 
> Google Docs. They were converted then to plain text (utf-8) which is the 
> format they are now in: plain text.

I wasn't aware that the problem disappears if you type Hebrew, and
appears only with these documents.  If so, can you send one such
document as an attachment?

Am I to understand that if you type the same exact text into Emacs,
there's no problem with the display?  If so, the reason must be in the
documents themselves: something in how the text is laid out there
causes this.  Being able to look at such a document is therefore
important for investigating this issue.

> I am currently trying the latest Emacs from git. I have also installed 
> gir1.2-harfbuzz-0.0. Is that the correct program?

Emacs needs HarfBuzz the library, not the program.  What does the
following produce inside the latest Emacs you built from Git:

  M-: (frame-parameter nil 'font-backend) RET




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 08:54:01 GMT) Full text and rfc822 format available.

Message #29 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 10:53:24 +0200
[Message part 1 (text/plain, inline)]
I have attached a sample document with problematic Hebrew taken from a 
few of the files.

> Am I to understand that if you type the same exact text into Emacs,
> there's no problem with the display?

Yes, that is correct.

>> gir1.2-harfbuzz-0.0. Is that the correct program?

(xft xfthb x)

> Emacs needs HarfBuzz the library, not the program.  What does the
> following produce inside the latest Emacs you built from Git:
>
>    M-: (frame-parameter nil 'font-backend) RET

[sample.org (text/plain, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 09:46:01 GMT) Full text and rfc822 format available.

Message #32 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 12:44:54 +0300
> Cc: 36171 <at> debbugs.gnu.org
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Thu, 13 Jun 2019 10:53:24 +0200
> 
> I have attached a sample document with problematic Hebrew taken from a 
> few of the files.

Ah, I see.  This document uses precomposed Hebrew characters from the
Alphabetic Presentation Forms block, and we didn't include those in
the regular expressions used for finding sequences we need to compose
for display.  I've now fixed this in Git, you should be able to
rebuild after syncing with Git, and then the problem should go away.

> (xft xfthb x)
> 
> > Emacs needs HarfBuzz the library, not the program.  What does the
> > following produce inside the latest Emacs you built from Git:
> >
> >    M-: (frame-parameter nil 'font-backend) RET

Yes, xfthb uses HarfBuzz to perform text shaping.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 12:55:02 GMT) Full text and rfc822 format available.

Message #35 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 14:54:19 +0200
[Message part 1 (text/plain, inline)]
I am very pleased we are making progress. If we can get this right, it 
is going to save me having to retype all the Hebrew! The text certainly 
looks much better but the problem is not as yet fully resolved. There 
remain circles in the text (see attachment). If I try to delete them, 
something is lost from the word, either a vowel or a dagesh. I have also 
taken a screen shot of the Emacs version. Please confirm this is the 
correct one I should be using.

On 2019/06/13 11:44, Eli Zaretskii wrote:
>> Cc: 36171 <at> debbugs.gnu.org
>> From: Raoul Comninos <revrari <at> mweb.co.za>
>> Date: Thu, 13 Jun 2019 10:53:24 +0200
>>
>> I have attached a sample document with problematic Hebrew taken from a
>> few of the files.
> Ah, I see.  This document uses precomposed Hebrew characters from the
> Alphabetic Presentation Forms block, and we didn't include those in
> the regular expressions used for finding sequences we need to compose
> for display.  I've now fixed this in Git, you should be able to
> rebuild after syncing with Git, and then the problem should go away.
>
>> (xft xfthb x)
>>
>>> Emacs needs HarfBuzz the library, not the program.  What does the
>>> following produce inside the latest Emacs you built from Git:
>>>
>>>     M-: (frame-parameter nil 'font-backend) RET
> Yes, xfthb uses HarfBuzz to perform text shaping.
>
> Thanks.
[circles.png (image/png, attachment)]
[emacs-version.png (image/png, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 13:15:02 GMT) Full text and rfc822 format available.

Message #38 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 16:14:00 +0300
> Cc: 36171 <at> debbugs.gnu.org
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Thu, 13 Jun 2019 14:54:19 +0200
> 
> I am very pleased we are making progress. If we can get this right, it 
> is going to save me having to retype all the Hebrew! The text certainly 
> looks much better but the problem is not as yet fully resolved. There 
> remain circles in the text (see attachment).

I don't see any such circles on my system.  I tried half a dozen
different fonts, and they all work correctly.  How many fonts did you
try?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 16:08:01 GMT) Full text and rfc822 format available.

Message #41 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Raoul Comninos <revrari <at> mweb.co.za>, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 18:07:06 +0200
>>>>> On Thu, 13 Jun 2019 16:14:00 +0300, Eli Zaretskii <eliz <at> gnu.org> said:

    >> Cc: 36171 <at> debbugs.gnu.org
    >> From: Raoul Comninos <revrari <at> mweb.co.za>
    >> Date: Thu, 13 Jun 2019 14:54:19 +0200
    >> 
    >> I am very pleased we are making progress. If we can get this right, it 
    >> is going to save me having to retype all the Hebrew! The text certainly 
    >> looks much better but the problem is not as yet fully resolved. There 
    >> remain circles in the text (see attachment).

    Eli> I don't see any such circles on my system.  I tried half a dozen
    Eli> different fonts, and they all work correctly.  How many fonts did you
    Eli> try?

I see circles on my GNU/Linux box, but only when using the xft font
backend, eg the second character from the left (visually) on the
'Aoristic perfect' line. If I put

(add-to-list 'default-frame-alist '(font-backend xfthb))

in my .emacs, then everything displays correctly. I thought HarfBuzz
was supposed to be preferred to xft?

Robert




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 16:08:02 GMT) Full text and rfc822 format available.

Message #44 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 19:07:32 +0300
[Please keep the bug address on the CC list.]

> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Thu, 13 Jun 2019 15:58:26 +0200
> 
> Are you speaking of the default Emacs font?

Yes.

> At present I am using Dejavu Sans Mono. How would I go about
> changing the font expressly for the Hebrew display? I removed
> fonts-sil-ezra. Would I need to use the unicode-fonts package?

If you are using Dejavu Sans Mono, please try a few more fonts.  No
need to set a special font for Hebrew, just invoke Emacs like this:

  emacs -Q -fn FONT-NAME

where FONT-NAME is the name of some font you have on your system.

Also, please go to some character that precedes a circle, type
"C-u C-x =" again, and show the output here, I want to make sure you
are using the xfthb font backend.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 16:44:01 GMT) Full text and rfc822 format available.

Message #47 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: revrari <at> mweb.co.za, YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>,
 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 19:42:50 +0300
> From: Robert Pluim <rpluim <at> gmail.com>
> Cc: Raoul Comninos <revrari <at> mweb.co.za>,  36171 <at> debbugs.gnu.org
> Date: Thu, 13 Jun 2019 18:07:06 +0200
> 
>     Eli> I don't see any such circles on my system.  I tried half a dozen
>     Eli> different fonts, and they all work correctly.  How many fonts did you
>     Eli> try?
> 
> I see circles on my GNU/Linux box, but only when using the xft font
> backend, eg the second character from the left (visually) on the
> 'Aoristic perfect' line. If I put
> 
> (add-to-list 'default-frame-alist '(font-backend xfthb))
> 
> in my .emacs, then everything displays correctly.

That's what I thought, thanks.  So these circles are a deficiency of
FLT shaping.

> I thought HarfBuzz was supposed to be preferred to xft?

It's supposed to, but, disappointedly,
https://lists.gnu.org/archive/html/emacs-devel/2019-06/msg00124.html
remains without a response, and therefore only the Windows build
currently prefers HarfBuzz.  I still hope Yamamoto-san will respond
and describe his proposal, and we could then implement the same
behavior on all platforms.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 18:40:02 GMT) Full text and rfc822 format available.

Message #50 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>, Robert Pluim <rpluim <at> gmail.com>
Cc: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 20:39:10 +0200
I have tried out several different fonts but there is no change to the 
problem.

I still see the circles. These can be turned off if I turn off 
auto-composition-mode. However, though the  circles then disappear, the 
vowels appear incorrectly placed. They are not as poorly placed as 
before, but still do not all properly align beneath the consonants. Some 
vowels are slightly off.

I added (add-to-list 'default-frame-alist '(font-backend xfthb)) to my 
.emacs file, but this made no difference.

Below is the information you requested.

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

             position: 50 of 9235 (1%), column: 7
            character: ְ (displayed as ְ) (codepoint 1456, #o2660, #x5b0)
              charset: unicode-bmp (Unicode Basic Multilingual Plane 
(U+0000..U+FFFF))
code point in charset: 0x05B0
               script: hebrew
               syntax: w     which means: word
             category: ^:Combining
             to input: type "C-x 8 RET 5b0" or "C-x 8 RET HEBREW POINT 
SHEVA"
          buffer code: #xD6 #xB0
            file code: #xD6 #xB0 (encoded by coding system utf-8-unix)
              display: composed to form "ְ" (see below)

Composed using this font:
  xft:-unknown-DejaVu Sans-normal-normal-normal-*-26-*-*-*-*-0-iso10646-1
by these glyphs:
  [1 1 9676 3748 23 1 22 17 4 nil]
  [1 1 1456 1297 0 7 9 -1 6 nil]

Character code properties: customize what to show
  name: HEBREW POINT SHEVA
  general-category: Mn (Mark, Nonspacing)
  decomposition: (1456) ('ְ')

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

             position: 41 of 9235 (0%), column: 0
            character: בּ‎ (displayed as בּ‎) (codepoint 64305, #o175461, 
#xfb31)
              charset: unicode-bmp (Unicode Basic Multilingual Plane 
(U+0000..U+FFFF))
code point in charset: 0xFB31
               script: hebrew
               syntax: w     which means: word
             category: .:Base, R:Right-to-left (strong)
             to input: type "C-x 8 RET fb31" or "C-x 8 RET HEBREW 
LETTER BET WITH DAGESH"
          buffer code: #xEF #xAC #xB1
            file code: #xEF #xAC #xB1 (encoded by coding system utf-8-unix)
              display: by this font (glyph code)
    xft:-unknown-DejaVu 
Sans-normal-normal-normal-*-26-*-*-*-*-0-iso10646-1 (#x13D1)

Character code properties: customize what to show
  name: HEBREW LETTER BET WITH DAGESH
  general-category: Lo (Letter, Other)
  decomposition: (1489 1468) ('ב' 'ּ')

There are text properties here:
  fontified            t

///////////////////////////////////////////////////////////////////////////////

On 2019/06/13 18:42, Eli Zaretskii wrote:
>> From: Robert Pluim <rpluim <at> gmail.com>
>> Cc: Raoul Comninos <revrari <at> mweb.co.za>,  36171 <at> debbugs.gnu.org
>> Date: Thu, 13 Jun 2019 18:07:06 +0200
>>
>>      Eli> I don't see any such circles on my system.  I tried half a dozen
>>      Eli> different fonts, and they all work correctly.  How many fonts did you
>>      Eli> try?
>>
>> I see circles on my GNU/Linux box, but only when using the xft font
>> backend, eg the second character from the left (visually) on the
>> 'Aoristic perfect' line. If I put
>>
>> (add-to-list 'default-frame-alist '(font-backend xfthb))
>>
>> in my .emacs, then everything displays correctly.
> That's what I thought, thanks.  So these circles are a deficiency of
> FLT shaping.
>
>> I thought HarfBuzz was supposed to be preferred to xft?
> It's supposed to, but, disappointedly,
> https://lists.gnu.org/archive/html/emacs-devel/2019-06/msg00124.html
> remains without a response, and therefore only the Windows build
> currently prefers HarfBuzz.  I still hope Yamamoto-san will respond
> and describe his proposal, and we could then implement the same
> behavior on all platforms.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Thu, 13 Jun 2019 18:58:02 GMT) Full text and rfc822 format available.

Message #53 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Thu, 13 Jun 2019 21:57:31 +0300
> Cc: 36171 <at> debbugs.gnu.org, YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Thu, 13 Jun 2019 20:39:10 +0200
> 
> I have tried out several different fonts but there is no change to the 
> problem.
> 
> I still see the circles.

That's because your Emacs uses the XFT/FLT shaping, not the HarfBuzz
shaping.

Try starting Emacs like this:

  emacs -Q -xrm Emacs.fontBackend:xfthb

and then see if the problem goes away.  You should see "xfthb" in the
"C-u C-x =" display, here you currently see "xft":

>    xft:-unknown-DejaVu Sans-normal-normal-normal-*-26-*-*-*-*-0-iso10646-1
     ^^^

> I added (add-to-list 'default-frame-alist '(font-backend xfthb)) to my 
> .emacs file, but this made no difference.

My crystal ball says that you added that too late in the init file.
You must do this at the very beginning.

In any case, for whatever reason, your Emacs still uses the XFT font
backend, not the HarfBuzz-powered xfthb.  I believe that's why you see
the circles: FLT doesn't support composition of the characters from
the Alphabetic Presentations Forms block.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Fri, 14 Jun 2019 11:52:02 GMT) Full text and rfc822 format available.

Message #56 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Fri, 14 Jun 2019 13:51:21 +0200
If I start Emacs with emacs -Q -xrm Emacs.fontBackend:xfthb I still see 
xft.

However, I have moved (add-to-list 'default-frame-alist '(font-backend 
xfthb)) to the front of my .emacs and this appears to have resolved the 
problem. I no longer see circles and the Hebrew font is properly displayed.

Thank you very much Eli for your help with this. I am going to test it 
today further just to make sure but so far however, all seems well.

With much appreciation.

Kind regards

Raoul Comninos

On 2019/06/13 20:57, Eli Zaretskii wrote:
>> Cc: 36171 <at> debbugs.gnu.org, YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
>> From: Raoul Comninos <revrari <at> mweb.co.za>
>> Date: Thu, 13 Jun 2019 20:39:10 +0200
>>
>> I have tried out several different fonts but there is no change to the
>> problem.
>>
>> I still see the circles.
> That's because your Emacs uses the XFT/FLT shaping, not the HarfBuzz
> shaping.
>
> Try starting Emacs like this:
>
>    emacs -Q -xrm Emacs.fontBackend:xfthb
>
> and then see if the problem goes away.  You should see "xfthb" in the
> "C-u C-x =" display, here you currently see "xft":
>
>>     xft:-unknown-DejaVu Sans-normal-normal-normal-*-26-*-*-*-*-0-iso10646-1
>       ^^^
>
>> I added (add-to-list 'default-frame-alist '(font-backend xfthb)) to my
>> .emacs file, but this made no difference.
> My crystal ball says that you added that too late in the init file.
> You must do this at the very beginning.
>
> In any case, for whatever reason, your Emacs still uses the XFT font
> backend, not the HarfBuzz-powered xfthb.  I believe that's why you see
> the circles: FLT doesn't support composition of the characters from
> the Alphabetic Presentations Forms block.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Fri, 14 Jun 2019 12:24:02 GMT) Full text and rfc822 format available.

Message #59 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Fri, 14 Jun 2019 15:22:52 +0300
> Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Fri, 14 Jun 2019 13:51:21 +0200
> 
> If I start Emacs with emacs -Q -xrm Emacs.fontBackend:xfthb I still see 
> xft.
> 
> However, I have moved (add-to-list 'default-frame-alist '(font-backend 
> xfthb)) to the front of my .emacs and this appears to have resolved the 
> problem. I no longer see circles and the Hebrew font is properly displayed.
> 
> Thank you very much Eli for your help with this. I am going to test it 
> today further just to make sure but so far however, all seems well.

And thanks to Robert for testing on GNU/Linux.

Please come back in a day or two and tell if you see any problems, I'd
like to close the bug report if no other problems remain.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Fri, 14 Jun 2019 14:41:01 GMT) Full text and rfc822 format available.

Message #62 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Fri, 14 Jun 2019 16:40:46 +0200
Thanks to you and also Robert. I will test it out and report to you in a 
day or two.

On 2019/06/14 14:22, Eli Zaretskii wrote:
>> Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
>> From: Raoul Comninos <revrari <at> mweb.co.za>
>> Date: Fri, 14 Jun 2019 13:51:21 +0200
>>
>> If I start Emacs with emacs -Q -xrm Emacs.fontBackend:xfthb I still see
>> xft.
>>
>> However, I have moved (add-to-list 'default-frame-alist '(font-backend
>> xfthb)) to the front of my .emacs and this appears to have resolved the
>> problem. I no longer see circles and the Hebrew font is properly displayed.
>>
>> Thank you very much Eli for your help with this. I am going to test it
>> today further just to make sure but so far however, all seems well.
> And thanks to Robert for testing on GNU/Linux.
>
> Please come back in a day or two and tell if you see any problems, I'd
> like to close the bug report if no other problems remain.
>
> Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36171; Package emacs. (Tue, 18 Jun 2019 18:02:03 GMT) Full text and rfc822 format available.

Message #65 received at 36171 <at> debbugs.gnu.org (full text, mbox):

From: Raoul Comninos <revrari <at> mweb.co.za>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Tue, 18 Jun 2019 20:00:57 +0200
I think we can sign off on this issue. I have not seen the problem recur 
in any of my documents.

Once again, thank you very much for all your help. It is a great relief 
for me to resolve this issue.

Kind regards

Raoul Comninos

On 2019/06/14 14:22, Eli Zaretskii wrote:
>> Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
>> From: Raoul Comninos <revrari <at> mweb.co.za>
>> Date: Fri, 14 Jun 2019 13:51:21 +0200
>>
>> If I start Emacs with emacs -Q -xrm Emacs.fontBackend:xfthb I still see
>> xft.
>>
>> However, I have moved (add-to-list 'default-frame-alist '(font-backend
>> xfthb)) to the front of my .emacs and this appears to have resolved the
>> problem. I no longer see circles and the Hebrew font is properly displayed.
>>
>> Thank you very much Eli for your help with this. I am going to test it
>> today further just to make sure but so far however, all seems well.
> And thanks to Robert for testing on GNU/Linux.
>
> Please come back in a day or two and tell if you see any problems, I'd
> like to close the bug report if no other problems remain.
>
> Thanks.




Reply sent to Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility. (Tue, 18 Jun 2019 18:11:01 GMT) Full text and rfc822 format available.

Notification sent to Raoul Comninos <revrari <at> mweb.co.za>:
bug acknowledged by developer. (Tue, 18 Jun 2019 18:11:02 GMT) Full text and rfc822 format available.

Message #70 received at 36171-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Raoul Comninos <revrari <at> mweb.co.za>
Cc: 36171-done <at> debbugs.gnu.org, rpluim <at> gmail.com
Subject: Re: bug#36171: 25.1; Emacs displays Hebrew text incorrectly
Date: Tue, 18 Jun 2019 21:09:52 +0300
> Cc: rpluim <at> gmail.com, 36171 <at> debbugs.gnu.org
> From: Raoul Comninos <revrari <at> mweb.co.za>
> Date: Tue, 18 Jun 2019 20:00:57 +0200
> 
> I think we can sign off on this issue. I have not seen the problem recur 
> in any of my documents.

Thanks, I'm closing the bug.

> Once again, thank you very much for all your help. It is a great relief 
> for me to resolve this issue.

This is also the first time we see a clear improvement from the very
recent addition of HarfBuzz support.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Wed, 17 Jul 2019 11:24:07 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 256 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.