GNU logs - #51766, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 11 Nov 2021 13:56:01 +0000
Resent-Message-ID: <handler.51766.B.163663893416660 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 51766 <at> debbugs.gnu.org
From: Ihor Radchenko <yantar92@HIDDEN>
Date: Thu, 11 Nov 2021 21:56:46 +0800
Message-ID: <87mtmalrs1.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=2607:f8b0:4864:20::531;
 envelope-from=yantar92@HIDDEN; helo=mail-pg1-x531.google.com
X-Spam_score_int: -10
X-Spam_score: -1.1
X-Spam_bar: -
X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.2 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: According to buffer-chars-modified-tick docstring: "By
 comparing
 the values returned by two individual calls of buffer-chars-modified-tick,
 you can tell whether a character change occurred in that buf [...] 
 Content analysis details:   (1.2 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -2.3 RCVD_IN_DNSWL_MED      RBL: Sender listed at https://www.dnswl.org/,
 medium trust [209.51.188.17 listed in list.dnswl.org]
 0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail
 provider (yantar92[at]gmail.com)
 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends
 in digit (yantar92[at]gmail.com)
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 -0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [209.51.188.17 listed in wl.mailspike.net]
 1.3 SPOOFED_FREEMAIL       No description available.
 0.9 SPOOF_GMAIL_MID        From Gmail but it doesn't seem to be...
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

According to buffer-chars-modified-tick docstring:
"By comparing the values returned by two individual calls of
buffer-chars-modified-tick, you can tell whether a character change
occurred in that buffer in between these calls"

However, the return value can change even when no visible change is made
to buffer text.

Steps to reproduce:
1. emacs -Q
2. Evaluate the following code:

(defun print-tick-before ()
  (when (eq this-command 'self-insert-command)
  (warn "Tick before: %S" (buffer-chars-modified-tick))))
(defun print-tick-after ()
    (when (eq this-command 'self-insert-command)
  (warn "Tick after: %S" (buffer-chars-modified-tick))))
(add-hook 'pre-command-hook #'print-tick-before)
(add-hook 'post-command-hook #'print-tick-after)

3. Insert a latin symbol ?a twice. The warning buffer will print
something like

Warning (emacs): Tick before: 1698 Disable showing Disable logging
Warning (emacs): Tick after: 1699 Disable showing Disable logging
Warning (emacs): Tick before: 1699 Disable showing Disable logging
Warning (emacs): Tick after: 1702 Disable showing Disable logging

Note that second and third line show the same buffer-chars-modified-tick
value.

4. Change input method (C-\) to russian-computer or i.e. arabic
5. Insert a non-latin symbol ?=D1=84 twice. The warning buffer will print
something like

Warning (emacs): Tick before: 1706 Disable showing Disable logging
Warning (emacs): Tick after: 1707 Disable showing Disable logging
Warning (emacs): Tick before: 1711 Disable showing Disable logging
Warning (emacs): Tick after: 1712 Disable showing Disable logging

Note that second and third line _do not_ show the same
buffer-chars-modified-tick value even though buffer text has not been
changed between the two self-insert commands

Expected behaviour: return value of buffer-chars-modified-tick does not
change when no changes in buffer text are made.

This issue causes breakage in latest version of Org. See
https://list.orgmode.org/87sfw2luhj.fsf@localhost/T/#you

Best,
Ihor


In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo version 1.16.0)
 of 2021-10-30 built on localhost
Repository revision: c3499b8ddc357544a58917bfd3846f88caf5d97c
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12013000
System Description: Gentoo/Linux

Configured using:
 'configure --prefix=3D/usr --build=3Dx86_64-pc-linux-gnu
 --host=3Dx86_64-pc-linux-gnu --mandir=3D/usr/share/man
 --infodir=3D/usr/share/info --datadir=3D/usr/share --sysconfdir=3D/etc
 --localstatedir=3D/var/lib --datarootdir=3D/usr/share
 --disable-silent-rules --docdir=3D/usr/share/doc/emacs-29.0.9999
 --htmldir=3D/usr/share/doc/emacs-29.0.9999/html --libdir=3D/usr/lib64
 --program-suffix=3D-emacs-29-vcs --includedir=3D/usr/include/emacs-29-vcs
 --infodir=3D/usr/share/info/emacs-29-vcs --localstatedir=3D/var
 --enable-locallisppath=3D/etc/emacs:/usr/share/emacs/site-lisp
 --without-compress-install --without-hesiod --without-pop
 --with-file-notification=3Dinotify --with-pdumper --enable-acl
 --with-dbus --with-modules --without-gameuser --with-libgmp
 --without-gpm --with-native-compilation --with-json --without-kerberos
 --without-kerberos5 --without-lcms2 --with-xml2 --without-mailutils
 --with-selinux --with-gnutls --without-libsystemd --with-threads
 --with-wide-int --with-zlib --with-sound=3Doss --with-x --without-ns
 --without-gconf --without-gsettings --without-toolkit-scroll-bars
 --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm
 --with-imagemagick --with-xft --with-cairo --with-harfbuzz
 --without-libotf --without-m17n-flt --with-x-toolkit=3Dno
 --with-dumping=3Dpdumper 'CFLAGS=3D-march=3Dnative -pipe -O2' CPPFLAGS=3D
 'LDFLAGS=3D-Wl,-O1 -Wl,--as-needed''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ IMAGEMAGICK JPEG
JSON LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY OLDXMENU
PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF WEBP X11 XDBE XIM XPM ZLIB

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  pdf-occur-global-minor-mode: t
  TeX-PDF-mode: t
  org-edna-mode: t
  eros-mode: t
  which-key-mode: t
  diredfl-global-mode: t
  dired-async-mode: t
  winner-mode: t
  recentf-mode: t
  helm-adaptive-mode: t
  helm-mode: t
  helm--remap-mouse-mode: t
  async-bytecomp-package-mode: t
  eval-sexp-fu-flash-mode: t
  el-patch-use-package-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  shell-dirtrack-mode: t
  unpackaged/magit-log-date-headers-mode: t
  hl-todo-mode: t
  pretty-symbols-mode: t
  company-mode: t
  persistent-scratch-autosave-mode: t
  savehist-mode: t
  boon-mode: t
  boon-local-mode: t
  global-hl-line-mode: t
  global-page-break-lines-mode: t
  page-break-lines-mode: t
  shackle-mode: t
  gcmh-mode: t
  override-global-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  global-prettify-symbols-mode: t
  prettify-symbols-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/home/yantar92/.emacs.d/straight/build/helm-org/helm-org hides /home/yantar=
92/.emacs.d/elpa/helm-org-20210324.1927/helm-org
/home/yantar92/.emacs.d/straight/build/helm-org/helm-org-autoloads hides /h=
ome/yantar92/.emacs.d/elpa/helm-org-20210324.1927/helm-org-autoloads
/home/yantar92/.emacs.d/straight/build/helm/helm-x-files hides /home/yantar=
92/.emacs.d/elpa/helm-20210827.1619/helm-x-files
/home/yantar92/.emacs.d/straight/build/helm/helm-utils hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-utils
/home/yantar92/.emacs.d/straight/build/helm/helm-types hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-types
/home/yantar92/.emacs.d/straight/build/helm/helm-tags hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-tags
/home/yantar92/.emacs.d/straight/build/helm/helm-sys hides /home/yantar92/.=
emacs.d/elpa/helm-20210827.1619/helm-sys
/home/yantar92/.emacs.d/straight/build/helm/helm-shell hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-shell
/home/yantar92/.emacs.d/straight/build/helm/helm-semantic hides /home/yanta=
r92/.emacs.d/elpa/helm-20210827.1619/helm-semantic
/home/yantar92/.emacs.d/straight/build/helm/helm-ring hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-ring
/home/yantar92/.emacs.d/straight/build/helm/helm-regexp hides /home/yantar9=
2/.emacs.d/elpa/helm-20210827.1619/helm-regexp
/home/yantar92/.emacs.d/straight/build/helm/helm-occur hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-occur
/home/yantar92/.emacs.d/straight/build/helm/helm-net hides /home/yantar92/.=
emacs.d/elpa/helm-20210827.1619/helm-net
/home/yantar92/.emacs.d/straight/build/helm/helm-mode hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-mode
/home/yantar92/.emacs.d/straight/build/helm/helm-misc hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-misc
/home/yantar92/.emacs.d/straight/build/helm/helm-man hides /home/yantar92/.=
emacs.d/elpa/helm-20210827.1619/helm-man
/home/yantar92/.emacs.d/straight/build/helm/helm-locate hides /home/yantar9=
2/.emacs.d/elpa/helm-20210827.1619/helm-locate
/home/yantar92/.emacs.d/straight/build/helm/helm-info hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-info
/home/yantar92/.emacs.d/straight/build/helm/helm-imenu hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-imenu
/home/yantar92/.emacs.d/straight/build/helm/helm-id-utils hides /home/yanta=
r92/.emacs.d/elpa/helm-20210827.1619/helm-id-utils
/home/yantar92/.emacs.d/straight/build/helm/helm-help hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-help
/home/yantar92/.emacs.d/straight/build/helm/helm-grep hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-grep
/home/yantar92/.emacs.d/straight/build/helm/helm-global-bindings hides /hom=
e/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-global-bindings
/home/yantar92/.emacs.d/straight/build/helm/helm-for-files hides /home/yant=
ar92/.emacs.d/elpa/helm-20210827.1619/helm-for-files
/home/yantar92/.emacs.d/straight/build/helm/helm-font hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-font
/home/yantar92/.emacs.d/straight/build/helm/helm-find hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-find
/home/yantar92/.emacs.d/straight/build/helm/helm-files hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-files
/home/yantar92/.emacs.d/straight/build/helm/helm-fd hides /home/yantar92/.e=
macs.d/elpa/helm-20210827.1619/helm-fd
/home/yantar92/.emacs.d/straight/build/helm/helm-external hides /home/yanta=
r92/.emacs.d/elpa/helm-20210827.1619/helm-external
/home/yantar92/.emacs.d/straight/build/helm/helm-eval hides /home/yantar92/=
.emacs.d/elpa/helm-20210827.1619/helm-eval
/home/yantar92/.emacs.d/straight/build/helm/helm-eshell hides /home/yantar9=
2/.emacs.d/elpa/helm-20210827.1619/helm-eshell
/home/yantar92/.emacs.d/straight/build/helm/helm-epa hides /home/yantar92/.=
emacs.d/elpa/helm-20210827.1619/helm-epa
/home/yantar92/.emacs.d/straight/build/helm/helm-elisp hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-elisp
/home/yantar92/.emacs.d/straight/build/helm/helm-elisp-package hides /home/=
yantar92/.emacs.d/elpa/helm-20210827.1619/helm-elisp-package
/home/yantar92/.emacs.d/straight/build/helm/helm-easymenu hides /home/yanta=
r92/.emacs.d/elpa/helm-20210827.1619/helm-easymenu
/home/yantar92/.emacs.d/straight/build/helm/helm-dabbrev hides /home/yantar=
92/.emacs.d/elpa/helm-20210827.1619/helm-dabbrev
/home/yantar92/.emacs.d/straight/build/helm/helm-config hides /home/yantar9=
2/.emacs.d/elpa/helm-20210827.1619/helm-config
/home/yantar92/.emacs.d/straight/build/helm/helm-command hides /home/yantar=
92/.emacs.d/elpa/helm-20210827.1619/helm-command
/home/yantar92/.emacs.d/straight/build/helm/helm-comint hides /home/yantar9=
2/.emacs.d/elpa/helm-20210827.1619/helm-comint
/home/yantar92/.emacs.d/straight/build/helm/helm-color hides /home/yantar92=
/.emacs.d/elpa/helm-20210827.1619/helm-color
/home/yantar92/.emacs.d/straight/build/helm/helm-buffers hides /home/yantar=
92/.emacs.d/elpa/helm-20210827.1619/helm-buffers
/home/yantar92/.emacs.d/straight/build/helm/helm-bookmark hides /home/yanta=
r92/.emacs.d/elpa/helm-20210827.1619/helm-bookmark
/home/yantar92/.emacs.d/straight/build/helm/helm-adaptive hides /home/yanta=
r92/.emacs.d/elpa/helm-20210827.1619/helm-adaptive
/home/yantar92/.emacs.d/straight/build/helm/helm-autoloads hides /home/yant=
ar92/.emacs.d/elpa/helm-20210827.1619/helm-autoloads
/home/yantar92/.emacs.d/straight/build/helm/helm-pkg hides /home/yantar92/.=
emacs.d/elpa/helm-20210827.1619/helm-pkg
/home/yantar92/.emacs.d/straight/build/helm-core/helm hides /home/yantar92/=
.emacs.d/elpa/helm-core-20210822.952/helm
/home/yantar92/.emacs.d/straight/build/helm-core/helm-source hides /home/ya=
ntar92/.emacs.d/elpa/helm-core-20210822.952/helm-source
/home/yantar92/.emacs.d/straight/build/helm-core/helm-multi-match hides /ho=
me/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-multi-match
/home/yantar92/.emacs.d/straight/build/helm-core/helm-lib hides /home/yanta=
r92/.emacs.d/elpa/helm-core-20210822.952/helm-lib
/home/yantar92/.emacs.d/straight/build/helm-core/helm-core-autoloads hides =
/home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-core-autoloads
/home/yantar92/.emacs.d/straight/build/helm-core/helm-core-pkg hides /home/=
yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-core-pkg
/home/yantar92/.emacs.d/straight/build/async/smtpmail-async hides /home/yan=
tar92/.emacs.d/elpa/async-20210823.528/smtpmail-async
/home/yantar92/.emacs.d/straight/build/async/dired-async hides /home/yantar=
92/.emacs.d/elpa/async-20210823.528/dired-async
/home/yantar92/.emacs.d/straight/build/async/async hides /home/yantar92/.em=
acs.d/elpa/async-20210823.528/async
/home/yantar92/.emacs.d/straight/build/async/async-bytecomp hides /home/yan=
tar92/.emacs.d/elpa/async-20210823.528/async-bytecomp
/home/yantar92/.emacs.d/straight/build/async/async-autoloads hides /home/ya=
ntar92/.emacs.d/elpa/async-20210823.528/async-autoloads
/home/yantar92/.emacs.d/straight/build/org-ql/org-ql hides /home/yantar92/.=
emacs.d/elpa/org-ql-20210713.233/org-ql
/home/yantar92/.emacs.d/straight/build/org-ql/org-ql-view hides /home/yanta=
r92/.emacs.d/elpa/org-ql-20210713.233/org-ql-view
/home/yantar92/.emacs.d/straight/build/org-ql/org-ql-search hides /home/yan=
tar92/.emacs.d/elpa/org-ql-20210713.233/org-ql-search
/home/yantar92/.emacs.d/straight/build/org-ql/org-ql-autoloads hides /home/=
yantar92/.emacs.d/elpa/org-ql-20210713.233/org-ql-autoloads
/home/yantar92/.emacs.d/straight/build/f/f hides /home/yantar92/.emacs.d/el=
pa/f-20210624.1103/f
/home/yantar92/.emacs.d/straight/build/f/f-autoloads hides /home/yantar92/.=
emacs.d/elpa/f-20210624.1103/f-autoloads
/home/yantar92/.emacs.d/straight/build/org-super-agenda/org-super-agenda hi=
des /home/yantar92/.emacs.d/elpa/org-super-agenda-20201211.918/org-super-ag=
enda
/home/yantar92/.emacs.d/straight/build/org-super-agenda/org-super-agenda-au=
toloads hides /home/yantar92/.emacs.d/elpa/org-super-agenda-20201211.918/or=
g-super-agenda-autoloads
/home/yantar92/.emacs.d/straight/build/ht/ht hides /home/yantar92/.emacs.d/=
elpa/ht-20210119.741/ht
/home/yantar92/.emacs.d/straight/build/ht/ht-autoloads hides /home/yantar92=
/.emacs.d/elpa/ht-20210119.741/ht-autoloads
/home/yantar92/.emacs.d/straight/build/ov/ov hides /home/yantar92/.emacs.d/=
elpa/ov-20200326.1042/ov
/home/yantar92/.emacs.d/straight/build/ov/ov-autoloads hides /home/yantar92=
/.emacs.d/elpa/ov-20200326.1042/ov-autoloads
/home/yantar92/.emacs.d/straight/build/peg/peg hides /home/yantar92/.emacs.=
d/elpa/peg-1.0/peg
/home/yantar92/.emacs.d/straight/build/peg/peg-tests hides /home/yantar92/.=
emacs.d/elpa/peg-1.0/peg-tests
/home/yantar92/.emacs.d/straight/build/peg/peg-autoloads hides /home/yantar=
92/.emacs.d/elpa/peg-1.0/peg-autoloads
/home/yantar92/.emacs.d/straight/build/popup/popup hides /home/yantar92/.em=
acs.d/elpa/popup-20210625.400/popup
/home/yantar92/.emacs.d/straight/build/popup/popup-autoloads hides /home/ya=
ntar92/.emacs.d/elpa/popup-20210625.400/popup-autoloads
/home/yantar92/.emacs.d/straight/build/transient/transient hides /home/yant=
ar92/.emacs.d/elpa/transient-20210819.2118/transient
/home/yantar92/.emacs.d/straight/build/transient/transient-autoloads hides =
/home/yantar92/.emacs.d/elpa/transient-20210819.2118/transient-autoloads
/home/yantar92/.emacs.d/straight/build/ts/ts hides /home/yantar92/.emacs.d/=
elpa/ts-20210813.1617/ts
/home/yantar92/.emacs.d/straight/build/ts/ts-autoloads hides /home/yantar92=
/.emacs.d/elpa/ts-20210813.1617/ts-autoloads
/home/yantar92/.emacs.d/straight/build/s/s hides /home/yantar92/.emacs.d/el=
pa/s-20210616.619/s
/home/yantar92/.emacs.d/straight/build/s/s-autoloads hides /home/yantar92/.=
emacs.d/elpa/s-20210616.619/s-autoloads
/home/yantar92/.emacs.d/straight/build/dash/dash hides /home/yantar92/.emac=
s.d/elpa/dash-20210826.1149/dash
/home/yantar92/.emacs.d/straight/build/dash/dash-autoloads hides /home/yant=
ar92/.emacs.d/elpa/dash-20210826.1149/dash-autoloads
/usr/share/emacs/site-lisp/cmake-mode hides /usr/share/emacs/site-lisp/cmak=
e/cmake-mode
/home/yantar92/.emacs.d/straight/build/dash/dash hides /usr/share/emacs/sit=
e-lisp/dash/dash
/usr/share/emacs/site-lisp/desktop-entry-mode hides /usr/share/emacs/site-l=
isp/desktop-file-utils/desktop-entry-mode
/home/yantar92/.emacs.d/straight/build/f/f hides /usr/share/emacs/site-lisp=
/f/f
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-lib hides /usr/share=
/emacs/site-lisp/notmuch/notmuch-lib
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-compat hides /usr/sh=
are/emacs/site-lisp/notmuch/notmuch-compat
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-parser hides /usr/sh=
are/emacs/site-lisp/notmuch/notmuch-parser
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch hides /usr/share/ema=
cs/site-lisp/notmuch/notmuch
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-query hides /usr/sha=
re/emacs/site-lisp/notmuch/notmuch-query
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-show hides /usr/shar=
e/emacs/site-lisp/notmuch/notmuch-show
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-tree hides /usr/shar=
e/emacs/site-lisp/notmuch/notmuch-tree
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-wash hides /usr/shar=
e/emacs/site-lisp/notmuch/notmuch-wash
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-hello hides /usr/sha=
re/emacs/site-lisp/notmuch/notmuch-hello
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-mua hides /usr/share=
/emacs/site-lisp/notmuch/notmuch-mua
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-address hides /usr/s=
hare/emacs/site-lisp/notmuch/notmuch-address
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-maildir-fcc hides /u=
sr/share/emacs/site-lisp/notmuch/notmuch-maildir-fcc
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-message hides /usr/s=
hare/emacs/site-lisp/notmuch/notmuch-message
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-crypto hides /usr/sh=
are/emacs/site-lisp/notmuch/notmuch-crypto
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-tag hides /usr/share=
/emacs/site-lisp/notmuch/notmuch-tag
/home/yantar92/.emacs.d/straight/build/notmuch/coolj hides /usr/share/emacs=
/site-lisp/notmuch/coolj
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-print hides /usr/sha=
re/emacs/site-lisp/notmuch/notmuch-print
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-jump hides /usr/shar=
e/emacs/site-lisp/notmuch/notmuch-jump
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-company hides /usr/s=
hare/emacs/site-lisp/notmuch/notmuch-company
/home/yantar92/.emacs.d/straight/build/notmuch/notmuch-draft hides /usr/sha=
re/emacs/site-lisp/notmuch/notmuch-draft
/home/yantar92/.emacs.d/straight/build/s/s hides /usr/share/emacs/site-lisp=
/s/s
/home/yantar92/.emacs.d/straight/build/with-editor/with-editor hides /usr/s=
hare/emacs/site-lisp/with-editor/with-editor
/home/yantar92/.emacs.d/straight/build/transient/transient hides /usr/share=
/emacs/29.0.50/lisp/transient
/home/yantar92/.emacs.d/straight/build/org/ob-C hides /usr/share/emacs/29.0=
.50/lisp/org/ob-C
/home/yantar92/.emacs.d/straight/build/org/ob-R hides /usr/share/emacs/29.0=
.50/lisp/org/ob-R
/home/yantar92/.emacs.d/straight/build/org/ob-awk hides /usr/share/emacs/29=
.0.50/lisp/org/ob-awk
/home/yantar92/.emacs.d/straight/build/org/ob-calc hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-calc
/home/yantar92/.emacs.d/straight/build/org/ob-clojure hides /usr/share/emac=
s/29.0.50/lisp/org/ob-clojure
/home/yantar92/.emacs.d/straight/build/org/ob-comint hides /usr/share/emacs=
/29.0.50/lisp/org/ob-comint
/home/yantar92/.emacs.d/straight/build/org/ob-core hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-core
/home/yantar92/.emacs.d/straight/build/org/ob-css hides /usr/share/emacs/29=
.0.50/lisp/org/ob-css
/home/yantar92/.emacs.d/straight/build/org/ob-ditaa hides /usr/share/emacs/=
29.0.50/lisp/org/ob-ditaa
/home/yantar92/.emacs.d/straight/build/org/ob-dot hides /usr/share/emacs/29=
.0.50/lisp/org/ob-dot
/home/yantar92/.emacs.d/straight/build/org/ob-emacs-lisp hides /usr/share/e=
macs/29.0.50/lisp/org/ob-emacs-lisp
/home/yantar92/.emacs.d/straight/build/org/ob-eshell hides /usr/share/emacs=
/29.0.50/lisp/org/ob-eshell
/home/yantar92/.emacs.d/straight/build/org/ob-eval hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-eval
/home/yantar92/.emacs.d/straight/build/org/ob-exp hides /usr/share/emacs/29=
.0.50/lisp/org/ob-exp
/home/yantar92/.emacs.d/straight/build/org/ob-forth hides /usr/share/emacs/=
29.0.50/lisp/org/ob-forth
/home/yantar92/.emacs.d/straight/build/org/ob-fortran hides /usr/share/emac=
s/29.0.50/lisp/org/ob-fortran
/home/yantar92/.emacs.d/straight/build/org/ob-gnuplot hides /usr/share/emac=
s/29.0.50/lisp/org/ob-gnuplot
/home/yantar92/.emacs.d/straight/build/org/ob-groovy hides /usr/share/emacs=
/29.0.50/lisp/org/ob-groovy
/home/yantar92/.emacs.d/straight/build/org/ob-haskell hides /usr/share/emac=
s/29.0.50/lisp/org/ob-haskell
/home/yantar92/.emacs.d/straight/build/org/ob-java hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-java
/home/yantar92/.emacs.d/straight/build/org/ob-js hides /usr/share/emacs/29.=
0.50/lisp/org/ob-js
/home/yantar92/.emacs.d/straight/build/org/ob-julia hides /usr/share/emacs/=
29.0.50/lisp/org/ob-julia
/home/yantar92/.emacs.d/straight/build/org/ob-latex hides /usr/share/emacs/=
29.0.50/lisp/org/ob-latex
/home/yantar92/.emacs.d/straight/build/org/ob-lilypond hides /usr/share/ema=
cs/29.0.50/lisp/org/ob-lilypond
/home/yantar92/.emacs.d/straight/build/org/ob-lisp hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-lisp
/home/yantar92/.emacs.d/straight/build/org/ob-lob hides /usr/share/emacs/29=
.0.50/lisp/org/ob-lob
/home/yantar92/.emacs.d/straight/build/org/ob-lua hides /usr/share/emacs/29=
.0.50/lisp/org/ob-lua
/home/yantar92/.emacs.d/straight/build/org/ob-makefile hides /usr/share/ema=
cs/29.0.50/lisp/org/ob-makefile
/home/yantar92/.emacs.d/straight/build/org/ob-matlab hides /usr/share/emacs=
/29.0.50/lisp/org/ob-matlab
/home/yantar92/.emacs.d/straight/build/org/ob-maxima hides /usr/share/emacs=
/29.0.50/lisp/org/ob-maxima
/home/yantar92/.emacs.d/straight/build/org/ob-ocaml hides /usr/share/emacs/=
29.0.50/lisp/org/ob-ocaml
/home/yantar92/.emacs.d/straight/build/org/ob-octave hides /usr/share/emacs=
/29.0.50/lisp/org/ob-octave
/home/yantar92/.emacs.d/straight/build/org/ob-org hides /usr/share/emacs/29=
.0.50/lisp/org/ob-org
/home/yantar92/.emacs.d/straight/build/org/ob-perl hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-perl
/home/yantar92/.emacs.d/straight/build/org/ob-plantuml hides /usr/share/ema=
cs/29.0.50/lisp/org/ob-plantuml
/home/yantar92/.emacs.d/straight/build/org/ob-processing hides /usr/share/e=
macs/29.0.50/lisp/org/ob-processing
/home/yantar92/.emacs.d/straight/build/org/ob-python hides /usr/share/emacs=
/29.0.50/lisp/org/ob-python
/home/yantar92/.emacs.d/straight/build/org/ob-ref hides /usr/share/emacs/29=
.0.50/lisp/org/ob-ref
/home/yantar92/.emacs.d/straight/build/org/ob-ruby hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-ruby
/home/yantar92/.emacs.d/straight/build/org/ob-sass hides /usr/share/emacs/2=
9.0.50/lisp/org/ob-sass
/home/yantar92/.emacs.d/straight/build/org/ob-scheme hides /usr/share/emacs=
/29.0.50/lisp/org/ob-scheme
/home/yantar92/.emacs.d/straight/build/org/ob-screen hides /usr/share/emacs=
/29.0.50/lisp/org/ob-screen
/home/yantar92/.emacs.d/straight/build/org/ob-sed hides /usr/share/emacs/29=
.0.50/lisp/org/ob-sed
/home/yantar92/.emacs.d/straight/build/org/ob-shell hides /usr/share/emacs/=
29.0.50/lisp/org/ob-shell
/home/yantar92/.emacs.d/straight/build/org/ob-sql hides /usr/share/emacs/29=
.0.50/lisp/org/ob-sql
/home/yantar92/.emacs.d/straight/build/org/ob-sqlite hides /usr/share/emacs=
/29.0.50/lisp/org/ob-sqlite
/home/yantar92/.emacs.d/straight/build/org/ob-table hides /usr/share/emacs/=
29.0.50/lisp/org/ob-table
/home/yantar92/.emacs.d/straight/build/org/ob-tangle hides /usr/share/emacs=
/29.0.50/lisp/org/ob-tangle
/home/yantar92/.emacs.d/straight/build/org/ob hides /usr/share/emacs/29.0.5=
0/lisp/org/ob
/home/yantar92/.emacs.d/straight/build/org/oc-basic hides /usr/share/emacs/=
29.0.50/lisp/org/oc-basic
/home/yantar92/.emacs.d/straight/build/org/oc-biblatex hides /usr/share/ema=
cs/29.0.50/lisp/org/oc-biblatex
/home/yantar92/.emacs.d/straight/build/org/oc-csl hides /usr/share/emacs/29=
.0.50/lisp/org/oc-csl
/home/yantar92/.emacs.d/straight/build/org/oc-natbib hides /usr/share/emacs=
/29.0.50/lisp/org/oc-natbib
/home/yantar92/.emacs.d/straight/build/org/oc hides /usr/share/emacs/29.0.5=
0/lisp/org/oc
/home/yantar92/.emacs.d/straight/build/org/ol-bbdb hides /usr/share/emacs/2=
9.0.50/lisp/org/ol-bbdb
/home/yantar92/.emacs.d/straight/build/org/ol-bibtex hides /usr/share/emacs=
/29.0.50/lisp/org/ol-bibtex
/home/yantar92/.emacs.d/straight/build/org/ol-docview hides /usr/share/emac=
s/29.0.50/lisp/org/ol-docview
/home/yantar92/.emacs.d/straight/build/org/ol-doi hides /usr/share/emacs/29=
.0.50/lisp/org/ol-doi
/home/yantar92/.emacs.d/straight/build/org/ol-eshell hides /usr/share/emacs=
/29.0.50/lisp/org/ol-eshell
/home/yantar92/.emacs.d/straight/build/org/ol-eww hides /usr/share/emacs/29=
.0.50/lisp/org/ol-eww
/home/yantar92/.emacs.d/straight/build/org/ol-gnus hides /usr/share/emacs/2=
9.0.50/lisp/org/ol-gnus
/home/yantar92/.emacs.d/straight/build/org/ol-info hides /usr/share/emacs/2=
9.0.50/lisp/org/ol-info
/home/yantar92/.emacs.d/straight/build/org/ol-irc hides /usr/share/emacs/29=
.0.50/lisp/org/ol-irc
/home/yantar92/.emacs.d/straight/build/org/ol-man hides /usr/share/emacs/29=
.0.50/lisp/org/ol-man
/home/yantar92/.emacs.d/straight/build/org/ol-mhe hides /usr/share/emacs/29=
.0.50/lisp/org/ol-mhe
/home/yantar92/.emacs.d/straight/build/org/ol-rmail hides /usr/share/emacs/=
29.0.50/lisp/org/ol-rmail
/home/yantar92/.emacs.d/straight/build/org/ol-w3m hides /usr/share/emacs/29=
.0.50/lisp/org/ol-w3m
/home/yantar92/.emacs.d/straight/build/org/ol hides /usr/share/emacs/29.0.5=
0/lisp/org/ol
/home/yantar92/.emacs.d/straight/build/org/org-agenda hides /usr/share/emac=
s/29.0.50/lisp/org/org-agenda
/home/yantar92/.emacs.d/straight/build/org/org-archive hides /usr/share/ema=
cs/29.0.50/lisp/org/org-archive
/home/yantar92/.emacs.d/straight/build/org/org-attach-git hides /usr/share/=
emacs/29.0.50/lisp/org/org-attach-git
/home/yantar92/.emacs.d/straight/build/org/org-attach hides /usr/share/emac=
s/29.0.50/lisp/org/org-attach
/home/yantar92/.emacs.d/straight/build/org/org-capture hides /usr/share/ema=
cs/29.0.50/lisp/org/org-capture
/home/yantar92/.emacs.d/straight/build/org/org-clock hides /usr/share/emacs=
/29.0.50/lisp/org/org-clock
/home/yantar92/.emacs.d/straight/build/org/org-colview hides /usr/share/ema=
cs/29.0.50/lisp/org/org-colview
/home/yantar92/.emacs.d/straight/build/org/org-compat hides /usr/share/emac=
s/29.0.50/lisp/org/org-compat
/home/yantar92/.emacs.d/straight/build/org/org-crypt hides /usr/share/emacs=
/29.0.50/lisp/org/org-crypt
/home/yantar92/.emacs.d/straight/build/org/org-ctags hides /usr/share/emacs=
/29.0.50/lisp/org/org-ctags
/home/yantar92/.emacs.d/straight/build/org/org-datetree hides /usr/share/em=
acs/29.0.50/lisp/org/org-datetree
/home/yantar92/.emacs.d/straight/build/org/org-duration hides /usr/share/em=
acs/29.0.50/lisp/org/org-duration
/home/yantar92/.emacs.d/straight/build/org/org-element hides /usr/share/ema=
cs/29.0.50/lisp/org/org-element
/home/yantar92/.emacs.d/straight/build/org/org-entities hides /usr/share/em=
acs/29.0.50/lisp/org/org-entities
/home/yantar92/.emacs.d/straight/build/org/org-faces hides /usr/share/emacs=
/29.0.50/lisp/org/org-faces
/home/yantar92/.emacs.d/straight/build/org/org-feed hides /usr/share/emacs/=
29.0.50/lisp/org/org-feed
/home/yantar92/.emacs.d/straight/build/org/org-footnote hides /usr/share/em=
acs/29.0.50/lisp/org/org-footnote
/home/yantar92/.emacs.d/straight/build/org/org-goto hides /usr/share/emacs/=
29.0.50/lisp/org/org-goto
/home/yantar92/.emacs.d/straight/build/org/org-habit hides /usr/share/emacs=
/29.0.50/lisp/org/org-habit
/home/yantar92/.emacs.d/straight/build/org/org-id hides /usr/share/emacs/29=
.0.50/lisp/org/org-id
/home/yantar92/.emacs.d/straight/build/org/org-indent hides /usr/share/emac=
s/29.0.50/lisp/org/org-indent
/home/yantar92/.emacs.d/straight/build/org/org-inlinetask hides /usr/share/=
emacs/29.0.50/lisp/org/org-inlinetask
/home/yantar92/.emacs.d/straight/build/org/org-install hides /usr/share/ema=
cs/29.0.50/lisp/org/org-install
/home/yantar92/.emacs.d/straight/build/org/org-keys hides /usr/share/emacs/=
29.0.50/lisp/org/org-keys
/home/yantar92/.emacs.d/straight/build/org/org-lint hides /usr/share/emacs/=
29.0.50/lisp/org/org-lint
/home/yantar92/.emacs.d/straight/build/org/org-list hides /usr/share/emacs/=
29.0.50/lisp/org/org-list
/home/yantar92/.emacs.d/straight/build/org/org-macro hides /usr/share/emacs=
/29.0.50/lisp/org/org-macro
/home/yantar92/.emacs.d/straight/build/org/org-macs hides /usr/share/emacs/=
29.0.50/lisp/org/org-macs
/home/yantar92/.emacs.d/straight/build/org/org-mobile hides /usr/share/emac=
s/29.0.50/lisp/org/org-mobile
/home/yantar92/.emacs.d/straight/build/org/org-mouse hides /usr/share/emacs=
/29.0.50/lisp/org/org-mouse
/home/yantar92/.emacs.d/straight/build/org/org-num hides /usr/share/emacs/2=
9.0.50/lisp/org/org-num
/home/yantar92/.emacs.d/straight/build/org/org-pcomplete hides /usr/share/e=
macs/29.0.50/lisp/org/org-pcomplete
/home/yantar92/.emacs.d/straight/build/org/org-plot hides /usr/share/emacs/=
29.0.50/lisp/org/org-plot
/home/yantar92/.emacs.d/straight/build/org/org-protocol hides /usr/share/em=
acs/29.0.50/lisp/org/org-protocol
/home/yantar92/.emacs.d/straight/build/org/org-refile hides /usr/share/emac=
s/29.0.50/lisp/org/org-refile
/home/yantar92/.emacs.d/straight/build/org/org-src hides /usr/share/emacs/2=
9.0.50/lisp/org/org-src
/home/yantar92/.emacs.d/straight/build/org/org-table hides /usr/share/emacs=
/29.0.50/lisp/org/org-table
/home/yantar92/.emacs.d/straight/build/org/org-tempo hides /usr/share/emacs=
/29.0.50/lisp/org/org-tempo
/home/yantar92/.emacs.d/straight/build/org/org-timer hides /usr/share/emacs=
/29.0.50/lisp/org/org-timer
/home/yantar92/.emacs.d/straight/build/org/org-version hides /usr/share/ema=
cs/29.0.50/lisp/org/org-version
/home/yantar92/.emacs.d/straight/build/org/org hides /usr/share/emacs/29.0.=
50/lisp/org/org
/home/yantar92/.emacs.d/straight/build/org/ox-ascii hides /usr/share/emacs/=
29.0.50/lisp/org/ox-ascii
/home/yantar92/.emacs.d/straight/build/org/ox-beamer hides /usr/share/emacs=
/29.0.50/lisp/org/ox-beamer
/home/yantar92/.emacs.d/straight/build/org/ox-html hides /usr/share/emacs/2=
9.0.50/lisp/org/ox-html
/home/yantar92/.emacs.d/straight/build/org/ox-icalendar hides /usr/share/em=
acs/29.0.50/lisp/org/ox-icalendar
/home/yantar92/.emacs.d/straight/build/org/ox-koma-letter hides /usr/share/=
emacs/29.0.50/lisp/org/ox-koma-letter
/home/yantar92/.emacs.d/straight/build/org/ox-latex hides /usr/share/emacs/=
29.0.50/lisp/org/ox-latex
/home/yantar92/.emacs.d/straight/build/org/ox-man hides /usr/share/emacs/29=
.0.50/lisp/org/ox-man
/home/yantar92/.emacs.d/straight/build/org/ox-md hides /usr/share/emacs/29.=
0.50/lisp/org/ox-md
/home/yantar92/.emacs.d/straight/build/org/ox-odt hides /usr/share/emacs/29=
.0.50/lisp/org/ox-odt
/home/yantar92/.emacs.d/straight/build/org/ox-org hides /usr/share/emacs/29=
.0.50/lisp/org/ox-org
/home/yantar92/.emacs.d/straight/build/org/ox-publish hides /usr/share/emac=
s/29.0.50/lisp/org/ox-publish
/home/yantar92/.emacs.d/straight/build/org/ox-texinfo hides /usr/share/emac=
s/29.0.50/lisp/org/ox-texinfo
/home/yantar92/.emacs.d/straight/build/org/ox hides /usr/share/emacs/29.0.5=
0/lisp/org/ox
/home/yantar92/.emacs.d/straight/build/org/org-loaddefs hides /usr/share/em=
acs/29.0.50/lisp/org/org-loaddefs
/home/yantar92/.emacs.d/straight/build/let-alist/let-alist hides /usr/share=
/emacs/29.0.50/lisp/emacs-lisp/let-alist
/home/yantar92/.emacs.d/straight/build/map/map hides /usr/share/emacs/29.0.=
50/lisp/emacs-lisp/map

Features:
(shadow emacsbug shell-pop shortdoc cl-print pyim-dhashcache
org-pdftools org-noter pdf-view-restore pdf-sync pdf-annot facemenu
pdf-outline pdf-links pdf-history pdf-occur tablist tablist-filter
semantic/wisent/comp semantic/wisent semantic/wisent/wisent
semantic/util-modes semantic/util semantic semantic/tag semantic/lex
semantic/fw mode-local cedet pdf-isearch pdf-misc pdf-tools pdf-view
pdf-cache pdf-info tq pdf-util pdf-macs magit-extras helm-imenu
dired-open helm-ring qrencode helm-command helm-elisp helm-eval
all-the-icons-dired dired-filter dired-hide-dotfiles network-stream
url-cache qp thai-util thai-word helm-font avy mule-util cal-move
ledger-mode ledger-check ledger-texi ledger-test ledger-sort
ledger-report ledger-reconcile ledger-occur ledger-fonts ledger-fontify
ledger-state ledger-complete ledger-schedule ledger-init ledger-xact
ledger-post ledger-exec ledger-navigate eshell esh-cmd esh-ext esh-opt
esh-proc esh-io esh-arg esh-module esh-groups esh-util ledger-context
ledger-commodities ledger-regex ox-org tabify elfeed-link cus-edit
cus-start cus-load w3m-form w3m-symbol w3m-bookmark w3m w3m-hist w3m-fb
bookmark-w3m w3m-ems w3m-favicon w3m-image tab-line w3m-proc w3m-util
mm-archive org-datetree org-learn latex latex-flymake flymake-proc
flymake tex-ispell tex-style tex sendmail boon-moves er-basic-expansions
expand-region-core expand-region-custom sort footnote mail-extr
boon-main boon-arguments multiple-cursors mc-separate-operations
rectangular-region-mode mc-mark-pop mc-edit-lines
mc-hide-unmatched-lines-mode mc-mark-more mc-cycle-cursors
multiple-cursors-core boon-regs boon-utils boon-hl misearch
multi-isearch org-duration cal-iso ffap org-table-sticky-header
org-appear ol-eww eww mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus
nnselect gnus-search eieio-opt speedbar ezimage dframe ol-docview
doc-view jka-compr ol-bbdb ol-w3m ol-doi org-link-doi tramp-archive
tramp-gvfs helm-x-files org-crypt helm-notmuch helm-notmuch-autoloads
ol-notmuch org-eldoc org-appear-autoloads doom-themes-ext-org
doom-themes doom-themes-base doom-themes-autoloads
org-table-sticky-header-autoloads posframe ob-async ob-async-autoloads
ob-latex ob-dot ob-calc calc-store calc-trail ob-gnuplot ob-ditaa ob-C
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs ob-python python tramp-sh ob-perl ob-org ob-shell
ob-mathematica org-tempo tempo org-archive ox-md ox-beamer ox-extra doct
ya-org-capture ya-org-capture-autoloads doct-autoloads
org-capture-pop-frame org-capture-pop-frame-autoloads org-protocol
org-analyzer-autoloads pomidor-autoloads alert-autoloads log4e-autoloads
gntp-autoloads org-clock org-autosort org-autosort-autoloads
helm-org-contacts helm-org-contacts-autoloads org-contacts gnus-art
mm-uu mml2015 gnus-sum gnus-group gnus-undo gnus-start gnus-dbus
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int
gnus-range gnus-win gnus nnheader helm-org-ql helm-org
helm-org-ql-autoloads helm-org-autoloads org-ql-search org-ql-view ov
org-super-agenda org-ql peg org-ql-autoloads peg-autoloads ov-autoloads
org-super-agenda-autoloads map-autoloads org-quick-peek
org-quick-peek-autoloads calfw-org calfw-org-autoloads calfw holidays
hol-loaddefs calfw-autoloads org-attach cdlatex texmathp
cdlatex-autoloads helm-recoll eieio-compat helm-for-files helm-bookmark
helm-info helm-external helm-recoll-autoloads org-capture-ref
org-ref-url-utils org-ref org-ref-helm-bibtex org-ref-helm helm-bibtex
helm-net helm-config org-ref-core reftex-cite reftex reftex-loaddefs
reftex-vars org-ref-glossary org-ref-bibtex org-ref-citeproc key-chord
doi-utils org-ref-utils org-ref-pdf ol-bibtex htmlize bibtex-completion
biblio biblio-download biblio-dissemin biblio-ieee biblio-hal
biblio-dblp biblio-crossref biblio-arxiv timezone biblio-doi biblio-core
ido parsebib org-ref-autoloads key-chord-autoloads ivy-autoloads
helm-bibtex-autoloads bibtex-completion-autoloads biblio-autoloads
biblio-core-autoloads parsebib-autoloads htmlize-autoloads
scimax-inkscape scimax-inkscape-autoloads org-pdftools-autoloads
org-noter-autoloads org-capture org-checklist org-habit org-edna
org-edna-autoloads org-inlinetask org-drill persist org-drill-autoloads
persist-autoloads speed-type speed-type-autoloads notmuch-calendar-x
notmuch-calendar-x-autoloads notmuch notmuch-tree notmuch-jump
notmuch-hello notmuch-show notmuch-print notmuch-crypto notmuch-mua
notmuch-message notmuch-draft notmuch-maildir-fcc notmuch-address
notmuch-company notmuch-parser notmuch-wash coolj notmuch-query
goto-addr icalendar diary-lib diary-loaddefs notmuch-tag notmuch-lib
notmuch-version notmuch-compat mm-view mml-smime smime dig w3m-load
w3m-autoloads notmuch-autoloads elfeed-score elfeed-score-maint
elfeed-score-scoring elfeed-score-serde elfeed-score-rule-stats
elfeed-org elfeed-org-autoloads quick-peek quick-peek-autoloads
elfeed-show elfeed-search vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-svn
vc-cvs vc-rcs vc hideshow display-fill-column-indicator eros
flycheck-tip error-tip notifications dbus flycheck-tip-autoloads
flycheck rainbow-delimiters highlight-numbers parent-mode easy-escape
yasnippet-snippets-autoloads yasnippet-snippets yasnippet elfeed-csv
elfeed elfeed-curl elfeed-log elfeed-db elfeed-lib url-queue xml-query
elfeed-score-rules elfeed-score-log elfeed-score-autoloads
elfeed-autoloads qrencode-el-autoloads keycast keycast-autoloads
gif-screencast gif-screencast-autoloads yaml-mode yaml-mode-autoloads
mingus libmpdee cl mingus-autoloads libmpdee-autoloads calctex calc-sel
calc-ext calctex-autoloads calc calc-loaddefs rect calc-macs
shell-pop-autoloads eterm-256color-autoloads xterm-color-autoloads vterm
term ehelp vterm-module term/xterm xterm vterm-autoloads ereader xml+
view shr kinsoku svg dom picture ereader-autoloads xml+-autoloads
diffpdf diffpdf-autoloads pdf-view-restore-autoloads pdf-tools-autoloads
tablist-autoloads wolfram-mode wolfram-mode-autoloads
ledger-mode-autoloads auctex-autoloads tex-site ebuild-mode skeleton
sh-script smie executable ebuild-mode-autoloads lua-mode
lua-mode-autoloads gnuplot-autoloads eros-autoloads nameless lisp-mnt
nameless-autoloads paredit paredit-autoloads which-key
which-key-autoloads helm-descbinds helm-descbinds-autoloads elisp-demos
elisp-demos-autoloads helpful edebug info-look help-fns radix-tree
elisp-refs helpful-autoloads elisp-refs-autoloads tldr tldr-autoloads
macrostep macrostep-autoloads font-lock-profiler
font-lock-profiler-autoloads font-lock-studio font-lock-studio-autoloads
memory-usage memory-usage-autoloads bug-hunter bug-hunter-autoloads
lorem-ipsum lorem-ipsum-autoloads debug backtrace yasnippet-autoloads
move-text move-text-autoloads aggressive-indent
aggressive-indent-autoloads visual-regexp-autoloads magit-bookmark
bookmark pp helm-bm helm-bm-autoloads bm bm-autoloads helm-dash
dash-docs use-package-dash-docs xml helm-dash-autoloads
dash-docs-autoloads disk-usage disk-usage-autoloads
dired-git-info-autoloads dired-hide-dotfiles-autoloads
dired-filter-autoloads diredfl diredfl-autoloads
all-the-icons-dired-autoloads dired-async dired-open-autoloads
dired-avfs dired-avfs-autoloads dired-narrow-autoloads dired-hacks-utils
dired-hacks-utils-autoloads dired+ image-dired image-mode exif
image-file image-converter dired-x dired-aux dired+-autoloads winner
windower emacs-windower-autoloads goggles pulse skip-buffers-mode
recentf tree-widget wid-edit helm-icons treemacs-icons treemacs-themes
treemacs-core-utils treemacs-logging treemacs-customization pfuture
inline helm-adaptive helm-mode helm-files tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat ls-lisp helm-buffers helm-occur
helm-tags helm-locate helm-grep helm-regexp helm-utils helm-help
helm-types helm async-bytecomp helm-global-bindings helm-source
helm-multi-match helm-lib async eval-sexp-fu eval-sexp-fu-autoloads
goggles-autoloads easy-escape-autoloads highlight-numbers-autoloads
parent-mode-autoloads rainbow-delimiters-autoloads highlight-parentheses
highlight-parentheses-autoloads flycheck-autoloads pkg-info-autoloads
epl-autoloads langtool compile langtool-autoloads el-patch
el-patch-autoloads flyspell ispell hi-lock ediff ediff-merg ediff-mult
ediff-wind ediff-diff ediff-help ediff-init ediff-util browse-at-remote
vc-git vc-dispatcher f browse-at-remote-autoloads forge-list
forge-commands forge-semi forge-bitbucket buck forge-gogs gogs
forge-gitea gtea forge-gitlab glab forge-github ghub-graphql treepy
gsexp ghub let-alist gnutls forge-notify forge-revnote forge-pullreq
forge-issue forge-topic yaml parse-time bug-reference forge-post
markdown-mode thingatpt forge-repo forge forge-core forge-db closql
emacsql-sqlite emacsql emacsql-compiler url-http url-auth url-gw nsm
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff git-commit log-edit message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail
rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
shell magit-mode transient magit-git magit-section magit-utils crm
forge-autoloads yaml-autoloads markdown-mode-autoloads ghub-autoloads
treepy-autoloads let-alist-autoloads closql-autoloads
emacsql-sqlite-autoloads emacsql-autoloads unpackaged smerge-mode
diff-mode diff ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar org-agenda ox-html table ox-ascii ox-publish ox org-element
org-persist xdg avl-tree ibuf-ext ibuffer ibuffer-loaddefs use-package
use-package-ensure use-package-delight ts ts-autoloads
unpackaged-autoloads magit-autoloads magit-section-autoloads
git-commit-autoloads with-editor-autoloads transient-autoloads
autorevert filenotify disp-table hl-todo pretty-symbols company-oddmuse
company-keywords company-etags etags fileloop generator xref project
company-gtags company-dabbrev-code company-dabbrev company-files
company-clang company-capf company-cmake company-semantic
company-template company-bbdb company persistent-scratch
persistent-scratch-autoloads savehist backup-walker-autoloads
company-autoloads helm-icons-autoloads treemacs-autoloads cfrs-autoloads
posframe-autoloads pfuture-autoloads ace-window-autoloads avy-autoloads
f-autoloads helm-autoloads helm-core-autoloads popup-autoloads
face-remap pyim pyim-hacks pyim-probe pyim-cregexp xr pyim-process
pyim-cstring pyim-autoselector pyim-punctuation pyim-outcome
pyim-indicator pyim-preview pyim-magic pyim-candidates pyim-codes
pyim-imobjs pyim-pinyin pyim-pymap pyim-entered pyim-dcache pyim-dict
pyim-page popup pyim-scheme pyim-common pyim-autoloads xr-autoloads
async-autoloads reverse-im quail reverse-im-autoloads hydra lv
boon-qwerty color olivetti straight-x boon boon-keys boon-core
boon-loaddefs boon-autoloads pcre2el-autoloads
multiple-cursors-autoloads expand-region-autoloads meta-functions org-id
org-refile meta-functions-autoloads hl-line memoize memoize-autoloads
info-colors info-colors-autoloads hl-todo-autoloads latex-pretty-symbols
latex-pretty-symbols-autoloads pretty-symbols-autoloads page-break-lines
page-break-lines-autoloads edmacro kmacro adaptive-wrap
adaptive-wrap-autoloads olivetti-autoloads shackle trace
shackle-autoloads all-the-icons all-the-icons-faces data-material
data-weathericons data-octicons data-fileicons data-faicons
data-alltheicons all-the-icons-autoloads org ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete
pcomplete comint ansi-color ring org-list org-faces org-entities
noutline outline org-version ob-emacs-lisp ob-core ob-eval org-cycle
org-table oc-basic bibtex iso8601 time-date ol org-fold org-fold-core
org-keys oc org-compat advice org-macs org-loaddefs format-spec
find-func cal-menu calendar cal-loaddefs modus-vivendi-theme
modus-operandi-theme modus-themes modus-themes-autoloads gcmh
gcmh-autoloads use-package-diminish s s-autoloads ht dash ht-autoloads
dash-autoloads pcase asoc asoc.el-autoloads no-littering
no-littering-autoloads hydra-autoloads lv-autoloads finder-inf
use-package-bind-key org-contrib-autoloads comp comp-cstr warnings rx
bind-key easy-mmode diminish diminish-autoloads use-package-core
use-package-autoloads bind-key-autoloads straight-autoloads cl-extra
help-mode straight server site-gentoo helm-easymenu info package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars seq gv subr-x byte-opt bytecomp
byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting font-render-setting cairo x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 6986171 2237182)
 (symbols 48 104486 98)
 (strings 32 752636 136785)
 (string-bytes 1 25502273)
 (vectors 16 561998)
 (vector-slots 8 24848211 822896)
 (floats 8 65913 13311)
 (intervals 56 655756 11352)
 (buffers 992 175))

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Ihor Radchenko <yantar92@HIDDEN>
Subject: bug#51766: Acknowledgement (29.0.50; Return value of
 buffer-chars-modified-tick changes when buffer text is not yet changed
 before inserting a character for non-latin input methods)
Message-ID: <handler.51766.B.163663893416660.ack <at> debbugs.gnu.org>
References: <87mtmalrs1.fsf@localhost>
X-Gnu-PR-Message: ack 51766
X-Gnu-PR-Package: emacs
Reply-To: 51766 <at> debbugs.gnu.org
Date: Thu, 11 Nov 2021 13:56:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 51766 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
51766: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D51766
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 11 Nov 2021 15:20:03 +0000
Resent-Message-ID: <handler.51766.B51766.163664396727644 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Date: Thu, 11 Nov 2021 17:19:15 +0200
Message-Id: <837dde200c.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87mtmalrs1.fsf@localhost> (message from Ihor Radchenko on Thu,
 11 Nov 2021 21:56:46 +0800)
References: <87mtmalrs1.fsf@localhost>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

> From: Ihor Radchenko <yantar92@HIDDEN>
> Date: Thu, 11 Nov 2021 21:56:46 +0800
> 
> According to buffer-chars-modified-tick docstring:
> "By comparing the values returned by two individual calls of
> buffer-chars-modified-tick, you can tell whether a character change
> occurred in that buffer in between these calls"
> 
> However, the return value can change even when no visible change is made
> to buffer text.
> 
> Steps to reproduce:
> 1. emacs -Q
> 2. Evaluate the following code:
> 
> (defun print-tick-before ()
>   (when (eq this-command 'self-insert-command)
>   (warn "Tick before: %S" (buffer-chars-modified-tick))))
> (defun print-tick-after ()
>     (when (eq this-command 'self-insert-command)
>   (warn "Tick after: %S" (buffer-chars-modified-tick))))
> (add-hook 'pre-command-hook #'print-tick-before)
> (add-hook 'post-command-hook #'print-tick-after)
> 
> 3. Insert a latin symbol ?a twice. The warning buffer will print
> something like
> 
> Warning (emacs): Tick before: 1698 Disable showing Disable logging
> Warning (emacs): Tick after: 1699 Disable showing Disable logging
> Warning (emacs): Tick before: 1699 Disable showing Disable logging
> Warning (emacs): Tick after: 1702 Disable showing Disable logging
> 
> Note that second and third line show the same buffer-chars-modified-tick
> value.
> 
> 4. Change input method (C-\) to russian-computer or i.e. arabic
> 5. Insert a non-latin symbol ?ф twice. The warning buffer will print
> something like
> 
> Warning (emacs): Tick before: 1706 Disable showing Disable logging
> Warning (emacs): Tick after: 1707 Disable showing Disable logging
> Warning (emacs): Tick before: 1711 Disable showing Disable logging
> Warning (emacs): Tick after: 1712 Disable showing Disable logging
> 
> Note that second and third line _do not_ show the same
> buffer-chars-modified-tick value even though buffer text has not been
> changed between the two self-insert commands
> 
> Expected behaviour: return value of buffer-chars-modified-tick does not
> change when no changes in buffer text are made.

How do you know there was no changes in the buffer?  You call your
function from pre/post-command-hook, but why is it guaranteed that
there was no change in the buffer between post-command-hook and the
following pre-command-hook?

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 11 Nov 2021 15:50:02 +0000
Resent-Message-ID: <handler.51766.B51766.163664575731325 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <837dde200c.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
Date: Thu, 11 Nov 2021 23:50:31 +0800
Message-ID: <87k0helmig.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Eli Zaretskii <eliz@HIDDEN> writes:

> How do you know there was no changes in the buffer?  You call your
> function from pre/post-command-hook, but why is it guaranteed that
> there was no change in the buffer between post-command-hook and the
> following pre-command-hook?

I also tested the bug by setting debug-on-entry for self-insert-command.
Try the following:
1. emacs -Q
2. M-\ russian-computer <RET>
3. M-\
4. M-: (buffer-chars-modified-tick) <RET>. Note the return value.
5. M-x debug-on-entry <RET> self-insert-command <RET>
6. Insert ?a
7. The debugger window appears. ?a is not yet inserted
8. In the debugger window: e M-p <RET> (call
   buffer-chars-modified-tick). The return value should be the same with
   4.
9. Continue execution of self-insert-command in the debugger (c). The ?a
   is inserted
10. Repeat 8. The return value correctly changes. Note the return value.
11. Continue execution to exit the debugger window (c)
12. Switch to russian input method (M-\)
13. Run M-: M-p <RET>. Note the return value. It is same with 10.
14. Run M-: (buffer-hash) <RET>. "(buffer-hash)" should by yanked to
    avoid triggering debugger. Note that hash value.
13. Type ?=D1=84 (?a on qwerty keyboard)
14. The debugger appears again. ?=D1=84 is _not_ yet inserted
15. Repeat 8. The return value is different from 10 even though the
    buffer text is not changed (and it can be confirmed if you run
    e (buffer-hash) <RET>)

Of course, there might be some kind of invisible change in buffer. I.e.
text is added and immediately deleted from the buffer without redisplay.
However, even if there is any change like that, before-change-functions
and after-change-functions are not triggered. That would be another bug
then.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 11 Nov 2021 17:36:02 +0000
Resent-Message-ID: <handler.51766.B51766.163665212918779 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163665212918779
          (code B ref 51766); Thu, 11 Nov 2021 17:36:02 +0000
Received: (at 51766) by debbugs.gnu.org; 11 Nov 2021 17:35:29 +0000
Received: from localhost ([127.0.0.1]:42164 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlDyz-0004so-AN
	for submit <at> debbugs.gnu.org; Thu, 11 Nov 2021 12:35:29 -0500
Received: from eggs.gnu.org ([209.51.188.92]:47412)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mlDyx-0004sc-T8
 for 51766 <at> debbugs.gnu.org; Thu, 11 Nov 2021 12:35:28 -0500
Received: from [2001:470:142:3::e] (port=36244 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlDys-00088i-L7; Thu, 11 Nov 2021 12:35:22 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=EvnVey7rDE3FLlDCoKnPJbUds7w7RB2pPwmybu5mW0s=; b=Cv40xZO9MLFg
 mvdOjl4S3w7GYji13sscWyy7LdHR5yMopsd0+zbmXzmk3JF8DWbmql4fhQrGGtTgjmkfw9/pffsek
 DPLA2U83ewqitoOIJOAgrKDGlBru6L6oN1bm5Khc2uU7nLoXONZIWuF2epqLj2qRO85tn3geQEtr3
 TYf7kZxBkPO6KyhdegY/4zavqeOuj80SPqu5l+C9s44OvM/zd7vkNZvd/zHlpB5hkVWlv4+zSV+mF
 aAteu8QQE2zJA4CACqnexOAjUHhANQiuNLfgOSZ53Gc5Ox2i3pQc/NIto/ajWrYHfut500D7VSTP0
 Nirv1G07wTYqim2l8ERHKg==;
Received: from [87.69.77.57] (port=3166 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlDys-0000Rw-8h; Thu, 11 Nov 2021 12:35:22 -0500
Date: Thu, 11 Nov 2021 19:35:19 +0200
Message-Id: <831r3m1tpk.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87k0helmig.fsf@localhost> (message from Ihor Radchenko on Thu,
 11 Nov 2021 23:50:31 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Thu, 11 Nov 2021 23:50:31 +0800
> 
> Of course, there might be some kind of invisible change in buffer. I.e.
> text is added and immediately deleted from the buffer without redisplay.

That's exactly what happens: quail.el deletes the inserted character
and then reinserts it (for reasons unrelated to this issue).  So the
count of the changes is not equal to the number of characters actually
inserted.  I see no problem here, since the documentation never
promises that the difference between the values returned by successive
calls to buffer-chars-modified-tick will be exactly equal to the
number of inserted or deleted characters.

So if Org relies on such an equality, it's a bug in Org (but I didn't
look at the relevant Org code, and don't have a clear idea of how
exactly it uses the above function for whatever it is caching).

> However, even if there is any change like that, before-change-functions
> and after-change-functions are not triggered. That would be another bug
> then.

quail.el inhibit buffer modifications in places, since otherwise you'd
have too many of them.  It wants to pretend that just one character
was inserted.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 12 Nov 2021 12:06:02 +0000
Resent-Message-ID: <handler.51766.B51766.163671872415595 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163671872415595
          (code B ref 51766); Fri, 12 Nov 2021 12:06:02 +0000
Received: (at 51766) by debbugs.gnu.org; 12 Nov 2021 12:05:24 +0000
Received: from localhost ([127.0.0.1]:43636 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlVJ6-00043T-Dl
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 07:05:24 -0500
Received: from mail-pf1-f173.google.com ([209.85.210.173]:40641)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1mlVJ2-00043C-Ve
 for 51766 <at> debbugs.gnu.org; Fri, 12 Nov 2021 07:05:22 -0500
Received: by mail-pf1-f173.google.com with SMTP id z6so8329989pfe.7
 for <51766 <at> debbugs.gnu.org>; Fri, 12 Nov 2021 04:05:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=EPTKoQkcZG0yRXYwdkH9d8zasDGrAms7HxA9q2yMGO4=;
 b=kXXArPvUPqap/H4QEtTbXhUNqhTEE0kpqkSpklxYYu9D75G7Wlb+Bth9YBbdHx6nGX
 MvxgS8aRNXwTiex7T40INEBBP6pinS0uttTbNWrnj1ZpujENFdiepuNXUBplpsJAL2is
 lJFEjdioFYGG3NqqQrV9HXuh0X1F98i8RBG2tC2D3ae7xCz5wjxlh21SBgEdlTo2s++o
 wD5OM/oAN7BKe3xSUQ/3J4jj5vaf2h8Jdtq9LxJoPPZ/gJ1CEAAePAla3f0Axl5vVqeS
 +XcsYWQa8/Kyg1PISyIVQithSMz5SKxQGegxs2PAwJP+oQiG8A6Cuva2xv8AH2tweGFe
 HyNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=EPTKoQkcZG0yRXYwdkH9d8zasDGrAms7HxA9q2yMGO4=;
 b=5fYijQEMsvorphpHlfvzTncRlO74Tx6xE/uUJghXL9rkPcUNZs2aWs+6BwFdbfuIto
 ja6Oq885jhueHSquYJCtsMb292z5N+SsasK1hvwpJchFBmWYKLw4iR4tCD9pxxhHgPp7
 54Cad2GNfAZl4quUM5bqWOClFlQ+LtkZMbKqqLXAuOT9f3Ka0RLrb0Ak8dnogifdSgL7
 p2BSdCB72Fw2REjA9RcMK7okfU05oxlQESOtywy/igUYJtHZxSRb+cLWHgbkvkyR3vDZ
 eADSjbPLrwrgxla8b61lWAF4GB+OoNIygSDlzXybgxcgT7UROzEQsEJKljpQjXQG1MDb
 Ym6w==
X-Gm-Message-State: AOAM532nm6J0egpQsakhM43V8QHJ77UXphAC89JwX12npsYfSM0cj3Yp
 nODWIjrG4CH1BwuyAYLJxuQ=
X-Google-Smtp-Source: ABdhPJxf01ZkCdNusVyjf2pFq2dD8csBe5jw0KC5CoqMWH72cF+UorwDg+qmObuSK3p3GlZhelcDyg==
X-Received: by 2002:aa7:818d:0:b0:49f:e7d4:bb55 with SMTP id
 g13-20020aa7818d000000b0049fe7d4bb55mr13585969pfi.60.1636718714899; 
 Fri, 12 Nov 2021 04:05:14 -0800 (PST)
Received: from localhost ([103.125.234.210])
 by smtp.gmail.com with ESMTPSA id o4sm10740556pjq.23.2021.11.12.04.05.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 12 Nov 2021 04:05:14 -0800 (PST)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <831r3m1tpk.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
Date: Fri, 12 Nov 2021 20:06:41 +0800
Message-ID: <8735o1r31q.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

> That's exactly what happens: quail.el deletes the inserted character
> and then reinserts it (for reasons unrelated to this issue).  So the
> count of the changes is not equal to the number of characters actually
> inserted.  I see no problem here, since the documentation never
> promises that the difference between the values returned by successive
> calls to buffer-chars-modified-tick will be exactly equal to the
> number of inserted or deleted characters.

I agree that Emacs does not break any promises here. Though it is
unfortunate. Feel free to close this bug report.

> So if Org relies on such an equality, it's a bug in Org (but I didn't
> look at the relevant Org code, and don't have a clear idea of how
> exactly it uses the above function for whatever it is caching).

Let me explain a little (hoping that you might have some idea about
alternative solutions without using buffer-chars-modified-tick).

Org has a caching mechanism (org-element-cache) that keeps parsed buffer
representation in memory and updates it on the fly as the buffer
changes. To make the mechanism work, Org must keep track of all the
changes in buffer and update the affected Org elements in memory.
Naturally, this is done using before/after-change-functions.

However, some third-party code carelessly uses
inhibit-modification-hooks and some edits may be missed by element
cache. If we just ignore the possibility of such edits, cache can be
broken badly. So, there is currently a control code that detects if
buffer has been changed outside the Org's change functions. The control
code uses buffer-chars-modified-tick.

The behaviour of quail.el makes the control code useless -
buffer-chars-modified-tick can no longer be reliably used to detect
unfavourable "stealthy" changes. AFAIK, the only alternative way to
detect the changes is buffer-hash/secure-hash. But calculating hash is
very too slow when I try to put it into before/after-change-functions. I
do not know any fast (as fast as buffer-chars-modified-tick) way to
detect buffer changes.

> quail.el inhibit buffer modifications in places, since otherwise you'd
> have too many of them.  It wants to pretend that just one character
> was inserted.

I understand the idea behind suppressing the modification hooks by
quail. Though it would be helpful if before-change-functions were called
before inserting+deleting a character by quail is done.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 12 Nov 2021 12:17:02 +0000
Resent-Message-ID: <handler.51766.B51766.163671938316669 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163671938316669
          (code B ref 51766); Fri, 12 Nov 2021 12:17:02 +0000
Received: (at 51766) by debbugs.gnu.org; 12 Nov 2021 12:16:23 +0000
Received: from localhost ([127.0.0.1]:43651 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlVTj-0004Kn-Ac
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 07:16:23 -0500
Received: from eggs.gnu.org ([209.51.188.92]:49782)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mlVTf-0004KY-Id
 for 51766 <at> debbugs.gnu.org; Fri, 12 Nov 2021 07:16:22 -0500
Received: from [2001:470:142:3::e] (port=56896 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlVTa-0007V3-Cu; Fri, 12 Nov 2021 07:16:14 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=nZFkcptkVnDk4Tn9xgxCv2vMXTwo89/u99Ho6uEY96U=; b=kQTqOqW1I6o5
 nFLio8WsRRFVE8NVkP6oV4Q15tYAbaKJpMxjKfVufZZV/rLC92sF6JCazZyIrChvaw50NgYkNgipM
 2UcmqlUle5YCvlxSjfLkurvOB9x0shRlmmdgJGS4uXNlE1DtSbQzobCWvuW8qV5VjoXuk+M2hC66c
 HlEdolGDmtd6O59EmIDsKYYz7N0bwmGUJ3ffn6lofCq7ehgxf5OohshChdft0vOyJ65gU1k3gHiuT
 Fvh0ipOtlL3/YdGVAMxgSANz7fO9UjF69r/GXUUNKzfApDopABIpKSpDVE/EVI43wcdR+dzzwHmQu
 DSBzOy6O61VtW5tOGRjQcw==;
Received: from [87.69.77.57] (port=4339 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlVTZ-00027H-Vr; Fri, 12 Nov 2021 07:16:14 -0500
Date: Fri, 12 Nov 2021 14:15:55 +0200
Message-Id: <834k8hzi10.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <8735o1r31q.fsf@localhost> (message from Ihor Radchenko on Fri,
 12 Nov 2021 20:06:41 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN> <8735o1r31q.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Fri, 12 Nov 2021 20:06:41 +0800
> 
> Org has a caching mechanism (org-element-cache) that keeps parsed buffer
> representation in memory and updates it on the fly as the buffer
> changes. To make the mechanism work, Org must keep track of all the
> changes in buffer and update the affected Org elements in memory.
> Naturally, this is done using before/after-change-functions.
> 
> However, some third-party code carelessly uses
> inhibit-modification-hooks and some edits may be missed by element
> cache. If we just ignore the possibility of such edits, cache can be
> broken badly. So, there is currently a control code that detects if
> buffer has been changed outside the Org's change functions. The control
> code uses buffer-chars-modified-tick.
> 
> The behaviour of quail.el makes the control code useless -
> buffer-chars-modified-tick can no longer be reliably used to detect
> unfavourable "stealthy" changes.

This last part I don't think I understand: why does quail's behavior
make the control code useless?  The value returned by
buffer-chars-modified-tick still increases in your recipe, so what
exactly is the aspect of that behavior that makes the control code
useless?  I think some additional details here are missing from your
description which could explain the issue.

> > quail.el inhibit buffer modifications in places, since otherwise you'd
> > have too many of them.  It wants to pretend that just one character
> > was inserted.
> 
> I understand the idea behind suppressing the modification hooks by
> quail. Though it would be helpful if before-change-functions were called
> before inserting+deleting a character by quail is done.

I don't understand this, either.  Are you saying that inserting a
character via an input method doesn't call buffer-modification hooks
even once?  If the hooks are called, then what exactly is the problem
with the hooks in this scenario?

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 12 Nov 2021 12:53:01 +0000
Resent-Message-ID: <handler.51766.B51766.163672153628607 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163672153628607
          (code B ref 51766); Fri, 12 Nov 2021 12:53:01 +0000
Received: (at 51766) by debbugs.gnu.org; 12 Nov 2021 12:52:16 +0000
Received: from localhost ([127.0.0.1]:43687 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlW2R-0007RK-Mc
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 07:52:15 -0500
Received: from mail-pg1-f181.google.com ([209.85.215.181]:37848)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1mlW2P-0007R1-RL
 for 51766 <at> debbugs.gnu.org; Fri, 12 Nov 2021 07:52:14 -0500
Received: by mail-pg1-f181.google.com with SMTP id s136so7927294pgs.4
 for <51766 <at> debbugs.gnu.org>; Fri, 12 Nov 2021 04:52:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=A2+6SjEdYlJq/uBw0f/fNWvurn/U5yWHWoZYX+o+CLQ=;
 b=q03I8wzBfqodydkcErBbTsi9DMamocH9fzlQctusP0lUPwEAClU0HCUBNWqFcFSi1o
 fFjqyQPSQIieGneD/MbMkKeUbErUVG5OjZMgksnN3hLdtXZdHNfQTaGx0u0YRnYm2Yf/
 I07oBDNa37ZU8qUvxXu/FLx/JO7x4SQ/X2NaGNs+R2OXU3pZgorj/NmY7QsZgKJoBeys
 pbn/gUiWGIKvAwKAgsH50puxSgeV5eTLr3E+tH/9oun39NnjYEhPYNKBy+TDzixmmiD3
 l+RyFB9WlvQvoQwx7wRThnhcy2k2JrTxbFDq2RaONG24Op0GoEX0vXrGwS+PavIHQv43
 Peqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=A2+6SjEdYlJq/uBw0f/fNWvurn/U5yWHWoZYX+o+CLQ=;
 b=LIg8Oo2DNh57lwjeOAGMyn5mIxSR3GMCf1Oo40BqsM49YxmWljrLwVwRBsMpDj7f6G
 eLE3gcZTI7B3VRuaTNKCQEv40S0A4QscXhsf5mpjf5Q9TKfyjpuMtAC4RELqMY9Ra939
 iVCWSlxx7VM2uBZpW2Oc7WVe6ShSMnDch3oldgjGijDqedyjnk6tqpvBHSRMntgi1nvL
 /uLLjeWEs+Fiv+dU3uJA+QKbrsJGFo5o2br++M8AaZAvC/hJsl/pm9pL3J/ixWThNsyP
 B3TpQXNuBEe3/EF6wp3sRSVW70vWmKGW1MrsmalSsWimqUVu/9IX5cqQSGyG1qEDupxk
 I0TA==
X-Gm-Message-State: AOAM533p+OslUAhp0EU2fn/SxpCboSOY6MduXUr9TWCoOHNG0mKztPbF
 72S+URsBIb1jsLR0jVxqdLVNYwDT+O6Q1Q==
X-Google-Smtp-Source: ABdhPJyVXDZdrL8UpJCEKXm80Mudy5b+dd3NaDBP9MLIQXLhG0Kp9RrLj0qBKXkgVCUDwlJxqN7VhQ==
X-Received: by 2002:a05:6a00:c95:b0:49f:c8de:9ae7 with SMTP id
 a21-20020a056a000c9500b0049fc8de9ae7mr13827754pfv.30.1636721527873; 
 Fri, 12 Nov 2021 04:52:07 -0800 (PST)
Received: from localhost ([103.125.234.210])
 by smtp.gmail.com with ESMTPSA id f2sm6838086pfe.132.2021.11.12.04.52.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 12 Nov 2021 04:52:07 -0800 (PST)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <834k8hzi10.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
Date: Fri, 12 Nov 2021 20:53:33 +0800
Message-ID: <87zgq9pmb6.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

> This last part I don't think I understand: why does quail's behavior
> make the control code useless?  The value returned by
> buffer-chars-modified-tick still increases in your recipe, so what
> exactly is the aspect of that behavior that makes the control code
> useless?  I think some additional details here are missing from your
> description which could explain the issue.

The control code makes sure that all the changes made in buffer are
processed by org-element-cache. It means that
org-element--after-change-function saves the buffer-chars-modified-tick
and the next org-element--before-change-function checks if the saved
value is unchanged. If the saved value is changed, the buffer has been
changed after org-element--after-change-function, but before next
org-element--before-change-function. Such change may be arbitrary and
the whole cache is potentially obsolete.

In code, the described roughly looks like:

(defun org-element--after-change-function (...)
 (setq org-element-chars-modified-tick (buffer-chars-modified-tick))
 (org-element-cache-submit-request ...))

(defun org-element--before-change-function (...)
 (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick))
     ;; Buffer has been changed without calling after-change-function
     ;; and we have no way to determine which part of buffer has been changed.
 ))

quail changes the buffer after org-element--after-change-function call,
but before org-element--before-change-function. So, all Org can see is
that something has been changed in buffer, but there is no way to tell
what it was. Org cannot distinguish between harmless buffer edits by
quail (they do not change buffer text) and other kinds of "silent"
changes.

> I don't understand this, either.  Are you saying that inserting a
> character via an input method doesn't call buffer-modification hooks
> even once?  If the hooks are called, then what exactly is the problem
> with the hooks in this scenario?

The hooks are called, but after quail already triggered
buffer-chars-modified-tick increase. If quail called
before-change-functions before buffer-chars-modified-tick increases, it
would be useful for my scenario. Though I am not sure how feasible it
is. Just an idea.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 12 Nov 2021 13:10:01 +0000
Resent-Message-ID: <handler.51766.B51766.163672258430265 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163672258430265
          (code B ref 51766); Fri, 12 Nov 2021 13:10:01 +0000
Received: (at 51766) by debbugs.gnu.org; 12 Nov 2021 13:09:44 +0000
Received: from localhost ([127.0.0.1]:43703 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlWJL-0007s5-QT
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 08:09:44 -0500
Received: from eggs.gnu.org ([209.51.188.92]:60526)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mlWJG-0007ro-Ss
 for 51766 <at> debbugs.gnu.org; Fri, 12 Nov 2021 08:09:42 -0500
Received: from [2001:470:142:3::e] (port=35816 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlWJB-00072v-N4; Fri, 12 Nov 2021 08:09:33 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=e9ieklnT1NTlqSL/3owBfQ857XsAr7AnK5zYn+pJ8ao=; b=fth8AwWUZnDb
 90ImWK4OxucVCUqoQecPI/Q+2kQBMSqUETSmGWGifcf3HYrRGgRSX7jzszcwmSN810i971+UB/HXm
 Rp5bDB4Tmh/UG2h1kyS6DHYe/dnKP20aqnqOrv7LyuAO2zrJb2g09vMx2HUWh3qLTnW2UBfAPyIIy
 g0m7sI/hk/jZN6AmWB443Hy2y0MKHLLmEkxFx16OSVkZXkuwEW8WL7Y7Jpye43zR8bwpNUk6RH+09
 /WHpLPZNPWk9sWqRMjBTXQPT62jtl7tzKubKhzj+whYzptU3eQN5E3+42dRF81g5BlcugRYVAvPCF
 gH2qLWQyfaKKJ9D25YKfzw==;
Received: from [87.69.77.57] (port=3719 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlWJB-0005Zn-BJ; Fri, 12 Nov 2021 08:09:33 -0500
Date: Fri, 12 Nov 2021 15:09:15 +0200
Message-Id: <831r3lzfk4.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87zgq9pmb6.fsf@localhost> (message from Ihor Radchenko on Fri,
 12 Nov 2021 20:53:33 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN> <87zgq9pmb6.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Fri, 12 Nov 2021 20:53:33 +0800
> 
> The control code makes sure that all the changes made in buffer are
> processed by org-element-cache. It means that
> org-element--after-change-function saves the buffer-chars-modified-tick
> and the next org-element--before-change-function checks if the saved
> value is unchanged. If the saved value is changed, the buffer has been
> changed after org-element--after-change-function, but before next
> org-element--before-change-function. Such change may be arbitrary and
> the whole cache is potentially obsolete.
> 
> In code, the described roughly looks like:
> 
> (defun org-element--after-change-function (...)
>  (setq org-element-chars-modified-tick (buffer-chars-modified-tick))
>  (org-element-cache-submit-request ...))
> 
> (defun org-element--before-change-function (...)
>  (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick))
>      ;; Buffer has been changed without calling after-change-function
>      ;; and we have no way to determine which part of buffer has been changed.
>  ))
> 
> quail changes the buffer after org-element--after-change-function call,
> but before org-element--before-change-function. So, all Org can see is
> that something has been changed in buffer, but there is no way to tell
> what it was. Org cannot distinguish between harmless buffer edits by
> quail (they do not change buffer text) and other kinds of "silent"
> changes.

OK, but why does this invalidate what Org does?  All it means, AFAIU,
is that in some cases Org will do unnecessary processing.  Those cases
are probably not too frequent.

IOW, why invalidating the cache unnecessarily is such a big deal?

> > I don't understand this, either.  Are you saying that inserting a
> > character via an input method doesn't call buffer-modification hooks
> > even once?  If the hooks are called, then what exactly is the problem
> > with the hooks in this scenario?
> 
> The hooks are called, but after quail already triggered
> buffer-chars-modified-tick increase. If quail called
> before-change-functions before buffer-chars-modified-tick increases, it
> would be useful for my scenario. Though I am not sure how feasible it
> is. Just an idea.

Would it help if Org looked at both buffer-modified-tick and
buffer-chars-modified-tick?

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 12 Nov 2021 13:39:01 +0000
Resent-Message-ID: <handler.51766.B51766.1636724318666 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.1636724318666
          (code B ref 51766); Fri, 12 Nov 2021 13:39:01 +0000
Received: (at 51766) by debbugs.gnu.org; 12 Nov 2021 13:38:38 +0000
Received: from localhost ([127.0.0.1]:43736 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlWlK-0000Ae-Fw
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 08:38:38 -0500
Received: from mail-pj1-f41.google.com ([209.85.216.41]:43607)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1mlWlG-0000AO-Ke
 for 51766 <at> debbugs.gnu.org; Fri, 12 Nov 2021 08:38:36 -0500
Received: by mail-pj1-f41.google.com with SMTP id
 nh10-20020a17090b364a00b001a69adad5ebso7628376pjb.2
 for <51766 <at> debbugs.gnu.org>; Fri, 12 Nov 2021 05:38:34 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=umJu5FDJ8nHJ5OwO9s7tBV+wo2LZ4I0hepzLH3F0ziY=;
 b=qKo5KT3UeR3SKaZgpc/9U01epP+IB6gQts5nMA7aWJv1JcfADIUi1/wRu5p910s+7A
 JjclTarJKx4tlWJVrw/4KkGjRZq/PcDQ79CjOZG+FEtwoO2mqiNn2FdD1dhZfSfGXXW1
 70Y3PnBfaJ7w9QhQZlcb+9uReeWBPFZ1ToXkFUmZGy0m9UY4lalvo4T+Q25jJzrRrrF9
 WfI4RWP/7kjXudEo3R8p00/xXCeyCIXH8+1HMpfc8NVnA9UnpkhjCdDa10CKz8uqU9z7
 4oef0eBZ4WDLUN4Gid2ENNhiX/QEpLJgbyvm0ujjDhwWBCYeVDDViEKbdt/BJGsgKIPx
 uOyw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=umJu5FDJ8nHJ5OwO9s7tBV+wo2LZ4I0hepzLH3F0ziY=;
 b=2nvTxBF2AQznOfjeFCYZf2z6XxAM2VwnL2h6hqX4k/F653zWM8bJTYygup1lCLCNnj
 A2l6ExCv9LxOEY+ZjhJK+pjG4cbO3LjWzMe8DHO2osdVUHXRpO0bTaDDS2PQQhn9qzX9
 VomNlttgnpLhUfzMAneTWbbZYBdTE/OUzUxyXChzsU7+eWm1KvsfWUfg7P+4YAq2jWDp
 faKdkPmWm6oCOg+7IvDl+q/rdXIBhsB4xd+Iwtwkm7kf84XQyNhEpQ8uS8jl/2d/COfJ
 bfPgERFzhhvOjfxf+olHt6N6/JI9LffCsZKxs9+38ImMjrTuQmxfUNmMsLyWdIqaIsk7
 epJg==
X-Gm-Message-State: AOAM532AYEwY5/KmDKLhQ5FTHuK2TODLMmMP5cm1C8gajmKI+VRaTBgW
 AoP/R4KLpiCyjAS1xHhOqyCIRDXObqzebQ==
X-Google-Smtp-Source: ABdhPJxVAmUI9ts9tcCPHzB/5z+EWxgSWefKp/HagAbYufl0y7CoEBrabC+XPU9ox9ObJVuLMAO4mQ==
X-Received: by 2002:a17:90b:4d09:: with SMTP id
 mw9mr17928459pjb.238.1636724308569; 
 Fri, 12 Nov 2021 05:38:28 -0800 (PST)
Received: from localhost ([103.125.234.210])
 by smtp.gmail.com with ESMTPSA id o134sm6437366pfg.1.2021.11.12.05.38.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 12 Nov 2021 05:38:27 -0800 (PST)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <831r3lzfk4.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
Date: Fri, 12 Nov 2021 21:39:54 +0800
Message-ID: <87wnldpk5x.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> quail changes the buffer after org-element--after-change-function call,
>> but before org-element--before-change-function. So, all Org can see is
>> that something has been changed in buffer, but there is no way to tell
>> what it was. Org cannot distinguish between harmless buffer edits by
>> quail (they do not change buffer text) and other kinds of "silent"
>> changes.
>
> OK, but why does this invalidate what Org does?  All it means, AFAIU,
> is that in some cases Org will do unnecessary processing.  Those cases
> are probably not too frequent.

Normally, buffer changes under inhibit-modification-hooks are not
frequent indeed. But not with quail + non-latin input methods. Every
single self-insert-command triggers such change.

> IOW, why invalidating the cache unnecessarily is such a big deal?

It is critical for Org to know which part of buffer was changed (i.e.
beg, end, length that are normally passed as arguments of
after-change-functions). org-element-cache can contain >100k elements
for especially large buffers. Manually checking which elements are
changed without knowing the changed region is inefficient. Clearing the
cache is not too much of a big deal, but causes slowdown when user runs
a query on Org buffers (i.e. in agenda or sparse trees) - the buffer has
to be re-parsed. When every edit triggers cache invalidation (that's
what happens when user uses non-latin input method), the slowdown is
pretty much guaranteed. Moreover, too frequent cache resets increase the
load on Emacs' garbage collector (cache size is typically a multiple of
buffer size). Overloading garbage collector leads to overall Emacs
slowdown.

>> The hooks are called, but after quail already triggered
>> buffer-chars-modified-tick increase. If quail called
>> before-change-functions before buffer-chars-modified-tick increases, it
>> would be useful for my scenario. Though I am not sure how feasible it
>> is. Just an idea.
>
> Would it help if Org looked at both buffer-modified-tick and
> buffer-chars-modified-tick?

When buffer-chars-modified-tick is changed, buffer-modified-tick is also
changed. AFAIU, buffer-chars-modified-tick registers a subset of buffer
modifications that actually change buffer text. buffer-modified-tick
also registers text property changes. So, I do not see how
buffer-modified-tick can help.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 12 Nov 2021 15:18:01 +0000
Resent-Message-ID: <handler.51766.B51766.163673027420692 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163673027420692
          (code B ref 51766); Fri, 12 Nov 2021 15:18:01 +0000
Received: (at 51766) by debbugs.gnu.org; 12 Nov 2021 15:17:54 +0000
Received: from localhost ([127.0.0.1]:45752 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlYJO-0005Nf-Bs
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 10:17:54 -0500
Received: from eggs.gnu.org ([209.51.188.92]:58218)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mlYJN-0005NT-0b
 for 51766 <at> debbugs.gnu.org; Fri, 12 Nov 2021 10:17:53 -0500
Received: from [2001:470:142:3::e] (port=45628 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlYJH-0008BL-Qz; Fri, 12 Nov 2021 10:17:47 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=aSAE/fiXcKXzsh7Q4lNNCSZWFwJG61hGjg6arXuKpvc=; b=bBwNyC+Zp9so
 7ilTdjpKcYTI6O57JFQ8JjEiONZktB/MjhJQNh/SN2LmjGwCJUrYymCAprq5uvYhMMnEQt6pzbc1S
 ubG+tS50/ZKVSPQYmH6ex8/9bC7/i9SWC9qMDIfwp3LIjNLfj+7byVF3L0tTLdEFKIg9Zc9aoYVxv
 Lcj5psesbVHXzW0VizouZJexeEJ0itA8THgBVyiu8Xvhh+6I0PEPEP/lePRe4psJv8cx6WwsJ24Kt
 FrmJmLmgDE3X4mkJOaYBFqY7oBWqi8pkQpB/EGjESuvQS+EWd4O/KxdeFhfO1QpSrC5Nj0oILgr/l
 SBwpqeL8tp7ZKDfT9cI4YA==;
Received: from [87.69.77.57] (port=3895 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlYJH-0006zH-FS; Fri, 12 Nov 2021 10:17:47 -0500
Date: Fri, 12 Nov 2021 17:17:29 +0200
Message-Id: <83zgq9xv1y.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87wnldpk5x.fsf@localhost> (message from Ihor Radchenko on Fri,
 12 Nov 2021 21:39:54 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN> <87wnldpk5x.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Fri, 12 Nov 2021 21:39:54 +0800
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> >> quail changes the buffer after org-element--after-change-function call,
> >> but before org-element--before-change-function. So, all Org can see is
> >> that something has been changed in buffer, but there is no way to tell
> >> what it was. Org cannot distinguish between harmless buffer edits by
> >> quail (they do not change buffer text) and other kinds of "silent"
> >> changes.
> >
> > OK, but why does this invalidate what Org does?  All it means, AFAIU,
> > is that in some cases Org will do unnecessary processing.  Those cases
> > are probably not too frequent.
> 
> Normally, buffer changes under inhibit-modification-hooks are not
> frequent indeed. But not with quail + non-latin input methods. Every
> single self-insert-command triggers such change.

"Such change" being what exactly? the situation where
buffer-chars-modified-tick changes between post-command-hook and the
following pre-command-hook? or something else?

I'm asking because I don't think I see the problem you are describing.
With the following code:

  (defun my-before (beg end)
    (message "buf %s beg %s end %s" (current-buffer) beg end))
  (defun my-after (beg end len)
    (message "buf %s beg %s end %s len %s" (current-buffer) beg end len))
  (add-hook 'before-change-functions 'my-before)
  (add-hook 'after-change-functions 'my-after)

if I activate the chinese-py input method, then inserting any
character via the input method produces a single call to my-before and
a single call to my-after, and with the expected values, for example:

  buf *scratch* beg 440 end 440
  buf *scratch* beg 440 end 441 len 0

So what exactly is the problem with these hooks when non-latin input
methods are used?  Or what am I missing?

> > IOW, why invalidating the cache unnecessarily is such a big deal?
> 
> It is critical for Org to know which part of buffer was changed (i.e.
> beg, end, length that are normally passed as arguments of
> after-change-functions). org-element-cache can contain >100k elements
> for especially large buffers. Manually checking which elements are
> changed without knowing the changed region is inefficient. Clearing the
> cache is not too much of a big deal, but causes slowdown when user runs
> a query on Org buffers (i.e. in agenda or sparse trees) - the buffer has
> to be re-parsed. When every edit triggers cache invalidation (that's
> what happens when user uses non-latin input method), the slowdown is
> pretty much guaranteed. Moreover, too frequent cache resets increase the
> load on Emacs' garbage collector (cache size is typically a multiple of
> buffer size). Overloading garbage collector leads to overall Emacs
> slowdown.

Perhaps Org developers should ask for infrastructure changes that will
allow Org to maintain such a cache reliably and not too expensively?
It sounds like Org currently applies all kinds of heuristics based on
assumptions about how the internals work and using hooks and features
that were never designed to support this kind of caching.  Jumping
through hoops in Lisp trying to implement something that might be much
easier or even trivial in C is not the best way of getting such jobs
done.

So perhaps someone could describe on emacs-devel what does Org need to
maintain this cache, and we could then see how to provide those
features to Org.

> >> The hooks are called, but after quail already triggered
> >> buffer-chars-modified-tick increase. If quail called
> >> before-change-functions before buffer-chars-modified-tick increases, it
> >> would be useful for my scenario. Though I am not sure how feasible it
> >> is. Just an idea.
> >
> > Would it help if Org looked at both buffer-modified-tick and
> > buffer-chars-modified-tick?
> 
> When buffer-chars-modified-tick is changed, buffer-modified-tick is also
> changed. AFAIU, buffer-chars-modified-tick registers a subset of buffer
> modifications that actually change buffer text. buffer-modified-tick
> also registers text property changes. So, I do not see how
> buffer-modified-tick can help.

If you look at the implementation, you will see that when Emacs
decides that the buffer's chars-modified-tick needs to be increased,
it simply assigns to it the value of the buffer's modified-tick at
that moment.  So by tracking the value of buffer-modified-tick you
could perhaps explain why buffer-chars-modified-tick jumps by more
than you expected.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 13 Nov 2021 09:09:02 +0000
Resent-Message-ID: <handler.51766.B51766.163679454031116 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163679454031116
          (code B ref 51766); Sat, 13 Nov 2021 09:09:02 +0000
Received: (at 51766) by debbugs.gnu.org; 13 Nov 2021 09:09:00 +0000
Received: from localhost ([127.0.0.1]:46473 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlp1u-00085k-09
	for submit <at> debbugs.gnu.org; Sat, 13 Nov 2021 04:09:00 -0500
Received: from mail-pl1-f175.google.com ([209.85.214.175]:39873)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1mlp1o-00085P-Vh
 for 51766 <at> debbugs.gnu.org; Sat, 13 Nov 2021 04:08:56 -0500
Received: by mail-pl1-f175.google.com with SMTP id t21so10336266plr.6
 for <51766 <at> debbugs.gnu.org>; Sat, 13 Nov 2021 01:08:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=ryFEtk06tIoYlj7YUOeH2p4EIZpUgJ8JMDi/glZtwzY=;
 b=ArmevQfpd5GusSsln4ZQyNOMoXbeCVl3yh6PiUjdHMkPMTDBqwQkqOqxmRgJL2SUmZ
 OugAKMA4gHghlb3oCmrTSKaErmIill/qdQAoLbvB6rJiK4/jytSixUBNoVFwrnmg+7Hi
 hnT2VAEbJib5lA/LYURepJQgHbkJIwLAe97ui+9jlpcxSKo9ZGNtzPQq7KCqo4NTFflQ
 8XP2k8OV5A8N/HGRsOR8DO3oCZC5so9jWql9kan3No4wcL6USAwj7nvxwkyLFdY2BLor
 n1m0JJRxqvJsOj0gjIp669CFY+oIF/HDRyxJbOR7rjrXe/Eftc8lPJOYUNdQFZN4rCKi
 5SBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=ryFEtk06tIoYlj7YUOeH2p4EIZpUgJ8JMDi/glZtwzY=;
 b=h7klrWl0ZinuECuiVRIFGr203uOh7h8NPBfmKEV6cGn10uOmT4MKOAyonl/Gf+irgh
 KI6Uq+EgFAxIS09VZVI1aeuqrM8GMZPKXjpJJtYBnKgK4ioKqphHUvDgDsTHyugwhN2T
 FqRpI4rseYpsIsCGT32WjWaXzmud2GyqUCT1OZAFwANl2jP01GsmbGlmBWO83BYWz2kg
 EHEUbbJnx9fZIYUjXPseTz0MamOZp9f6P7FMGL9Q/yhKsXjSAIVut9C7lImTdwPK8E/I
 ne6qUmdqoC8cupkeEyLKiXV8JFV9e54qHmERL6dKqUDprugwlFKLQZMZhqhCQ8OtCxVG
 DXUA==
X-Gm-Message-State: AOAM531EGWte71CDuise3Qs3zg1T+PbQlWp/IdUL1uZmpFQF36DPNI1H
 5Q4+tYZ4uqSbR10oeRoGhhIRb/cq46Gn8Q==
X-Google-Smtp-Source: ABdhPJzs/haJz/SQvdEUA2YeQl0UySOMRogdgR1VhoQ2e+J8gX2Mw2DBJXpB9O2rsH5IvVV0XdfR+w==
X-Received: by 2002:a17:903:245:b0:13f:7872:9382 with SMTP id
 j5-20020a170903024500b0013f78729382mr15275395plh.26.1636794527181; 
 Sat, 13 Nov 2021 01:08:47 -0800 (PST)
Received: from localhost ([103.125.234.210])
 by smtp.gmail.com with ESMTPSA id e10sm10030308pfv.140.2021.11.13.01.08.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 13 Nov 2021 01:08:46 -0800 (PST)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <83zgq9xv1y.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
Date: Sat, 13 Nov 2021 17:10:11 +0800
Message-ID: <87r1bkpgjw.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

> "Such change" being what exactly? the situation where
> buffer-chars-modified-tick changes between post-command-hook and the
> following pre-command-hook? or something else?

The former.

> So what exactly is the problem with these hooks when non-latin input
> methods are used?  Or what am I missing?

There is no problem with the hooks in your example. However, consider
the following:

(let ((inhibit-modification-hooks t))
  (insert "Insertion that will never trigger before/after-change-functions"))

Org cache is bound to track all the changes in buffer. Any missed change
will lead to cache corruption. So, situations like the above must be
tracked somehow. This tracking can be done using
buffer-chars-modified-tick or buffer-hash/secure-hash. The latter is too
slow.

If I understand your earlier explanation correctly, quail for non-latin
input (I tested with russian-computer) does something like

(let ((inhibit-modification-hooks t))
  (insert ?char)
  (backward-delete-char)) ;; This increases buffer-chars-modified-tick
(insert ?translated_char_according_to_input_method)

The change hooks will only be called for the last insertion. However,
the first insertion+deletion will change buffer-chars-modified-tick.

The quail's insertion+deletion itself is not a problem for Org cache -
it does not really alter the buffer text and cannot break the cache. The
problem is that it cannot be distinguished from the first example - both
cases will trigger buffer-chars-modified-tick increase.

> Perhaps Org developers should ask for infrastructure changes that will
> allow Org to maintain such a cache reliably and not too expensively?
> It sounds like Org currently applies all kinds of heuristics based on
> assumptions about how the internals work and using hooks and features
> that were never designed to support this kind of caching.  Jumping
> through hoops in Lisp trying to implement something that might be much
> easier or even trivial in C is not the best way of getting such jobs
> done.
>
> So perhaps someone could describe on emacs-devel what does Org need to
> maintain this cache, and we could then see how to provide those
> features to Org.

I am one of the Org developers.

The only assumption I had it that Emacs does not frequently change
buffer text without triggering modification hooks. Clearly, the
assumption was wrong.

Ideally, a way to track _all_ buffer modifications regardless of
inhibit-modification-hooks would be useful. Currently, Org has to work
around the possibilities that text can be inserted without triggering
modification hooks: (1) when
inhibit-modification-hooks/before-change-functions/after-change-functions
are let-bound to nil; (2) when changes are made in indirect buffer with
different buffer-local values of before/after-change-functions.

Alternatively, Emacs could support language parsers. Org cache
implements editing syntax tree generated by Org element parser. It is
very similar to what tree-sitter editing API does: https://tree-sitter.github.io/tree-sitter/using-parsers#editing

Native support for storing, modifying, and querying syntax trees using
efficient data structures could be a great addition to Emacs from Org's
perspective. Though it is not an easy feature to implement.

AFAIR, something similar to my last suggestion has been already
proposed: tree-sitter support. I can also propose the first idea about
reliable buffer change tracking if you think that it is something
reasonable.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 13 Nov 2021 10:12:02 +0000
Resent-Message-ID: <handler.51766.B51766.16367983185348 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.16367983185348
          (code B ref 51766); Sat, 13 Nov 2021 10:12:02 +0000
Received: (at 51766) by debbugs.gnu.org; 13 Nov 2021 10:11:58 +0000
Received: from localhost ([127.0.0.1]:46577 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlq0r-0001OC-N0
	for submit <at> debbugs.gnu.org; Sat, 13 Nov 2021 05:11:58 -0500
Received: from eggs.gnu.org ([209.51.188.92]:58592)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mlq0m-0001Nw-KS
 for 51766 <at> debbugs.gnu.org; Sat, 13 Nov 2021 05:11:55 -0500
Received: from [2001:470:142:3::e] (port=46954 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlq0h-0004AN-EZ; Sat, 13 Nov 2021 05:11:47 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=L+XgwkivAdSwZxNO5aCOYBze1zcwo7bFJOxaZxXWQy4=; b=YPmwQBRVV/Jb
 ghnCCIx9GC3Z63WYzM6eoX5FEbr87VbexQPgNh2xLybECfdtEKqBJe3yP9lmLNXE5yB/TyJHD3Coy
 BhhMykNP8FCppKbHTw9JMtIx3sZaXcvVdRQ6CMlM/srgCHpocfqkdUores/ncvxKmGOJzASW0mINR
 0h4x2mIyGK9EEP8Wjr88dTu4lk85aagq2tlvmep+LmhCBpQkKvuSn3K8qiWBEXkBK/PUdGj9qBeW4
 vdHbXq8HNj83JrNV77ROOXZiagHqJLADDFRGe5aa/89gfjOOipgujF+3awIMFGEjM5Mg5ZUInDa/b
 GjLAxIEslxCVR9KcAC1B5g==;
Received: from [87.69.77.57] (port=1914 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mlq0h-0005Lz-2c; Sat, 13 Nov 2021 05:11:47 -0500
Date: Sat, 13 Nov 2021 12:11:31 +0200
Message-Id: <83pmr4xt4c.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87r1bkpgjw.fsf@localhost> (message from Ihor Radchenko on Sat,
 13 Nov 2021 17:10:11 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN> <87r1bkpgjw.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Sat, 13 Nov 2021 17:10:11 +0800
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > "Such change" being what exactly? the situation where
> > buffer-chars-modified-tick changes between post-command-hook and the
> > following pre-command-hook? or something else?
> 
> The former.
> 
> > So what exactly is the problem with these hooks when non-latin input
> > methods are used?  Or what am I missing?
> 
> There is no problem with the hooks in your example. However, consider
> the following:
> 
> (let ((inhibit-modification-hooks t))
>   (insert "Insertion that will never trigger before/after-change-functions"))

So the problem is that Org uses the modification hooks as the primary
mechanism (with which quail presents no problem), and
buffer-chars-modified-tick as the fallback, and that when some code
inhibits the modification hooks, then the primary mechanism cannot
work and quail breaks the fallback?

> The quail's insertion+deletion itself is not a problem for Org cache -
> it does not really alter the buffer text and cannot break the cache. The
> problem is that it cannot be distinguished from the first example - both
> cases will trigger buffer-chars-modified-tick increase.

You didn't answer my question regarding buffer-modified-tick: it can
explain to Org why buffer-chars-modified-tick jumped unexpectedly, and
thus (hopefully) resolve this situation.  If that helps, you could
perhaps turn the table and use buffer-chars-modified-tick is the
primary method of discovering changes, not as fallback.

> > Perhaps Org developers should ask for infrastructure changes that will
> > allow Org to maintain such a cache reliably and not too expensively?
> > It sounds like Org currently applies all kinds of heuristics based on
> > assumptions about how the internals work and using hooks and features
> > that were never designed to support this kind of caching.  Jumping
> > through hoops in Lisp trying to implement something that might be much
> > easier or even trivial in C is not the best way of getting such jobs
> > done.
> >
> > So perhaps someone could describe on emacs-devel what does Org need to
> > maintain this cache, and we could then see how to provide those
> > features to Org.
> 
> I am one of the Org developers.
> 
> The only assumption I had it that Emacs does not frequently change
> buffer text without triggering modification hooks. Clearly, the
> assumption was wrong.

I meant the assumptions about what buffer-chars-modified-tick does and
what its value means.

> Ideally, a way to track _all_ buffer modifications regardless of
> inhibit-modification-hooks would be useful.

But Org is not interested in just any moidification, AFAIU.  It is
only interested in modifications that change the buffer text.  Isn't
that true?  Or what else is Org interested in for this purpose.

> Alternatively, Emacs could support language parsers.

I meant support on the low level, where changes to buffer text are
considered and indicated.  As I indicate below, the integration of
tree-sitter simply uses the existing change indications, so I'm not
sure how would a parser support help you in this matter.

> Org cache
> implements editing syntax tree generated by Org element parser. It is
> very similar to what tree-sitter editing API does: https://tree-sitter.github.io/tree-sitter/using-parsers#editing
> 
> Native support for storing, modifying, and querying syntax trees using
> efficient data structures could be a great addition to Emacs from Org's
> perspective. Though it is not an easy feature to implement.
> 
> AFAIR, something similar to my last suggestion has been already
> proposed: tree-sitter support. I can also propose the first idea about
> reliable buffer change tracking if you think that it is something
> reasonable.

The Emacs code related to tree-sitter already uses change indications,
and AFAIR didn't require any changes to the existing infrastructure.
I wonder why Org cannot settle with what we have, if your needs are
similar enough to those of tree-sitter.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 13 Nov 2021 11:29:01 +0000
Resent-Message-ID: <handler.51766.B51766.163680290114443 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163680290114443
          (code B ref 51766); Sat, 13 Nov 2021 11:29:01 +0000
Received: (at 51766) by debbugs.gnu.org; 13 Nov 2021 11:28:21 +0000
Received: from localhost ([127.0.0.1]:46664 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlrCn-0003ks-9Y
	for submit <at> debbugs.gnu.org; Sat, 13 Nov 2021 06:28:21 -0500
Received: from mail-pj1-f52.google.com ([209.85.216.52]:44992)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1mlrCk-0003kc-0x
 for 51766 <at> debbugs.gnu.org; Sat, 13 Nov 2021 06:28:19 -0500
Received: by mail-pj1-f52.google.com with SMTP id
 o6-20020a17090a0a0600b001a64b9a11aeso9757983pjo.3
 for <51766 <at> debbugs.gnu.org>; Sat, 13 Nov 2021 03:28:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=EFMmPOLGS3u78zRooj/cHCyM7/X8b3yHHGC9AntuNYI=;
 b=NGDpdaQhACLelAOjBwSUDFIeqWfEBaCfeKomG4DgRYh7Ck31DSzsiDuemfhKUE79uv
 2eXWnLu2NrQ6mrsI6asXxy6oIMwifd+TzuHgu79hCf+Bg+ym2rbu1nrHInqmtEZ89OfV
 Wecst+yiDPEfZhh9hh7viUy3ijjszgZHmE+772i7mR608AQYhnp3ZyY5Kmfg5DhnJSq6
 EeuXIuX0W7R5UGNwDUPEj7V4/Hi8awBj1DEbXiL+rWDHL+VDHRh8nk60Sjvm3vW8E74C
 1p8DkU5uHY6utWZdwg5JS9IBrF4iACeHm9gyOQhKfp0ROEuYkUGfI2xOSPkSKS4Vb6gc
 xN+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=EFMmPOLGS3u78zRooj/cHCyM7/X8b3yHHGC9AntuNYI=;
 b=v/5PfPkP1FYjsVhm+1sGDw0RYHH1BygLxNBLPEJVLz1tHcSe7Q/Q7IgK/i/F9BGOXB
 HN6j9VPYY+zCtr8ux4SIL2PfAFlTBjZc4DEu+ena+gS7I9IPsXDEJ4Qd453h+M5lpOP/
 OliPiguspGz+E7wP072gDni+b6bvbJTjd54K/JqPhH9YJW7V5TJJRDvhydgEM65ImJhM
 xlrlkRStz1KUAmVw68Tj6Ad4/9j5/f7F4XDYtjjjbPmHuUvquGZikrE5bkNNgweQFNV1
 lf3iaZd1oQRq5oL50ACUq5a1+USKzM8NAXcRE1q0Hnf5B2PWMDDzDTcmk1o7IrYAeMcx
 R4ng==
X-Gm-Message-State: AOAM531uP9EMb27YF7oqKR0MDX53NQw276wnz4i29sKXgc5DfgJ2Bs/a
 kt/tjq1tsKRX6BoWmgzYCjVjRpxmMa07qA==
X-Google-Smtp-Source: ABdhPJxiwshJNMEc4qKho0DvkJikcmL8CSTnYoNMHqdn/lASnV9ZGvD3oHTbXnuMMB6/2sw9ol+2DQ==
X-Received: by 2002:a17:90b:4c85:: with SMTP id
 my5mr26020772pjb.26.1636802892115; 
 Sat, 13 Nov 2021 03:28:12 -0800 (PST)
Received: from localhost ([103.125.234.210])
 by smtp.gmail.com with ESMTPSA id i5sm7204008pgo.36.2021.11.13.03.28.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 13 Nov 2021 03:28:11 -0800 (PST)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <83pmr4xt4c.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <83pmr4xt4c.fsf@HIDDEN>
Date: Sat, 13 Nov 2021 19:29:36 +0800
Message-ID: <87o86opa3j.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> (let ((inhibit-modification-hooks t))
>>   (insert "Insertion that will never trigger before/after-change-functions"))
>
> So the problem is that Org uses the modification hooks as the primary
> mechanism (with which quail presents no problem), and
> buffer-chars-modified-tick as the fallback, and that when some code
> inhibits the modification hooks, then the primary mechanism cannot
> work and quail breaks the fallback?

Not exactly. Org uses modification hooks as the only mechanism to
process buffer changes because Org needs to know the region where the
buffer text changes. buffer-chars-modified-tick is used for error
detection - when buffer text is changed, but Org modification hooks are
not called for some reason. quail triggers false positives during the error
detection.

>> The quail's insertion+deletion itself is not a problem for Org cache -
>> it does not really alter the buffer text and cannot break the cache. The
>> problem is that it cannot be distinguished from the first example - both
>> cases will trigger buffer-chars-modified-tick increase.
>
> You didn't answer my question regarding buffer-modified-tick: it can
> explain to Org why buffer-chars-modified-tick jumped unexpectedly, and
> thus (hopefully) resolve this situation.  If that helps, you could
> perhaps turn the table and use buffer-chars-modified-tick is the
> primary method of discovering changes, not as fallback.
> ...
>> The only assumption I had it that Emacs does not frequently change
>> buffer text without triggering modification hooks. Clearly, the
>> assumption was wrong.
>
> I meant the assumptions about what buffer-chars-modified-tick does and
> what its value means.

It seems that we have some misunderstanding here. Org does not care
about the value of buffer-chars-modified-tick - just whether
buffer-chars-modified-tick is changed or not (see the above).

Even if Org used the value of buffer-chars-modified-tick, it would not
be useful. There is no information about buffer region where the edits
happened if we just look at buffer-chars-modified-tick. AFAIK, only
after-change-functions have access to the changed region bounds.

>> Ideally, a way to track _all_ buffer modifications regardless of
>> inhibit-modification-hooks would be useful.
>
> But Org is not interested in just any moidification, AFAIU.  It is
> only interested in modifications that change the buffer text.  Isn't
> that true?  Or what else is Org interested in for this purpose.

You are right. Org is interested in modifications that change buffer
text. Also, Org is interested to be not affected by
inhibit-modification-hooks. Making sure that Org modification hooks run
for every modification is automatically making sure that no
modifications that do change buffer text are missed. I thought that it
can be the simplest approach to fix the issue.

>> Alternatively, Emacs could support language parsers.
>
> I meant support on the low level, where changes to buffer text are
> considered and indicated.  As I indicate below, the integration of
> tree-sitter simply uses the existing change indications, so I'm not
> sure how would a parser support help you in this matter.

I probably went too far with my suggestion here. I meant not only
handling changes, but also adding API for working with syntax trees using
C-level functions. It is out of scope of this bug report.

> The Emacs code related to tree-sitter already uses change indications,
> and AFAIR didn't require any changes to the existing infrastructure.
> I wonder why Org cannot settle with what we have, if your needs are
> similar enough to those of tree-sitter.

I just checked https://github.com/casouri/emacs/blob/ts/src/insdel.c
That ts Emacs branch directly modifies C-level Emacs buffer editing
primitives (see the calls to ts_record_change). So, it is not affected
by inhibit-modification-hooks.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 13 Nov 2021 13:39:02 +0000
Resent-Message-ID: <handler.51766.B51766.16368107204043 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.16368107204043
          (code B ref 51766); Sat, 13 Nov 2021 13:39:02 +0000
Received: (at 51766) by debbugs.gnu.org; 13 Nov 2021 13:38:40 +0000
Received: from localhost ([127.0.0.1]:46737 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mltEt-000138-Td
	for submit <at> debbugs.gnu.org; Sat, 13 Nov 2021 08:38:40 -0500
Received: from eggs.gnu.org ([209.51.188.92]:59616)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mltEq-00012s-SO
 for 51766 <at> debbugs.gnu.org; Sat, 13 Nov 2021 08:38:38 -0500
Received: from [2001:470:142:3::e] (port=50594 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mltEl-00057i-Kd; Sat, 13 Nov 2021 08:38:31 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=xfHD4bgy9lKEHUuzNphoTQsCTqT/syIvLzX1FGiEWgk=; b=abxOHNQLJw6r
 oFkmNsaLkfDsEtIvIrhQawAmId4UiqQgiHFL1udgjxfDfyGfR/J7lvzJY7I5lI7yv5omH0pCNeMNb
 UvQ4WiU7fOKdC2KaPbSWLuzKYTWgodXQA4Zgz+IZr1NLLzMYnSeCFXbaz16y47Y4MPYPsFCGVhBf6
 zx5pOF8f42B/YuBw43sZbUllgcNzkEQaXocuoKnq2AAiuQUYysDYMxeizPTxKy4bLKY4VDK0o0hGV
 aBTN4tywsR2W67PbU83mjJI9CPUqRjrP68JT98m7+tTCXHlO4zf8dswb/tZ5pwGa3zosXyc4oVfTo
 PMbwk5SYLxQOyMwhi/RqjA==;
Received: from [87.69.77.57] (port=3276 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mltEj-0007b6-3e; Sat, 13 Nov 2021 08:38:31 -0500
Date: Sat, 13 Nov 2021 15:38:13 +0200
Message-Id: <83k0hcxjju.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87o86opa3j.fsf@localhost> (message from Ihor Radchenko on Sat,
 13 Nov 2021 19:29:36 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <83pmr4xt4c.fsf@HIDDEN> <87o86opa3j.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Sat, 13 Nov 2021 19:29:36 +0800
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> >> (let ((inhibit-modification-hooks t))
> >>   (insert "Insertion that will never trigger before/after-change-functions"))
> >
> > So the problem is that Org uses the modification hooks as the primary
> > mechanism (with which quail presents no problem), and
> > buffer-chars-modified-tick as the fallback, and that when some code
> > inhibits the modification hooks, then the primary mechanism cannot
> > work and quail breaks the fallback?
> 
> Not exactly. Org uses modification hooks as the only mechanism to
> process buffer changes because Org needs to know the region where the
> buffer text changes. buffer-chars-modified-tick is used for error
> detection - when buffer text is changed, but Org modification hooks are
> not called for some reason. quail triggers false positives during the error
> detection.

Is that any different from what I said, which is that you need error
detection only when the modification hooks are not called?  And that
the quail behavior is only the issue when using this error detection,
i.e. when the modification hooks are not called?

> >> The quail's insertion+deletion itself is not a problem for Org cache -
> >> it does not really alter the buffer text and cannot break the cache. The
> >> problem is that it cannot be distinguished from the first example - both
> >> cases will trigger buffer-chars-modified-tick increase.
> >
> > You didn't answer my question regarding buffer-modified-tick: it can
> > explain to Org why buffer-chars-modified-tick jumped unexpectedly, and
> > thus (hopefully) resolve this situation.  If that helps, you could
> > perhaps turn the table and use buffer-chars-modified-tick is the
> > primary method of discovering changes, not as fallback.
> > ...
> >> The only assumption I had it that Emacs does not frequently change
> >> buffer text without triggering modification hooks. Clearly, the
> >> assumption was wrong.
> >
> > I meant the assumptions about what buffer-chars-modified-tick does and
> > what its value means.
> 
> It seems that we have some misunderstanding here. Org does not care
> about the value of buffer-chars-modified-tick - just whether
> buffer-chars-modified-tick is changed or not (see the above).

But if buffer-modified-tick completely explains the change in
buffer-chars-modified-tick, you can conclude that
buffer-chars-modified-tick didn't change for your purposes, and then
all's well, no?

> Even if Org used the value of buffer-chars-modified-tick, it would not
> be useful. There is no information about buffer region where the edits
> happened if we just look at buffer-chars-modified-tick. AFAIK, only
> after-change-functions have access to the changed region bounds.

So what does Org do if the modification hooks were not called, and
buffer-chars-modified-tick says the text was changed?

> > But Org is not interested in just any moidification, AFAIU.  It is
> > only interested in modifications that change the buffer text.  Isn't
> > that true?  Or what else is Org interested in for this purpose.
> 
> You are right. Org is interested in modifications that change buffer
> text. Also, Org is interested to be not affected by
> inhibit-modification-hooks.

Then maybe this is the missing infrastructure you'd like to see
implemented.

> > The Emacs code related to tree-sitter already uses change indications,
> > and AFAIR didn't require any changes to the existing infrastructure.
> > I wonder why Org cannot settle with what we have, if your needs are
> > similar enough to those of tree-sitter.
> 
> I just checked https://github.com/casouri/emacs/blob/ts/src/insdel.c
> That ts Emacs branch directly modifies C-level Emacs buffer editing
> primitives (see the calls to ts_record_change). So, it is not affected
> by inhibit-modification-hooks.

That's what I meant by "existing infrastructure".

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 13 Nov 2021 14:42:02 +0000
Resent-Message-ID: <handler.51766.B51766.163681452027360 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.163681452027360
          (code B ref 51766); Sat, 13 Nov 2021 14:42:02 +0000
Received: (at 51766) by debbugs.gnu.org; 13 Nov 2021 14:42:00 +0000
Received: from localhost ([127.0.0.1]:46817 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mluEC-00077E-CX
	for submit <at> debbugs.gnu.org; Sat, 13 Nov 2021 09:42:00 -0500
Received: from mail-pj1-f54.google.com ([209.85.216.54]:55948)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1mluEA-000771-8Z
 for 51766 <at> debbugs.gnu.org; Sat, 13 Nov 2021 09:41:59 -0500
Received: by mail-pj1-f54.google.com with SMTP id v23so9118544pjr.5
 for <51766 <at> debbugs.gnu.org>; Sat, 13 Nov 2021 06:41:58 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=laN2nZPljR3c3/mQEluY8eWtpKJoPnHi8KmlzZ0mUjo=;
 b=XldTVeqSIjkncGfL5U03acFXHazvTpx2gb/lSM2j9Gl2dvfbGj9WY3iS4NlD07wVhW
 yRmjIdZPjNKkoZFwYxlPzS7Og2YpJailh9s35vsY5uPxEygKpHg7Mitj0onoIEoe8MyB
 WxFL8Wb59uB9lq/9+3Sws5d1WeHP5//CgWLD1wbgXb6fN7lDMLI615OAQEJavKzHD7Na
 g7lsNL/yhSMi/xjAiYbhc13B6ReiIeGf+IznPNxLGY+MZhGrgM5exX/lXOF660vFoVGW
 /YLKOV9SP1NaCqpBJugYYNAZBelBe1x1hkFjLD5PGRINnZzZlc5HaCKgd5/rBz3dKpvy
 byGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=laN2nZPljR3c3/mQEluY8eWtpKJoPnHi8KmlzZ0mUjo=;
 b=FWKnELOTCNhdW7RNbCjJIEj2VN2/hMO5leFvD3IStn2N5ZV+OEdizlVXWkn//PKc1i
 v63ZueK90BEyQ2iTtQgWazx05Wqng2Snn/n/pCXtf075Xq2KarFAdHDyyppG0dw4LWxZ
 +V5zUiVABKV3tFwfiYl9N7aoY2/hpTlivxfcykg2QF6/Qd/QqUjq4iXeaJp5+aftFO5o
 qvYQQaxL3umIesoKFWmf0fADtFq1jD3ynkYOXU6SflaJNDzwil2bKXURjwF4QwrOBluv
 0oRO7qQpUj8ebG40ijq4kKRwyIa+sCSjfB+M4bWAWZxUw1feLEplwkEgtCGexue47m18
 kLrw==
X-Gm-Message-State: AOAM533M8ED3mWbEkvf57PTqQbpFoNR/JMSIcJLfMn9YDnfMtkVEcGYM
 7uBHhE6P+Hqucl8ODzDTmYv67FObhIHzZg==
X-Google-Smtp-Source: ABdhPJwN/99VZwhvXDm4ZDiYxldzG9Z1PUdTidsSBaq6xB2JlfCCu85aOyYFM9fDiMh7bsoLF/4jXQ==
X-Received: by 2002:a17:90b:4acd:: with SMTP id
 mh13mr46768549pjb.230.1636814512462; 
 Sat, 13 Nov 2021 06:41:52 -0800 (PST)
Received: from localhost ([103.125.234.210])
 by smtp.gmail.com with ESMTPSA id c20sm10886573pfl.201.2021.11.13.06.41.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 13 Nov 2021 06:41:51 -0800 (PST)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <83k0hcxjju.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <83pmr4xt4c.fsf@HIDDEN>
 <87o86opa3j.fsf@localhost> <83k0hcxjju.fsf@HIDDEN>
Date: Sat, 13 Nov 2021 22:43:17 +0800
Message-ID: <87sfw0glq2.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> Not exactly. Org uses modification hooks as the only mechanism to
>> process buffer changes because Org needs to know the region where the
>> buffer text changes. buffer-chars-modified-tick is used for error
>> detection - when buffer text is changed, but Org modification hooks are
>> not called for some reason. quail triggers false positives during the error
>> detection.
>
> Is that any different from what I said, which is that you need error
> detection only when the modification hooks are not called?  And that
> the quail behavior is only the issue when using this error detection,
> i.e. when the modification hooks are not called?

Your understanding is correct.

>> It seems that we have some misunderstanding here. Org does not care
>> about the value of buffer-chars-modified-tick - just whether
>> buffer-chars-modified-tick is changed or not (see the above).
>
> But if buffer-modified-tick completely explains the change in
> buffer-chars-modified-tick, you can conclude that
> buffer-chars-modified-tick didn't change for your purposes, and then
> all's well, no?

I looked into it again and tried to play with non-cyrillic input looking
at the values of buffer-chars-modified-tick and buffer-modified-tick.
You are right, there seems to be a special relation between the values
when I use non-latin input method
(buffer-chars-modified-tick=buffer-modified-tick). Thanks!

However, I am not sure if equality of the chars-modified-tick and
modified-tick is unique to non-changing edits. Can test in the wild
though.

> So what does Org do if the modification hooks were not called, and
> buffer-chars-modified-tick says the text was changed?

The cache is potentially invalid, so it is dropped altogether by
org-element-cache-reset.

>> > But Org is not interested in just any moidification, AFAIU.  It is
>> > only interested in modifications that change the buffer text.  Isn't
>> > that true?  Or what else is Org interested in for this purpose.
>> 
>> You are right. Org is interested in modifications that change buffer
>> text. Also, Org is interested to be not affected by
>> inhibit-modification-hooks.
>
> Then maybe this is the missing infrastructure you'd like to see
> implemented.

Yes, I think. In practical terms, it may something like a new pair of
hooks: before/after-change-no-inhibit-functions. The hooks work exactly
like before/after-change-functions, but cannot be suppressed by
let-binding inhibit-modification-hooks and
before/after-change-functions. If necessary they can still be explicitly
let-bound to nil, but it should be discouraged. WDYT?

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 13 Nov 2021 15:26:01 +0000
Resent-Message-ID: <handler.51766.B51766.1636817115428 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.1636817115428
          (code B ref 51766); Sat, 13 Nov 2021 15:26:01 +0000
Received: (at 51766) by debbugs.gnu.org; 13 Nov 2021 15:25:15 +0000
Received: from localhost ([127.0.0.1]:47898 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mluu3-00006q-9n
	for submit <at> debbugs.gnu.org; Sat, 13 Nov 2021 10:25:15 -0500
Received: from eggs.gnu.org ([209.51.188.92]:47012)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mluu0-00006X-O1
 for 51766 <at> debbugs.gnu.org; Sat, 13 Nov 2021 10:25:14 -0500
Received: from [2001:470:142:3::e] (port=53928 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mluts-0001gO-V7; Sat, 13 Nov 2021 10:25:05 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=2K6yzyWM33Y2U6+ivOKFo42WAqvn/wLCfDiDVlY+JTo=; b=mx2Si46Wg1q+
 tb+GYoCI0cmNy3aQU3UdybxlKbiDifPn4GO6mRTbw5boQybSBJbtxMFtDF0r3kTPGaLZyI5UgbnfY
 xf7agaCdDB5AYjtitRnu01wUnylcnI412AcufLFcm4gS2IePEsYMeaJkOOI66gPsT89IsMaFOOKs8
 Zsv29Kl8yT5fj2VhnKoAy1NRS3p5U4xzvf+kTWYxAo+bJlAQeK8nFGxw8u8ZkMya1anvtrCA0xZDc
 DXJ+TPCHUqFy4Vo8coKp2laNjKm3UpZkcvvfFDT+wf1GvSPkVBmoPgSZ060XCEsrjkbnzaylEc+GG
 bDIraUdyejAhxFvLOSElMw==;
Received: from [87.69.77.57] (port=1918 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mluts-0003SQ-Fi; Sat, 13 Nov 2021 10:25:04 -0500
Date: Sat, 13 Nov 2021 17:24:49 +0200
Message-Id: <837ddcxem6.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87sfw0glq2.fsf@localhost> (message from Ihor Radchenko on Sat,
 13 Nov 2021 22:43:17 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <83pmr4xt4c.fsf@HIDDEN>
 <87o86opa3j.fsf@localhost> <83k0hcxjju.fsf@HIDDEN> <87sfw0glq2.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: 51766 <at> debbugs.gnu.org
> Date: Sat, 13 Nov 2021 22:43:17 +0800
> 
> > But if buffer-modified-tick completely explains the change in
> > buffer-chars-modified-tick, you can conclude that
> > buffer-chars-modified-tick didn't change for your purposes, and then
> > all's well, no?
> 
> I looked into it again and tried to play with non-cyrillic input looking
> at the values of buffer-chars-modified-tick and buffer-modified-tick.
> You are right, there seems to be a special relation between the values
> when I use non-latin input method
> (buffer-chars-modified-tick=buffer-modified-tick). Thanks!

That's what the implementation does: it copies the value from the
latter to the former.

> However, I am not sure if equality of the chars-modified-tick and
> modified-tick is unique to non-changing edits. Can test in the wild
> though.

I'd be surprised if the relation were violated.  But weirder things
have happened, so...

> > Then maybe this is the missing infrastructure you'd like to see
> > implemented.
> 
> Yes, I think. In practical terms, it may something like a new pair of
> hooks: before/after-change-no-inhibit-functions. The hooks work exactly
> like before/after-change-functions, but cannot be suppressed by
> let-binding inhibit-modification-hooks and
> before/after-change-functions. If necessary they can still be explicitly
> let-bound to nil, but it should be discouraged. WDYT?

Something like that, yes.  Just with shorter names ;-)

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Stefan Monnier <monnier@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 17 Jun 2022 02:55:01 +0000
Resent-Message-ID: <handler.51766.B51766.165543447725140 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: Eli Zaretskii <eliz@HIDDEN>, 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165543447725140
          (code B ref 51766); Fri, 17 Jun 2022 02:55:01 +0000
Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 02:54:37 +0000
Received: from localhost ([127.0.0.1]:43470 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o2284-0006XP-Op
	for submit <at> debbugs.gnu.org; Thu, 16 Jun 2022 22:54:36 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:12432)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1o227z-0006X9-7t
 for 51766 <at> debbugs.gnu.org; Thu, 16 Jun 2022 22:54:35 -0400
Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
 by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id BADF18073F;
 Thu, 16 Jun 2022 22:54:25 -0400 (EDT)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 20A8380009;
 Thu, 16 Jun 2022 22:54:24 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1655434464;
 bh=rqzqpksyF2942+9RDxYhpNCo4iPWZnpGG+lxVPWxNbs=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=m6jCLslpABE/dTX3ir2bvJCXuH5c1/l6obEfPjiIXaDlCqHOWJpU8rk6C32GswVy1
 LYLjqY6dyfocRNKdmUfVLa5AIsFW/IAY893xeBg1LKMDwZNw0YquWyoMYz5dcjDe2o
 H8TUC/JUsxHsGXOQRuatpBgWG/QlMOlwCdKMAaQ0OTJdzUjlxQNzUG9kKCK4hRh7lh
 RTJUyBvXwbrtOHBO6Vl8c+oBTowsJ6SA9Dfsqj8RrX//T3h7iRLJmn9fU+g60gF8Jy
 Jwo22s56i4ojgj21ghNn7VRGJ5qpmsiC57wG6UBtYaQm4kJQ02qogYcP1N7A8MAAZI
 n2Daz26A/u4ww==
Received: from alfajor (unknown [45.72.221.51])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B985E120163;
 Thu, 16 Jun 2022 22:54:23 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
Message-ID: <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost>
Date: Thu, 16 Jun 2022 22:54:22 -0400
In-Reply-To: <87r1bkpgjw.fsf@localhost> (Ihor Radchenko's message of "Sat, 13
 Nov 2021 17:10:11 +0800")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL -0.057 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain T_SCC_BODY_TEXT_LINE    -0.01 -
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> (let ((inhibit-modification-hooks t))
>   (insert "Insertion that will never trigger before/after-change-functions"))

This is a severely broken piece of code.  I don't think anyone should
try and handle this in any other way than by screaming bloody murder
when it detects the consequences.

> (defun org-element--after-change-function (...)
>  (setq org-element-chars-modified-tick (buffer-chars-modified-tick))
>  (org-element-cache-submit-request ...))
>
> (defun org-element--before-change-function (...)
>  (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick))
>      ;; Buffer has been changed without calling after-change-function
>      ;; and we have no way to determine which part of buffer has been changed.
>  ))

So this `unless` is intended to detect the case where we should scream
bloody murder, right?

Why do you need it?  AFAICT it should only be needed for debugging
purposes until the offender is found and shamed publicly.
[ I have a weird feeling that I might be one of the offenders.  ]

> Ideally, a way to track _all_ buffer modifications regardless of
> inhibit-modification-hooks would be useful.

I don't think this *can* exist: if we add a mechanism which ignores
`inhibit-modification-hooks` it will still need some way to ignore some
changes so we'll need another `inhibit-<foo>` variable to "silence"
those changes and we'll be back at square one.

I think the better way to proceed is to figure out why/when
significant changes are made while `inhibit-modification-hooks` is
non-nil, since that's the origin of your problems, AFAICT.


        Stefan

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 17 Jun 2022 05:38:02 +0000
Resent-Message-ID: <handler.51766.B51766.16554442218390 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Stefan Monnier <monnier@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, yantar92@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.16554442218390
          (code B ref 51766); Fri, 17 Jun 2022 05:38:02 +0000
Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 05:37:01 +0000
Received: from localhost ([127.0.0.1]:43524 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o24fF-0002BB-1a
	for submit <at> debbugs.gnu.org; Fri, 17 Jun 2022 01:37:01 -0400
Received: from eggs.gnu.org ([209.51.188.92]:57318)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1o24f5-0002Aq-Jh
 for 51766 <at> debbugs.gnu.org; Fri, 17 Jun 2022 01:36:59 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:56848)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o24ez-0007jb-Bh; Fri, 17 Jun 2022 01:36:45 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=Dpr3hQv2hhkXJeWR/sLGcBbS+9Tj48560R5ihAvpg6I=; b=eBVZnDWucpWb
 owO4GdEUqgvD7e3rV16COUOklqehH4i61dPlw7y3Dq1ry7khorx3Ixrm9nk2h8NCohO+kDjH6ZuNL
 Alq7qGD8uUGgPPGHdwd2Ra0PB5Pgu1rvaZlYZAzZsLr3g/etoSBqb/DQOqWI7QuunT7DihIHSLGb+
 3bfkw2Az55zZkwZfxAOeM/48nJIO3dx0hETCRyhR+hWcyZwlfe9Sxxut+p/dHrwKORqRvgcm4el1C
 LTSwSnRNj3l6APALpzWvGO5LhXt49pghr2mUoGMFai0CXyaBWobcao+JKo2y8EHt3JWZssoEOJ2o0
 1O5cKi57Qqu4ITibAzveBw==;
Received: from [87.69.77.57] (port=4431 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o24ew-0001RB-HH; Fri, 17 Jun 2022 01:36:44 -0400
Date: Fri, 17 Jun 2022 08:36:37 +0300
Message-Id: <83r13nq2ai.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <jwvpmj86mba.fsf-monnier+emacs@HIDDEN> (message from Stefan
 Monnier on Thu, 16 Jun 2022 22:54:22 -0400)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Stefan Monnier <monnier@HIDDEN>
> Cc: Eli Zaretskii <eliz@HIDDEN>,  51766 <at> debbugs.gnu.org
> Date: Thu, 16 Jun 2022 22:54:22 -0400
> 
> I think the better way to proceed is to figure out why/when
> significant changes are made while `inhibit-modification-hooks` is
> non-nil, since that's the origin of your problems, AFAICT.

I thought that was clear from the rest of the discussion: it's quail's
input methods that cause the issue, because quail tries to pretend
that just one character was inserted, when in fact the user could type
several characters.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 17 Jun 2022 10:05:02 +0000
Resent-Message-ID: <handler.51766.B51766.165546027911125 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Stefan Monnier <monnier@HIDDEN>
Cc: Eli Zaretskii <eliz@HIDDEN>, 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165546027911125
          (code B ref 51766); Fri, 17 Jun 2022 10:05:02 +0000
Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 10:04:39 +0000
Received: from localhost ([127.0.0.1]:43805 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o28qF-0002tM-5Y
	for submit <at> debbugs.gnu.org; Fri, 17 Jun 2022 06:04:39 -0400
Received: from mail-pl1-f178.google.com ([209.85.214.178]:44017)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1o28qC-0002t6-GF
 for 51766 <at> debbugs.gnu.org; Fri, 17 Jun 2022 06:04:37 -0400
Received: by mail-pl1-f178.google.com with SMTP id r1so3468836plo.10
 for <51766 <at> debbugs.gnu.org>; Fri, 17 Jun 2022 03:04:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=xV/WpZ74it4tk1i8Ju7EgvDraANm7Z+/W2sym0KIReA=;
 b=aaXBSqTX/PYf8hNULhcKzf23xIyK2p7J/SZAKb1Lg932J1Wuygd/uKDJ6vSQp9XBqR
 2u9/r9B6tIoOjRqU9oGzBNWw22DayQ47qrv1+slUdsaPuSlu1yGiIhh33Ajy0riEzq7p
 xiRbrM8J2IajsS+NvYn3GH0D1/el2xmqT1rNQq3aNu1b+EIHb/PIYKyel9+zXcUn/o6P
 nwpJEbyDPwmY+HDwG1k4MEZrv5dsagPwmsrFJa8QR/y2+1DfpVySZKUiOBev5mBwKmCS
 i8vyMneACUy3DSpgz+W/bLuqLb45fvbXuZLcNeOIjceHAKPRMerhnEHB1gBq8KdrKjFC
 xkoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=xV/WpZ74it4tk1i8Ju7EgvDraANm7Z+/W2sym0KIReA=;
 b=Mc3EqX5lRNWxy078/Ws0N1mPd343PvuKMxcU8BALheud8n4QXRrYB/01hdY1mTmfaX
 Zh+ZYazF8yCK8MkL+g2AfTMJ+LS4x96sc1E/9kwb3lH0d0KVFTdtZ5+KwyOnPJ7pW9hO
 uXY24H/zmR+e44XkG5UPNShKzwP06H5dfNK+cxDHAzsS5CGWQRYGDWQ0DEwZq2jKP35r
 afBbsy4EoWZgZKG+tHO8jryagS2eWh3iOE1XrSx62I8AhSmYL/W3Z/eLFGsMNmsuo3z1
 SH2/DXhyvb03AuhEszQ65y21k6NeI+nn1eiaueqm6HCynzkmutdD/0O/hXBvh+6JEiUa
 ETKg==
X-Gm-Message-State: AJIora84gfRay1nk40U1HuB8Twyq3FS2S4jQ+0+kMtVdiMvHrKNigH61
 2CTnupn5D9ath58+5TYRabA=
X-Google-Smtp-Source: AGRyM1szvqfbO4WLrmnwJHSXc8ASM75eoQ9O1RV+alGxEihXlAA2ZzMneudsntgCq0Hehu1BAH6Zzg==
X-Received: by 2002:a17:90b:4d90:b0:1e3:3025:66fe with SMTP id
 oj16-20020a17090b4d9000b001e3302566femr20624540pjb.145.1655460270522; 
 Fri, 17 Jun 2022 03:04:30 -0700 (PDT)
Received: from localhost ([66.154.105.4]) by smtp.gmail.com with ESMTPSA id
 b9-20020a170902bd4900b0016782c55790sm1152636plx.232.2022.06.17.03.04.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 Jun 2022 03:04:29 -0700 (PDT)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
Date: Fri, 17 Jun 2022 18:05:36 +0800
Message-ID: <87y1xv7ggf.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Stefan Monnier <monnier@HIDDEN> writes:

>> (let ((inhibit-modification-hooks t))
>>   (insert "Insertion that will never trigger before/after-change-functions"))
>
> This is a severely broken piece of code.  I don't think anyone should
> try and handle this in any other way than by screaming bloody murder
> when it detects the consequences.

>> (defun org-element--before-change-function (...)
>>  (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick))
>>      ;; Buffer has been changed without calling after-change-function
>>      ;; and we have no way to determine which part of buffer has been changed.
>>  ))
>
> So this `unless` is intended to detect the case where we should scream
> bloody murder, right?
>
> Why do you need it?  AFAICT it should only be needed for debugging
> purposes until the offender is found and shamed publicly.
> [ I have a weird feeling that I might be one of the offenders.  ]
>
>> Ideally, a way to track _all_ buffer modifications regardless of
>> inhibit-modification-hooks would be useful.

Well. This was also my assumption. I initially implemented the checking
code to detect internal errors in Org.

Then we received 12 bug reports on this. The offenders were identified
in some cases. They were:
- polymode, valign, and ox-hugo
- Doom (unrelated to this particular issue, but Doom is let-binding major-mode...)
- Emacs quail and also replace-match in some cases (because of
  false-positives caused by changing buffer-chars-modified-tick)

ox-hugo and polymode fixed the issue.

valign relies on disabling modification hooks because it is otherwise
difficult to figure out pixel width of a string in current buffer:
https://github.com/casouri/valign/issues/30

In other cases, the offenders were hard to identify because of remote
debugging difficulty.

What I want to say is that the problem is more widespread than you may
think. And the consequences of missed modifications can cause very real
data corruption in Org files (some editing Org commands are relying on
cache being valid; if syntax boundaries are incorrect, the editing can
mess up the text). It must be avoided at all costs, regardless of the
recommended Elisp programming practices.

On top of the misbehaving third-party code, there is an issue described
in bug#46982. This discussion reminded me about
clone-indirect-buffer-hook, but it is only executed by
`clone-indirect-buffer', not by `make-indirect-buffer'. The latter is
used ubiquitously. See
https://github.com/search?q=make-indirect-buffer&type=code
Again, some unsuspecting users can experience data corruption just
because someone carelessly uses `make-indirect-buffer' in the code.

> I don't think this *can* exist: if we add a mechanism which ignores
> `inhibit-modification-hooks` it will still need some way to ignore some
> changes so we'll need another `inhibit-<foo>` variable to "silence"
> those changes and we'll be back at square one.
>
> I think the better way to proceed is to figure out why/when
> significant changes are made while `inhibit-modification-hooks` is
> non-nil, since that's the origin of your problems, AFAICT.

See the above. There is real-world code doing things that make
Emacs ignore after-change-functions.

Combined with the issue revealed in this bug report, I am left with no
Emacs tools to handle the problematic buffer modifications.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 17 Jun 2022 10:51:01 +0000
Resent-Message-ID: <handler.51766.B51766.165546301717511 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165546301717511
          (code B ref 51766); Fri, 17 Jun 2022 10:51:01 +0000
Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 10:50:17 +0000
Received: from localhost ([127.0.0.1]:43860 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o29YO-0004YI-CF
	for submit <at> debbugs.gnu.org; Fri, 17 Jun 2022 06:50:16 -0400
Received: from eggs.gnu.org ([209.51.188.92]:41100)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1o29YM-0004Xx-Nt
 for 51766 <at> debbugs.gnu.org; Fri, 17 Jun 2022 06:50:15 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43918)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o29YH-0000EC-DU; Fri, 17 Jun 2022 06:50:09 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=9OlnoOg214spvTJdrJzzHFl2FyY/XqpIDI102T8sse4=; b=WPAXM80gIsGm
 yodlEvwb2w2sz9QPm2m8RhFWGeCkH4/k8H8f2R/9zt+HUZ1OADZ2DMl85eFFD0001uJr0eN6Iq3T2
 1LSP2EAbpWSxmTr7ybrTm3o58/ajy7chy1pZ/KpWzM+yd6aWKXuPH7AaDt44lz1/2VK2Yf7QmVie3
 uDDuc8Ix13mDj4WsKA9PUbN5YHPMChKh0tOvVbmwqtbTFD+2GyWjcaCIfQCu5IjoGkiGhEnXhOO8A
 FU1mzSuqjq/tt6JR43Z0fOSGLm/hFw0/995T6ErcRIiXtvjz4eumS+afq5ivKIhJOIlC4iu155bQp
 RSPWr5TicRjNttwG8VqP+w==;
Received: from [87.69.77.57] (port=4126 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o29YG-0001AO-Qp; Fri, 17 Jun 2022 06:50:09 -0400
Date: Fri, 17 Jun 2022 13:50:04 +0300
Message-Id: <838rpvpns3.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87y1xv7ggf.fsf@localhost> (message from Ihor Radchenko on Fri,
 17 Jun 2022 18:05:36 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: Eli Zaretskii <eliz@HIDDEN>,  51766 <at> debbugs.gnu.org
> Date: Fri, 17 Jun 2022 18:05:36 +0800
> 
> Combined with the issue revealed in this bug report, I am left with no
> Emacs tools to handle the problematic buffer modifications.

You aren't supposed to try to do that in Lisp.  I suggest to describe
a generalization of the use cases you are aware of which you think
need this, and then we could think about implementing some or all of
it in C.

> valign relies on disabling modification hooks because it is otherwise
> difficult to figure out pixel width of a string in current buffer:
> https://github.com/casouri/valign/issues/30

That discussion is very short and lacking in detail, but up front, why
doesn't valign use the primitives we provide for determining the pixel
width of a string?

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Stefan Monnier <monnier@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 17 Jun 2022 13:17:02 +0000
Resent-Message-ID: <handler.51766.B51766.165547180715960 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, yantar92@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165547180715960
          (code B ref 51766); Fri, 17 Jun 2022 13:17:02 +0000
Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 13:16:47 +0000
Received: from localhost ([127.0.0.1]:44332 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o2BqB-00048y-AN
	for submit <at> debbugs.gnu.org; Fri, 17 Jun 2022 09:16:47 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:30154)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1o2Bq7-00041f-Ck
 for 51766 <at> debbugs.gnu.org; Fri, 17 Jun 2022 09:16:45 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id B348A4410EA;
 Fri, 17 Jun 2022 09:16:37 -0400 (EDT)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 54262440C7C;
 Fri, 17 Jun 2022 09:16:36 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1655471796;
 bh=WC7co+3i4YCtMO45kmiph6AWjV80mOI15zZbubNntF8=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=ks0BRVelTBWPMAsTcNgzRht1BSzFRhQhYlBK+LGowFqnfYhRseTLJ1UUy2wIxS4YQ
 8nR2rAXerg3+eA6/6zUTTxeueYLdXEY09mMvYf7PK5y5HI8dE/s0epduhc7vtqTd8p
 N3qfYs1Q9CB50T9ySjA7noHg+b8hQbOvdiE7TYe+jNyjTUw0iQJSFf6r/OLOX/V6Zz
 LQJD9bkP2ebC+Wbl/IOvi6qhaXet9tsPIGLpt1Fmt5qjJHvheYgdVZUYAmHTpjvQuz
 jhxndBXV7LKiXRWpKlXU4mTK3qdzI/LuuVp4QvYMwNwATYm4ksDFCR1Sic/rjjjXlJ
 p/qRU0rkxlRwQ==
Received: from pastel (unknown [45.72.221.51])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 2384812047F;
 Fri, 17 Jun 2022 09:16:36 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
Message-ID: <jwvsfo3xwl1.fsf-monnier+emacs@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <83r13nq2ai.fsf@HIDDEN>
Date: Fri, 17 Jun 2022 09:16:34 -0400
In-Reply-To: <83r13nq2ai.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 17 Jun
 2022 08:36:37 +0300")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL -0.061 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain T_SCC_BODY_TEXT_LINE    -0.01 -
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Eli Zaretskii [2022-06-17 08:36:37] wrote:
>> From: Stefan Monnier <monnier@HIDDEN>
>> Cc: Eli Zaretskii <eliz@HIDDEN>,  51766 <at> debbugs.gnu.org
>> Date: Thu, 16 Jun 2022 22:54:22 -0400
>> 
>> I think the better way to proceed is to figure out why/when
>> significant changes are made while `inhibit-modification-hooks` is
>> non-nil, since that's the origin of your problems, AFAICT.
>
> I thought that was clear from the rest of the discussion: it's quail's
> input methods that cause the issue, because quail tries to pretend
> that just one character was inserted, when in fact the user could type
> several characters.

AFAIK in the case of Quail the char-modified-ticks changes (so there's
some insertions/deletions going on) while `inhibit-modification-hooks`
is set, but the state of the buffer at the next
`before-change-functions` is correct, e.g. the buffer-hash is unchanged.

IOW in the cse of Quail the Org mode code doesn't need to flush the
whole parser's state, which means that the code that flushes the parser
state when char-modified-ticks is modified silently was written to
defend against *other* problems.


        Stefan

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Stefan Monnier <monnier@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 17 Jun 2022 13:29:01 +0000
Resent-Message-ID: <handler.51766.B51766.165547251319477 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: Eli Zaretskii <eliz@HIDDEN>, 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165547251319477
          (code B ref 51766); Fri, 17 Jun 2022 13:29:01 +0000
Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 13:28:33 +0000
Received: from localhost ([127.0.0.1]:44385 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o2C1Y-000542-K2
	for submit <at> debbugs.gnu.org; Fri, 17 Jun 2022 09:28:32 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:48958)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1o2C1W-00053q-GI
 for 51766 <at> debbugs.gnu.org; Fri, 17 Jun 2022 09:28:31 -0400
Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1C4A3441268;
 Fri, 17 Jun 2022 09:28:25 -0400 (EDT)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 590234410EA;
 Fri, 17 Jun 2022 09:28:23 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1655472503;
 bh=i/vDSExZcgIUpyn48B0KkvA/YVClW/bkAM2pgA/tlvM=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=NozX5ByO5zjOG+/1UlyKjadQBzNVyfGz2ASD5F0/Y4yB0EYZdFtVzOj1YM+6gxyQX
 Upqfo/cs8CiXBgbeM2k3ZzC3nSbABWmmXbl//UuGTJo9kplMh0lr/Ge1V94C0fAV73
 n2x3Niq87UguDQf2UlrohhxSLD7MJtQvBDiFNAC15G6ME5EtwjlgGHPjTwTW1gFIvl
 d/lPF889Uh69aCPcpAjq7oP89rjQo9LLyBL1oOGcf1tNPZnSnkmulNkOcc2ZF/eExI
 GnnCujacgDzaQDW4/0v2OO22UPo3y7Ydd+oYQ144sjw+OTuwCTpub0WGSfhfAl23d8
 VYgJF5sjczBUA==
Received: from pastel (unknown [45.72.221.51])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 28D211201CF;
 Fri, 17 Jun 2022 09:28:23 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
Message-ID: <jwvmtebxwd2.fsf-monnier+emacs@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost>
Date: Fri, 17 Jun 2022 09:28:21 -0400
In-Reply-To: <87y1xv7ggf.fsf@localhost> (Ihor Radchenko's message of "Fri, 17
 Jun 2022 18:05:36 +0800")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL -0.061 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain T_SCC_BODY_TEXT_LINE    -0.01 -
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Well.  This was also my assumption.  I initially implemented the checking
> code to detect internal errors in Org.
>
> Then we received 12 bug reports on this. The offenders were identified
> in some cases. They were:
> - polymode, valign, and ox-hugo
> - Doom (unrelated to this particular issue, but Doom is let-binding major-mode...)
> - Emacs quail and also replace-match in some cases (because of
>   false-positives caused by changing buffer-chars-modified-tick)

IIUC Quail and replace-match shouldn't be in the above list: they're
caught by your debugging check but they're false positives.

> valign relies on disabling modification hooks because it is otherwise
> difficult to figure out pixel width of a string in current buffer:
> https://github.com/casouri/valign/issues/30

AFAIK this is *also* a false positive: the buffer is only temporarily
modified within the `with-silent-modifications` and reverted to its
previous state before the end of that macro, so there's no need to flush
your parser's state.

>> I don't think this *can* exist: if we add a mechanism which ignores
>> `inhibit-modification-hooks` it will still need some way to ignore some
>> changes so we'll need another `inhibit-<foo>` variable to "silence"
>> those changes and we'll be back at square one.
>>
>> I think the better way to proceed is to figure out why/when
>> significant changes are made while `inhibit-modification-hooks` is
>> non-nil, since that's the origin of your problems, AFAICT.
>
> See the above. There is real-world code doing things that make
> Emacs ignore after-change-functions.

I don't see how this relates to what I'm saying: what I'm saying is that
for the same reason there's code that has very valid reasons to inhibit
`after-change-functions`, there will be code that has very valid reasons
to inhibit some new `after-really-every-change-functions`, and then
there will inevitably also be code that abuses this.

The only real solution is to "push back" and get those abuses fixed.

One thing you could do, for example is replace your char-modified-tick
check with one based on buffer-size: it won't catch all cases, but it
won't suffer from false positives, so you can really scream bloody
murder when it happens.


        Stefan

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 04:13:02 +0000
Resent-Message-ID: <handler.51766.B51766.165578472917788 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165578472917788
          (code B ref 51766); Tue, 21 Jun 2022 04:13:02 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 04:12:09 +0000
Received: from localhost ([127.0.0.1]:58907 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3VFJ-0004cq-If
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 00:12:09 -0400
Received: from mail-oa1-f54.google.com ([209.85.160.54]:41526)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1o3VFE-0004cD-R8
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 00:12:08 -0400
Received: by mail-oa1-f54.google.com with SMTP id
 586e51a60fabf-101bb9275bcso11982073fac.8
 for <51766 <at> debbugs.gnu.org>; Mon, 20 Jun 2022 21:12:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=wo3D5qE1+jXahL8rkDSFSahjvyvBEYgWtIhbApX6g60=;
 b=Mxpdpq3WF88CuMzStcc1j8HBYpN5X20hGoHDbSouDiY7wVxn068JYP+x2OuWPbuLkp
 wz/5I0uTrb9DImZmhx2jqEM9SkJYQMbzSM6K4/IRcwgeK0ClsKHZ0eaLp7FhZqca323w
 QmjabftWGTGqSrLN2A8TJs0DFkcQKbZk1GXEFJq5l9bgtNpjW3GeF0pNk162WJX/kOYk
 Nn0Le67DBaxCPDy6u/50KBqjmAy7ZNX6IHXkgTj4KOuz0UOQgOD4p6qntbuno2zm8byN
 8Ux4XFNX/ftlxKS6mcIHTniyb2vms8/jD+KM0Pl+UVt46VSgFYP5dOnrWPKwgfGN1u9h
 g5kA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=wo3D5qE1+jXahL8rkDSFSahjvyvBEYgWtIhbApX6g60=;
 b=s58OyMj1+HnG5dvV/uvGh5HinjOAwoRkiqCJolBvYTdD9rmqTKwtt8M+2PnyudZ4kj
 W+7FNSJjnOyCv9nSQnAk7H1HU71S0ohgO98MnbQz4jTaerR0SNqur1xdWfrEbm6C881z
 +DT2nNZtbTop0vGbyrJFuMlWIey591XoYD7mx3X4LbcE6GekiMWRy6uAoPiYiIog9ZJk
 5leUdhxhe6Gjx4XgNwi00dqvdtBUY4ZjMRtzGlDVrUFCndELPqU1Y1Pg1Y+4r6tYgADP
 T0urWggKwFZL6oL2N57hSUzUkjfmjcoZ4jLeWInyBuVLNd1VKbE4G84Psb6oCLt5SX1Z
 w7jg==
X-Gm-Message-State: AJIora8gkL13Nxa4bh6A9u8HtIugWae5uYVVmfms71X5R0jKmIBF10Lv
 oX7ay+EAfM6qEvCqbOQPVXQ=
X-Google-Smtp-Source: AGRyM1uY+J/6zcprA42znSTwy4PywLHfTaY3t13r/V1ylmlSruEoQKVlEu8KSgyMMpFbMFpYolr0Kg==
X-Received: by 2002:a05:6871:5cf:b0:101:cfbb:6b33 with SMTP id
 v15-20020a05687105cf00b00101cfbb6b33mr6401037oan.198.1655784719230; 
 Mon, 20 Jun 2022 21:11:59 -0700 (PDT)
Received: from localhost ([207.126.88.10]) by smtp.gmail.com with ESMTPSA id
 dv4-20020a056870d88400b00101c83352c6sm5040585oab.34.2022.06.20.21.11.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 Jun 2022 21:11:58 -0700 (PDT)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <838rpvpns3.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
Date: Tue, 21 Jun 2022 12:13:09 +0800
Message-ID: <8735fyslgq.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> valign relies on disabling modification hooks because it is otherwise
>> difficult to figure out pixel width of a string in current buffer:
>> https://github.com/casouri/valign/issues/30
>
> That discussion is very short and lacking in detail, but up front, why
> doesn't valign use the primitives we provide for determining the pixel
> width of a string?

Because string width in different buffers may be different depending on
the fontification, frame font size, face remapping,
wrap-prefix/line-prefix string properties (AFAIK, the built-in
string-pixel-width will return incorrect value on string with such
properties), invisibility specs in the buffer, line numbers mode, etc
We have implemented a number of workarounds in org-string-width on main,
but I am not 100% sure that I covered all the edge cases.

The most accurate way to measure the real string width inside current
buffer is actually inserting it and requesting the measurement.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 04:14:01 +0000
Resent-Message-ID: <handler.51766.B51766.165578483117934 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Stefan Monnier <monnier@HIDDEN>
Cc: Eli Zaretskii <eliz@HIDDEN>, 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165578483117934
          (code B ref 51766); Tue, 21 Jun 2022 04:14:01 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 04:13:51 +0000
Received: from localhost ([127.0.0.1]:58911 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3VGw-0004fC-VP
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 00:13:51 -0400
Received: from mail-ot1-f50.google.com ([209.85.210.50]:40647)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1o3VGu-0004ey-Gj
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 00:13:50 -0400
Received: by mail-ot1-f50.google.com with SMTP id
 s20-20020a056830439400b0060c3e43b548so9811462otv.7
 for <51766 <at> debbugs.gnu.org>; Mon, 20 Jun 2022 21:13:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=Mf4W25bG2HkjNbpF3A2eIlJLCf3u4kmiIFfRxiBO0uw=;
 b=bBzwcY2a63SyAbWBgw+qmjfe9MVmuf7K+WjTa19w6cH+XQU9wcyuTl1Z6lo3HiizpD
 C1Akg7JEHi5mcmNubMmQctQpNgrHLLQHx1FcWuRe5J0UZB4U8kQ47NgzDcNpfKxOOdXV
 VagtFe7Zz4hhsTaQG/R032xEpNJEs0NMxNHwOr3k7ueXdonxoCgSw8t7yJMqDx7ebmxI
 Z1/Vxs3vg2EaRXe58RtD9+hQtTGOT1kfVhU7a2Ny9weXarSTmzeXng3iCh6koflalQYC
 q+DJVgUXs9oaLZgmVPOxBuVpvHaixy/936sN4qUnYZWUvz3thZxmQ6B+nBdS+GrOHiR8
 w5Wg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=Mf4W25bG2HkjNbpF3A2eIlJLCf3u4kmiIFfRxiBO0uw=;
 b=nX5VxXyaOFM6zkof1QXNsWY7tdll/jmO1PwqWdpUAoDEg/TMb8QEFyJmi4KHPNrGkE
 fz5IKfd+jLtohwAWrsg+yauk2j9mQkYgE0kTaCqKQvvQgyhmD28ENjBtdGSXULEr6X7y
 RRxet9vtyphZF5Ng8Dc9gCM8zguzVGtpqyPtqiGVwJ6eigZ0R5T4OQHYqs7yidxGeCiO
 kT3cFcc2X1scedvpzEruCwIwWBB7FeKXTf3y3dLVuMZz5A8cRGUvEpxYa1XJ4+8VssZ7
 Xa7+5wb01NeIsWhtKHR+ZbZgGVuw0g7fb7t1HMXDRfuRxoHEXauCmtn5YJdWPGqfB1L7
 cPFQ==
X-Gm-Message-State: AJIora8Od5aYQQ1NzZQP9TQEf4DQHGnVO8fkcGiwYEJYJlwSF329v0SM
 49f2HbqqwPjd1zWY8XNiUE69J2quWzi6jQ==
X-Google-Smtp-Source: AGRyM1tDRJjNpaHXZffvaA2/uypcRQhtXu2LCevxe0oG7ciacxtk6IuZbVf6O7bH+ZvdqPAtqzIfgw==
X-Received: by 2002:a05:6830:2a01:b0:606:d153:1ba0 with SMTP id
 y1-20020a0568302a0100b00606d1531ba0mr11322241otu.35.1655784822972; 
 Mon, 20 Jun 2022 21:13:42 -0700 (PDT)
Received: from localhost ([207.126.88.10]) by smtp.gmail.com with ESMTPSA id
 m186-20020aca3fc3000000b00328c9e63389sm8437885oia.11.2022.06.20.21.13.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 Jun 2022 21:13:42 -0700 (PDT)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <jwvmtebxwd2.fsf-monnier+emacs@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <jwvmtebxwd2.fsf-monnier+emacs@HIDDEN>
Date: Tue, 21 Jun 2022 12:14:54 +0800
Message-ID: <87zgi6r6td.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Stefan Monnier <monnier@HIDDEN> writes:

>> See the above. There is real-world code doing things that make
>> Emacs ignore after-change-functions.
>
> I don't see how this relates to what I'm saying: what I'm saying is that
> for the same reason there's code that has very valid reasons to inhibit
> `after-change-functions`, there will be code that has very valid reasons
> to inhibit some new `after-really-every-change-functions`, and then
> there will inevitably also be code that abuses this.
>
> The only real solution is to "push back" and get those abuses fixed.
>
> One thing you could do, for example is replace your char-modified-tick
> check with one based on buffer-size: it won't catch all cases, but it
> won't suffer from false positives, so you can really scream bloody
> murder when it happens.

Checking the buffer-size is a great idea. Thanks!
It should be reliable enough for Org purposes.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 10:17:02 +0000
Resent-Message-ID: <handler.51766.B51766.165580660821404 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165580660821404
          (code B ref 51766); Tue, 21 Jun 2022 10:17:02 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 10:16:48 +0000
Received: from localhost ([127.0.0.1]:59168 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3awB-0005ZA-PX
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 06:16:47 -0400
Received: from eggs.gnu.org ([209.51.188.92]:36428)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1o3awA-0005Yv-4g
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 06:16:46 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:41754)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o3aw4-0003ou-Sb; Tue, 21 Jun 2022 06:16:40 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=yi/Q040Csa1NWShCO7tb04zJLrd2iKgEkJ7yYGOcFMo=; b=kJxXZaSeDaC6
 rCu8/woGi1uj5AMZ/mPsDSdQou1GYdURO2UijNFB2xIRmmjJgxKSZtF+fcX8JcHF4RKplhdXLtzgs
 07HaUIy8iae5sofDVWLfkjWyH+vsA/+Hy1hSQkwnUAymTuXJRQNB222y2rHJri3Gtrc6rMxOY1C19
 Wvj01qsCx8q44eymsWBF3Phb3KG/S2aTR6KbGDS0fXxxUJWXKi+pXRefjSMXHJyAimrUlnGPX2zye
 qwz/cec3d1RNJfBC/ZlDIcaVmny0bkyiYBSUmVGMJUq1BUD1n4qmPiWdwvspseZkreY5jIo4WzG/9
 ZVpOomNh5djNcJpsDa69cA==;
Received: from [87.69.77.57] (port=1101 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o3aw4-0005xC-3m; Tue, 21 Jun 2022 06:16:40 -0400
Date: Tue, 21 Jun 2022 13:16:28 +0300
Message-Id: <83fsjyl3sz.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <8735fyslgq.fsf@localhost> (message from Ihor Radchenko on Tue,
 21 Jun 2022 12:13:09 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN> <8735fyslgq.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: monnier@HIDDEN,  51766 <at> debbugs.gnu.org
> Date: Tue, 21 Jun 2022 12:13:09 +0800
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > That discussion is very short and lacking in detail, but up front, why
> > doesn't valign use the primitives we provide for determining the pixel
> > width of a string?
> 
> Because string width in different buffers may be different depending on
> the fontification, frame font size, face remapping,
> wrap-prefix/line-prefix string properties (AFAIK, the built-in
> string-pixel-width will return incorrect value on string with such
> properties), invisibility specs in the buffer, line numbers mode, etc
> We have implemented a number of workarounds in org-string-width on main,
> but I am not 100% sure that I covered all the edge cases.

If you need such high accuracy, may I suggest window-text-pixel-size?

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 11:00:02 +0000
Resent-Message-ID: <handler.51766.B51766.16558091722267 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.16558091722267
          (code B ref 51766); Tue, 21 Jun 2022 11:00:02 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 10:59:32 +0000
Received: from localhost ([127.0.0.1]:59217 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3bbY-0000aU-0q
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 06:59:32 -0400
Received: from mail-oa1-f50.google.com ([209.85.160.50]:44801)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1o3bbU-0000aH-Vr
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 06:59:30 -0400
Received: by mail-oa1-f50.google.com with SMTP id
 586e51a60fabf-101e1a33fe3so7673617fac.11
 for <51766 <at> debbugs.gnu.org>; Tue, 21 Jun 2022 03:59:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=Tnbalz+YvwUhGHM93X0ptnFN02Jq2o7pgLG+JvFl6fg=;
 b=XC+/c78PfOOs0CkCPKLl/rSb4vBeOUXCSCtcZpbRWZiY5zHuWld8kkx3WNfGmzLw87
 hokb7OdrqlPXvBSYdwkCqhJX+PEsN2W73WtaF1RSAyZA+xotvg8IM36lub7rpjX1xeiw
 znw2PJLR1KDUECZ/Wz1AYxOyiXfBolSte+4pZmT0AcmTUzgWQ8USBkv+O/qnycMSx/zP
 xbWANBR0/SNB2P9sEm/59ehv0WkodRkUVGXhX/a34gCTRCrQ9XB638g9UU8ri1qDw55a
 WI9yYnKXYHBWkjPUnsZH20gRO9FiKYbKjNsHHe56/GCSocb8Yy+CtM96twzbCz8EJFfs
 SFdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=Tnbalz+YvwUhGHM93X0ptnFN02Jq2o7pgLG+JvFl6fg=;
 b=CNjgVRg2gnLaaCjFHx+h9+lO8KAieFtOnMuHDAlSLQwE3Ncpd/A2f5uwNBKg4uNl8g
 xCVjuwrJuh1QSkEE1taz0p+j2THBz0ZJynOfECA5LKHvaHK7AcW0eiPcdlx7SQBALxF2
 UB5uHlc8l7rcLMMJ2XA3S2GTQNKY7u1yYMKlkoIRXlDCTnTO49ZqHltPyzySVNwwj6/3
 druhM4vQBu3r8P1WIJaxk3N4BVo/pjYWsj6M14urURkR4pH18ynWGCBVY27G1FeBfRIj
 VNMk18VHzfnL+ETrsMfny5PTEyNwPNfFlOQc2wo7gOmRjtmMQRJJa7v6GFiiXD+m0HOe
 SDiA==
X-Gm-Message-State: AJIora/TcDw+kyDyAt9dbn+NPlyDdKT1HK1CPnVeBggVm4zdOTnBVTi3
 gRWe7H/O59ZOYhpsJwQGnTI=
X-Google-Smtp-Source: AGRyM1usSs3gvoIvG19OwDyKkvohzm1IZoWCkpMoAb6F9MBnYDjANRO6WgITXoIjniubZG4gMxnVWg==
X-Received: by 2002:a05:6870:6027:b0:101:696e:d594 with SMTP id
 t39-20020a056870602700b00101696ed594mr19699243oaa.245.1655809163208; 
 Tue, 21 Jun 2022 03:59:23 -0700 (PDT)
Received: from localhost ([207.126.88.10]) by smtp.gmail.com with ESMTPSA id
 u19-20020a056870951300b000f309d52933sm8810144oal.47.2022.06.21.03.59.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 Jun 2022 03:59:22 -0700 (PDT)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <83fsjyl3sz.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
 <8735fyslgq.fsf@localhost> <83fsjyl3sz.fsf@HIDDEN>
Date: Tue, 21 Jun 2022 19:00:34 +0800
Message-ID: <87r13iqo19.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> > That discussion is very short and lacking in detail, but up front, why
>> > doesn't valign use the primitives we provide for determining the pixel
>> > width of a string?
>> 
>> Because string width in different buffers may be different depending on
>> the fontification, frame font size, face remapping,
>> wrap-prefix/line-prefix string properties (AFAIK, the built-in
>> string-pixel-width will return incorrect value on string with such
>> properties), invisibility specs in the buffer, line numbers mode, etc
>> We have implemented a number of workarounds in org-string-width on main,
>> but I am not 100% sure that I covered all the edge cases.
>
> If you need such high accuracy, may I suggest window-text-pixel-size?

window-text-pixel-size suffers from the same issues with
wrap-prefix/line-prefix and line numbers mode.

Also, in order to use it in current buffer on not-yet-inserted string,
you need to insert it. That where the issue in valign originated from.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 12:18:02 +0000
Resent-Message-ID: <handler.51766.B51766.165581386420120 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165581386420120
          (code B ref 51766); Tue, 21 Jun 2022 12:18:02 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 12:17:44 +0000
Received: from localhost ([127.0.0.1]:59358 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3cpE-0005EQ-Eo
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 08:17:44 -0400
Received: from eggs.gnu.org ([209.51.188.92]:33722)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1o3cpB-0005Dz-1D
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 08:17:41 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43502)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o3cp5-0007Jd-Km; Tue, 21 Jun 2022 08:17:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=PQr1EyjUvT39AMqJ+K7NE+Lzh5YSL7XPSnpxahrcB5E=; b=cmam8QVjqYxd
 nIzLioKItLe+578Zt9sjIrDQ3CanD7C3XbEJrL5EbIiuFwzGHGT5Cs1dE8g8vVY6g/oFBRRIsobtj
 PdUm596H5Uizq7rdX41LMdnvmOfo2quLLhw8FZo9E4f52N64TqYwEjRZ0E21IgmXrnX1vSrN6RzQi
 vE22sITRwBMV5eqD3ApASJZGPdgljOR5VyA9jKHDjZV3XXZVBjbbntiNfdgevNzusW1vwPAvl8QJO
 1I32VqU89TnAeEqxALGmoiN/xqbyHwBniDIi1qWFijOh289X05UEaxnNFGK1mcXP19t0l8eBmL08S
 kUiqLRCF8HjyyUORunrPZw==;
Received: from [87.69.77.57] (port=4549 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o3cp4-0004h4-BZ; Tue, 21 Jun 2022 08:17:35 -0400
Date: Tue, 21 Jun 2022 15:17:22 +0300
Message-Id: <834k0eky7h.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87r13iqo19.fsf@localhost> (message from Ihor Radchenko on Tue,
 21 Jun 2022 19:00:34 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
 <8735fyslgq.fsf@localhost> <83fsjyl3sz.fsf@HIDDEN> <87r13iqo19.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: monnier@HIDDEN,  51766 <at> debbugs.gnu.org
> Date: Tue, 21 Jun 2022 19:00:34 +0800
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> >> > That discussion is very short and lacking in detail, but up front, why
> >> > doesn't valign use the primitives we provide for determining the pixel
> >> > width of a string?
> >> 
> >> Because string width in different buffers may be different depending on
> >> the fontification, frame font size, face remapping,
> >> wrap-prefix/line-prefix string properties (AFAIK, the built-in
> >> string-pixel-width will return incorrect value on string with such
> >> properties), invisibility specs in the buffer, line numbers mode, etc
> >> We have implemented a number of workarounds in org-string-width on main,
> >> but I am not 100% sure that I covered all the edge cases.
> >
> > If you need such high accuracy, may I suggest window-text-pixel-size?
> 
> window-text-pixel-size suffers from the same issues with
> wrap-prefix/line-prefix and line numbers mode.

What issue are those?

> Also, in order to use it in current buffer on not-yet-inserted string,
> you need to insert it. That where the issue in valign originated from.

The usual method is to use a temporary buffer.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 12:39:01 +0000
Resent-Message-ID: <handler.51766.B51766.165581508822319 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165581508822319
          (code B ref 51766); Tue, 21 Jun 2022 12:39:01 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 12:38:08 +0000
Received: from localhost ([127.0.0.1]:59386 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3d8x-0005nu-Qe
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 08:38:08 -0400
Received: from mail-oa1-f46.google.com ([209.85.160.46]:37433)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1o3d8v-0005nO-9x
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 08:38:06 -0400
Received: by mail-oa1-f46.google.com with SMTP id
 586e51a60fabf-fb6b4da1dfso18179667fac.4
 for <51766 <at> debbugs.gnu.org>; Tue, 21 Jun 2022 05:38:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=V/K/yOPYT5YcrBKtVFXa5/DN8ftRe1G5E6PhpWar0lY=;
 b=KP+x9AN3Pi+mf6opUFwlb4CWXRW5wOcKXUm8/3swVID2Y4DP56WlzZCty9m+g1GLJG
 F6Y92ORPN+txMEbhoqXzvjoUs+JErSUQ555qgNynWD7ynZwhIaOjMasC6/TmFkOP2Jq9
 lIW5BKtmRMf5zbPwuqicgEAvKrD74SVZd4YRWCEJNyN4N/77pWdnee2CC2Yk2LCgXqeA
 zne1FeGbGtbTEW6/DiWNny2f/V2F4ka+7VMod3gVRVm67G1ya+GGw0hE5zPv5dPF+sPL
 4mv3oN/neQVTx9oNlfyKO2TQjqbAfzW8YEZoG3Y3GgNXjK+jEzo3GoErarZ+coiQF3Uz
 t3KQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=V/K/yOPYT5YcrBKtVFXa5/DN8ftRe1G5E6PhpWar0lY=;
 b=q7I11mYChnh2XzOaw3s6Yzk3/G+JyjECrODHqjloqKlGJkul4ftwt0hmzjL30KK0C2
 MPYfmezOpC3Hg2nSBVag9YZAyAxtcjDZFqRE24w99R+Vbm1JbHfiO7iLdqKFNSnUin2h
 xIl8ZcSMkZxRRJDKJNz8dWR5kZE+E+t/1Avq3rPZzBWG0eum/hsJZvwg4qzWLRBieRDc
 MsrI2Xvm1rPe3hSJ+krWOzwe8new8I9eHF6rwqhDKckE4Y4XzIVP9bng42jZ1NQmJGro
 6/0xVy/WZFL0dyFmoMTAfAj2gKtRya9VPHQy1nwpkdo/8SF93QKvyvQOtYtrqtB51USQ
 j33w==
X-Gm-Message-State: AJIora8483OU6RIEOmlxTL8BJ+s0+iyEA8I57GXpe92B9abyBkSIN0MI
 nIpwO6pzCWFKUtu8zQmLuxM=
X-Google-Smtp-Source: AGRyM1v/9s7RuRKAbq7NH15VXT8D858BUCuocygjakyGJNaH6Ofn1IrC8UvQCFoP1blqHbwyDI+Dlw==
X-Received: by 2002:a05:6870:631a:b0:101:6275:67e7 with SMTP id
 s26-20020a056870631a00b00101627567e7mr15881230oao.114.1655815079404; 
 Tue, 21 Jun 2022 05:37:59 -0700 (PDT)
Received: from localhost ([207.126.88.10]) by smtp.gmail.com with ESMTPSA id
 u10-20020a4ab5ca000000b0041ba884d42csm9421168ooo.42.2022.06.21.05.37.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 Jun 2022 05:37:58 -0700 (PDT)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <834k0eky7h.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
 <8735fyslgq.fsf@localhost> <83fsjyl3sz.fsf@HIDDEN>
 <87r13iqo19.fsf@localhost> <834k0eky7h.fsf@HIDDEN>
Date: Tue, 21 Jun 2022 20:39:10 +0800
Message-ID: <87a6a6qjgx.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> >> Because string width in different buffers may be different depending on
>> >> the fontification, frame font size, face remapping,
>> >> wrap-prefix/line-prefix string properties (AFAIK, the built-in
>> >> string-pixel-width will return incorrect value on string with such
>> >> properties), invisibility specs in the buffer, line numbers mode, etc
>> >> We have implemented a number of workarounds in org-string-width on main,
>> >> but I am not 100% sure that I covered all the edge cases.
>> >
>> > If you need such high accuracy, may I suggest window-text-pixel-size?
>> 
>> window-text-pixel-size suffers from the same issues with
>> wrap-prefix/line-prefix and line numbers mode.
>
> What issue are those?

The length of line-prefix is added to the return value of
window-text-pixel-size, which makes it wrong when the intention is
measuring the actual string width.

Not that window-text-pixel-size is misbehaving here - line-prefix does
need to be included if the intention is to query the actual full width
of text in buffer. The same goes for line numbers mode - line numbers
are a part of window size.

However, using window-text-pixel-size becomes awkward as the means to
measure expected string width when it is going to be inserted into
current buffer.

>> Also, in order to use it in current buffer on not-yet-inserted string,
>> you need to insert it. That where the issue in valign originated from.
>
> The usual method is to use a temporary buffer.

Yes, and it is not accurate because temporary buffer may not have the
same local environment for face remapping, invisibility specs,
char-property-alias, etc

To show all the trickery, let me share org-string-width I had to
implement for the purposes of Org mode. I did not want this function to
be complex and every single extra LOC there is fixing some edge case,
test failure, or bug report:

(defun org-string-width (string &optional pixels)
  "Return width of STRING when displayed in the current buffer.
Return width in pixels when PIXELS is non-nil."
  (if (and (version< emacs-version "28") (not pixels))
      ;; FIXME: Fallback to old limited version, because
      ;; `window-pixel-width' is buggy in older Emacs.
      (org--string-width-1 string)
    ;; Wrap/line prefix will make `window-text-pizel-size' return too
    ;; large value including the prefix.
    (remove-text-properties 0 (length string)
                            '(wrap-prefix t line-prefix t)
                            string)
    ;; Face should be removed to make sure that all the string symbols
    ;; are using default face with constant width.  Constant char width
    ;; is critical to get right string width from pixel width (not needed
    ;; when PIXELS are requested though).
    (unless pixels
      (remove-text-properties 0 (length string) '(face t) string))
    (let (;; We need to remove the folds to make sure that folded table
          ;; alignment is not messed up.
          (current-invisibility-spec
           (or (and (not (listp buffer-invisibility-spec))
                    buffer-invisibility-spec)
               (let (result)
                 (dolist (el buffer-invisibility-spec)
                   (unless (or (memq el
                                     '(org-fold-drawer
                                       org-fold-block
                                       org-fold-outline))
                               (and (listp el)
                                    (memq (car el)
                                          '(org-fold-drawer
                                            org-fold-block
                                            org-fold-outline))))
                     (push el result)))
                 result)))
          (current-char-property-alias-alist char-property-alias-alist))
      (with-temp-buffer
        (setq-local display-line-numbers nil)
        (setq-local buffer-invisibility-spec
                    (if (listp current-invisibility-spec)
                        (mapcar (lambda (el)
                                  ;; Consider elipsis to have 0 width.
                                  ;; It is what Emacs 28+ does, but we have
                                  ;; to force it in earlier Emacs versions.
                                  (if (and (consp el) (cdr el))
                                      (list (car el))
                                    el))
                                current-invisibility-spec)
                      current-invisibility-spec))
        (setq-local char-property-alias-alist
                    current-char-property-alias-alist)
        (let (pixel-width symbol-width)
          (with-silent-modifications
            (setf (buffer-string) string)
            (setq pixel-width
                  (if (get-buffer-window (current-buffer))
                      (car (window-text-pixel-size
                            nil (line-beginning-position) (point-max)))
                    (set-window-buffer nil (current-buffer))
                    (car (window-text-pixel-size
                          nil (line-beginning-position) (point-max)))))
            (unless pixels
              (setf (buffer-string) "a")
              (setq symbol-width
                    (if (get-buffer-window (current-buffer))
                        (car (window-text-pixel-size
                              nil (line-beginning-position) (point-max)))
                      (set-window-buffer nil (current-buffer))
                      (car (window-text-pixel-size
                            nil (line-beginning-position) (point-max)))))))
          (if pixels
              pixel-width
            (/ pixel-width symbol-width)))))))

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 12:48:02 +0000
Resent-Message-ID: <handler.51766.B51766.165581566323454 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165581566323454
          (code B ref 51766); Tue, 21 Jun 2022 12:48:02 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 12:47:43 +0000
Received: from localhost ([127.0.0.1]:59414 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3dIF-00066C-3R
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 08:47:43 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39452)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1o3dID-00065U-HO
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 08:47:41 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43744)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o3dI8-0003dg-79; Tue, 21 Jun 2022 08:47:36 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=cs5EAnaQOYz5b8lbejPqSBs9yR9jxxPPvzvsdKrCJyA=; b=VjHUbPrmtw+g
 OBm7kN3XkGH2X4Gr7h7GR7jsl+zTGN5Xo+eNX81Y3UdObjuy0AxTzfPkYzo0cGHRNLlXw0OiqBfOO
 jejC3GZXwPdfi2HvX5uWpVeTTBlYX4Drx6cgp7yLy/Ksun7ZyjnSID4f0TjKBPvuZ+vWtkf+bPKUE
 w+HejJZajfiUdCKdZBRBz7xV+S/sGcVqNl1yJCE+TXJz2BF/INUN63r1h3TdsGV6pDkyHkR1ydSYQ
 9JK6Lro2IoY+QcGuxX1fdjyQwMUBjTcicomb40Yfa9OVxy+8AqdDPou3d99i5vSLzb3h6zeVUAtgX
 otSBOhNCxhZ9rcyBOi0+vQ==;
Received: from [87.69.77.57] (port=2594 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1o3dI7-0003k7-Lv; Tue, 21 Jun 2022 08:47:35 -0400
Date: Tue, 21 Jun 2022 15:47:24 +0300
Message-Id: <83y1xqji8z.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87a6a6qjgx.fsf@localhost> (message from Ihor Radchenko on Tue,
 21 Jun 2022 20:39:10 +0800)
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
 <8735fyslgq.fsf@localhost> <83fsjyl3sz.fsf@HIDDEN>
 <87r13iqo19.fsf@localhost> <834k0eky7h.fsf@HIDDEN> <87a6a6qjgx.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: monnier@HIDDEN,  51766 <at> debbugs.gnu.org
> Date: Tue, 21 Jun 2022 20:39:10 +0800
> 
> >> > If you need such high accuracy, may I suggest window-text-pixel-size?
> >> 
> >> window-text-pixel-size suffers from the same issues with
> >> wrap-prefix/line-prefix and line numbers mode.
> >
> > What issue are those?
> [...]
> To show all the trickery, let me share org-string-width I had to
> implement for the purposes of Org mode. I did not want this function to
> be complex and every single extra LOC there is fixing some edge case,
> test failure, or bug report:

Ah, so you do use window-text-pixel-size...  Then we are in violent
agreement.

Anyway, the beginning of this sub-thread, specifically about valign,
was in the context of Lisp programs that do buffer modifications under
with-silent-modifications or equivalent, and valign seems to do that
because it just needs to measure the pixel width of a string, and it
does that by inserting the string and then removing it.  So in that
case, the "buffer modifications" are indeed null and void, and Org
shouldn't be bothered by such "modifications", because the buffer
really remains unmodified.  Right?

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations (was: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods)
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 21 Jun 2022 13:03:01 +0000
Resent-Message-ID: <handler.51766.B51766.165581656125139 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 51766 <at> debbugs.gnu.org, monnier@HIDDEN
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165581656125139
          (code B ref 51766); Tue, 21 Jun 2022 13:03:01 +0000
Received: (at 51766) by debbugs.gnu.org; 21 Jun 2022 13:02:41 +0000
Received: from localhost ([127.0.0.1]:59418 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o3dWj-0006XP-H5
	for submit <at> debbugs.gnu.org; Tue, 21 Jun 2022 09:02:41 -0400
Received: from mail-pg1-f179.google.com ([209.85.215.179]:39624)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1o3dWg-0006XD-Cg
 for 51766 <at> debbugs.gnu.org; Tue, 21 Jun 2022 09:02:40 -0400
Received: by mail-pg1-f179.google.com with SMTP id q140so13056659pgq.6
 for <51766 <at> debbugs.gnu.org>; Tue, 21 Jun 2022 06:02:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version; bh=h5YwIdkW9oGy8Q9gioUDBtzYwa6UGmmH4XPwdJL65Lo=;
 b=pFh9Z48+u2WLknzYwCM3ZpI9yCDHCz2cTQuO1314e75D9wKNgBFxLcERyhLvqKxHk4
 RJUvR46YuZL1KixaHjhcSNFyNqwAqJzp2ctgMuLYNGzBGWKK2hN15AjP3P2qvLs4w24m
 jk2hTdNhjHs0s2vLRZ8Pjg7zCrQvaycbq7yctqS4tjf8JkBqBgF6R+PK/yrbfkwMQFzN
 /IEctKSLaBU0nJoqeNi1vfROFTKDCFD+mTBA3DNFxdMa3VyUMhqTUUz/G3GRg9jadvEI
 aD+L2m/7ZJgig0FGgwK6XyGIQKVvxIGaujHW8Kk/BBhWKvs/R5aQa8tCTfCqVXw3C6GV
 AZtA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date
 :message-id:mime-version;
 bh=h5YwIdkW9oGy8Q9gioUDBtzYwa6UGmmH4XPwdJL65Lo=;
 b=RQEY5c6Qap/kSPdet5iZPsbogj2ogrrXrNYSSJ3s9lYxloFLBonfYKFWKKdpVCNImW
 BHushzGnP/3p9WZ5UsnL9TSlN/DNWSzHd04MlFk2knE11skoTpK4Dnxqp7Z67EdGlE0K
 Jt9aHFbS6nq5/3RODeYTk8cL3mCG2o/eTBKi71a1cS4fTQHmFsRLQlwbmaJDpZXsRxl4
 sBXNpIurYWM47L2Zh4sv8cUvsJ2RSlzr5MO13LuoHYsicSNEeWw5bsB0qXLWszXq3EA/
 jq2DkkEIdbhbDKnqyOEYMleHATIxI/mM/dEHT6De8cYxUtoD74pAe2kQV3LnnaXclg3u
 8FxQ==
X-Gm-Message-State: AJIora+84fAD7KumdPTxrnOeBByRmWr5jqMAPNxxgFrhOGSwkXRbl71q
 Jkh/mD0yOx1XcjVZhd+w48s=
X-Google-Smtp-Source: AGRyM1toKTJy0HTUq3eUA20pXMC3LooZsSjbUI2aDOZBO4Ji2xHgVDWnv1PoQJdLLDwQqOP2eo99nw==
X-Received: by 2002:a63:4f04:0:b0:408:8206:5bcd with SMTP id
 d4-20020a634f04000000b0040882065bcdmr26461540pgb.105.1655816552445; 
 Tue, 21 Jun 2022 06:02:32 -0700 (PDT)
Received: from localhost ([155.94.207.39]) by smtp.gmail.com with ESMTPSA id
 ij11-20020a170902ab4b00b0016a0bf0ce32sm7474495plb.70.2022.06.21.06.02.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 Jun 2022 06:02:31 -0700 (PDT)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <83y1xqji8z.fsf@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
 <8735fyslgq.fsf@localhost> <83fsjyl3sz.fsf@HIDDEN>
 <87r13iqo19.fsf@localhost> <834k0eky7h.fsf@HIDDEN>
 <87a6a6qjgx.fsf@localhost> <83y1xqji8z.fsf@HIDDEN>
Date: Tue, 21 Jun 2022 21:03:43 +0800
Message-ID: <874k0eqic0.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.2 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

Note that I changed the topic in this particular branch. My aim here is
to make you aware about the issues with current Emacs tools to measure
pixel-precise string width. Maybe, things can be improved on Emacs side
in this regard.

The below is going back to the initial topic.

Eli Zaretskii <eliz@HIDDEN> writes:

> Anyway, the beginning of this sub-thread, specifically about valign,
> was in the context of Lisp programs that do buffer modifications under
> with-silent-modifications or equivalent, and valign seems to do that
> because it just needs to measure the pixel width of a string, and it
> does that by inserting the string and then removing it.  So in that
> case, the "buffer modifications" are indeed null and void, and Org
> shouldn't be bothered by such "modifications", because the buffer
> really remains unmodified.  Right?

In short, you are right. To clarify the problem on my side goes like:

1. Org has a real issue with bad third-party code inhibiting
   before/after-change function + modifications in indirect buffers not
   always triggering before/after-change
2. Because the issue is critical and can cause data corruption, we
   cannot just ignore it
3. The first attempt to detect "stealthy" modifications was using
   buffer-chars-modified-tick
4. But this method is not reliable because (a) quail does some legit
   edits under inhibit-modification-hooks; (b) some other code, like
   valign also does legit edits under inhibit-modification-hooks
   These buffer modifications are harmless from Org perspective.
5. However, We end up with numerous false-positives using (3) and I am
   clueless how to reliably detect or work around harmful "stealthy"
   edits

   - The suggestion to compare buffer size is helpful, but not 100%
     reliable
   - My other idea to request before/after-change function variants are
     too specific to the problem at hand and may be not good for Emacs
     in a whole

In any case, bug#51766 should not be considered a bug because quail does
modify the buffer and changes in buffer-chars-modified-tick are legit.

Best,
Ihor

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#51766: string-pixel-width limitations
Resent-From: Stefan Monnier <monnier@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Wed, 22 Jun 2022 23:50:02 +0000
Resent-Message-ID: <handler.51766.B51766.165594180017471 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 51766
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: Eli Zaretskii <eliz@HIDDEN>, 51766 <at> debbugs.gnu.org
Received: via spool by 51766-submit <at> debbugs.gnu.org id=B51766.165594180017471
          (code B ref 51766); Wed, 22 Jun 2022 23:50:02 +0000
Received: (at 51766) by debbugs.gnu.org; 22 Jun 2022 23:50:00 +0000
Received: from localhost ([127.0.0.1]:36855 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1o4A6h-0004Xj-U2
	for submit <at> debbugs.gnu.org; Wed, 22 Jun 2022 19:50:00 -0400
Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:37077)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <monnier@HIDDEN>) id 1o4A6d-0004XT-5q
 for 51766 <at> debbugs.gnu.org; Wed, 22 Jun 2022 19:49:58 -0400
Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 6D9DB1003CC;
 Wed, 22 Jun 2022 19:49:49 -0400 (EDT)
Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1])
 by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id CFF9D1001D2;
 Wed, 22 Jun 2022 19:49:47 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca;
 s=mail; t=1655941787;
 bh=AoQcocZdhyohR5liNle22P+iN8FETyJUrUy6/NmfZHA=;
 h=From:To:Cc:Subject:References:Date:In-Reply-To:From;
 b=OT6aerOcJCOKtf7pKeiyEyl0j+K++IrbUg/jze2GqFARlYBatb0cxEuXqeTYqG999
 cbVfHVP608bWDruAlR5E24nuUX9517UAeaww+W/0e2OoEuqL6CCSt7k/u+IwjrXy9t
 hrPZR/vS7AZkVegnHbiCgNf2OnVkkRXEoaBlWfJq0vJCwlfDR/IIxzRtOJNble5uZe
 hakAuR6PGiI1xrP33DUWquNCQ9DB1WLpsV0HUUo3PvS8ssJk+vRcGy8BFQXqESRqOu
 Apns667i7LnU7oGo/X69T3zuN3g6AfKJ4O5bIXLD9HGbytuQcCzBjNive/Q1mkKhRK
 7pTkmm7+lK4pg==
Received: from alfajor (196.214.25.93.rev.sfr.net [93.25.214.196])
 by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 06F8F120377;
 Wed, 22 Jun 2022 19:49:46 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
Message-ID: <jwvh74cqn5x.fsf-monnier+emacs@HIDDEN>
References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@HIDDEN>
 <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@HIDDEN>
 <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@HIDDEN>
 <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@HIDDEN>
 <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@HIDDEN>
 <87r1bkpgjw.fsf@localhost> <jwvpmj86mba.fsf-monnier+emacs@HIDDEN>
 <87y1xv7ggf.fsf@localhost> <838rpvpns3.fsf@HIDDEN>
 <8735fyslgq.fsf@localhost> <83fsjyl3sz.fsf@HIDDEN>
 <87r13iqo19.fsf@localhost> <834k0eky7h.fsf@HIDDEN>
 <87a6a6qjgx.fsf@localhost> <83y1xqji8z.fsf@HIDDEN>
 <874k0eqic0.fsf@localhost>
Date: Wed, 22 Jun 2022 19:49:44 -0400
In-Reply-To: <874k0eqic0.fsf@localhost> (Ihor Radchenko's message of "Tue, 21
 Jun 2022 21:03:43 +0800")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-SPAM-INFO: Spam detection results:  0
 ALL_TRUSTED                -1 Passed through trusted hosts only via SMTP
 AWL 0.087 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain T_SCC_BODY_TEXT_LINE    -0.01 -
X-SPAM-LEVEL: 
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> 1. Org has a real issue with bad third-party code inhibiting
>    before/after-change function + modifications in indirect buffers not
>    always triggering before/after-change
> 2. Because the issue is critical and can cause data corruption, we
>    cannot just ignore it

The bug is not yours.  You don't have to ignore it, but you don't have
to fix it either.

>    - The suggestion to compare buffer size is helpful, but not 100%
>      reliable

To the extent that this just helps to detect other people's bug, I don't
think it needs to be 100%.


        Stefan

Last modified: Thu, 23 Jun 2022 00:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.