GNU bug report logs - #41195
27.0.91; Using syntax-ppss-flush-cache in syntax-propertize-function infloops

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41195 in the body.
You can then email your comments to 41195 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.91; Using syntax-ppss-flush-cache in syntax-propertize-function
 infloops
Date: Tue, 12 May 2020 02:32:37 +0300

In theory, syntax-propertize-function implementations shouldn't change
buffer syntax the way that would affect the current syntax-ppss cache.
Or if they do, they should clear the cache accordingly.

Unfortunately, using syntax-ppss-flush-cache for that purpose doesn't
work: it makes Emacs infloop.

To reproduce, apply this patch:

diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 5ec3e94275..efe38049ab 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -2362,6 +2362,7 @@ js-syntax-propertize
                            (eval-when-compile (append "=({[,:;" 
'(nil))))))
            (put-text-property (match-beginning 1) (match-end 1)
                               'syntax-table (string-to-syntax "\"/"))
+           (syntax-ppss-flush-cache (match-beginning 0))
            (js-syntax-propertize-regexp end)))))
     ("\`\(#\)!" (1 "< b"))
     ("<" (0 (ignore

and then edit a file with js-mode containing

  a = /xyz/

Message #10 received at 41195-close <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 41195-close <at> debbugs.gnu.org
Subject: Re: bug#41195: 27.0.91; Using syntax-ppss-flush-cache in
 syntax-propertize-function infloops
Date: Tue, 12 May 2020 16:55:18 -0400

Version: 28.1

> In theory, syntax-propertize-function implementations shouldn't change
> buffer syntax the way that would affect the current syntax-ppss cache.
> Or if they do, they should clear the cache accordingly.
> Unfortunately, using syntax-ppss-flush-cache for that purpose doesn't
> work: it makes Emacs infloop.

I thought at first that the fix should go into `emacs-27`, but then you
correctly pointed out that the docstring of `syntax-propertize-function`
warned that `syntax-ppss-flush-cache` should not be called, so this bug
is really a known and documented limitation.

But indeed, it's a limitation that we should lift, so I installed the
patch below into `master`.


        Stefan


diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
index 3294378754a..46dc8d9ade8 100644
--- a/lisp/emacs-lisp/syntax.el
+++ b/lisp/emacs-lisp/syntax.el
@@ -63,9 +63,10 @@ syntax-propertize-function
 cannot be handled just by the buffer's syntax-table.
 
 The specified function may call `syntax-ppss' on any position
-before END, but it should not call `syntax-ppss-flush-cache',
-which means that it should not call `syntax-ppss' on some
-position and later modify the buffer on some earlier position.")
+before END, but if it calls `syntax-ppss' on some
+position and later modifies the buffer on some earlier position,
+then it is its responsability to call `syntax-ppss-flush-cache' to flush
+the now obsolete ppss info from the cache.")
 
 (defvar syntax-propertize-chunk-size 500)
 
@@ -320,6 +321,11 @@ syntax-propertize-via-font-lock
 (defvar-local syntax-ppss-table nil
   "Syntax-table to use during `syntax-ppss', if any.")
 
+(defvar-local syntax-propertize--inhibit-flush nil
+  "If non-nil, `syntax-ppss-flush-cache' only flushes the ppss cache.
+Otherwise it flushes both the ppss cache and the properties
+set by `syntax-propertize'")
+
 (defun syntax-propertize (pos)
   "Ensure that syntax-table properties are set until POS (a buffer point)."
   (when (< syntax-propertize--done pos)
@@ -375,8 +381,13 @@ syntax-propertize
               ;; (message "syntax-propertizing from %s to %s" start end)
               (remove-text-properties start end
                                       '(syntax-table nil syntax-multiline nil))
-              ;; Avoid recursion!
-              (let ((syntax-propertize--done most-positive-fixnum))
+              ;; Make sure we only let-bind it buffer-locally.
+              (make-local-variable 'syntax-propertize--inhibit-flush)
+              ;; Let-bind `syntax-propertize--done' to avoid infinite recursion!
+              (let ((syntax-propertize--done most-positive-fixnum)
+                    ;; Let `syntax-propertize-function' call
+                    ;; `syntax-ppss-flush-cache' without worries.
+                    (syntax-propertize--inhibit-flush t))
                 (funcall syntax-propertize-function start end)))))))))
 
 ;;; Link syntax-propertize with syntax.c.
@@ -455,7 +466,8 @@ 'syntax-ppss-after-change-function
 (defun syntax-ppss-flush-cache (beg &rest ignored)
   "Flush the cache of `syntax-ppss' starting at position BEG."
   ;; Set syntax-propertize to refontify anything past beg.
-  (setq syntax-propertize--done (min beg syntax-propertize--done))
+  (unless syntax-propertize--inhibit-flush
+    (setq syntax-propertize--done (min beg syntax-propertize--done)))
   ;; Flush invalid cache entries.
   (dolist (cell (list syntax-ppss-wide syntax-ppss-narrow))
     (pcase cell

Message #13 received at 41195-close <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 41195-close <at> debbugs.gnu.org
Subject: Re: bug#41195: 27.0.91; Using syntax-ppss-flush-cache in
 syntax-propertize-function infloops
Date: Wed, 13 May 2020 00:05:04 +0300

On 12.05.2020 23:55, Stefan Monnier wrote:
> I thought at first that the fix should go into `emacs-27`, but then you
> correctly pointed out that the docstring of `syntax-propertize-function`
> warned that `syntax-ppss-flush-cache` should not be called, so this bug
> is really a known and documented limitation.
> 
> But indeed, it's a limitation that we should lift, so I installed the
> patch below into `master`.

OK. Thank you.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.