GNU bug report logs - #65470
29.1.50; js-ts-mode: regex pattern can cause incorrect parenthesis matching

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 65470 in the body.
You can then email your comments to 65470 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.1.50; js-ts-mode: regex pattern can cause incorrect parenthesis
 matching
Date: Wed, 23 Aug 2023 11:05:17 +0200

With the mode js-ts-mode, matching tokens (such as '()', '[]') can be
incorrectly paired by `show-paren-mode`. This is trivially reproducible
with this simple example:

(/foobar)/)

The first parenthesis is matched with the second one, which is inside a
regular expression pattern (between slashes), and the last one is not
paired.

The behavior should be the same as for string, the content of the regex
pattern should have no influence on the structure of the code. The first
parenthesis should match with the third one. Here, the first parenthesis
is matched with the last one:

("foobar)")

js-mode behaves correctly in both cases.



In GNU Emacs 29.1.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.17.8) of 2023-07-30 built on inspiron-5567
Repository revision: 0002d4f31666719c5e972ca65efffc70750db613
Repository branch: makepkg
Windowing system distributor 'The X.Org Foundation', version 11.0.12302000
System Description: Arch Linux

Configured using:
 'configure --prefix=/opt --sysconfdir=/etc --libexecdir=/opt/lib
 --localstatedir=/var --mandir=/opt/share/man --with-gameuser=:games
 --with-modules --without-libotf --without-m17n-flt --without-gconf
 --with-native-compilation=yes --with-native-compilation=aot
 --with-xinput2 --with-x-toolkit=gtk3 --without-xaw3d --with-sound=no
 --with-tree-sitter --without-gpm --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
 PKG_CONFIG_PATH=/home/arch-aug/_opam/lib/pkgconfig
 'CXXFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG
RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP
X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: fr_FR.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Info

Minor modes in effect:
  windmove-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  delete-selection-mode: t
  global-subword-mode: t
  subword-mode: t
  save-place-mode: t
  smartparens-global-mode: t
  treemacs-filewatch-mode: t
  treemacs-follow-mode: t
  treemacs-git-mode: t
  treemacs-fringe-indicator-mode: t
  global-hl-line-mode: t
  editorconfig-mode: t
  drag-stuff-global-mode: t
  drag-stuff-mode: t
  doom-modeline-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  global-auto-revert-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  adaptive-wrap-prefix-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  global-anzu-mode: t
  anzu-mode: t
  which-key-mode: t
  projectile-mode: t
  global-corfu-mode: t
  corfu-mode: t
  TeX-PDF-mode: t
  TeX-source-correlate-mode: t
  vertico-mode: t
  marginalia-mode: t
  override-global-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  savehist-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  isearch-fold-quotes-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  buffer-read-only: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  global-visual-line-mode: t
  visual-line-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/arch-aug/.config/emacs-neo/straight/build/transient/transient 
hides /opt/share/emacs/29.1.50/lisp/transient
/home/arch-aug/.config/emacs-neo/straight/build/jsonrpc/jsonrpc hides 
/opt/share/emacs/29.1.50/lisp/jsonrpc
/home/arch-aug/.config/emacs-neo/straight/build/external-completion/external-completion 
hides /opt/share/emacs/29.1.50/lisp/external-completion
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-lint 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-lint
/home/arch-aug/.config/emacs-neo/straight/build/bind-key/bind-key hides 
/opt/share/emacs/29.1.50/lisp/use-package/bind-key
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-diminish 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-diminish
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-jump 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-jump
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-core 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-core
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-bind-key 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-bind-key
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-delight 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-delight
/home/arch-aug/.config/emacs-neo/straight/build/use-package/use-package-ensure 
hides /opt/share/emacs/29.1.50/lisp/use-package/use-package-ensure
/home/arch-aug/.config/emacs-neo/straight/build/xref/xref hides 
/opt/share/emacs/29.1.50/lisp/progmodes/xref
/home/arch-aug/.config/emacs-neo/straight/build/project/project hides 
/opt/share/emacs/29.1.50/lisp/progmodes/project
/home/arch-aug/.config/emacs-neo/straight/build/flymake/flymake hides 
/opt/share/emacs/29.1.50/lisp/progmodes/flymake
/home/arch-aug/.config/emacs-neo/straight/build/eglot/eglot hides 
/opt/share/emacs/29.1.50/lisp/progmodes/eglot
/home/arch-aug/.config/emacs-neo/straight/build/soap-client/soap-inspect 
hides /opt/share/emacs/29.1.50/lisp/net/soap-inspect
/home/arch-aug/.config/emacs-neo/straight/build/soap-client/soap-client 
hides /opt/share/emacs/29.1.50/lisp/net/soap-client
/home/arch-aug/.config/emacs-neo/straight/build/eldoc/eldoc hides 
/opt/share/emacs/29.1.50/lisp/emacs-lisp/eldoc
/home/arch-aug/.config/emacs-neo/straight/build/let-alist/let-alist 
hides /opt/share/emacs/29.1.50/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr emacsbug cus-start windmove magit-extras
magit-delta xterm-color dabbrev textutils cape combobulate
combobulate-yaml combobulate-css combobulate-js-ts combobulate-python
combobulate-html combobulate-ui combobulate-display combobulate-contrib
combobulate-manipulation python combobulate-navigation combobulate-misc
combobulate-interface combobulate-rules combobulate-settings tempo
misearch multi-isearch noutline outline shortdoc vertico-directory js
c-ts-common mule-util checkdoc doom-snippets doom-snippets-lib yasnippet
rainbow-delimiters ws-butler delsel cap-words superword subword
saveplace flex-mode jison-mode bison-mode cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
bison-mode-autoloads dired-x cmake-mode rst cmake-mode-autoloads utop
utop-minor-mode tuareg tuareg-compat tuareg-opam caml-help find-file
utop-autoloads flycheck-ocaml flycheck find-func
flycheck-ocaml-autoloads merlin-eldoc-autoloads merlin-xref merlin-cap
merlin caml-types merlin-autoloads dune dune-autoloads tuareg-autoloads
caml-autoloads pkgbuild-mode tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat parse-time iso8601 sh-script smie
executable pkgbuild-mode-autoloads geiser-guile info-look geiser-debug
geiser-repl geiser-image geiser-capf geiser-doc geiser-menu
geiser-autodoc geiser-edit etags fileloop generator geiser-completion
geiser-eval geiser-connection tq geiser-syntax scheme geiser-impl
help-fns radix-tree geiser-log geiser-popup view geiser-custom
geiser-base geiser-guile-autoloads geiser geiser-autoloads lua-mode
lua-mode-autoloads debbugs soap-client url-http url-auth url-gw nsm
rng-xsd rng-dt rng-util xsd-regexp debbugs-autoloads
soap-client-autoloads expand-region-autoloads jinx-autoloads smartparens
loadhist smartparens-autoloads builder vc-git vc-dispatcher dir-var
builder-autoloads treemacs treemacs-header-line treemacs-compatibility
treemacs-mode treemacs-bookmarks treemacs-tags xref treemacs-interface
treemacs-persistence treemacs-filewatch-mode treemacs-follow-mode
treemacs-rendering treemacs-annotations treemacs-async
treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator treemacs-scope pulse color treemacs-faces
treemacs-icons treemacs-themes treemacs-core-utils pfuture inline
hl-line ht treemacs-logging treemacs-customization treemacs-macros
treemacs-autoloads cfrs-autoloads posframe-autoloads ht-autoloads
hydra-autoloads lv-autoloads pfuture-autoloads ace-window-autoloads
avy-autoloads hare-mode-autoloads eglot-autoloads
external-completion-autoloads jsonrpc-autoloads flymake-autoloads
project-autoloads xref-autoloads eldoc-autoloads flycheck-autoloads
let-alist-autoloads pkg-info-autoloads epl-autoloads vertico-search
general general-autoloads combobulate-autoloads treesit editorconfig
editorconfig-core editorconfig-core-handle editorconfig-fnmatch pcase
editorconfig-autoloads ws-butler-autoloads bel-coloration
rainbow-delimiters-autoloads drag-stuff drag-stuff-autoloads
doom-modeline doom-modeline-segments doom-modeline-env
doom-modeline-core shrink-path f f-shortdoc s doom-modeline-autoloads
shrink-path-autoloads f-autoloads s-autoloads all-the-icons
all-the-icons-faces data-material data-weathericons data-octicons
data-fileicons data-faicons data-alltheicons all-the-icons-autoloads
doom-themes-ext-org doom-themes-ext-treemacs doom-themes-ext-neotree
doom-themes-ext-visual-bell face-remap doom-dracula-theme doom-themes
doom-themes-base doom-themes-autoloads magit-delta-autoloads
xterm-color-autoloads magit-bookmark magit-submodule magit-obsolete
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit package browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-handlers url-parse
auth-source json map url-vars magit-repos magit-apply magit-wip
magit-log which-func imenu magit-diff smerge-mode diff-mode git-commit
log-edit message sendmail mailcap yank-media puny dired dired-loaddefs
rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config
gnus-util time-date mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils
gmm-utils mailheader pcvs-util add-log magit-core magit-autorevert
autorevert filenotify magit-margin magit-transient magit-process
with-editor shell pcomplete server magit-mode transient magit-git
magit-base magit-section format-spec eieio eieio-core dash
magit-autoloads magit-section-autoloads git-commit-autoloads
with-editor-autoloads transient-autoloads dash-autoloads consult-vertico
consult recentf tree-widget wid-edit bookmark pp consult-autoloads
adaptive-wrap adaptive-wrap-autoloads undo-tree diff queue
undo-tree-autoloads queue-autoloads anzu anzu-autoloads which-key
which-key-autoloads comp comp-cstr rx projectile lisp-mnt grep ibuf-ext
ibuffer ibuffer-loaddefs projectile-autoloads doom-snippets-autoloads
yasnippet-autoloads finder-inf cape-autoloads corfu corfu-autoloads
auctex-latexmk advice latex latex-flymake flymake-proc flymake project
byte-opt compile text-property-search comint ansi-osc ansi-color ring
warnings icons thingatpt tex-ispell tex-style auctex-latexmk-autoloads
tex dbus xml crm texmathp auctex-autoloads tex-site vertico
vertico-autoloads marginalia compat edmacro kmacro marginalia-autoloads
compat-autoloads use-package-bind-key bind-key easy-mmode orderless
orderless-autoloads use-package-ensure use-package-autoloads info
bind-key-autoloads straight-autoloads cl-seq cl-extra help-mode straight
subr-x cl-macs gv use-package-core cl-loaddefs cl-lib bytecomp
byte-compile savehist display-line-numbers elec-pair cus-load rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 855111 185665)
 (symbols 48 62979 2)
 (strings 32 199519 18907)
 (string-bytes 1 6782525)
 (vectors 16 90652)
 (vector-slots 8 2300969 313710)
 (floats 8 1625 1345)
 (intervals 56 20355 3201)
 (buffers 984 25))

Message #8 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Augustin Chéneau (BTuin) <btuin <at> mailo.com>,
 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Wed, 23 Aug 2023 16:23:13 +0300

On 23/08/2023 12:05, Augustin Chéneau (BTuin) wrote:
> 
> With the mode js-ts-mode, matching tokens (such as '()', '[]') can be
> incorrectly paired by `show-paren-mode`. This is trivially reproducible
> with this simple example:
> 
> (/foobar)/)
> 
> The first parenthesis is matched with the second one, which is inside a
> regular expression pattern (between slashes), and the last one is not
> paired.
> 
> The behavior should be the same as for string, the content of the regex
> pattern should have no influence on the structure of the code. The first
> parenthesis should match with the third one. Here, the first parenthesis
> is matched with the last one:
> 
> ("foobar)")
> 
> js-mode behaves correctly in both cases.

Sounds like js-ts-mode also needs a syntax-propertize-function, similar 
to c-ts-mode, ruby-ts-mode and rust-ts-mode.

Others (typescript-ts-mode?) probably need it as well, at least modes 
for those languages that have dedicated regexp or heredoc syntax.

Message #11 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>, Theodor Thornhill <theo <at> thornhill.no>,
 Jostein Kjonigsen <jostein <at> kjonigsen.net>, Yuan Fu <casouri <at> gmail.com>
Cc: btuin <at> mailo.com, 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Thu, 24 Aug 2023 08:59:17 +0300

> Date: Wed, 23 Aug 2023 16:23:13 +0300
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 23/08/2023 12:05, Augustin Chéneau (BTuin) wrote:
> > 
> > With the mode js-ts-mode, matching tokens (such as '()', '[]') can be
> > incorrectly paired by `show-paren-mode`. This is trivially reproducible
> > with this simple example:
> > 
> > (/foobar)/)
> > 
> > The first parenthesis is matched with the second one, which is inside a
> > regular expression pattern (between slashes), and the last one is not
> > paired.
> > 
> > The behavior should be the same as for string, the content of the regex
> > pattern should have no influence on the structure of the code. The first
> > parenthesis should match with the third one. Here, the first parenthesis
> > is matched with the last one:
> > 
> > ("foobar)")
> > 
> > js-mode behaves correctly in both cases.
> 
> Sounds like js-ts-mode also needs a syntax-propertize-function, similar 
> to c-ts-mode, ruby-ts-mode and rust-ts-mode.
> 
> Others (typescript-ts-mode?) probably need it as well, at least modes 
> for those languages that have dedicated regexp or heredoc syntax.

Would someone please add syntax-propertize-function in modes that need
it?  I think this should be done on the emacs-29 branch.

TIA

Message #14 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Theodor Thornhill <theo <at> thornhill.no>
To: Eli Zaretskii <eliz <at> gnu.org>, Dmitry Gutov <dmitry <at> gutov.dev>, Jostein
 Kjonigsen <jostein <at> kjonigsen.net>, Yuan Fu <casouri <at> gmail.com>
Cc: btuin <at> mailo.com, 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause
 incorrect parenthesis matching
Date: Thu, 24 Aug 2023 21:47:09 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Date: Wed, 23 Aug 2023 16:23:13 +0300
>> From: Dmitry Gutov <dmitry <at> gutov.dev>
>> 
>> On 23/08/2023 12:05, Augustin Chéneau (BTuin) wrote:
>> > 
>> > With the mode js-ts-mode, matching tokens (such as '()', '[]') can be
>> > incorrectly paired by `show-paren-mode`. This is trivially reproducible
>> > with this simple example:
>> > 
>> > (/foobar)/)
>> > 
>> > The first parenthesis is matched with the second one, which is inside a
>> > regular expression pattern (between slashes), and the last one is not
>> > paired.
>> > 
>> > The behavior should be the same as for string, the content of the regex
>> > pattern should have no influence on the structure of the code. The first
>> > parenthesis should match with the third one. Here, the first parenthesis
>> > is matched with the last one:
>> > 
>> > ("foobar)")
>> > 
>> > js-mode behaves correctly in both cases.
>> 
>> Sounds like js-ts-mode also needs a syntax-propertize-function, similar 
>> to c-ts-mode, ruby-ts-mode and rust-ts-mode.
>> 
>> Others (typescript-ts-mode?) probably need it as well, at least modes 
>> for those languages that have dedicated regexp or heredoc syntax.
>
> Would someone please add syntax-propertize-function in modes that need
> it?  I think this should be done on the emacs-29 branch.
>
> TIA

I'll add it to my list, but if someone will grab it that's fine, as I'm
a little short on time the next couple of weeks :-(

Theo

Message #17 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Theodor Thornhill <theo <at> thornhill.no>, Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjonigsen <jostein <at> kjonigsen.net>, Yuan Fu <casouri <at> gmail.com>
Cc: btuin <at> mailo.com, 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 25 Aug 2023 03:18:54 +0300

[Message part 1 (text/plain, inline)]

On 24/08/2023 22:47, Theodor Thornhill wrote:
> Eli Zaretskii<eliz <at> gnu.org>  writes:
> 
>>> Date: Wed, 23 Aug 2023 16:23:13 +0300
>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>>
>>> On 23/08/2023 12:05, Augustin Chéneau (BTuin) wrote:
>>>> With the mode js-ts-mode, matching tokens (such as '()', '[]') can be
>>>> incorrectly paired by `show-paren-mode`. This is trivially reproducible
>>>> with this simple example:
>>>>
>>>> (/foobar)/)
>>>>
>>>> The first parenthesis is matched with the second one, which is inside a
>>>> regular expression pattern (between slashes), and the last one is not
>>>> paired.
>>>>
>>>> The behavior should be the same as for string, the content of the regex
>>>> pattern should have no influence on the structure of the code. The first
>>>> parenthesis should match with the third one. Here, the first parenthesis
>>>> is matched with the last one:
>>>>
>>>> ("foobar)")
>>>>
>>>> js-mode behaves correctly in both cases.
>>> Sounds like js-ts-mode also needs a syntax-propertize-function, similar
>>> to c-ts-mode, ruby-ts-mode and rust-ts-mode.
>>>
>>> Others (typescript-ts-mode?) probably need it as well, at least modes
>>> for those languages that have dedicated regexp or heredoc syntax.
>> Would someone please add syntax-propertize-function in modes that need
>> it?  I think this should be done on the emacs-29 branch.
>>
>> TIA
> I'll add it to my list, but if someone will grab it that's fine, as I'm
> a little short on time the next couple of weeks 🙁

This one seems to work for js-ts-mode.

typescript is a bit more fiddly (two separate modes, one with jsx and 
one without), but should be able to follow the similar pattern.

[js-ts--syntax-propertize.diff (text/x-patch, attachment)]

Message #20 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
To: 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 24 Aug 2023 21:31:33 +0200

[Message part 1 (text/plain, inline)]

I have created a patch heavily inspired from ruby-ts-mode.

It fixes the trivial examples and it works as expected in a more complex 
file.
I'm not sure if the commit message is formatted properly.

[0001-Apply-syntax-properties-to-regex-delimiters-in-js-ts.patch (text/x-patch, attachment)]

Message #23 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: btuin <at> mailo.com, jostein <at> kjonigsen.net, casouri <at> gmail.com,
 65470 <at> debbugs.gnu.org, theo <at> thornhill.no
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 25 Aug 2023 08:30:32 +0300

> Date: Fri, 25 Aug 2023 03:18:54 +0300
> Cc: btuin <at> mailo.com, 65470 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> >>> Others (typescript-ts-mode?) probably need it as well, at least modes
> >>> for those languages that have dedicated regexp or heredoc syntax.
> >> Would someone please add syntax-propertize-function in modes that need
> >> it?  I think this should be done on the emacs-29 branch.
> >>
> >> TIA
> > I'll add it to my list, but if someone will grab it that's fine, as I'm
> > a little short on time the next couple of weeks 🙁
> 
> This one seems to work for js-ts-mode.
> 
> typescript is a bit more fiddly (two separate modes, one with jsx and 
> one without), but should be able to follow the similar pattern.

Thanks.  If no issues are found with this within the next couple of
days, please install on the emacs-29 branch.

Message #26 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Theodor Thornhill <theo <at> thornhill.no>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Dmitry Gutov <dmitry <at> gutov.dev>, btuin <at> mailo.com, Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 25 Aug 2023 08:59:49 +0200

> On 24 Aug 2023, at 21:47, Theodor Thornhill <theo <at> thornhill.no> wrote:
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
>> 
>> Would someone please add syntax-propertize-function in modes that need
>> it?  I think this should be done on the emacs-29 branch.
>> 
>> TIA
> 
> I'll add it to my list, but if someone will grab it that's fine, as I'm
> a little short on time the next couple of weeks :-(
> 
> Theo

I’ve just been forced over to a Mac at work, and I can’t (yet) for the life of me figure out how to build Emacs from source on this platform. It seems Emacs is not picking up libraries I’ve installed using homebrew?

If anyone can help me get past this roadblock, I might be able to get a patch going. (It’s a ARM-based Mac, if that matters.)

—
Jostein

Message #29 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 25 Aug 2023 09:17:07 +0200

[Message part 1 (text/plain, inline)]

Le 25/08/2023 à 02:18, Dmitry Gutov a écrit :
> On 24/08/2023 22:47, Theodor Thornhill wrote:
>> Eli Zaretskii<eliz <at> gnu.org>  writes:
>>
>>>> Date: Wed, 23 Aug 2023 16:23:13 +0300
>>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>>>
>>>> On 23/08/2023 12:05, Augustin Chéneau (BTuin) wrote:
>>>>> With the mode js-ts-mode, matching tokens (such as '()', '[]') can be
>>>>> incorrectly paired by `show-paren-mode`. This is trivially 
>>>>> reproducible
>>>>> with this simple example:
>>>>>
>>>>> (/foobar)/)
>>>>>
>>>>> The first parenthesis is matched with the second one, which is 
>>>>> inside a
>>>>> regular expression pattern (between slashes), and the last one is not
>>>>> paired.
>>>>>
>>>>> The behavior should be the same as for string, the content of the 
>>>>> regex
>>>>> pattern should have no influence on the structure of the code. The 
>>>>> first
>>>>> parenthesis should match with the third one. Here, the first 
>>>>> parenthesis
>>>>> is matched with the last one:
>>>>>
>>>>> ("foobar)")
>>>>>
>>>>> js-mode behaves correctly in both cases.
>>>> Sounds like js-ts-mode also needs a syntax-propertize-function, similar
>>>> to c-ts-mode, ruby-ts-mode and rust-ts-mode.
>>>>
>>>> Others (typescript-ts-mode?) probably need it as well, at least modes
>>>> for those languages that have dedicated regexp or heredoc syntax.
>>> Would someone please add syntax-propertize-function in modes that need
>>> it?  I think this should be done on the emacs-29 branch.
>>>
>>> TIA
>> I'll add it to my list, but if someone will grab it that's fine, as I'm
>> a little short on time the next couple of weeks 🙁
> 
> This one seems to work for js-ts-mode.
> 
> typescript is a bit more fiddly (two separate modes, one with jsx and 
> one without), but should be able to follow the similar pattern.

Thanks for the patch, but I still have some issues. I believe that the
START and END arguments of `put-text-property` are off by one. The
attached modified patch seems to work better.

[js-ts--syntax-propertize-v2.diff (text/x-patch, attachment)]

Message #32 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: 65470 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>,
 Dmitry Gutov <dmitry <at> gutov.dev>, btuin <at> mailo.com, Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 25 Aug 2023 11:27:50 -0700

> On Aug 24, 2023, at 11:59 PM, Jostein Kjønigsen <jostein <at> secure.kjonigsen.net> wrote:
> 
> 
> 
>> On 24 Aug 2023, at 21:47, Theodor Thornhill <theo <at> thornhill.no> wrote:
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>>> 
>>> Would someone please add syntax-propertize-function in modes that need
>>> it?  I think this should be done on the emacs-29 branch.
>>> 
>>> TIA
>> 
>> I'll add it to my list, but if someone will grab it that's fine, as I'm
>> a little short on time the next couple of weeks :-(
>> 
>> Theo
> 
> I’ve just been forced over to a Mac at work, and I can’t (yet) for the life of me figure out how to build Emacs from source on this platform. It seems Emacs is not picking up libraries I’ve installed using homebrew?
> 
> If anyone can help me get past this roadblock, I might be able to get a patch going. (It’s a ARM-based Mac, if that matters.)

It’s been quite a while since I needed to setup this, so I don’t quite remember. But setting LD_LIBRARY_PATH (Homebrew’s path) and PKG_CONFIG_PATH (/usr/local/lib/pkgconfig) should help the build process and Emacs find the libraries.

Yuan

Message #35 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
Cc: 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Sat, 26 Aug 2023 04:52:35 +0300

[Message part 1 (text/plain, inline)]

Hi Augustin!

On 25/08/2023 10:17, Augustin Chéneau (BTuin) wrote:
> Thanks for the patch, but I still have some issues. I believe that the
> START and END arguments of `put-text-property` are off by one. The
> attached modified patch seems to work better.

Thanks for trying it out. Sorry about missing your first patch: 
apparently while it was the first time-wise, it was help up in moderation.

Anyway, you make a good point about the case with the regexp: since I 
matched the pattern inside, the bounds of the node are off by one 
compared to the case with the jsx node. Your patch moves those bounds, 
but then they'll be wrong for the latter case.

Here's an update which tries to handle both, but I guess if we add more 
node types later we'll just have to stop trying to handle them the same 
way (and do it closer to ruby-ts--syntax-propertize).

Also added the new return_statement case for jsx.

[js-ts--syntax-propertize-v2.diff (text/x-patch, attachment)]

Message #38 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 65470 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>,
 Dmitry Gutov <dmitry <at> gutov.dev>, btuin <at> mailo.com, Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Sat, 26 Aug 2023 11:22:39 +0200

> On 25 Aug 2023, at 20:28, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>>> On Aug 24, 2023, at 11:59 PM, Jostein Kjønigsen <jostein <at> secure.kjonigsen.net> wrote:
>>> 
>>> 
>>> 
>>>> On 24 Aug 2023, at 21:47, Theodor Thornhill <theo <at> thornhill.no> wrote:
>>> 
>>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>> 
>>>> 
>>>> Would someone please add syntax-propertize-function in modes that need
>>>> it?  I think this should be done on the emacs-29 branch.
>>>> 
>>>> TIA
>>> 
>>> I'll add it to my list, but if someone will grab it that's fine, as I'm
>>> a little short on time the next couple of weeks :-(
>>> 
>>> Theo
>> 
>> I’ve just been forced over to a Mac at work, and I can’t (yet) for the life of me figure out how to build Emacs from source on this platform. It seems Emacs is not picking up libraries I’ve installed using homebrew?
>> 
>> If anyone can help me get past this roadblock, I might be able to get a patch going. (It’s a ARM-based Mac, if that matters.)
> 
> It’s been quite a while since I needed to setup this, so I don’t quite remember. But setting LD_LIBRARY_PATH (Homebrew’s path) and PKG_CONFIG_PATH (/usr/local/lib/pkgconfig) should help the build process and Emacs find the libraries.
> 
> Yuan

I figured it out in the end. For anyone suffering a similar problem:

I had all packages I needed installed, but not pkg-config, leading configure not being able to locate any of them. 

Installing pkg-config too and running “make bootstrap” got me a working build. 

With that out of the way, I’ll see if I can come up with some patches for the typescript ts-modes sometime the next week. 

—
Jostein

Message #41 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Fu Yuan <casouri <at> gmail.com>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: 65470 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>,
 Dmitry Gutov <dmitry <at> gutov.dev>, btuin <at> mailo.com, Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Sat, 26 Aug 2023 08:29:17 -0700

>> 
>>>> On Aug 24, 2023, at 11:59 PM, Jostein Kjønigsen <jostein <at> secure.kjonigsen.net> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On 24 Aug 2023, at 21:47, Theodor Thornhill <theo <at> thornhill.no> wrote:
>>>> 
>>>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>>> 
>>>>> 
>>>>> Would someone please add syntax-propertize-function in modes that need
>>>>> it?  I think this should be done on the emacs-29 branch.
>>>>> 
>>>>> TIA
>>>> 
>>>> I'll add it to my list, but if someone will grab it that's fine, as I'm
>>>> a little short on time the next couple of weeks :-(
>>>> 
>>>> Theo
>>> 
>>> I’ve just been forced over to a Mac at work, and I can’t (yet) for the life of me figure out how to build Emacs from source on this platform. It seems Emacs is not picking up libraries I’ve installed using homebrew?
>>> 
>>> If anyone can help me get past this roadblock, I might be able to get a patch going. (It’s a ARM-based Mac, if that matters.)
>> 
>> It’s been quite a while since I needed to setup this, so I don’t quite remember. But setting LD_LIBRARY_PATH (Homebrew’s path) and PKG_CONFIG_PATH (/usr/local/lib/pkgconfig) should help the build process and Emacs find the libraries.
>> 
>> Yuan
> 
> I figured it out in the end. For anyone suffering a similar problem:
> 
> I had all packages I needed installed, but not pkg-config, leading configure not being able to locate any of them. 
> 
> Installing pkg-config too and running “make bootstrap” got me a working build. 
> 
> With that out of the way, I’ll see if I can come up with some patches for the typescript ts-modes sometime the next week. 

Good to hear that! Exiles you are at it, could you also have a look at JSX? I think the latest tree-sitter grammar breaks some queries.

Yuan

Message #44 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Fu Yuan <casouri <at> gmail.com>
Cc: 65470 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>,
 Dmitry Gutov <dmitry <at> gutov.dev>, btuin <at> mailo.com, Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Sat, 26 Aug 2023 23:13:15 +0200

> On 26 Aug 2023, at 17:29, Fu Yuan <casouri <at> gmail.com> wrote:
> 
> 
>>> 
>>>>> On Aug 24, 2023, at 11:59 PM, Jostein Kjønigsen <jostein <at> secure.kjonigsen.net> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 24 Aug 2023, at 21:47, Theodor Thornhill <theo <at> thornhill.no> wrote:
>>>>> 
>>>>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>>>> 
>>>>>> 
>>>>>> Would someone please add syntax-propertize-function in modes that need
>>>>>> it?  I think this should be done on the emacs-29 branch.
>>>>>> 
>>>>>> TIA
>>>>> 
>>>>> I'll add it to my list, but if someone will grab it that's fine, as I'm
>>>>> a little short on time the next couple of weeks :-(
>>>>> 
>>>>> Theo
>>>> 
>>>> I’ve just been forced over to a Mac at work, and I can’t (yet) for the life of me figure out how to build Emacs from source on this platform. It seems Emacs is not picking up libraries I’ve installed using homebrew?
>>>> 
>>>> If anyone can help me get past this roadblock, I might be able to get a patch going. (It’s a ARM-based Mac, if that matters.)
>>> 
>>> It’s been quite a while since I needed to setup this, so I don’t quite remember. But setting LD_LIBRARY_PATH (Homebrew’s path) and PKG_CONFIG_PATH (/usr/local/lib/pkgconfig) should help the build process and Emacs find the libraries.
>>> 
>>> Yuan
>> 
>> I figured it out in the end. For anyone suffering a similar problem:
>> 
>> I had all packages I needed installed, but not pkg-config, leading configure not being able to locate any of them. 
>> 
>> Installing pkg-config too and running “make bootstrap” got me a working build. 
>> 
>> With that out of the way, I’ll see if I can come up with some patches for the typescript ts-modes sometime the next week. 
> 
> Good to hear that! Exiles you are at it, could you also have a look at JSX? I think the latest tree-sitter grammar breaks some queries.
> 
> Yuan

I don’t really work with JSX and don’t think have a “idiomatic” code base to try it on where I would notice brokenness. 

I think this would be better handled by people more experienced with both JSX and our JSX code base than I am. 

—
Jostein

Message #47 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>,
 Fu Yuan <casouri <at> gmail.com>
Cc: btuin <at> mailo.com, Jostein Kjønigsen <jostein <at> kjonigsen.net>,
 Eli Zaretskii <eliz <at> gnu.org>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Sun, 27 Aug 2023 00:45:19 +0300

On 27/08/2023 00:13, Jostein Kjønigsen wrote:
> I don’t really work with JSX and don’t think have a “idiomatic” code base to try it on where I would notice brokenness.
> 
> I think this would be better handled by people more experienced with both JSX and our JSX code base than I am.

Just checking that viewing some JSX code in .tsx files doesn't show 
errors in Messages might be good enough.

I tried that with javascript, didn't notice any problems so far.

Message #50 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 31 Aug 2023 12:41:13 +0300

> Date: Sun, 27 Aug 2023 00:45:19 +0300
> Cc: Theodor Thornhill <theo <at> thornhill.no>, Eli Zaretskii <eliz <at> gnu.org>,
>  Jostein Kjønigsen <jostein <at> kjonigsen.net>, btuin <at> mailo.com,
>  65470 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 27/08/2023 00:13, Jostein Kjønigsen wrote:
> > I don’t really work with JSX and don’t think have a “idiomatic” code base to try it on where I would notice brokenness.
> > 
> > I think this would be better handled by people more experienced with both JSX and our JSX code base than I am.
> 
> Just checking that viewing some JSX code in .tsx files doesn't show 
> errors in Messages might be good enough.
> 
> I tried that with javascript, didn't notice any problems so far.

I'm a bit confused by this discussion.  Should I install the patch on
the emacs-29 branch now, or are you still discussing whether it's
correct?

Message #53 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 31 Aug 2023 14:15:22 +0300

On 31/08/2023 12:41, Eli Zaretskii wrote:
>> Date: Sun, 27 Aug 2023 00:45:19 +0300
>> Cc: Theodor Thornhill<theo <at> thornhill.no>, Eli Zaretskii<eliz <at> gnu.org>,
>>   Jostein Kjønigsen<jostein <at> kjonigsen.net>,btuin <at> mailo.com,
>>   65470 <at> debbugs.gnu.org
>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>
>> On 27/08/2023 00:13, Jostein Kjønigsen wrote:
>>> I don’t really work with JSX and don’t think have a “idiomatic” code base to try it on where I would notice brokenness.
>>>
>>> I think this would be better handled by people more experienced with both JSX and our JSX code base than I am.
>> Just checking that viewing some JSX code in .tsx files doesn't show
>> errors in Messages might be good enough.
>>
>> I tried that with javascript, didn't notice any problems so far.
> I'm a bit confused by this discussion.  Should I install the patch on
> the emacs-29 branch now, or are you still discussing whether it's
> correct?

I think we were discussing the typescript modes here.

Personally I was waiting for some additional confirmation that my v2 for 
js-ts-mode is correct, but I guess we have a silent approval, so we can 
install it.

Message #56 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 31 Aug 2023 15:53:11 +0300

> Date: Thu, 31 Aug 2023 14:15:22 +0300
> Cc: jostein <at> secure.kjonigsen.net, casouri <at> gmail.com, theo <at> thornhill.no,
>  jostein <at> kjonigsen.net, btuin <at> mailo.com, 65470 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 31/08/2023 12:41, Eli Zaretskii wrote:
> > I'm a bit confused by this discussion.  Should I install the patch on
> > the emacs-29 branch now, or are you still discussing whether it's
> > correct?
> 
> I think we were discussing the typescript modes here.
> 
> Personally I was waiting for some additional confirmation that my v2 for 
> js-ts-mode is correct, but I guess we have a silent approval, so we can 
> install it.

Good to hear.  Then please install it, and thanks.

Message #59 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 1 Sep 2023 04:42:26 +0300

On 31/08/2023 15:53, Eli Zaretskii wrote:
>> Date: Thu, 31 Aug 2023 14:15:22 +0300
>> Cc:jostein <at> secure.kjonigsen.net,casouri <at> gmail.com,theo <at> thornhill.no,
>>   jostein <at> kjonigsen.net,btuin <at> mailo.com,65470 <at> debbugs.gnu.org
>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>
>> On 31/08/2023 12:41, Eli Zaretskii wrote:
>>> I'm a bit confused by this discussion.  Should I install the patch on
>>> the emacs-29 branch now, or are you still discussing whether it's
>>> correct?
>> I think we were discussing the typescript modes here.
>>
>> Personally I was waiting for some additional confirmation that my v2 for
>> js-ts-mode is correct, but I guess we have a silent approval, so we can
>> install it.
> Good to hear.  Then please install it, and thanks.

All right, installed one for js-ts-mode.

Message #62 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 1 Sep 2023 17:45:17 +0200

Le 31/08/2023 à 13:15, Dmitry Gutov a écrit :
> On 31/08/2023 12:41, Eli Zaretskii wrote:
>>> Date: Sun, 27 Aug 2023 00:45:19 +0300
>>> Cc: Theodor Thornhill<theo <at> thornhill.no>, Eli Zaretskii<eliz <at> gnu.org>,
>>>   Jostein Kjønigsen<jostein <at> kjonigsen.net>,btuin <at> mailo.com,
>>>   65470 <at> debbugs.gnu.org
>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>>
>>> On 27/08/2023 00:13, Jostein Kjønigsen wrote:
>>>> I don’t really work with JSX and don’t think have a “idiomatic” code 
>>>> base to try it on where I would notice brokenness.
>>>>
>>>> I think this would be better handled by people more experienced with 
>>>> both JSX and our JSX code base than I am.
>>> Just checking that viewing some JSX code in .tsx files doesn't show
>>> errors in Messages might be good enough.
>>>
>>> I tried that with javascript, didn't notice any problems so far.
>> I'm a bit confused by this discussion.  Should I install the patch on
>> the emacs-29 branch now, or are you still discussing whether it's
>> correct?
> 
> I think we were discussing the typescript modes here.
> 
> Personally I was waiting for some additional confirmation that my v2 for 
> js-ts-mode is correct, but I guess we have a silent approval, so we can 
> install it.

Whoops, sorry.  I confirm this fixed my issue with js-ts-mode.
Thank you for your patch!

Message #67 received at 65470-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
Cc: dmitry <at> gutov.dev, 65470-done <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Fri, 01 Sep 2023 18:58:55 +0300

> Cc: 65470 <at> debbugs.gnu.org
> Date: Fri, 1 Sep 2023 17:45:17 +0200
> From: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
> 
> > Personally I was waiting for some additional confirmation that my v2 for 
> > js-ts-mode is correct, but I guess we have a silent approval, so we can 
> > install it.
> 
> Whoops, sorry.  I confirm this fixed my issue with js-ts-mode.
> Thank you for your patch!

Thanks, I'm therefore closing the bug.

Message #70 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Augustin Chéneau (BTuin) <btuin <at> mailo.com>
Cc: 65470 <at> debbugs.gnu.org
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 1 Sep 2023 22:21:08 +0300

On 01/09/2023 18:45, Augustin Chéneau (BTuin) wrote:
>>>>
>>> I'm a bit confused by this discussion.  Should I install the patch on
>>> the emacs-29 branch now, or are you still discussing whether it's
>>> correct?
>>
>> I think we were discussing the typescript modes here.
>>
>> Personally I was waiting for some additional confirmation that my v2 
>> for js-ts-mode is correct, but I guess we have a silent approval, so 
>> we can install it.
> 
> Whoops, sorry.  I confirm this fixed my issue with js-ts-mode.
> Thank you for your patch!

Thanks for testing!

Message #73 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Tue, 5 Sep 2023 21:31:25 +0200

[Message part 1 (text/plain, inline)]

> On 1 Sep 2023, at 03:42, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 31/08/2023 15:53, Eli Zaretskii wrote:
>>> Date: Thu, 31 Aug 2023 14:15:22 +0300
>>> Cc:jostein <at> secure.kjonigsen.net,casouri <at> gmail.com,theo <at> thornhill.no,
>>>  jostein <at> kjonigsen.net,btuin <at> mailo.com,65470 <at> debbugs.gnu.org
>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>> 
>>> On 31/08/2023 12:41, Eli Zaretskii wrote:
>>>> I'm a bit confused by this discussion.  Should I install the patch on
>>>> the emacs-29 branch now, or are you still discussing whether it's
>>>> correct?
>>> I think we were discussing the typescript modes here.
>>> 
>>> Personally I was waiting for some additional confirmation that my v2 for
>>> js-ts-mode is correct, but I guess we have a silent approval, so we can
>>> install it.
>> Good to hear.  Then please install it, and thanks.
> 
> All right, installed one for js-ts-mode.

Sorry for the somewhat slow response on this. Life has been busy for me as well :)

Based on the infamous “copy, paste & adapt” methodology I’ve come up with a patch for typescript-ts-mode and tsx-ts-mode which seems to work for the simple test case provided above.

However the patch for js-ts-mode has lots of … interesting stuff about jsx I have not included, for no other reason than not understanding what type of use-cases they are meant to support.

If someone can provide me some examples for the JSX use-cases, I can try to make room for TSX-variants of the same code.

The patch so far is attached. It’s intentionally -not- optimized to leave room for TSX cases which may arise, so hopefully no need to nitpick this yet.

[0001-typescript-ts-mode-Fix-syntax-properties-for-regexp-.patch (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

—
Jostein

Message #78 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: btuin <at> mailo.com, dmitry <at> gutov.dev, casouri <at> gmail.com, 65470 <at> debbugs.gnu.org,
 theo <at> thornhill.no
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 07 Sep 2023 11:59:37 +0300

> From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
> Date: Tue, 5 Sep 2023 21:31:25 +0200
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>  Yuan Fu <casouri <at> gmail.com>,
>  Theodor Thornhill <theo <at> thornhill.no>,
>  Jostein Kjønigsen <jostein <at> kjonigsen.net>,
>  btuin <at> mailo.com,
>  65470 <at> debbugs.gnu.org
> 
> Sorry for the somewhat slow response on this. Life has been busy for me as well :)
> 
> Based on the infamous “copy, paste & adapt” methodology I’ve come up with a patch for typescript-ts-mode and tsx-ts-mode which seems to work for the simple test case provided above.
> 
> However the patch for js-ts-mode has lots of … interesting stuff about jsx I have not included, for no other reason than not understanding what type of use-cases they are meant to support.
> 
> If someone can provide me some examples for the JSX use-cases, I can try to make room for TSX-variants of the same code.
> 
> The patch so far is attached. It’s intentionally -not- optimized to leave room for TSX cases which may arise, so hopefully no need to nitpick this yet.

Thanks, Jostein.

Any objections to my installing this on the emacs-29 branch, anyone?

Message #81 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: btuin <at> mailo.com, dmitry <at> gutov.dev, casouri <at> gmail.com, 65470 <at> debbugs.gnu.org,
 theo <at> thornhill.no
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 7 Sep 2023 02:02:17 -0700

Eli Zaretskii <eliz <at> gnu.org> writes:

> Any objections to my installing this on the emacs-29 branch, anyone?

No objections here.

Message #84 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Thu, 7 Sep 2023 15:09:39 +0300

On 05/09/2023 22:31, Jostein Kjønigsen wrote:
> The patch so far is attached. It’s intentionally -not- optimized to 
leave room for TSX cases which may arise, so hopefully no need to 
nitpick this yet.

I was going to nitpick it, but then read the rest of your message ;-(

> However the patch for js-ts-mode has lots of … interesting stuff about jsx I have not included, for no other reason than not understanding what type of use-cases they are meant to support.
> 
> If someone can provide me some examples for the JSX use-cases, I can try to make room for TSX-variants of the same code.

The idea was to "enclose" every TSX in "generic string" syntax so that 
whatever unpaired characters are inside (such as (, ", ...), won't 
affect syntax-ppss status on the outside. I'm not sure how critical that 
is, but I guess some users might encounter such situations.

Examples like:

  ReactDOM.render(
    <div className="">
      <h1>Hello, Welcome to React and TypeScript ;-(</h1>
    </div>,
    // type closing paren here and see that it's matched to opener above
    document.getElementById("root")
  );

Message #87 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Mon, 11 Sep 2023 21:37:33 +0200

[Message part 1 (text/plain, inline)]

> On 7 Sep 2023, at 14:09, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 05/09/2023 22:31, Jostein Kjønigsen wrote:
> > The patch so far is attached. It’s intentionally -not- optimized to leave room for TSX cases which may arise, so hopefully no need to nitpick this yet.
> 
> I was going to nitpick it, but then read the rest of your message ;-(
> 
>> However the patch for js-ts-mode has lots of … interesting stuff about jsx I have not included, for no other reason than not understanding what type of use-cases they are meant to support.
>> If someone can provide me some examples for the JSX use-cases, I can try to make room for TSX-variants of the same code.
> 
> The idea was to "enclose" every TSX in "generic string" syntax so that whatever unpaired characters are inside (such as (, ", ...), won't affect syntax-ppss status on the outside. I'm not sure how critical that is, but I guess some users might encounter such situations.
> 
> Examples like:
> 
>  ReactDOM.render(
>    <div className="">
>      <h1>Hello, Welcome to React and TypeScript ;-(</h1>
>    </div>,
>    // type closing paren here and see that it's matched to opener above
>    document.getElementById("root")
>  );
> 

Ok. That makes sense, and explains everything. I’ve tried implementing the same thing for tsx-ts-mode too in the attached patch. (You may start nitpicking now).

I first tried using a more naive and wide approach only trying to capture full (jsx_element) blocks without any further constraints.
What I found was that when having nested elements (which one tend to have all the time), the syntax-propertizing would cancel itself for every second nesting.

So to fix that, I’ve tried to anchor the query to top-level constructs which typically does not nest. Looking at the query for js.el, I can only assume that you were trying to too solve the same problem, Dmitry?

But without further ado… Here’s the patch. Do your worst, gentlemen :D

[0001-typescript-ts-mode-Fix-syntax-properties-for-regexp-.patch (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

—
Jostein

Message #90 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Tue, 12 Sep 2023 01:23:18 +0300

Hi Jostein!

The patch looks good overall (though I haven't tested it).

On 11/09/2023 22:37, Jostein Kjønigsen wrote:

> So to fix that, I’ve tried to anchor the query to top-level 
constructs which typically does not nest. Looking at the query for 
js.el, I can only assume that you were trying to too solve the same 
problem, Dmitry?

That's right.

> +                             ((arguments (jsx_element) @jsx))
> +                             ((parenthesized_expression (jsx_element) @jsx))))))

These two contexts actually seem missing in my patch for js-ts-mode. :-)

OTOH, it seems like the ones that _are_ included there, are missing 
here. Don't they?

  ((variable_declarator value: (jsx_element) @jsx))

corresponds do

  let v = <div>...</div>

  ((assignment_expression right: (jsx_element) @jsx))

corresponds do

  v = <p>...<p>

(with variable declared previously), and

  ((return_statement (jsx_element) @jsx))

corresponds to

  function f() {
    return <div>...<div>;
  }

Message #93 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Tue, 12 Sep 2023 08:29:41 +0200

[Message part 1 (text/plain, inline)]

On 12 Sep 2023, at 00:23, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> Hi Jostein!
> 
> The patch looks good overall (though I haven't tested it).
> 
> On 11/09/2023 22:37, Jostein Kjønigsen wrote:
> 
> > So to fix that, I’ve tried to anchor the query to top-level constructs which typically does not nest. Looking at the query for js.el, I can only assume that you were trying to too solve the same problem, Dmitry?
> 
> That's right.
> 
>> +                             ((arguments (jsx_element) @jsx))
>> +                             ((parenthesized_expression (jsx_element) @jsx))))))
> 
> These two contexts actually seem missing in my patch for js-ts-mode. :-)
> 
> OTOH, it seems like the ones that _are_ included there, are missing here. Don't they?
> 
>  ((variable_declarator value: (jsx_element) @jsx))
> 
> corresponds do
> 
>  let v = <div>...</div>
> 
>  ((assignment_expression right: (jsx_element) @jsx))
> 
> corresponds do
> 
>  v = <p>...<p>
> 
> (with variable declared previously), and
> 
>  ((return_statement (jsx_element) @jsx))
> 
> corresponds to
> 
>  function f() {
>    return <div>...<div>;
>  }

Good feedback. I’ve tested and included those as well. Attached is a new patch.

[0001-typescript-ts-mode-Fix-syntax-properties-for-regexp-.patch (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

—
Jostein

Message #96 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Wed, 13 Sep 2023 02:14:59 +0300

On 12/09/2023 09:29, Jostein Kjønigsen wrote:
> Good feedback. I’ve tested and included those as well. Attached is a new patch.

Thanks! Installed. It would've been better with a commit message too 
(next time).

Speaking of the TSX difficulties, though, which Yuan hinted at recently: 
I think the current fix (tsx-ts-mode--font-lock-compatibility-bb1f97b) 
is not working. To be clear, that's unrelated to the current patch.

Basically, the query for ((member_expression) @capture) succeeds, but 
then the more complex one fails with

Structure error at: 22
(jsx_opening_element [(member_expression (identifier)) (identifier)] 
@typescript-ts-jsx-tag-face) (jsx_closing_element [(member_expression 
(identifier)) (identifier)] @typescript-ts-jsx-tag-face) 
(jsx_self_closing_element [(member_expression (identifier)) 
(identifier)] @typescript-ts-jsx-tag-face)

This patch seems to fix that and bring JSX syntax highlighting back:

diff --git a/lisp/progmodes/typescript-ts-mode.el 
b/lisp/progmodes/typescript-ts-mode.el
index 57382c9cb31..7108ff49b28 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -163,7 +163,7 @@ tsx-ts-mode--font-lock-compatibility-bb1f97b
   ;; but then raises an error if the wrong node type is used. So it is
   ;; important to check with the new node type (member_expression)
   (condition-case nil
-      (progn (treesit-query-capture language '((member_expression) 
@capture))
+      (progn (treesit-query-capture language '(jsx_opening_element 
(member_expression) @capture))
 	     '((jsx_opening_element
 		[(member_expression (identifier)) (identifier)]
 		@typescript-ts-jsx-tag-face)

Message #99 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 15 Sep 2023 14:11:12 +0200

> On 13 Sep 2023, at 01:14, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 12/09/2023 09:29, Jostein Kjønigsen wrote:
>> Good feedback. I’ve tested and included those as well. Attached is a new patch.
> 
> Thanks! Installed. It would've been better with a commit message too (next time).
> 
> Speaking of the TSX difficulties, though, which Yuan hinted at recently: I think the current fix (tsx-ts-mode--font-lock-compatibility-bb1f97b) is not working. To be clear, that's unrelated to the current patch.
> 
> Basically, the query for ((member_expression) @capture) succeeds, but then the more complex one fails with
> 
> Structure error at: 22
> (jsx_opening_element [(member_expression (identifier)) (identifier)] @typescript-ts-jsx-tag-face) (jsx_closing_element [(member_expression (identifier)) (identifier)] @typescript-ts-jsx-tag-face) (jsx_self_closing_element [(member_expression (identifier)) (identifier)] @typescript-ts-jsx-tag-face)
> 
> This patch seems to fix that and bring JSX syntax highlighting back:
> 
> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
> index 57382c9cb31..7108ff49b28 100644
> --- a/lisp/progmodes/typescript-ts-mode.el
> +++ b/lisp/progmodes/typescript-ts-mode.el
> @@ -163,7 +163,7 @@ tsx-ts-mode--font-lock-compatibility-bb1f97b
>   ;; but then raises an error if the wrong node type is used. So it is
>   ;; important to check with the new node type (member_expression)
>   (condition-case nil
> -      (progn (treesit-query-capture language '((member_expression) @capture))
> +      (progn (treesit-query-capture language '(jsx_opening_element (member_expression) @capture))
> 	     '((jsx_opening_element
> 		[(member_expression (identifier)) (identifier)]
> 		@typescript-ts-jsx-tag-face)
> 

I really don’t know this part of the code very well. If you’ve identified an issue/bug for a specific TSX use-case, and found a fix for it, I have no objections to it being applied.

—
Jostein

Message #102 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Yuan Fu <casouri <at> gmail.com>, 65470 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, btuin <at> mailo.com,
 Eli Zaretskii <eliz <at> gnu.org>,
 Jostein Kjønigsen <jostein <at> kjonigsen.net>
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Fri, 15 Sep 2023 16:35:55 +0300

On 15/09/2023 15:11, Jostein Kjønigsen wrote:
>> This patch seems to fix that and bring JSX syntax highlighting back:
>>
>> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
>> index 57382c9cb31..7108ff49b28 100644
>> --- a/lisp/progmodes/typescript-ts-mode.el
>> +++ b/lisp/progmodes/typescript-ts-mode.el
>> @@ -163,7 +163,7 @@ tsx-ts-mode--font-lock-compatibility-bb1f97b
>>    ;; but then raises an error if the wrong node type is used. So it is
>>    ;; important to check with the new node type (member_expression)
>>    (condition-case nil
>> -      (progn (treesit-query-capture language '((member_expression) @capture))
>> +      (progn (treesit-query-capture language '(jsx_opening_element (member_expression) @capture))
>> 	     '((jsx_opening_element
>> 		[(member_expression (identifier)) (identifier)]
>> 		@typescript-ts-jsx-tag-face)
>>
> I really don’t know this part of the code very well. If you’ve identified an issue/bug for a specific TSX use-case, and found a fix for it, I have no objections to it being applied.

No problem, pushed that to emacs-29, thanks.

Just wanted to notify in advance in case somebody had an opinion on this 
change.

Message #105 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Sat, 16 Sep 2023 08:54:32 +0300

> Date: Fri, 15 Sep 2023 16:35:55 +0300
> Cc: Eli Zaretskii <eliz <at> gnu.org>, Yuan Fu <casouri <at> gmail.com>,
>  Theodor Thornhill <theo <at> thornhill.no>,
>  Jostein Kjønigsen <jostein <at> kjonigsen.net>, btuin <at> mailo.com,
>  65470 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 15/09/2023 15:11, Jostein Kjønigsen wrote:
> >> This patch seems to fix that and bring JSX syntax highlighting back:
> >>
> >> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
> >> index 57382c9cb31..7108ff49b28 100644
> >> --- a/lisp/progmodes/typescript-ts-mode.el
> >> +++ b/lisp/progmodes/typescript-ts-mode.el
> >> @@ -163,7 +163,7 @@ tsx-ts-mode--font-lock-compatibility-bb1f97b
> >>    ;; but then raises an error if the wrong node type is used. So it is
> >>    ;; important to check with the new node type (member_expression)
> >>    (condition-case nil
> >> -      (progn (treesit-query-capture language '((member_expression) @capture))
> >> +      (progn (treesit-query-capture language '(jsx_opening_element (member_expression) @capture))
> >> 	     '((jsx_opening_element
> >> 		[(member_expression (identifier)) (identifier)]
> >> 		@typescript-ts-jsx-tag-face)
> >>
> > I really don’t know this part of the code very well. If you’ve identified an issue/bug for a specific TSX use-case, and found a fix for it, I have no objections to it being applied.
> 
> No problem, pushed that to emacs-29, thanks.

This seems to cause:

    INFO     Scraping files for loaddefs...85%
  typescript-ts-mode:0: Warning: Not registering prefix "ts".  Affects: ("tsx-ts-mode--indent-compatibility-b893426" "tsx-ts-mode--font-lock-compatibility-bb1f97b" "tsx-ts--s-p-query" "tsx-ts--syntax-propertize" "ts-ts--s-p-query" "ts-ts--syntax-propertize" "ts-ts--syntax-propertize-captures")

Stefan, what do we usually do with such warnings?

Message #108 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dmitry <at> gutov.dev
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, monnier <at> iro.umontreal.ca, btuin <at> mailo.com
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Sat, 16 Sep 2023 14:20:16 +0300

> Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
>  jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
> Date: Sat, 16 Sep 2023 08:54:32 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > Date: Fri, 15 Sep 2023 16:35:55 +0300
> > Cc: Eli Zaretskii <eliz <at> gnu.org>, Yuan Fu <casouri <at> gmail.com>,
> >  Theodor Thornhill <theo <at> thornhill.no>,
> >  Jostein Kjønigsen <jostein <at> kjonigsen.net>, btuin <at> mailo.com,
> >  65470 <at> debbugs.gnu.org
> > From: Dmitry Gutov <dmitry <at> gutov.dev>
> > 
> > On 15/09/2023 15:11, Jostein Kjønigsen wrote:
> > >> This patch seems to fix that and bring JSX syntax highlighting back:
> > >>
> > >> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
> > >> index 57382c9cb31..7108ff49b28 100644
> > >> --- a/lisp/progmodes/typescript-ts-mode.el
> > >> +++ b/lisp/progmodes/typescript-ts-mode.el
> > >> @@ -163,7 +163,7 @@ tsx-ts-mode--font-lock-compatibility-bb1f97b
> > >>    ;; but then raises an error if the wrong node type is used. So it is
> > >>    ;; important to check with the new node type (member_expression)
> > >>    (condition-case nil
> > >> -      (progn (treesit-query-capture language '((member_expression) @capture))
> > >> +      (progn (treesit-query-capture language '(jsx_opening_element (member_expression) @capture))
> > >> 	     '((jsx_opening_element
> > >> 		[(member_expression (identifier)) (identifier)]
> > >> 		@typescript-ts-jsx-tag-face)
> > >>
> > > I really don’t know this part of the code very well. If you’ve identified an issue/bug for a specific TSX use-case, and found a fix for it, I have no objections to it being applied.
> > 
> > No problem, pushed that to emacs-29, thanks.
> 
> This seems to cause:
> 
>     INFO     Scraping files for loaddefs...85%
>   typescript-ts-mode:0: Warning: Not registering prefix "ts".  Affects: ("tsx-ts-mode--indent-compatibility-b893426" "tsx-ts-mode--font-lock-compatibility-bb1f97b" "tsx-ts--s-p-query" "tsx-ts--syntax-propertize" "ts-ts--s-p-query" "ts-ts--syntax-propertize" "ts-ts--syntax-propertize-captures")
> 
> Stefan, what do we usually do with such warnings?

This also causes byte-compilation warnings on a system that has no
tree-sitter installed:

  In end of data:
  progmodes/typescript-ts-mode.el:498:17: Warning: the function ‘treesit-node-end’ is not known to be defined.
  progmodes/typescript-ts-mode.el:497:17: Warning: the function ‘treesit-node-start’ is not known to be defined.
  progmodes/typescript-ts-mode.el:474:6: Warning: the function ‘treesit-query-compile’ is not known to be defined.

Could you please fix these?

Message #111 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dmitry <at> gutov.dev, casouri <at> gmail.com
Cc: btuin <at> mailo.com, jostein <at> secure.kjonigsen.net, 65470 <at> debbugs.gnu.org,
 theo <at> thornhill.no, monnier <at> iro.umontreal.ca
Subject: Re: bug#65470: 29.1.50;
 js-ts-mode: regex pattern can cause incorrect parenthesis matching
Date: Sat, 16 Sep 2023 14:40:02 +0300

> Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
>  jostein <at> secure.kjonigsen.net, monnier <at> iro.umontreal.ca, btuin <at> mailo.com
> Date: Sat, 16 Sep 2023 14:20:16 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
> >  jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
> > Date: Sat, 16 Sep 2023 08:54:32 +0300
> > From: Eli Zaretskii <eliz <at> gnu.org>
> > 
> > > Date: Fri, 15 Sep 2023 16:35:55 +0300
> > > Cc: Eli Zaretskii <eliz <at> gnu.org>, Yuan Fu <casouri <at> gmail.com>,
> > >  Theodor Thornhill <theo <at> thornhill.no>,
> > >  Jostein Kjønigsen <jostein <at> kjonigsen.net>, btuin <at> mailo.com,
> > >  65470 <at> debbugs.gnu.org
> > > From: Dmitry Gutov <dmitry <at> gutov.dev>
> > > 
> > > On 15/09/2023 15:11, Jostein Kjønigsen wrote:
> > > >> This patch seems to fix that and bring JSX syntax highlighting back:
> > > >>
> > > >> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
> > > >> index 57382c9cb31..7108ff49b28 100644
> > > >> --- a/lisp/progmodes/typescript-ts-mode.el
> > > >> +++ b/lisp/progmodes/typescript-ts-mode.el
> > > >> @@ -163,7 +163,7 @@ tsx-ts-mode--font-lock-compatibility-bb1f97b
> > > >>    ;; but then raises an error if the wrong node type is used. So it is
> > > >>    ;; important to check with the new node type (member_expression)
> > > >>    (condition-case nil
> > > >> -      (progn (treesit-query-capture language '((member_expression) @capture))
> > > >> +      (progn (treesit-query-capture language '(jsx_opening_element (member_expression) @capture))
> > > >> 	     '((jsx_opening_element
> > > >> 		[(member_expression (identifier)) (identifier)]
> > > >> 		@typescript-ts-jsx-tag-face)
> > > >>
> > > > I really don’t know this part of the code very well. If you’ve identified an issue/bug for a specific TSX use-case, and found a fix for it, I have no objections to it being applied.
> > > 
> > > No problem, pushed that to emacs-29, thanks.
> > 
> > This seems to cause:
> > 
> >     INFO     Scraping files for loaddefs...85%
> >   typescript-ts-mode:0: Warning: Not registering prefix "ts".  Affects: ("tsx-ts-mode--indent-compatibility-b893426" "tsx-ts-mode--font-lock-compatibility-bb1f97b" "tsx-ts--s-p-query" "tsx-ts--syntax-propertize" "ts-ts--s-p-query" "ts-ts--syntax-propertize" "ts-ts--syntax-propertize-captures")
> > 
> > Stefan, what do we usually do with such warnings?
> 
> This also causes byte-compilation warnings on a system that has no
> tree-sitter installed:
> 
>   In end of data:
>   progmodes/typescript-ts-mode.el:498:17: Warning: the function ‘treesit-node-end’ is not known to be defined.
>   progmodes/typescript-ts-mode.el:497:17: Warning: the function ‘treesit-node-start’ is not known to be defined.
>   progmodes/typescript-ts-mode.el:474:6: Warning: the function ‘treesit-query-compile’ is not known to be defined.
> 
> Could you please fix these?

I fixed this myself by simply declaring the functions the
byte-compiler was complaining about, but I'd be happier if someone
could explain these warnings, since the offending functions are not
called anywhere in the code I could see.  What did I miss?

Message #114 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 Dmitry Gutov <dmitry <at> gutov.dev>, jostein <at> secure.kjonigsen.net, btuin <at> mailo.com,
 jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause
 incorrect parenthesis matching
Date: Sat, 16 Sep 2023 09:59:11 -0400

>> No problem, pushed that to emacs-29, thanks.
>
> This seems to cause:
>
>     INFO     Scraping files for loaddefs...85%
>   typescript-ts-mode:0: Warning: Not registering prefix "ts".  Affects: ("tsx-ts-mode--indent-compatibility-b893426" "tsx-ts-mode--font-lock-compatibility-bb1f97b" "tsx-ts--s-p-query" "tsx-ts--syntax-propertize" "ts-ts--s-p-query" "ts-ts--syntax-propertize" "ts-ts--syntax-propertize-captures")
>
> Stefan, what do we usually do with such warnings?

We usually ignored them?
[ They only affect the features controlled by
  `help-enable-(completion-)autoload`.  ]

A better option, is to try and use fewer namespace prefixes.
e.g. make sure all the definitions start with `ts-`, `ts-ts-`,
`tsts-`, or somesuch (except for the autoloaded ones which can break
that rule).

Maybe rename `typescript-ts-mode-*` to `ts-ts-mode-*` (and
make `typescript-ts-mode` an alias of `ts-ts-mode`).


        Stefan


diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 7c85a7b6fe9..8c50b55535b 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -87,7 +87,7 @@ tsx-ts-mode--indent-compatibility-b893426
       (progn (treesit-query-capture 'tsx '((jsx_fragment) @capture))
              `(((match "<" "jsx_fragment") parent 0)
                ((parent-is "jsx_fragment") parent typescript-ts-mode-indent-offset)))
-    (error
+    (error ;; FIXME: Use more precise error name.
      `(((match "<" "jsx_text") parent 0)
        ((parent-is "jsx_text") parent typescript-ts-mode-indent-offset)))))
 
@@ -414,8 +414,11 @@ typescript-ts-mode--sexp-nodes
   "Nodes that designate sexps in TypeScript.
 See `treesit-thing-settings' for more information.")
 
+;; FIXME: Do we really need to autoload this?
 ;;;###autoload
 (define-derived-mode typescript-ts-base-mode prog-mode "TypeScript"
+  ;; FIXME: This is the same docstring as `typescript-ts-mode',
+  ;; we should document the difference here!
   "Major mode for editing TypeScript."
   :group 'typescript
   :syntax-table typescript-ts-mode--syntax-table

Message #117 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>, Eli Zaretskii <eliz <at> gnu.org>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, btuin <at> mailo.com, jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Sat, 16 Sep 2023 23:05:35 +0300

On 16/09/2023 16:59, Stefan Monnier wrote:
>>> No problem, pushed that to emacs-29, thanks.
>> This seems to cause:
>>
>>      INFO     Scraping files for loaddefs...85%
>>    typescript-ts-mode:0: Warning: Not registering prefix "ts".  Affects: ("tsx-ts-mode--indent-compatibility-b893426" "tsx-ts-mode--font-lock-compatibility-bb1f97b" "tsx-ts--s-p-query" "tsx-ts--syntax-propertize" "ts-ts--s-p-query" "ts-ts--syntax-propertize" "ts-ts--syntax-propertize-captures")
>>
>> Stefan, what do we usually do with such warnings?
> We usually ignored them?
> [ They only affect the features controlled by
>    `help-enable-(completion-)autoload`.  ]
> 
> A better option, is to try and use fewer namespace prefixes.
> e.g. make sure all the definitions start with `ts-`, `ts-ts-`,
> `tsts-`, or somesuch (except for the autoloaded ones which can break
> that rule).
> 
> Maybe rename `typescript-ts-mode-*` to `ts-ts-mode-*` (and
> make `typescript-ts-mode` an alias of `ts-ts-mode`).

Thanks. I haven't seen the warnings myself, but I sorted the new 
addition into two existing prefixes in that file: typescript-ts and tsx-ts.

> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
> index 7c85a7b6fe9..8c50b55535b 100644
> --- a/lisp/progmodes/typescript-ts-mode.el
> +++ b/lisp/progmodes/typescript-ts-mode.el
> @@ -87,7 +87,7 @@ tsx-ts-mode--indent-compatibility-b893426
>         (progn (treesit-query-capture 'tsx '((jsx_fragment) @capture))
>                `(((match "<" "jsx_fragment") parent 0)
>                  ((parent-is "jsx_fragment") parent typescript-ts-mode-indent-offset)))
> -    (error
> +    (error ;; FIXME: Use more precise error name.
>        `(((match "<" "jsx_text") parent 0)
>          ((parent-is "jsx_text") parent typescript-ts-mode-indent-offset)))))
>   
> @@ -414,8 +414,11 @@ typescript-ts-mode--sexp-nodes
>     "Nodes that designate sexps in TypeScript.
>   See `treesit-thing-settings' for more information.")
>   
> +;; FIXME: Do we really need to autoload this?
>   ;;;###autoload

Don't think so. Though I'm not sure if we should change that in emacs-29.

>   (define-derived-mode typescript-ts-base-mode prog-mode "TypeScript"
> +  ;; FIXME: This is the same docstring as `typescript-ts-mode',
> +  ;; we should document the difference here!
>     "Major mode for editing TypeScript."
>     :group 'typescript
>     :syntax-table typescript-ts-mode--syntax-table
> 

Fixed two other things on the release branch, seemed safe enough.

Message #120 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Eli Zaretskii <eliz <at> gnu.org>, casouri <at> gmail.com
Cc: btuin <at> mailo.com, jostein <at> secure.kjonigsen.net, 65470 <at> debbugs.gnu.org,
 theo <at> thornhill.no, monnier <at> iro.umontreal.ca
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Sat, 16 Sep 2023 23:07:50 +0300

On 16/09/2023 14:40, Eli Zaretskii wrote:
>> This also causes byte-compilation warnings on a system that has no
>> tree-sitter installed:
>>
>>    In end of data:
>>    progmodes/typescript-ts-mode.el:498:17: Warning: the function ‘treesit-node-end’ is not known to be defined.
>>    progmodes/typescript-ts-mode.el:497:17: Warning: the function ‘treesit-node-start’ is not known to be defined.
>>    progmodes/typescript-ts-mode.el:474:6: Warning: the function ‘treesit-query-compile’ is not known to be defined.
>>
>> Could you please fix these?
> I fixed this myself by simply declaring the functions the
> byte-compiler was complaining about, but I'd be happier if someone
> could explain these warnings, since the offending functions are not
> called anywhere in the code I could see.  What did I miss?

treesit-node-start/end are called in tsx-ts--syntax-propertize-captures 
(note the rename I just performed), and treesit-query-compile is called 
in *-ts--s-p-query's init forms.

Both added in commit 1fb2fb501f3 reviewed by yours truly, so, sorry 
about that.

Message #123 received at 65470 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: casouri <at> gmail.com, 65470 <at> debbugs.gnu.org, theo <at> thornhill.no,
 jostein <at> secure.kjonigsen.net, monnier <at> iro.umontreal.ca, btuin <at> mailo.com,
 jostein <at> kjonigsen.net
Subject: Re: bug#65470: 29.1.50; js-ts-mode: regex pattern can cause incorrect
 parenthesis matching
Date: Sun, 17 Sep 2023 08:22:00 +0300

> Date: Sat, 16 Sep 2023 23:05:35 +0300
> Cc: jostein <at> secure.kjonigsen.net, casouri <at> gmail.com, theo <at> thornhill.no,
>  jostein <at> kjonigsen.net, btuin <at> mailo.com, 65470 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 16/09/2023 16:59, Stefan Monnier wrote:
> >>> No problem, pushed that to emacs-29, thanks.
> >> This seems to cause:
> >>
> >>      INFO     Scraping files for loaddefs...85%
> >>    typescript-ts-mode:0: Warning: Not registering prefix "ts".  Affects: ("tsx-ts-mode--indent-compatibility-b893426" "tsx-ts-mode--font-lock-compatibility-bb1f97b" "tsx-ts--s-p-query" "tsx-ts--syntax-propertize" "ts-ts--s-p-query" "ts-ts--syntax-propertize" "ts-ts--syntax-propertize-captures")
> >>
> >> Stefan, what do we usually do with such warnings?
> > We usually ignored them?
> > [ They only affect the features controlled by
> >    `help-enable-(completion-)autoload`.  ]
> > 
> > A better option, is to try and use fewer namespace prefixes.
> > e.g. make sure all the definitions start with `ts-`, `ts-ts-`,
> > `tsts-`, or somesuch (except for the autoloaded ones which can break
> > that rule).
> > 
> > Maybe rename `typescript-ts-mode-*` to `ts-ts-mode-*` (and
> > make `typescript-ts-mode` an alias of `ts-ts-mode`).
> 
> Thanks. I haven't seen the warnings myself, but I sorted the new 
> addition into two existing prefixes in that file: typescript-ts and tsx-ts.

Thanks, compiles cleanly now.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.