GNU bug report logs - #35140
12.1; Scan errors in doctex mode with ^^A-comments after braces

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35140 in the body.
You can then email your comments to 35140 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Markus Kohm <komascript <at> gmx.info>
To: bug-auctex <at> gnu.org
Subject: 12.1; Scan errors in doctex mode with ^^A-comments after braces
Date: Thu, 04 Apr 2019 09:17:47 +0200

[Message part 1 (text/plain, inline)]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.

Be sure to consult the FAQ section in the manual before submitting
a bug report.  In addition check if the bug is reproducable with an
up-to-date version of AUCTeX.  So please upgrade to the version
available from http://www.gnu.org/software/auctex/ if your
installation is older than the one available from the web site.

If the bug is triggered by a specific (La)TeX file, you should try
to produce a minimal sample file showing the problem and include it
in your report.

Your report will be posted for the auctex package at the GNU bug
tracker.  Visit http://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=auctex
to browse existing AUCTeX bugs.
------------------------------------------------------------------------

The LaTeX package doc, that is usually used via class ltxdoc for the
documentation part of dtx-files, declares ^^A as comment character.
But auctex's doctex mode fails scanning for balanced {...} pairs, if
} is immediately followed by ^^A. You can test it, with this example file:

--- test.dtx ---
% \begin{macro}{\test}^^A This is an official dtx-comment
% This does not work. Try to double-click to the closing brace before the
% comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
% \end{macro}
%
% \begin{macro}{\test}%^^A This is a workaround for an official dtx-comment
% This does work, but the \% after the closing brace is not wanted.
% \end{macro}
% \endinput
% Local Variables:
% mode: doctex
% TeX-master: t
% End:
--- end of file ---

Emacs  : GNU Emacs 26.1 (build 1, x86_64-suse-linux-gnu, GTK+ Version 3.24.4)
Package: 12.1

current state:
==============
(setq
 AUCTeX-date "2017-12-10"
 window-system 'x
 LaTeX-version "2e"
 TeX-style-path '("/usr/share/emacs/site-lisp/auctex/auto"
		  "/usr/share/emacs/site-lisp/auctex/style"
		  "/home/mjk/.emacs.d/auctex/auto"
		  "/home/mjk/.emacs.d/auctex/style" "auto" "style")
 TeX-auto-save t
 TeX-parse-self t
 TeX-master t
 TeX-command-list '(("biber" "biber %s.bcf" TeX-run-BibTeX nil t)
		    ("TeX" "%(PDF)%(tex) %`%S%(PDFout)%(mode)%' %t"
		     TeX-run-TeX nil
		     (plain-tex-mode texinfo-mode ams-tex-mode) :help
		     "Run plain TeX")
		    ("LaTeX" "%`%l%(mode)%' %t" TeX-run-TeX nil
		     (latex-mode doctex-mode) :help "Run LaTeX")
		    ("Makeinfo" "makeinfo %t" TeX-run-compile nil
		     (texinfo-mode) :help "Run Makeinfo with Info output")
		    ("Makeinfo HTML" "makeinfo --html %t" TeX-run-compile
nil
		     (texinfo-mode) :help "Run Makeinfo with HTML output")
		    ("AmSTeX" "%(PDF)amstex %`%S%(PDFout)%(mode)%' %t"
		     TeX-run-TeX nil (ams-tex-mode) :help "Run AMSTeX")
		    ("ConTeXt" "texexec --once --texutil %(execopts)%t"
		     TeX-run-TeX nil (context-mode) :help "Run ConTeXt
once")
		    ("ConTeXt Full" "texexec %(execopts)%t" TeX-run-TeX
nil
		     (context-mode) :help "Run ConTeXt until completion")
		    ("BibTeX" "bibtex %s" TeX-run-BibTeX nil t :help
		     "Run BibTeX")
		    ("View" "%V" TeX-run-discard-or-function t t :help
		     "Run Viewer")
		    ("Print" "%p" TeX-run-command t t :help "Print the
file")
		    ("Queue" "%q" TeX-run-background nil t :help
		     "View the printer queue" :visible TeX-queue-command)
		    ("File" "%(o?)dvips %d -o %f " TeX-run-command t t
:help
		     "Generate PostScript file")
		    ("Index" "makeindex %s" TeX-run-command nil t :help
		     "Create index file")
		    ("Check" "lacheck %s" TeX-run-compile nil (latex-mode)
		     :help "Check LaTeX file for correctness")
		    ("Spell" "(TeX-ispell-document \"\")" TeX-run-function
nil
		     t :help "Spell-check the document")

[doctexfailure.dtx (text/x-tex, attachment)]

[auctex-scanerror.png (image/png, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: auctex-devel <at> gnu.org
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Tue, 07 May 2019 12:39:35 +0900

[Message part 1 (text/plain, inline)]

Hi Markus and all developpers,

>>>>> Markus Kohm <komascript <at> gmx.info> writes:

> The LaTeX package doc, that is usually used via class ltxdoc for the
> documentation part of dtx-files, declares ^^A as comment character.
> But auctex's doctex mode fails scanning for balanced {...} pairs, if
> } is immediately followed by ^^A. You can test it, with this example file:

The reason of this error is the syntax-table property put on the closing
curly brace.  In order to achieve different fontification on the docTeX
documentation and the comment after ^^A, the function
`font-latex-doctex-^^A' puts the "comment ender" syntax on the closing
"}", which hides the proper syntax "closing paren paired with {" and
thus leads to the reported error.

Actually, `font-latex-doctex-^^A' is essentially just a copy of
`doctex-font-lock-^^A' in `tex-mode.el', so the emacs built-in
doctex-mode shares exactly the same problem with respect to the given
test file at double-clicking.

It was hard to me to resolve this problem keeping the closing "}" having
"comment ender" syntax, so my proposal is to borrow an idea used at
fontifying the verbatim-like environments.  I.e, to shift the position
of "comment ender" syntax 1 character rightward and put another "comment
starter" syntax on the second, not the first, "^" of ^^A.  The wrong
fontification which the first "^" receives is overwrote afterwards by an
additional entry in `font-latex-doctex-keywords'.  See the attached
patch.

Strictly speaking, this patch sets the boundary of two different types
of comments in "SOMETHING^^A" at wrong position, not between "G" and the
first "^", but between the first and second "^".  However, the treatment
of comments of AUCTeX is rather regexp centric and slight in syntax, I
expect this change doesn't bring serious bad side effects at editing
docTeX documents such as filling and indenting.

How do you think about this idea?  Any comments are appreciated.

Regards,
Ikumi Keita

> --- test.dtx ---
> % \begin{macro}{\test}^^A This is an official dtx-comment
> % This does not work. Try to double-click to the closing brace before the
> % comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
> % \end{macro}
> %
> % \begin{macro}{\test}%^^A This is a workaround for an official dtx-comment
> % This does work, but the \% after the closing brace is not wanted.
> % \end{macro}
> % \endinput
> % Local Variables:
> % mode: doctex
> % TeX-master: t
> % End:
> --- end of file ---

[0001-Improve-comment-syntax-handling-in-docTeX-mode.patch (text/x-diff, attachment)]

Message #11 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: 35140 <at> debbugs.gnu.org
Cc: Markus Kohm <komascript <at> gmx.info>
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Tue, 01 Mar 2022 18:34:51 +0900

[Message part 1 (text/plain, inline)]

Hello Markus and all developpers,

I'd like to discuss this issue again.

After reconsideration, I came to think that there is little point in
discriminating two kinds of comments, i.e. those after "%" and those
after "^^A", as long as AUCTeX is concerned. It requires complicated
operation in syntax propertization (`font-latex-doctex-^^A') whereas it
calls for unwanted troubles like this bug#35140 and comments written in
`font-latex-find-matching-close':
,----
|         ;; XXX: Do not look up syntax-table properties since they may
|         ;; be misleading, e.g. in the case of "{foo}^^A" where the
|         ;; closing brace gets a comment end syntax.
`----

Thus I'm inclined to an idea just to fontify "^^A" type comments with
normal comment face, not bothering with changing its syntax. Attached is
a patch that implements this idea.

What do others think about this?

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

[patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

>>>>> Ikumi Keita <ikumi <at> ikumi.que.jp> writes:
>>>>> Markus Kohm <komascript <at> gmx.info> writes:

>> The LaTeX package doc, that is usually used via class ltxdoc for the
>> documentation part of dtx-files, declares ^^A as comment character.
>> But auctex's doctex mode fails scanning for balanced {...} pairs, if
>> } is immediately followed by ^^A. You can test it, with this example file:

> The reason of this error is the syntax-table property put on the closing
> curly brace.  In order to achieve different fontification on the docTeX
> documentation and the comment after ^^A, the function
> `font-latex-doctex-^^A' puts the "comment ender" syntax on the closing
> "}", which hides the proper syntax "closing paren paired with {" and
> thus leads to the reported error.

> Actually, `font-latex-doctex-^^A' is essentially just a copy of
> `doctex-font-lock-^^A' in `tex-mode.el', so the emacs built-in
> doctex-mode shares exactly the same problem with respect to the given
> test file at double-clicking.

> It was hard to me to resolve this problem keeping the closing "}" having
> "comment ender" syntax, so my proposal is to borrow an idea used at
> fontifying the verbatim-like environments.  I.e, to shift the position
> of "comment ender" syntax 1 character rightward and put another "comment
> starter" syntax on the second, not the first, "^" of ^^A.  The wrong
> fontification which the first "^" receives is overwrote afterwards by an
> additional entry in `font-latex-doctex-keywords'.  See the attached
> patch.

> Strictly speaking, this patch sets the boundary of two different types
> of comments in "SOMETHING^^A" at wrong position, not between "G" and the
> first "^", but between the first and second "^".  However, the treatment
> of comments of AUCTeX is rather regexp centric and slight in syntax, I
> expect this change doesn't bring serious bad side effects at editing
> docTeX documents such as filling and indenting.

> How do you think about this idea?  Any comments are appreciated.

> Regards,
> Ikumi Keita

>> --- test.dtx ---
>> % \begin{macro}{\test}^^A This is an official dtx-comment
>> % This does not work. Try to double-click to the closing brace before the
>> % comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
>> % \end{macro}
>> %
>> % \begin{macro}{\test}%^^A This is a workaround for an official dtx-comment
>> % This does work, but the \% after the closing brace is not wanted.
>> % \end{macro}
>> % \endinput
>> % Local Variables:
>> % mode: doctex
>> % TeX-master: t
>> % End:
>> --- end of file ---

Message #14 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Tue, 01 Mar 2022 12:52:39 +0100

Hi Keita,

Ikumi Keita <ikumi <at> ikumi.que.jp> writes:

> Thus I'm inclined to an idea just to fontify "^^A" type comments with
> normal comment face, not bothering with changing its syntax. Attached is
> a patch that implements this idea.
>
> What do others think about this?

I don't think AUCTeX suffers if the syntax after "^^A" isn't change so
we could do it.  One other idea would be to raise this issue also on
Emacs devel list since Emacs has the same issue as well or ping Stefan
M. to have a look at this thread before we make a final decision.  What
do you think?

> +(defun font-latex-doctex-match-^^A (limit)
> +  "In docTeX mode, match comment started by ^^A before LIMIT."
> +  (catch 'found
> +    (while (TeX-search-forward-unescaped "^^A" limit t)

While we're at it, can we change this to:

  (while (TeX-re-search-forward-unescaped "\\^\\^[AX]" limit t)

as well?  doc v3 also accepts ^^X as comment starter.

> +      (when (eq (char-after (line-beginning-position)) ?\%)
> +        (forward-line 1)
> +        ;; Adjust `font-latex--updated-region-end' if necessary.
> +        (let ((p (point)))
> +          (if (< font-latex--updated-region-end limit)
> +              (setq font-latex--updated-region-end limit))
> +          (when (< font-latex--updated-region-end p)
> +            (font-lock-unfontify-region
> +             font-latex--updated-region-end p)
> +            (setq font-latex--updated-region-end p))
> +          (set-match-data (list (match-beginning 0) p)))
> +        (throw 'found t)))))
>  
>  ;; Copy and adaptation of `doctex-font-lock-syntactic-face-function'
>  ;; in `tex-mode.el' of CVS Emacs (March 2004)

Best, Arash

Message #17 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Arash Esbati <arash <at> gnu.org>
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Tue, 01 Mar 2022 23:50:34 +0900

Hi Arash, thanks for your response.

>>>>> Arash Esbati <arash <at> gnu.org> writes:
> I don't think AUCTeX suffers if the syntax after "^^A" isn't change so
> we could do it.  One other idea would be to raise this issue also on
> Emacs devel list since Emacs has the same issue as well or ping Stefan
> M. to have a look at this thread before we make a final decision.  What
> do you think?

That sounds nice. I'll do that tomorrow.

>> +(defun font-latex-doctex-match-^^A (limit)
>> +  "In docTeX mode, match comment started by ^^A before LIMIT."
>> +  (catch 'found
>> +    (while (TeX-search-forward-unescaped "^^A" limit t)

> While we're at it, can we change this to:

>   (while (TeX-re-search-forward-unescaped "\\^\\^[AX]" limit t)

> as well?  doc v3 also accepts ^^X as comment starter.

Oh, I didn't know that. Then I have to modify there along your
suggestion.

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

Message #20 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: emacs-devel <at> gnu.org
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Wed, 02 Mar 2022 18:01:18 +0900

Greetings, Emacs developers.

I'd like to ask what to do with bug#35140. This was originally posted as
AUCTeX bug, but the same issue applies for Emacs built-in tex-mode.el as
well, as described below. That's the reason why I'm posting to
emacs-devel.

Please have a look at
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35140#5
to see what this bug is.

The reason of this error is the syntax-table property put on the closing
curly brace. In order to achieve different fontification on the docTeX
documentation and the comment after ^^A, the function
`font-latex-doctex-^^A' puts the "comment ender" syntax on the closing
"}", which hides the proper syntax "closing paren paired with {" and
thus leads to the reported error.

Actually, `font-latex-doctex-^^A' is essentially just a copy of
`doctex-font-lock-^^A' in `tex-mode.el', so the Emacs built-in
doctex-mode shares exactly the same problem with respect to the given
test file at double-clicking.

Are there any smart ways to address this issue?

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

Message #23 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, Markus Kohm <komascript <at> gmx.info>,
 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Wed, 02 Mar 2022 09:00:16 -0500

>> Thus I'm inclined to an idea just to fontify "^^A" type comments with
>> normal comment face, not bothering with changing its syntax. Attached is
>> a patch that implements this idea.
>>
>> What do others think about this?
>
> I don't think AUCTeX suffers if the syntax after "^^A" isn't change so
> we could do it.  One other idea would be to raise this issue also on
> Emacs devel list since Emacs has the same issue as well or ping Stefan
> M. to have a look at this thread before we make a final decision.  What
> do you think?

I'm not sufficiently informed about DocTeX to know what's the right
thing to do here.  Changing the code so it only changes the face but
without messing with syntax tables would definitely be simpler.

But one thing catches my attention.  If having the `}` marked as an
end-of-comment "fails scanning for balanced {...} pairs" with the
current setup, wouldn't the same kind of problem appear with

    # \begin{macro}^^A Funny, eh? (

if the `^^A.*` part isn't marked in the syntax as separate?

In which operation does the "fails scanning for balanced {...} pairs"
happen?


        Stefan


PS: It would be nicer if AUCTeX could just use the tex-mode.el code
rather the copy&paste it.

Message #26 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Wed, 02 Mar 2022 23:47:56 +0900

Hello Stefan, thanks for your response.

>>>>> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> In which operation does the "fails scanning for balanced {...} pairs"
> happen?

That's described in the example file test.dtx provided in the original
post by Markus; double-clicking by mouse:
--- test.dtx ---
% \begin{macro}{\test}^^A This is an official dtx-comment
% This does not work. Try to double-click to the closing brace before the
% comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
% \end{macro}
%
% \begin{macro}{\test}%^^A This is a workaround for an official dtx-comment
% This does work, but the \% after the closing brace is not wanted.
% \end{macro}
% \endinput
% Local Variables:
% mode: doctex
% TeX-master: t
% End:
--- end of file ---

> I'm not sufficiently informed about DocTeX to know what's the right
> thing to do here.  Changing the code so it only changes the face but
> without messing with syntax tables would definitely be simpler.

Agreed.

> But one thing catches my attention.  If having the `}` marked as an
> end-of-comment "fails scanning for balanced {...} pairs" with the
> current setup, wouldn't the same kind of problem appear with

>     # \begin{macro}^^A Funny, eh? (

> if the `^^A.*` part isn't marked in the syntax as separate?

I'm not sure I understand what you are worring about. Could you
ellaborate it?

> PS: It would be nicer if AUCTeX could just use the tex-mode.el code
> rather the copy&paste it.

I think that we can do that for `font-latex-doctex-^^A' and
`doctex-font-lock-^^A'. I guess AUCTeX needed separate function because
of compatibility for older emacsen including xemacs.

On the other hand, AUCTeX still needs its own function
`font-latex-doctex-syntactic-face-function' different from
`doctex-font-lock-syntactic-face-function' because we use different
faces for its return value.

Regards,
Ikumi Keita

Message #29 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Wed, 02 Mar 2022 13:15:02 -0500

Ikumi Keita [2022-03-02 23:47:56] wrote:
> Hello Stefan, thanks for your response.
>>>>>> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>> In which operation does the "fails scanning for balanced {...} pairs"
>> happen?
>
> That's described in the example file test.dtx provided in the original
> post by Markus; double-clicking by mouse:

Duh!  Sorry for being dense, I didn't actually read the example's text.
Can't believe it was staring at me!

> --- test.dtx ---
> % \begin{macro}{\test}^^A This is an official dtx-comment
> % This does not work. Try to double-click to the closing brace before the
> % comment and you will get: |Scan error: "Unbalanced parantheses", 22, 22|
> % \end{macro}

So, IIUC the same kind of problem already happens "everywhere" in Emacs
in slightly less trivial cases, such as

    %% \somemacro{some
    %%            multiline arg}

where double clicking on the braces will not do what we want (and will
often signal that same "Unbalanced parantheses" error).

This doesn't imply the OP's problem is negligible, of course, but
I think it does reduce its severity.

It also means that the approach you propose of only changing the `face`
and not the syntax tables is probably good enough.

> I think that we can do that for `font-latex-doctex-^^A' and
> `doctex-font-lock-^^A'. I guess AUCTeX needed separate function because
> of compatibility for older emacsen including xemacs.
>
> On the other hand, AUCTeX still needs its own function
> `font-latex-doctex-syntactic-face-function' different from
> `doctex-font-lock-syntactic-face-function' because we use different
> faces for its return value.

Indeed, practical issue tend to get in the way.
BTW, I noticed in the patch that `font-latex.el` appears to still make
use of `font-lock-syntactic-keywords`.  This var has been obsolete
since Emacs-24.1 and it's like to disappear in Emacs-29, so it would be
good to fix this issue (maybe using `syntax-propertize-via-font-lock`).


        Stefan

Message #32 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, Markus Kohm <komascript <at> gmx.info>,
 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Wed, 02 Mar 2022 21:11:32 +0100

Stefan Monnier via bug-auctex via Bug reporting list for AUCTeX <bug-auctex <at> gnu.org> writes:

> BTW, I noticed in the patch that `font-latex.el` appears to still make
> use of `font-lock-syntactic-keywords`.  This var has been obsolete
> since Emacs-24.1 and it's like to disappear in Emacs-29, so it would
> be good to fix this issue (maybe using
> `syntax-propertize-via-font-lock`).

Thanks for the heads-up, I've done that just now.

  https://git.savannah.gnu.org/cgit/auctex.git/commit/?id=ab3bfaf1033f0d6e0ecfe3172794a6363450b207

Bye,
Tassilo

Message #35 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Markus Kohm <komascript <at> gmx.info>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Thu, 03 Mar 2022 16:05:55 +0900

Hi Stefan,

>>>>> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> It also means that the approach you propose of only changing the `face`
> and not the syntax tables is probably good enough.

Thanks, then I'll go that route.

By the way,
> So, IIUC the same kind of problem already happens "everywhere" in Emacs
> in slightly less trivial cases, such as

>     %% \somemacro{some
>     %%            multiline arg}

> where double clicking on the braces will not do what we want (and will
> often signal that same "Unbalanced parantheses" error).

Surprisingly, it isn't the case. When I double-click on the right brace,
emacs spans active region successfully between "{" and "}", both in
latex-mode and doctex-mode buffer (at least for AUCTeX, I haven't tried
in Emacs built-in tex mode.)

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

Message #38 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, Markus Kohm <komascript <at> gmx.info>,
 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Thu, 03 Mar 2022 09:22:51 +0100

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> I'm not sufficiently informed about DocTeX to know what's the right
> thing to do here.  Changing the code so it only changes the face but
> without messing with syntax tables would definitely be simpler.

Thanks for your comment.  I'm not sure if you want to change this in
tex-mode.el as well, but the current code of
`doctex-syntax-propertize-rules' should be changed like this:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/textmodes/tex-mode.el b/lisp/textmodes/tex-mode.el
index ab94036d01..d43eb5f4cc 100644
--- a/lisp/textmodes/tex-mode.el
+++ b/lisp/textmodes/tex-mode.el
@@ -3019,7 +3019,7 @@ doctex-font-lock-syntactic-face-function
     (syntax-propertize-precompile-rules
      latex-syntax-propertize-rules
      ;; For DocTeX comment-in-doc.
-     ("\\(\\^\\)\\^A" (1 (doctex-font-lock-^^A))))))
+     ("\\(\\^\\)\\^[AX]" (1 (doctex-font-lock-^^A))))))

 (defvar doctex-font-lock-keywords
   (append tex-font-lock-keywords
--8<---------------cut here---------------end--------------->8---

^^X is also a comment starter beginning with doc.sty v3 (which will be
part of next LaTeX release).

Best, Arash

Message #41 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, Markus Kohm <komascript <at> gmx.info>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Thu, 03 Mar 2022 09:45:32 +0100

Hi Tassilo,

Tassilo Horn <tsdh <at> gnu.org> writes:

> Thanks for the heads-up, I've done that just now.
>
>   https://git.savannah.gnu.org/cgit/auctex.git/commit/?id=ab3bfaf1033f0d6e0ecfe3172794a6363450b207

Thanks for the quick fix.  Have you also tried to run 'make' in the
tests directory?  I get errors for three files, e.g. font-latex-test.el:

  Test font-latex-three-dollars condition:
      (error "Font-lock trying to use keywords before setting them up")

  Test font-latex-unclosed-dollars condition:
      (error "Font-lock trying to use keywords before setting them up")

It seems that calling `font-lock-ensure' after `LaTeX-mode' helps:

--8<---------------cut here---------------start------------->8---
diff --git a/tests/latex/font-latex-test.el b/tests/latex/font-latex-test.el
index 9cc7ac52..837f0b47 100644
--- a/tests/latex/font-latex-test.el
+++ b/tests/latex/font-latex-test.el
@@ -39,6 +39,7 @@
               (insert "% $$$ $$$
 $a$")
               (LaTeX-mode)
+              (font-lock-ensure)
               (goto-char (point-min))
               (setq font-latex--updated-region-end (point-max))
               (font-latex-match-dollar-math (point-max))))))
@@ -48,6 +49,7 @@ $a$")
   (let ((TeX-install-font-lock #'font-latex-setup))
     (with-temp-buffer
       (LaTeX-mode)
+      (font-lock-ensure)

       (insert "a$")
       (goto-char (point-min))
--8<---------------cut here---------------end--------------->8---

Is this the course of action or something smarter?

Best, Arash

Message #44 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, Markus Kohm <komascript <at> gmx.info>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 35140 <at> debbugs.gnu.org
Subject: Re: bug#35140: 12.1; Scan errors in doctex mode with ^^A-comments
 after braces
Date: Thu, 03 Mar 2022 10:09:26 +0100

Arash Esbati <arash <at> gnu.org> writes:

Hi Arash,

>> Thanks for the heads-up, I've done that just now.
>>
>>   https://git.savannah.gnu.org/cgit/auctex.git/commit/?id=ab3bfaf1033f0d6e0ecfe3172794a6363450b207
>
> Thanks for the quick fix.  Have you also tried to run 'make' in the
> tests directory?  I get errors for three files, e.g. font-latex-test.el:
>
>   Test font-latex-three-dollars condition:
>       (error "Font-lock trying to use keywords before setting them up")
>
>   Test font-latex-unclosed-dollars condition:
>       (error "Font-lock trying to use keywords before setting them up")
>
> It seems that calling `font-lock-ensure' after `LaTeX-mode' helps:

See my proposed fix in Message-Id: 87a6e7laly.fsf <at> gnu.org.

Bye,
Tassilo

Message #47 received at 35140 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: 35140 <at> debbugs.gnu.org
Cc: Markus Kohm <komascript <at> gmx.info>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#35140: 12.1;
 Scan errors in doctex mode with ^^A-comments after braces
Date: Sat, 05 Mar 2022 16:59:41 +0900

>>>>> Ikumi Keita <ikumi <at> ikumi.que.jp> writes:
>>>>> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>> It also means that the approach you propose of only changing the `face`
>> and not the syntax tables is probably good enough.

> Thanks, then I'll go that route.

Done. Markus, very sorry to have kept you waiting so long period!

Regards,
Ikumi Keita

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.