GNU bug report logs - #48078
Verbatim environments with key-value arguments

Previous Next

Package: auctex;

Reported by: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>

Date: Wed, 28 Apr 2021 09:12:01 UTC

Severity: normal

Done: Tassilo Horn <tsdh <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 48078 in the body.
You can then email your comments to 48078 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Wed, 28 Apr 2021 09:12:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-auctex <at> gnu.org. (Wed, 28 Apr 2021 09:12:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>
To: bug-auctex <at> gnu.org
Subject: Verbatim environments with key-value arguments
Date: Wed, 28 Apr 2021 11:11:07 +0200
[Message part 1 (text/plain, inline)]
Hi,

I am writing styles for the minted and listings libraries of tcolorbox,
which introduce the tcblisting verbatim environment.  Here is a small
example showing things:

[tcblisting.tex (text/x-tex, inline)]
\documentclass[preview,svgnames,x11names]{standalone}

\RequirePackage[newfloat]{minted}
\RequirePackage{tcolorbox}
\tcbuselibrary{most,minted}

\begin{document}
\begin{tcblisting}{%
listing only,%
title={Example shell session},
minted language=shell-session,%
minted options={},%
}
$ echo 'Hello, World!'
Hello, World!
\end{tcblisting}
\end{document}

% Local Variables:
% mode: latex
% TeX-command-extra-options: "-shell-escape"
% TeX-master: t
% End:
[Message part 3 (text/plain, inline)]
As you can see, tcblisting takes key-value arguments.  This breaks the
"mandatory arguments to verbatim environments" part of
`font-latex-syntactic-keywords', which is defined as:

    ;; After the optional argument, there may also be
    ;; another mandatory argument(s) (e.g. with VerbatimOut or
    ;; the minted envs or defined with `lstnewenvironment').
    "\\(?:{[^}]*}\\)*"

Since the key-value arguments _can_ contain braces (e.g., title={Example
shell session}), the regex does not match properly and messes up the
fontification.

Feel free to add "tcblisting" to `LaTeX-verbtaim-environments' and check
out the example above: You will see that everything after title={Example
shell session} is fontified as verbatim.

I would be happy to try and improve the regex so that it handles
key-value arguments, but I first wanted to ask if this is indeed and the
best approach.

Best regards,

-- 
Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>
Key fingerprint = F7C3 734D 2381 DAEB 4C6D  9CF7 744A 4F0B 4F1C 9371
$ gpg --keyserver   hkps://hkps.pool.sks-keyservers.net \
      --search-keys 744A4F0B4F1C9371

Information forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Wed, 28 Apr 2021 19:44:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>
Cc: bug-auctex <at> gnu.org, 48078-done <at> debbugs.gnu.org
Subject: Re: bug#48078: Verbatim environments with key-value arguments
Date: Wed, 28 Apr 2021 20:41:16 +0200
Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com> writes:

Hi Dario,

> \begin{tcblisting}{%
> listing only,%
> title={Example shell session},
> minted language=shell-session,%
> minted options={},%
> }
> $ echo 'Hello, World!'
> Hello, World!
> \end{tcblisting}
>
> Feel free to add "tcblisting" to `LaTeX-verbtaim-environments' and
> check out the example above: You will see that everything after
> title={Example shell session} is fontified as verbatim.
>
> I would be happy to try and improve the regex so that it handles
> key-value arguments,

To handle all situations correctly is impossible.  You can't properly
match nested constructs like balanced parens/braces using regular
expressions.  For that, you'd need an actual parser.

But you can at least allow a fixed number of balanced braces, and I've
just done that in order to allow one level of nested {...} in mandatory
arguments.  (We already allowed for one level of nested brackets in an
optional argument.)

I'll make a new ELPA release.

Thanks for the report and Bye,
Tassilo




Reply sent to Tassilo Horn <tsdh <at> gnu.org>:
You have taken responsibility. (Wed, 28 Apr 2021 19:44:02 GMT) Full text and rfc822 format available.

Notification sent to Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>:
bug acknowledged by developer. (Wed, 28 Apr 2021 19:44:03 GMT) Full text and rfc822 format available.

Information forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Thu, 29 Apr 2021 17:34:03 GMT) Full text and rfc822 format available.

Message #16 received at 48078 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>, 48078 <at> debbugs.gnu.org
Subject: Re: bug#48078: Verbatim environments with key-value arguments
Date: Thu, 29 Apr 2021 19:32:56 +0200
Hi Tassilo,

Tassilo Horn <tsdh <at> gnu.org> writes:

> But you can at least allow a fixed number of balanced braces, and I've
> just done that in order to allow one level of nested {...} in mandatory
> arguments.  (We already allowed for one level of nested brackets in an
> optional argument.)

Thank you for the quick fix.  I also removed some unnecessary escaping
of characters in the regexp for the optional arguments.  While we're at,
I think we should replace

    "\\(?:\\[[^][]*\\(?:\\[[^][]*\\][^][]*\\)*\\]\\)?"

with

    "\\(?:\\[[^][]*\\(?:{[^}{]*}[^}{]*\\)*\\]\\)?"

at some point.  Rationale: One can have a single ] in an optional
argument which is hidden inside {}.  In this case, our current
implementation will fail.  I can live with the situation for the time
being, we can fix it when we get a complaint :-)

Best, Arash




Information forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Thu, 29 Apr 2021 19:29:02 GMT) Full text and rfc822 format available.

Message #19 received at 48078 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Arash Esbati <arash <at> gnu.org>
Cc: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>, 48078 <at> debbugs.gnu.org
Subject: Re: bug#48078: Verbatim environments with key-value arguments
Date: Thu, 29 Apr 2021 21:05:16 +0200
Arash Esbati <arash <at> gnu.org> writes:

Hi Arash,

> Thank you for the quick fix.  I also removed some unnecessary escaping
> of characters in the regexp for the optional arguments.

I had to check the docs to understand that. :-)

> While we're at, I think we should replace
>
>     "\\(?:\\[[^][]*\\(?:\\[[^][]*\\][^][]*\\)*\\]\\)?"
>
> with
>
>     "\\(?:\\[[^][]*\\(?:{[^}{]*}[^}{]*\\)*\\]\\)?"
>
> at some point.

But your advertised new regex doesn't even match the normal nested
bracket case, e.g., [foo=[1,2], bar={~}].

> Rationale: One can have a single ] in an optional argument which is
> hidden inside {}.

Not everything that can be written should be written. ;-)

Bye,
Tassilo




Information forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Thu, 29 Apr 2021 21:41:02 GMT) Full text and rfc822 format available.

Message #22 received at 48078 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>, 48078 <at> debbugs.gnu.org
Subject: Re: bug#48078: Verbatim environments with key-value arguments
Date: Thu, 29 Apr 2021 23:39:49 +0200
Hi Tassilo,

Tassilo Horn <tsdh <at> gnu.org> writes:

> Arash Esbati <arash <at> gnu.org> writes:
>
>> Thank you for the quick fix.  I also removed some unnecessary escaping
>> of characters in the regexp for the optional arguments.
>
> I had to check the docs to understand that. :-)

I have to check the docs every time I deal with regexp's :-)

>> While we're at, I think we should replace
>>
>>     "\\(?:\\[[^][]*\\(?:\\[[^][]*\\][^][]*\\)*\\]\\)?"
>>
>> with
>>
>>     "\\(?:\\[[^][]*\\(?:{[^}{]*}[^}{]*\\)*\\]\\)?"
>>
>> at some point.
>
> But your advertised new regex doesn't even match the normal nested
> bracket case, e.g., [foo=[1,2], bar={~}].

It has to match [foo={[1,2]}, bar={~}].  In your example above, LaTeX
scanner stops matching for the optional argument after ] and everything
after that would be part of the environment.

>> Rationale: One can have a single ] in an optional argument which is
>> hidden inside {}.
>
> Not everything that can be written should be written. ;-)

A slight delay might be acceptable, sometimes :-)

Best, Arash




Information forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Sat, 01 May 2021 06:45:01 GMT) Full text and rfc822 format available.

Message #25 received at 48078 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Arash Esbati <arash <at> gnu.org>
Cc: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>, 48078 <at> debbugs.gnu.org
Subject: Re: bug#48078: Verbatim environments with key-value arguments
Date: Sat, 01 May 2021 08:42:47 +0200
Arash Esbati <arash <at> gnu.org> writes:

Hi Arash,

>>> While we're at, I think we should replace
>>>
>>>     "\\(?:\\[[^][]*\\(?:\\[[^][]*\\][^][]*\\)*\\]\\)?"
>>>
>>> with
>>>
>>>     "\\(?:\\[[^][]*\\(?:{[^}{]*}[^}{]*\\)*\\]\\)?"
>>>
>>> at some point.
>>
>> But your advertised new regex doesn't even match the normal nested
>> bracket case, e.g., [foo=[1,2], bar={~}].
>
> It has to match [foo={[1,2]}, bar={~}].  In your example above, LaTeX
> scanner stops matching for the optional argument after ] and
> everything after that would be part of the environment.

Ah, so my example is not even valid LaTeX syntax?  If that's the case,
feel free to commit your improvement, preferably with a fontification
test

Bye,
Tassilo




Information forwarded to bug-auctex <at> gnu.org:
bug#48078; Package auctex. (Sat, 01 May 2021 21:50:02 GMT) Full text and rfc822 format available.

Message #28 received at 48078 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: Dario Gjorgjevski <dario.gjorgjevski <at> gmail.com>, 48078 <at> debbugs.gnu.org
Subject: Re: bug#48078: Verbatim environments with key-value arguments
Date: Sat, 01 May 2021 23:49:15 +0200
Hi Tassilo,

Tassilo Horn <tsdh <at> gnu.org> writes:

> Ah, so my example is not even valid LaTeX syntax?  If that's the case,
> feel free to commit your improvement, preferably with a fontification
> test

Until now, there were no user complaints and I'm not aware of any use
cases where a single ] can be used in an optional argrument of a
verbatim environment.  I think I can wait until there is a need to touch
the regexp, then we can fix it incl. a test.  I hope that's Ok for you
as well.

Best, Arash




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 30 May 2021 11:24:07 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 304 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.