GNU bug report logs - #56757
29.0.50; Python f-string font-lock logic is incomplete

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 56757 in the body.
You can then email your comments to 56757 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dima Kogan <dima <at> secretsauce.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; Python f-string font-lock logic is incomplete
Date: Mon, 25 Jul 2022 02:06:36 -0700

Hi. Currently python-mode has syntax highlighting to display the part of
the string being interpolated (the {...} stuff) differently from the
rest of the string. The emacs python-mode has logic to detect f-strings
to turn this highlighting on/off properly. It looks like this logic is
incomplete, however: it doesn't handle ''' and """ and r strings. (It
doesn't handle b strings either, but fb'asdf' is not legal Python, so
that's fine).

This is legal Python, and {x} is expanded to '5' in each case when the
program is executed.

  x = 5
  ss = (f'''aaa {x} bbb''',
        fr'''aaa {x} bbb''',
        rf'''aaa {x} bbb''',
        f"""aaa {x} bbb""",
        fr"""aaa {x} bbb""",
        rf"""aaa {x} bbb""",
        f'aaa {x} bbb',        # this is highlighted properly
        fr'aaa {x} bbb',
        rf'aaa {x} bbb',
        f"aaa {x} bbb",        # this is highlighted properly
        fr"aaa {x} bbb",
        rf"aaa {x} bbb",)
  for s in ss:
        print(s)

However the {x} is highlighted as code only on the lines marked above.
The other lines should have this highlighting also.

Thanks!

Message #8 received at 56757 <at> debbugs.gnu.org (full text, mbox):

From: Laurence Warne <laurencewarne <at> gmail.com>
To: 56757 <at> debbugs.gnu.org
Subject: Re: Python f-string font-lock logic is incomplete
Date: Sat, 30 Jul 2022 15:58:00 +0100

[Message part 1 (text/plain, inline)]

Hi, I've attached a patch which looks good to me locally using the above
examples and others.

In addition to the above, mixed capitalizations are also covered:
https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
.

Thanks!

[Message part 2 (text/html, inline)]

[0001-Apply-syntax-highlighting-for-all-python-f-strings.patch (text/x-patch, attachment)]

Message #11 received at 56757 <at> debbugs.gnu.org (full text, mbox):

From: Laurence Warne <laurencewarne <at> gmail.com>
To: 56757 <at> debbugs.gnu.org
Subject: Re: bug#56757: 29.0.50; Python f-string font-lock logic is incomplete
Date: Sat, 30 Jul 2022 16:51:19 +0100

[Message part 1 (text/plain, inline)]

Hi, I've attached a patch which looks good to me locally using the above
examples and others.

In addition to the above, mixed capitalizations are also covered:
https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
.

Thanks!

[Message part 2 (text/html, inline)]

[0001-Apply-syntax-highlighting-for-all-python-f-strings.patch (text/x-patch, attachment)]

Message #18 received at 56757 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Laurence Warne <laurencewarne <at> gmail.com>
Cc: 56757 <at> debbugs.gnu.org
Subject: Re: bug#56757: 29.0.50; Python f-string font-lock logic is incomplete
Date: Mon, 05 Sep 2022 21:13:03 +0200

Laurence Warne <laurencewarne <at> gmail.com> writes:

> Hi, I've attached a patch which looks good to me locally using the above examples
> and others.
>
> In addition to the above, mixed capitalizations are also covered:
> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals.

I think your patch makes sense, but it leads to failures in:

1 unexpected results:
   FAILED  python-font-lock-escape-sequence-multiline-string

Could you have a look at that and possibly adjust the test too (if it's
the test that's wrong here)?

Message #23 received at 56757 <at> debbugs.gnu.org (full text, mbox):

From: Laurence Warne <laurencewarne <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 56757 <at> debbugs.gnu.org
Subject: Re: bug#56757: 29.0.50; Python f-string font-lock logic is incomplete
Date: Tue, 6 Sep 2022 11:08:32 +0100

[Message part 1 (text/plain, inline)]

Hi, I've attached a patch fixing up this test. The difference occurred with:
rf"""\x12 S \n \u1234 \U00010348 \N{Plus-Minus Sign}"""
(and equivalent variants: the same string but starting with fr""", fr''',
rf''').

The patch treats "{Plus-Minus Sign}" as an embedded expression within an
f-string, causing it to be fontified differently (previously it was
fontified the same as the rest of the string).  This is invalid syntax
since "Plus-Minus Sign" is not a valid expression, so I think it's the
correct behaviour since it emphasises that "{Plus-Minus Sign}" will be
treated as an embedded expression even though it's invalid.

I hope that makes sense, thanks, Laurence

On Mon, Sep 5, 2022 at 8:13 PM Lars Ingebrigtsen <larsi <at> gnus.org> wrote:

> Laurence Warne <laurencewarne <at> gmail.com> writes:
>
> > Hi, I've attached a patch which looks good to me locally using the above
> examples
> > and others.
> >
> > In addition to the above, mixed capitalizations are also covered:
> >
> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
> .
>
> I think your patch makes sense, but it leads to failures in:
>
> 1 unexpected results:
>    FAILED  python-font-lock-escape-sequence-multiline-string
>
> Could you have a look at that and possibly adjust the test too (if it's
> the test that's wrong here)?
>
>

[Message part 2 (text/html, inline)]

[0001-Apply-syntax-highlighting-for-all-python-f-strings.patch (text/x-patch, attachment)]

Message #26 received at 56757 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Laurence Warne <laurencewarne <at> gmail.com>
Cc: 56757 <at> debbugs.gnu.org
Subject: Re: bug#56757: 29.0.50; Python f-string font-lock logic is incomplete
Date: Tue, 06 Sep 2022 12:28:34 +0200

Laurence Warne <laurencewarne <at> gmail.com> writes:

> I hope that makes sense, thanks, Laurence

It does indeed.  I've now pushed your patch to Emacs 29.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.