GNU bug report logs - #69714
30.0.50; ert-font-lock doesn't handle list of faces

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 69714 in the body.
You can then email your comments to 69714 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: bug-gnu-emacs <at> gnu.org
Cc: Vladimir Kazanov <vekazanov <at> gmail.com>
Subject: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Sun, 10 Mar 2024 16:31:58 -0400

I'm trying to use this package to test out my tree-sitter mode, but am
running into an issue with lists of faces.  It's possible that the
face for a location in the buffer will contain a list of 1 or more
faces.  For example, when I use the ":override 'prepend" keyword in
the call to treesit-font-lock-rules, even if only a single face is
specified for the rule that matches that section of the buffer, this
will result in a list of one entry (i.e., "(face-name)").

When this happens, ert-font-lock fails to recognize that this matches
the face "face-name" (e.g., "^ face-name" will fail to match in this
case).  I feel the tool should recognize a list containing a single
face as matching the face.  Even worse however, it appears
ert-font-lock doesn't support a list of faces in the comment.  I tried
to work around the original issue by using "^ (face-name)", but the
tool silently ignores this, as it doesn't match the internal regular
expression (which ended up allowing my test to pass without actually
checking anything).

I can't figure out a way to use this tool in its current state due to
its lack of support for a list of faces.  Also, I find that since it
silently ignores incorrect comment syntax (e.g., "^face-name", "^
(face-name)"), it gives a false illusion that it's actually performing
those checks (and the checks are passing), when it's really just
ignoring them.  Maybe any comment line starting with a "^" or "<-"
should be considered an assertion check and to fail if the rest of the
syntax is not as expected.  Maybe it should also fail the test if no
assertion checks are found in a source file or string.

Even if the tool would allow a list of a single face to match the
supplied face in the comment, I think it should also allow for
multiple faces to be listed in the comment.  I have other places where
multiple faces are used (e.g., "(font-lock-constant-face
font-lock-variable-name-face)" to highlight a constant variable),
which would not be testable with the current state of the package.

Message #8 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: brownts <at> troybrown.dev
Cc: 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Mon, 11 Mar 2024 08:36:22 +0000

Hi,

Thanks for reporting this! I have a bunch of ert-font-lock
improvements in my local repo getting ready for submission, and can
look into your suggestions as well.

Do you have your unit test code somewhere in a public repo? It'd be
great to think of further improvements to support your use case.

Thanks,
Vlad

On Sun, 10 Mar 2024 at 20:33, Troy Brown <brownts <at> troybrown.dev> wrote:
>
> I'm trying to use this package to test out my tree-sitter mode, but am
> running into an issue with lists of faces.  It's possible that the
> face for a location in the buffer will contain a list of 1 or more
> faces.  For example, when I use the ":override 'prepend" keyword in
> the call to treesit-font-lock-rules, even if only a single face is
> specified for the rule that matches that section of the buffer, this
> will result in a list of one entry (i.e., "(face-name)").
>
> When this happens, ert-font-lock fails to recognize that this matches
> the face "face-name" (e.g., "^ face-name" will fail to match in this
> case).  I feel the tool should recognize a list containing a single
> face as matching the face.  Even worse however, it appears
> ert-font-lock doesn't support a list of faces in the comment.  I tried
> to work around the original issue by using "^ (face-name)", but the
> tool silently ignores this, as it doesn't match the internal regular
> expression (which ended up allowing my test to pass without actually
> checking anything).
>
> I can't figure out a way to use this tool in its current state due to
> its lack of support for a list of faces.  Also, I find that since it
> silently ignores incorrect comment syntax (e.g., "^face-name", "^
> (face-name)"), it gives a false illusion that it's actually performing
> those checks (and the checks are passing), when it's really just
> ignoring them.  Maybe any comment line starting with a "^" or "<-"
> should be considered an assertion check and to fail if the rest of the
> syntax is not as expected.  Maybe it should also fail the test if no
> assertion checks are found in a source file or string.
>
> Even if the tool would allow a list of a single face to match the
> supplied face in the comment, I think it should also allow for
> multiple faces to be listed in the comment.  I have other places where
> multiple faces are used (e.g., "(font-lock-constant-face
> font-lock-variable-name-face)" to highlight a constant variable),
> which would not be testable with the current state of the package.
>
>
>


-- 
Regards,

Vladimir Kazanov

Message #11 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: brownts <at> troybrown.dev
Cc: 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Tue, 12 Mar 2024 20:46:54 +0000

[Message part 1 (text/plain, inline)]

Hi,

I've attached a patch that handles face lists, fails on files without
assertions and expands the parser a bit to support multiple carets per
line.

For faces it does the following:

1. Turn symbols into single element lists.
2. Parses face lists from the assertions.
3. Compare face lists using equas.

Could you please check if this works for you?

Thanks

On Mon, 11 Mar 2024 at 08:36, Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
>
> Hi,
>
> Thanks for reporting this! I have a bunch of ert-font-lock
> improvements in my local repo getting ready for submission, and can
> look into your suggestions as well.
>
> Do you have your unit test code somewhere in a public repo? It'd be
> great to think of further improvements to support your use case.
>
> Thanks,
> Vlad
>
> On Sun, 10 Mar 2024 at 20:33, Troy Brown <brownts <at> troybrown.dev> wrote:
> >
> > I'm trying to use this package to test out my tree-sitter mode, but am
> > running into an issue with lists of faces.  It's possible that the
> > face for a location in the buffer will contain a list of 1 or more
> > faces.  For example, when I use the ":override 'prepend" keyword in
> > the call to treesit-font-lock-rules, even if only a single face is
> > specified for the rule that matches that section of the buffer, this
> > will result in a list of one entry (i.e., "(face-name)").
> >
> > When this happens, ert-font-lock fails to recognize that this matches
> > the face "face-name" (e.g., "^ face-name" will fail to match in this
> > case).  I feel the tool should recognize a list containing a single
> > face as matching the face.  Even worse however, it appears
> > ert-font-lock doesn't support a list of faces in the comment.  I tried
> > to work around the original issue by using "^ (face-name)", but the
> > tool silently ignores this, as it doesn't match the internal regular
> > expression (which ended up allowing my test to pass without actually
> > checking anything).
> >
> > I can't figure out a way to use this tool in its current state due to
> > its lack of support for a list of faces.  Also, I find that since it
> > silently ignores incorrect comment syntax (e.g., "^face-name", "^
> > (face-name)"), it gives a false illusion that it's actually performing
> > those checks (and the checks are passing), when it's really just
> > ignoring them.  Maybe any comment line starting with a "^" or "<-"
> > should be considered an assertion check and to fail if the rest of the
> > syntax is not as expected.  Maybe it should also fail the test if no
> > assertion checks are found in a source file or string.
> >
> > Even if the tool would allow a list of a single face to match the
> > supplied face in the comment, I think it should also allow for
> > multiple faces to be listed in the comment.  I have other places where
> > multiple faces are used (e.g., "(font-lock-constant-face
> > font-lock-variable-name-face)" to highlight a constant variable),
> > which would not be testable with the current state of the package.
> >
> >
> >
>
>
> --
> Regards,
>
> Vladimir Kazanov



-- 
Regards,

Vladimir Kazanov

[0001-Improve-ert-font-lock-assertion-parser-Bug-69714.patch (text/x-patch, attachment)]

Message #14 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Wed, 13 Mar 2024 12:14:38 -0400

Hi Vlad, sorry for the delayed response.

I haven't pushed my change which uses this package yet, as I was
struggling to get it working and didn't want to push failing tests.  I
just discovered the package and was working on a regression test for a
bug fix involving font locking.  This seemed like the perfect reason
to use your package.  At the moment I only have this one
work-in-progress test, but I expect to use it more going forward.

I did check out your patch and for my immediate needs, it worked
perfectly.  Thanks!  Additionally, I did experiment a little with the
multi-caret functionality, which is nice as I have a use for that.  I
also experimented with the negation functionality (although I don't
have an immediate need for that), and did notice a couple things.  The
first was that the assertion would be ignored if there was a space
between the negation symbol and the face.  Also, if the actual and
expected faces didn't match and the negation flag was being used, it
acted like the negation was not indicated at all and failed the test.
I've included a diff below containing the changes I made which seemed
to address those concerns.

diff --git a/lisp/emacs-lisp/ert-font-lock.el b/lisp/emacs-lisp/ert-font-lock.el
index 06c90add9d3..1a5fe96fb09 100644
--- a/lisp/emacs-lisp/ert-font-lock.el
+++ b/lisp/emacs-lisp/ert-font-lock.el
@@ -61,6 +61,7 @@ ert-font-lock--assertion-line-re
    (group (zero-or-more (seq "^" (zero-or-more whitespace))))
    ;; optional negation of the face specification
    (group (optional "!"))
+   (zero-or-more whitespace)
    ;; face symbol name or a list of symbols
    (group (or (regexp ert-font-lock--face-symbol-re)
               (regexp ert-font-lock--face-symbol-list-re))))
@@ -354,7 +355,7 @@ ert-font-lock--check-faces
       (when (symbolp expected-face)
         (setq expected-face (list expected-face)))

-      (when (not (equal actual-face expected-face))
+      (when (and (not negation) (not (equal actual-face expected-face)))
         (ert-fail
          (list (format "Expected face %S, got %S on line %d column %d"
                        expected-face actual-face line-checked column-checked)


Thanks,

Troy.

On Tue, Mar 12, 2024 at 4:47 PM Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
>
> Hi,
>
> I've attached a patch that handles face lists, fails on files without
> assertions and expands the parser a bit to support multiple carets per
> line.
>
> For faces it does the following:
>
> 1. Turn symbols into single element lists.
> 2. Parses face lists from the assertions.
> 3. Compare face lists using equas.
>
> Could you please check if this works for you?
>
> Thanks
>
> On Mon, 11 Mar 2024 at 08:36, Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> >
> > Hi,
> >
> > Thanks for reporting this! I have a bunch of ert-font-lock
> > improvements in my local repo getting ready for submission, and can
> > look into your suggestions as well.
> >
> > Do you have your unit test code somewhere in a public repo? It'd be
> > great to think of further improvements to support your use case.
> >
> > Thanks,
> > Vlad
> >
> > On Sun, 10 Mar 2024 at 20:33, Troy Brown <brownts <at> troybrown.dev> wrote:
> > >
> > > I'm trying to use this package to test out my tree-sitter mode, but am
> > > running into an issue with lists of faces.  It's possible that the
> > > face for a location in the buffer will contain a list of 1 or more
> > > faces.  For example, when I use the ":override 'prepend" keyword in
> > > the call to treesit-font-lock-rules, even if only a single face is
> > > specified for the rule that matches that section of the buffer, this
> > > will result in a list of one entry (i.e., "(face-name)").
> > >
> > > When this happens, ert-font-lock fails to recognize that this matches
> > > the face "face-name" (e.g., "^ face-name" will fail to match in this
> > > case).  I feel the tool should recognize a list containing a single
> > > face as matching the face.  Even worse however, it appears
> > > ert-font-lock doesn't support a list of faces in the comment.  I tried
> > > to work around the original issue by using "^ (face-name)", but the
> > > tool silently ignores this, as it doesn't match the internal regular
> > > expression (which ended up allowing my test to pass without actually
> > > checking anything).
> > >
> > > I can't figure out a way to use this tool in its current state due to
> > > its lack of support for a list of faces.  Also, I find that since it
> > > silently ignores incorrect comment syntax (e.g., "^face-name", "^
> > > (face-name)"), it gives a false illusion that it's actually performing
> > > those checks (and the checks are passing), when it's really just
> > > ignoring them.  Maybe any comment line starting with a "^" or "<-"
> > > should be considered an assertion check and to fail if the rest of the
> > > syntax is not as expected.  Maybe it should also fail the test if no
> > > assertion checks are found in a source file or string.
> > >
> > > Even if the tool would allow a list of a single face to match the
> > > supplied face in the comment, I think it should also allow for
> > > multiple faces to be listed in the comment.  I have other places where
> > > multiple faces are used (e.g., "(font-lock-constant-face
> > > font-lock-variable-name-face)" to highlight a constant variable),
> > > which would not be testable with the current state of the package.
> > >
> > >
> > >
> >
> >
> > --
> > Regards,
> >
> > Vladimir Kazanov
>
>
>
> --
> Regards,
>
> Vladimir Kazanov

Message #17 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: Troy Brown <brownts <at> troybrown.dev>
Cc: 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Wed, 13 Mar 2024 17:04:23 +0000

[Message part 1 (text/plain, inline)]

Thanks a lot!

The suggestions really do make sense.

Here's the final integrated patch, complete with updated tests and
docs. If you're fine with it then I'll ask somebody to install it on
master.

PS I've got to write an additional announcement in the main mailing
list inviting people to check the updated version out.

On Wed, 13 Mar 2024 at 16:14, Troy Brown <brownts <at> troybrown.dev> wrote:
>
> Hi Vlad, sorry for the delayed response.
>
> I haven't pushed my change which uses this package yet, as I was
> struggling to get it working and didn't want to push failing tests.  I
> just discovered the package and was working on a regression test for a
> bug fix involving font locking.  This seemed like the perfect reason
> to use your package.  At the moment I only have this one
> work-in-progress test, but I expect to use it more going forward.
>
> I did check out your patch and for my immediate needs, it worked
> perfectly.  Thanks!  Additionally, I did experiment a little with the
> multi-caret functionality, which is nice as I have a use for that.  I
> also experimented with the negation functionality (although I don't
> have an immediate need for that), and did notice a couple things.  The
> first was that the assertion would be ignored if there was a space
> between the negation symbol and the face.  Also, if the actual and
> expected faces didn't match and the negation flag was being used, it
> acted like the negation was not indicated at all and failed the test.
> I've included a diff below containing the changes I made which seemed
> to address those concerns.
>
> diff --git a/lisp/emacs-lisp/ert-font-lock.el b/lisp/emacs-lisp/ert-font-lock.el
> index 06c90add9d3..1a5fe96fb09 100644
> --- a/lisp/emacs-lisp/ert-font-lock.el
> +++ b/lisp/emacs-lisp/ert-font-lock.el
> @@ -61,6 +61,7 @@ ert-font-lock--assertion-line-re
>     (group (zero-or-more (seq "^" (zero-or-more whitespace))))
>     ;; optional negation of the face specification
>     (group (optional "!"))
> +   (zero-or-more whitespace)
>     ;; face symbol name or a list of symbols
>     (group (or (regexp ert-font-lock--face-symbol-re)
>                (regexp ert-font-lock--face-symbol-list-re))))
> @@ -354,7 +355,7 @@ ert-font-lock--check-faces
>        (when (symbolp expected-face)
>          (setq expected-face (list expected-face)))
>
> -      (when (not (equal actual-face expected-face))
> +      (when (and (not negation) (not (equal actual-face expected-face)))
>          (ert-fail
>           (list (format "Expected face %S, got %S on line %d column %d"
>                         expected-face actual-face line-checked column-checked)
>
>
> Thanks,
>
> Troy.
>
> On Tue, Mar 12, 2024 at 4:47 PM Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> >
> > Hi,
> >
> > I've attached a patch that handles face lists, fails on files without
> > assertions and expands the parser a bit to support multiple carets per
> > line.
> >
> > For faces it does the following:
> >
> > 1. Turn symbols into single element lists.
> > 2. Parses face lists from the assertions.
> > 3. Compare face lists using equas.
> >
> > Could you please check if this works for you?
> >
> > Thanks
> >
> > On Mon, 11 Mar 2024 at 08:36, Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > Thanks for reporting this! I have a bunch of ert-font-lock
> > > improvements in my local repo getting ready for submission, and can
> > > look into your suggestions as well.
> > >
> > > Do you have your unit test code somewhere in a public repo? It'd be
> > > great to think of further improvements to support your use case.
> > >
> > > Thanks,
> > > Vlad
> > >
> > > On Sun, 10 Mar 2024 at 20:33, Troy Brown <brownts <at> troybrown.dev> wrote:
> > > >
> > > > I'm trying to use this package to test out my tree-sitter mode, but am
> > > > running into an issue with lists of faces.  It's possible that the
> > > > face for a location in the buffer will contain a list of 1 or more
> > > > faces.  For example, when I use the ":override 'prepend" keyword in
> > > > the call to treesit-font-lock-rules, even if only a single face is
> > > > specified for the rule that matches that section of the buffer, this
> > > > will result in a list of one entry (i.e., "(face-name)").
> > > >
> > > > When this happens, ert-font-lock fails to recognize that this matches
> > > > the face "face-name" (e.g., "^ face-name" will fail to match in this
> > > > case).  I feel the tool should recognize a list containing a single
> > > > face as matching the face.  Even worse however, it appears
> > > > ert-font-lock doesn't support a list of faces in the comment.  I tried
> > > > to work around the original issue by using "^ (face-name)", but the
> > > > tool silently ignores this, as it doesn't match the internal regular
> > > > expression (which ended up allowing my test to pass without actually
> > > > checking anything).
> > > >
> > > > I can't figure out a way to use this tool in its current state due to
> > > > its lack of support for a list of faces.  Also, I find that since it
> > > > silently ignores incorrect comment syntax (e.g., "^face-name", "^
> > > > (face-name)"), it gives a false illusion that it's actually performing
> > > > those checks (and the checks are passing), when it's really just
> > > > ignoring them.  Maybe any comment line starting with a "^" or "<-"
> > > > should be considered an assertion check and to fail if the rest of the
> > > > syntax is not as expected.  Maybe it should also fail the test if no
> > > > assertion checks are found in a source file or string.
> > > >
> > > > Even if the tool would allow a list of a single face to match the
> > > > supplied face in the comment, I think it should also allow for
> > > > multiple faces to be listed in the comment.  I have other places where
> > > > multiple faces are used (e.g., "(font-lock-constant-face
> > > > font-lock-variable-name-face)" to highlight a constant variable),
> > > > which would not be testable with the current state of the package.
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Vladimir Kazanov
> >
> >
> >
> > --
> > Regards,
> >
> > Vladimir Kazanov



-- 
Regards,

Vladimir Kazanov

[0001-Improve-ert-font-lock-assertion-parser-Bug-69714.patch (text/x-patch, attachment)]

Message #20 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Wed, 13 Mar 2024 13:48:48 -0400

The new patch looks good to me.

One other thing I thought I'd mention.  I have places in my test where
I want to verify that there is no font lock face.  I've been able to
specify "nil" for the face and that works perfectly to check this.  I
thought I'd mention this because I wasn't sure if it was intentional
behavior, but I do find that useful.


Thanks,

Troy.

On Wed, Mar 13, 2024 at 1:04 PM Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
>
> Thanks a lot!
>
> The suggestions really do make sense.
>
> Here's the final integrated patch, complete with updated tests and
> docs. If you're fine with it then I'll ask somebody to install it on
> master.
>
> PS I've got to write an additional announcement in the main mailing
> list inviting people to check the updated version out.
>
> On Wed, 13 Mar 2024 at 16:14, Troy Brown <brownts <at> troybrown.dev> wrote:
> >
> > Hi Vlad, sorry for the delayed response.
> >
> > I haven't pushed my change which uses this package yet, as I was
> > struggling to get it working and didn't want to push failing tests.  I
> > just discovered the package and was working on a regression test for a
> > bug fix involving font locking.  This seemed like the perfect reason
> > to use your package.  At the moment I only have this one
> > work-in-progress test, but I expect to use it more going forward.
> >
> > I did check out your patch and for my immediate needs, it worked
> > perfectly.  Thanks!  Additionally, I did experiment a little with the
> > multi-caret functionality, which is nice as I have a use for that.  I
> > also experimented with the negation functionality (although I don't
> > have an immediate need for that), and did notice a couple things.  The
> > first was that the assertion would be ignored if there was a space
> > between the negation symbol and the face.  Also, if the actual and
> > expected faces didn't match and the negation flag was being used, it
> > acted like the negation was not indicated at all and failed the test.
> > I've included a diff below containing the changes I made which seemed
> > to address those concerns.
> >
> > diff --git a/lisp/emacs-lisp/ert-font-lock.el b/lisp/emacs-lisp/ert-font-lock.el
> > index 06c90add9d3..1a5fe96fb09 100644
> > --- a/lisp/emacs-lisp/ert-font-lock.el
> > +++ b/lisp/emacs-lisp/ert-font-lock.el
> > @@ -61,6 +61,7 @@ ert-font-lock--assertion-line-re
> >     (group (zero-or-more (seq "^" (zero-or-more whitespace))))
> >     ;; optional negation of the face specification
> >     (group (optional "!"))
> > +   (zero-or-more whitespace)
> >     ;; face symbol name or a list of symbols
> >     (group (or (regexp ert-font-lock--face-symbol-re)
> >                (regexp ert-font-lock--face-symbol-list-re))))
> > @@ -354,7 +355,7 @@ ert-font-lock--check-faces
> >        (when (symbolp expected-face)
> >          (setq expected-face (list expected-face)))
> >
> > -      (when (not (equal actual-face expected-face))
> > +      (when (and (not negation) (not (equal actual-face expected-face)))
> >          (ert-fail
> >           (list (format "Expected face %S, got %S on line %d column %d"
> >                         expected-face actual-face line-checked column-checked)
> >
> >
> > Thanks,
> >
> > Troy.
> >
> > On Tue, Mar 12, 2024 at 4:47 PM Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I've attached a patch that handles face lists, fails on files without
> > > assertions and expands the parser a bit to support multiple carets per
> > > line.
> > >
> > > For faces it does the following:
> > >
> > > 1. Turn symbols into single element lists.
> > > 2. Parses face lists from the assertions.
> > > 3. Compare face lists using equas.
> > >
> > > Could you please check if this works for you?
> > >
> > > Thanks
> > >
> > > On Mon, 11 Mar 2024 at 08:36, Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > Thanks for reporting this! I have a bunch of ert-font-lock
> > > > improvements in my local repo getting ready for submission, and can
> > > > look into your suggestions as well.
> > > >
> > > > Do you have your unit test code somewhere in a public repo? It'd be
> > > > great to think of further improvements to support your use case.
> > > >
> > > > Thanks,
> > > > Vlad
> > > >
> > > > On Sun, 10 Mar 2024 at 20:33, Troy Brown <brownts <at> troybrown.dev> wrote:
> > > > >
> > > > > I'm trying to use this package to test out my tree-sitter mode, but am
> > > > > running into an issue with lists of faces.  It's possible that the
> > > > > face for a location in the buffer will contain a list of 1 or more
> > > > > faces.  For example, when I use the ":override 'prepend" keyword in
> > > > > the call to treesit-font-lock-rules, even if only a single face is
> > > > > specified for the rule that matches that section of the buffer, this
> > > > > will result in a list of one entry (i.e., "(face-name)").
> > > > >
> > > > > When this happens, ert-font-lock fails to recognize that this matches
> > > > > the face "face-name" (e.g., "^ face-name" will fail to match in this
> > > > > case).  I feel the tool should recognize a list containing a single
> > > > > face as matching the face.  Even worse however, it appears
> > > > > ert-font-lock doesn't support a list of faces in the comment.  I tried
> > > > > to work around the original issue by using "^ (face-name)", but the
> > > > > tool silently ignores this, as it doesn't match the internal regular
> > > > > expression (which ended up allowing my test to pass without actually
> > > > > checking anything).
> > > > >
> > > > > I can't figure out a way to use this tool in its current state due to
> > > > > its lack of support for a list of faces.  Also, I find that since it
> > > > > silently ignores incorrect comment syntax (e.g., "^face-name", "^
> > > > > (face-name)"), it gives a false illusion that it's actually performing
> > > > > those checks (and the checks are passing), when it's really just
> > > > > ignoring them.  Maybe any comment line starting with a "^" or "<-"
> > > > > should be considered an assertion check and to fail if the rest of the
> > > > > syntax is not as expected.  Maybe it should also fail the test if no
> > > > > assertion checks are found in a source file or string.
> > > > >
> > > > > Even if the tool would allow a list of a single face to match the
> > > > > supplied face in the comment, I think it should also allow for
> > > > > multiple faces to be listed in the comment.  I have other places where
> > > > > multiple faces are used (e.g., "(font-lock-constant-face
> > > > > font-lock-variable-name-face)" to highlight a constant variable),
> > > > > which would not be testable with the current state of the package.
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Vladimir Kazanov
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Vladimir Kazanov
>
>
>
> --
> Regards,
>
> Vladimir Kazanov

Message #23 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: Troy Brown <brownts <at> troybrown.dev>
Cc: 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: 30.0.50; ert-font-lock doesn't handle list of faces
Date: Wed, 13 Mar 2024 18:20:23 +0000

[Message part 1 (text/plain, inline)]

Yeah, this is unintentional but I like it. "nil" is happily parsed by
both the (read) call and the guarding regex.

There's a little thing that needs to be fixed - avoid turning 'nil
into '(nil) when comparing lists. It works but for the wrong reason.
Anyway, I'll update the docs to suggest using nil and !nil in
assertions, and also add a test.

The patch looks ready. I'll wait for a few days and if nobody notices
I will give the maintainers a shout.


On Wed, 13 Mar 2024 at 17:49, Troy Brown <brownts <at> troybrown.dev> wrote:
>
> The new patch looks good to me.
>
> One other thing I thought I'd mention.  I have places in my test where
> I want to verify that there is no font lock face.  I've been able to
> specify "nil" for the face and that works perfectly to check this.  I
> thought I'd mention this because I wasn't sure if it was intentional
> behavior, but I do find that useful.
>
>
> Thanks,
>
> Troy.
>
> On Wed, Mar 13, 2024 at 1:04 PM Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> >
> > Thanks a lot!
> >
> > The suggestions really do make sense.
> >
> > Here's the final integrated patch, complete with updated tests and
> > docs. If you're fine with it then I'll ask somebody to install it on
> > master.
> >
> > PS I've got to write an additional announcement in the main mailing
> > list inviting people to check the updated version out.
> >
> > On Wed, 13 Mar 2024 at 16:14, Troy Brown <brownts <at> troybrown.dev> wrote:
> > >
> > > Hi Vlad, sorry for the delayed response.
> > >
> > > I haven't pushed my change which uses this package yet, as I was
> > > struggling to get it working and didn't want to push failing tests.  I
> > > just discovered the package and was working on a regression test for a
> > > bug fix involving font locking.  This seemed like the perfect reason
> > > to use your package.  At the moment I only have this one
> > > work-in-progress test, but I expect to use it more going forward.
> > >
> > > I did check out your patch and for my immediate needs, it worked
> > > perfectly.  Thanks!  Additionally, I did experiment a little with the
> > > multi-caret functionality, which is nice as I have a use for that.  I
> > > also experimented with the negation functionality (although I don't
> > > have an immediate need for that), and did notice a couple things.  The
> > > first was that the assertion would be ignored if there was a space
> > > between the negation symbol and the face.  Also, if the actual and
> > > expected faces didn't match and the negation flag was being used, it
> > > acted like the negation was not indicated at all and failed the test.
> > > I've included a diff below containing the changes I made which seemed
> > > to address those concerns.
> > >
> > > diff --git a/lisp/emacs-lisp/ert-font-lock.el b/lisp/emacs-lisp/ert-font-lock.el
> > > index 06c90add9d3..1a5fe96fb09 100644
> > > --- a/lisp/emacs-lisp/ert-font-lock.el
> > > +++ b/lisp/emacs-lisp/ert-font-lock.el
> > > @@ -61,6 +61,7 @@ ert-font-lock--assertion-line-re
> > >     (group (zero-or-more (seq "^" (zero-or-more whitespace))))
> > >     ;; optional negation of the face specification
> > >     (group (optional "!"))
> > > +   (zero-or-more whitespace)
> > >     ;; face symbol name or a list of symbols
> > >     (group (or (regexp ert-font-lock--face-symbol-re)
> > >                (regexp ert-font-lock--face-symbol-list-re))))
> > > @@ -354,7 +355,7 @@ ert-font-lock--check-faces
> > >        (when (symbolp expected-face)
> > >          (setq expected-face (list expected-face)))
> > >
> > > -      (when (not (equal actual-face expected-face))
> > > +      (when (and (not negation) (not (equal actual-face expected-face)))
> > >          (ert-fail
> > >           (list (format "Expected face %S, got %S on line %d column %d"
> > >                         expected-face actual-face line-checked column-checked)
> > >
> > >
> > > Thanks,
> > >
> > > Troy.
> > >
> > > On Tue, Mar 12, 2024 at 4:47 PM Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I've attached a patch that handles face lists, fails on files without
> > > > assertions and expands the parser a bit to support multiple carets per
> > > > line.
> > > >
> > > > For faces it does the following:
> > > >
> > > > 1. Turn symbols into single element lists.
> > > > 2. Parses face lists from the assertions.
> > > > 3. Compare face lists using equas.
> > > >
> > > > Could you please check if this works for you?
> > > >
> > > > Thanks
> > > >
> > > > On Mon, 11 Mar 2024 at 08:36, Vladimir Kazanov <vekazanov <at> gmail.com> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > Thanks for reporting this! I have a bunch of ert-font-lock
> > > > > improvements in my local repo getting ready for submission, and can
> > > > > look into your suggestions as well.
> > > > >
> > > > > Do you have your unit test code somewhere in a public repo? It'd be
> > > > > great to think of further improvements to support your use case.
> > > > >
> > > > > Thanks,
> > > > > Vlad
> > > > >
> > > > > On Sun, 10 Mar 2024 at 20:33, Troy Brown <brownts <at> troybrown.dev> wrote:
> > > > > >
> > > > > > I'm trying to use this package to test out my tree-sitter mode, but am
> > > > > > running into an issue with lists of faces.  It's possible that the
> > > > > > face for a location in the buffer will contain a list of 1 or more
> > > > > > faces.  For example, when I use the ":override 'prepend" keyword in
> > > > > > the call to treesit-font-lock-rules, even if only a single face is
> > > > > > specified for the rule that matches that section of the buffer, this
> > > > > > will result in a list of one entry (i.e., "(face-name)").
> > > > > >
> > > > > > When this happens, ert-font-lock fails to recognize that this matches
> > > > > > the face "face-name" (e.g., "^ face-name" will fail to match in this
> > > > > > case).  I feel the tool should recognize a list containing a single
> > > > > > face as matching the face.  Even worse however, it appears
> > > > > > ert-font-lock doesn't support a list of faces in the comment.  I tried
> > > > > > to work around the original issue by using "^ (face-name)", but the
> > > > > > tool silently ignores this, as it doesn't match the internal regular
> > > > > > expression (which ended up allowing my test to pass without actually
> > > > > > checking anything).
> > > > > >
> > > > > > I can't figure out a way to use this tool in its current state due to
> > > > > > its lack of support for a list of faces.  Also, I find that since it
> > > > > > silently ignores incorrect comment syntax (e.g., "^face-name", "^
> > > > > > (face-name)"), it gives a false illusion that it's actually performing
> > > > > > those checks (and the checks are passing), when it's really just
> > > > > > ignoring them.  Maybe any comment line starting with a "^" or "<-"
> > > > > > should be considered an assertion check and to fail if the rest of the
> > > > > > syntax is not as expected.  Maybe it should also fail the test if no
> > > > > > assertion checks are found in a source file or string.
> > > > > >
> > > > > > Even if the tool would allow a list of a single face to match the
> > > > > > supplied face in the comment, I think it should also allow for
> > > > > > multiple faces to be listed in the comment.  I have other places where
> > > > > > multiple faces are used (e.g., "(font-lock-constant-face
> > > > > > font-lock-variable-name-face)" to highlight a constant variable),
> > > > > > which would not be testable with the current state of the package.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Vladimir Kazanov
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Vladimir Kazanov
> >
> >
> >
> > --
> > Regards,
> >
> > Vladimir Kazanov



-- 
Regards,

Vladimir Kazanov

[0001-Improve-ert-font-lock-assertion-parser-Bug-69714.patch (text/x-patch, attachment)]

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH] Improve ert-font-lock assertion parser (Bug#69714)
Date: Fri, 15 Mar 2024 11:47:27 +0000

[Message part 1 (text/plain, inline)]

Dear maintainers,

I've come up with a number of improvements for ert-font-lock as
requested in Bug#69714 (which can be closed now).

The attached patch provides the fix, more unit tests and updated
documentation. I don't think we need to update NEWS as ert-font-lock
was announced there already.

Thank you

-- 
Regards,

Vladimir Kazanov

[0001-Improve-ert-font-lock-assertion-parser-Bug-69714.patch (text/x-patch, attachment)]

Message #31 received at 69714-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: 69714-done <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
 (Bug#69714)
Date: Thu, 28 Mar 2024 11:41:09 +0200

> From: Vladimir Kazanov <vekazanov <at> gmail.com>
> Date: Fri, 15 Mar 2024 11:47:27 +0000
> 
> I've come up with a number of improvements for ert-font-lock as
> requested in Bug#69714 (which can be closed now).
> 
> The attached patch provides the fix, more unit tests and updated
> documentation. I don't think we need to update NEWS as ert-font-lock
> was announced there already.

Thanks, installed on master, and closing the bug.

Message #34 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Sat, 30 Mar 2024 13:52:05 +0100

Vladimir, thank you for your patch. However, this regexp:

> (defconst ert-font-lock--face-symbol-re
>   (rx (one-or-more (or alphanumeric "-" "_" ".")))
>   "A face symbol matching regex.")
> 
> (defconst ert-font-lock--face-symbol-list-re
>   (rx "("
>       (* whitespace)
>       (one-or-more
>        (seq (regexp ert-font-lock--face-symbol-re)
>             (* whitespace)))
>       ")")
>   "A face symbol list matching regex.")

is rather inefficient. (Relint complained about it, which is why I am here).

The problem is that

      (one-or-more
       (seq (one-or-more (or alphanumeric "-" "_" "."))
            (* whitespace)))

can match a string that consists of alphanumeric characters, say, in many different ways because of the nested `one-or-more`; the (* whitespace) can match the empty string anywhere.
Try, for instance:

  (string-match ert-font-lock--face-symbol-list-re
                (concat "(" (make-string 20 ?a)))

which should return nil as it doesn't match.
Now try raising the number from 20 to 25, then 30.

If you want the regexp to match a list of one or more symbols, what about this:

  (rx "("
      (* whitespace)
      (regexp ert-font-lock--face-symbol-re)
      (* (+ whitespace)
         (regexp ert-font-lock--face-symbol-re))
      ")")

Message #37 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Sun, 31 Mar 2024 17:56:38 +0000

[Message part 1 (text/plain, inline)]

Hi Mattias,

Thanks for noticing this!

> If you want the regexp to match a list of one or more symbols, what about this:
>
>   (rx "("
>       (* whitespace)
>       (regexp ert-font-lock--face-symbol-re)
>       (* (+ whitespace)
>          (regexp ert-font-lock--face-symbol-re))
>       ")")
>

Got it, thanks. I forgot about relint! Regex performance is easy to
get wrong. I'd really like peg.el to be merged and used for this kind
of parsing.

Anyway, can you please take a look at the patch? It fixes the problem
noticed by relint as well as a couple of other minor things, and adds
tests to make sure regexes work the way I want them to.

Thank you
Vlad


-- 
Regards,

Vladimir Kazanov

[v1-0001-Fix-symbol-list-matching-regexps.patch (text/x-patch, attachment)]

Message #40 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Mon, 1 Apr 2024 10:04:44 +0200

31 mars 2024 kl. 19.56 skrev Vladimir Kazanov <vekazanov <at> gmail.com>:

>    (rx "("
>        (* whitespace)
> -      (one-or-more
> -       (seq (regexp ert-font-lock--face-symbol-re)
> -            (* whitespace)))
> +      (opt (regexp ert-font-lock--face-symbol-re))
> +      (zero-or-more
> +       (seq (+ whitespace)
> +            (regexp ert-font-lock--face-symbol-re)))
> +      (* whitespace)
>        ")")

Since you want to match zero or more symbols, this can be improved further:

   (rx "("
       (* whitespace)
       (* (regexp ert-font-lock--face-symbol-re)
          (+ whitespace)))
       ")")

(Note that most rx operators have an implicit `seq` inside.)

You may want to use ´rx-define` instead of `defconst` as well -- it avoids some use of the `regexp` construct and removes some load-time string building.

Message #43 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Mon, 1 Apr 2024 10:17:50 +0200

1 apr. 2024 kl. 10.04 skrev Mattias Engdegård <mattias.engdegard <at> gmail.com>:

>   (rx "("
>       (* whitespace)
>       (* (regexp ert-font-lock--face-symbol-re)
>          (+ whitespace)))
>       ")")

Sorry, that was not very well thought-out, was it? Try

  (rx "("
      (* whitespace)
      (? (regexp ert-font-lock--face-symbol-re)
         (* (+ whitespace)
            (regexp ert-font-lock--face-symbol-re))
         (* whitespace))
      ")")

instead. Not as pretty but should actually work.

Message #46 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Mon, 1 Apr 2024 09:34:50 +0100

[Message part 1 (text/plain, inline)]

>   (rx "("
>       (* whitespace)
>       (? (regexp ert-font-lock--face-symbol-re)
>          (* (+ whitespace)
>             (regexp ert-font-lock--face-symbol-re))
>          (* whitespace))
>       ")")
>
> instead. Not as pretty but should actually work.
>

Yeah, somewhat cleaner. Added to the patch.

Not sure using rx-define et al makes sense for 3 regular expressions I
need to test directly.

Here's patch v2.

--
Regards,

Vladimir Kazanov

[v2-0001-Fix-symbol-list-matching-regexps.patch (application/x-patch, attachment)]

Message #49 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Mon, 1 Apr 2024 11:08:30 +0200

1 apr. 2024 kl. 10.34 skrev Vladimir Kazanov <vekazanov <at> gmail.com>:

> Not sure using rx-define et al makes sense for 3 regular expressions I
> need to test directly.

Well, ert-font-lock--face-symbol-list-re is only used in other regexps. (And in tests, but you can always write `(rx ert-font-lock--face-symbol-list-re)`.)

But it's not very important.

> Here's patch v2.

Looks fine!

Message #52 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Vladimir Kazanov <vekazanov <at> gmail.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Mon, 1 Apr 2024 10:12:10 +0100

> Looks fine!

Thanks a lot! Can you install it for me? Or should I wait for Eli to
have time to handle this?

-- 
Regards,

Vladimir Kazanov

Message #55 received at 69714 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Vladimir Kazanov <vekazanov <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 69714 <at> debbugs.gnu.org
Subject: Re: bug#69714: [PATCH] Improve ert-font-lock assertion parser
Date: Mon, 1 Apr 2024 11:15:03 +0200

> Can you install it for me?

Certainly, now done.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.