GNU bug report logs - #25348
`display` property faces are prioritized above overlays

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25348 in the body.
You can then email your comments to 25348 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Travis Foster <tsfoster <at> mtu.edu>
To: bug-gnu-emacs <at> gnu.org
Subject: `display` property faces are prioritized above overlays
Date: Tue, 3 Jan 2017 14:17:07 -0800

[Message part 1 (text/plain, inline)]

When I do the following (for example):

(put-text-property (point) (1+ (point)) 'display (propertize "." 'face
'header-line))

The character at point becomes a dot with a gray background. If I then
enable hl-line-mode, the line turns green, as expected. However, the dot
still has a gray background; it does not turn green with the rest of the
line.

It appears that hl-line-mode uses an overlay to highlight the line. From
the overlay documentation, "Currently, all overlays take priority over text
properties." But it seems like in this case, the display text property is
taking priority over the the overlay. Since I'm embedding the face into the
display string, I expect it to take priority over any faces that are
applied to the text, but not over any overlays which affect it.

[Message part 2 (text/html, inline)]

Message #8 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Travis Foster <tsfoster <at> mtu.edu>, 25348 <at> debbugs.gnu.org
Subject: RE: bug#25348: `display` property faces are prioritized above overlays
Date: Tue, 3 Jan 2017 14:29:55 -0800 (PST)

> When I do the following (for example):
> (put-text-property (point) (1+ (point))
>                    'display (propertize "." 'face 'header-line))
...
> But it seems like in this case, the display text property
> is taking priority over the the overlay.

You are using a "replacing" `display'-property spec.
See (elisp) `Replacing Specs'.
http://www.gnu.org/software/emacs/manual/html_node/elisp/Replacing-Specs.html

Your text that has the property is entirely replaced (for display)
by what is specified for property `display'.

Message #11 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Travis Foster <tsfoster <at> mtu.edu>
To: Drew Adams <drew.adams <at> oracle.com>, 25348 <at> debbugs.gnu.org
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Tue, 3 Jan 2017 14:52:26 -0800

[Message part 1 (text/plain, inline)]

Yes, the text in the buffer is replaced by its display text, with the face.
Then, I would expect the overlay to apply to all the text on the line,
including the replacement text. Since the properties in overlays are
supposed to take priority over the properties in the buffer itself, I don't
think it makes sense for the replacement to take priority over the
overlay. Is that wrong? It seems that the replacement should occur, and
then the resulting text should be modified further by the overlay.

On Tue, Jan 3, 2017 at 2:29 PM, Drew Adams <drew.adams <at> oracle.com> wrote:

> > When I do the following (for example):
> > (put-text-property (point) (1+ (point))
> >                    'display (propertize "." 'face 'header-line))
> ...
> > But it seems like in this case, the display text property
> > is taking priority over the the overlay.
>
> You are using a "replacing" `display'-property spec.
> See (elisp) `Replacing Specs'.
> http://www.gnu.org/software/emacs/manual/html_node/elisp/
> Replacing-Specs.html
>
> Your text that has the property is entirely replaced (for display)
> by what is specified for property `display'.
>

[Message part 2 (text/html, inline)]

Message #14 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Travis Foster <tsfoster <at> mtu.edu>
Cc: 25348 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Wed, 04 Jan 2017 18:06:31 +0200

> From: Travis Foster <tsfoster <at> mtu.edu>
> Date: Tue, 3 Jan 2017 14:52:26 -0800
> 
> Yes, the text in the buffer is replaced by its display text, with the face. Then, I would expect the overlay to apply
> to all the text on the line, including the replacement text. Since the properties in overlays are supposed to take
> priority over the properties in the buffer itself, I don't think it makes sense for the replacement to take priority
> over the overlay. Is that wrong?

Drew is right: the priority of overlays over text properties only
comes into play when both text properties and overlays are set on the
same region of text.  In your case, the 'face' property is put on a
display string, whereas the hl-line overlay is on buffer text.  So
priority considerations don't apply here.

> It seems that the replacement should occur, and then the resulting
> text should be modified further by the overlay.

Emacs uses the face from the overlay only for text to which this
overlay is applied.  The display string is therefore using its own
face definitions, which completely override those from the hl-line
overlay.

If you define a face for the display string that only specifies a
foreground color, then Emacs will use the hl-line overlay for the
background color.

This is normal operation of the Emacs display engine, it has been like
that since Emacs 21.  IOW, this is not a bug.

Message #17 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Travis Foster <tsfoster <at> mtu.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, 25348 <at> debbugs.gnu.org
Cc: Drew Adams <drew.adams <at> oracle.com>
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Wed, 4 Jan 2017 11:25:54 -0800

[Message part 1 (text/plain, inline)]

> Emacs uses the face from the overlay only for text to which this
> overlay is applied.  The display string is therefore using its own
> face definitions, which completely override those from the hl-line
> overlay.

I just figured that the overlay applies to a range of text, and the
replacement text is within that range ... so it seems like it should be
affected. I guess not, though. So what you're saying is, since the
replacement text is not technically part of the buffer text, it doesn't
count as being within the range of the overlay, and it isn't affected by
the overlay at all? But that's not the case either, because as you also
stated, face attributes from the overlay are applied to the display string,
as long as the display string doesn't already specify those attributes. So,
it seems that the overlay is applied to the display string, but it just
takes a lower priority than the display string's text properties. If that
was the design, that's fine, but it does conflict with the documentation
stating that overlays always take priority over text properties.

I haven't looked at the code for this, so I might be wrong, but what
appears to be happening is this:
1. The overlay is applied to the buffer text, and the overlay face takes
priority over the buffer text's faces
2. If the overlay had a display property, that would take priority over the
buffer text's display property, but since the overlay has no such property,
this doesn't happen
3. After the overlay is applied, the display property is applied, and its
faces take priority over the existing faces, including those supplied by
the overlay

So the priority goes: display property faces > overlay faces > buffer
faces. Am I on the right track?

[Message part 2 (text/html, inline)]

Message #20 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Travis Foster <tsfoster <at> mtu.edu>
Cc: 25348 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Wed, 04 Jan 2017 21:55:55 +0200

> From: Travis Foster <tsfoster <at> mtu.edu>
> Date: Wed, 4 Jan 2017 11:25:54 -0800
> Cc: Drew Adams <drew.adams <at> oracle.com>
> 
> I just figured that the overlay applies to a range of text, and the replacement text is within that range ... so it
> seems like it should be affected.

The replacement text comes from a string, not from the buffer, so it
is NOT within the range of buffer text to which the overlay is applied.

> So what you're saying is, since the replacement text is
> not technically part of the buffer text, it doesn't count as being within the range of the overlay, and it isn't
> affected by the overlay at all?

Yes.

> But that's not the case either, because as you also stated, face attributes from
> the overlay are applied to the display string, as long as the display string doesn't already specify those
> attributes.

If the face of the display string doesn't specify the background
color, the only sensible thing to do is to use the background of the
"underlying" buffer text.  (Note that in this case the issue of
priority doesn't arise either, because only one of the sources
specifies the background color.)

> So, it seems that the overlay is applied to the display string, but it just takes a lower priority than the
> display string's text properties.

That's your interpretation, but it is incorrect.  As a matter of fact,
the priority is not applicable in this case, because the two faces are
applied to two different objects: one is buffer text, the other is the
text of the display string.

> it does conflict with the documentation stating that overlays always
> take priority over text properties.

Not in my view and interpretation, no.

> I haven't looked at the code for this, so I might be wrong, but what appears to be happening is this:
> 1. The overlay is applied to the buffer text, and the overlay face takes priority over the buffer text's faces
> 2. If the overlay had a display property, that would take priority over the buffer text's display property, but since
> the overlay has no such property, this doesn't happen
> 3. After the overlay is applied, the display property is applied, and its faces take priority over the existing faces,
> including those supplied by the overlay

That's not how the code works, if you want to talk about the
implementation.  What actually happens is this:

  . The display engine displays the buffer text with the overlay face
    applied, one character at a time, until it bumps into the display
    string.
  . The display engine then stops displaying buffer text, pushes its
    internal state onto a stack, then starts displaying the display
    string, using the face of that string.  If the face doesn't
    specify all the face attributes, it is merged with the face of the
    "underlying" buffer text, in this case the face of the hl-line
    overlay.
  . When the text of the display string is exhausted, the display
    engine pops the previous state from the stack, jumps to buffer
    position beyond the text replaced by the display string, and
    continues displaying buffer text using the overlay face.

> So the priority goes: display property faces > overlay faces > buffer faces. Am I on the right track?

No, because the priority is not being considered in this situation.
The priority is only considered when the same text has both text
properties that specify a face and one or more overlays that also
specify a face.

Message #23 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Travis Foster <tsfoster <at> mtu.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, 25348 <at> debbugs.gnu.org
Cc: Drew Adams <drew.adams <at> oracle.com>
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Wed, 4 Jan 2017 13:59:00 -0800

[Message part 1 (text/plain, inline)]

> If the face of the display string doesn't specify the background
> color, the only sensible thing to do is to use the background of the
> "underlying" buffer text.  (Note that in this case the issue of
> priority doesn't arise either, because only one of the sources
> specifies the background color.)

Alright ... and the "underlying" face it appears to use is the one applied
to the first character of the replacee text. That makes sense, I guess.

> > So the priority goes: display property faces > overlay faces > buffer
faces. Am I on the right track?
>
> No, because the priority is not being considered in this situation.
> The priority is only considered when the same text has both text
> properties that specify a face and one or more overlays that also
> specify a face.

Apologies, I was using 'priority' in the more general sense of "which faces
will show up on the screen", rather than in the implementation sense of the
overlay `priority` property and priority calculation. Perhaps I should use
another word instead, like 'precedence' or something.

> > it does conflict with the documentation stating that overlays always
> > take priority over text properties.
>
> Not in my view and interpretation, no.

But it appears to, I think. All I'm saying here is that, based entirely on
the wording of the documentation and with no insight into the
implementation, the most intuitive interpretation is that the display text
property should be expanded first, and then the faces and properties from
the overlay should be applied on top of that. I realize that this wouldn't
quite work, since trying to apply an overlay on top of already replaced
text would be problematic, but it's still surprising that the precedence
for faces isn't [overlay display string > overlay text > buffer display
string > buffer text]. I don't know, maybe I'm wrong.

So, alright, this isn't a bug. What if I don't want this behavior, though?
Is there anything I can do to my display properties so that they don't show
through overlays? I sort of doubt it at this point, but I might as well ask.

[Message part 2 (text/html, inline)]

Message #26 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Travis Foster <tsfoster <at> mtu.edu>
Cc: 25348 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Thu, 05 Jan 2017 18:27:55 +0200

> From: Travis Foster <tsfoster <at> mtu.edu>
> Date: Wed, 4 Jan 2017 13:59:00 -0800
> Cc: Drew Adams <drew.adams <at> oracle.com>
> 
> So, alright, this isn't a bug. What if I don't want this behavior, though? Is there anything I can do to my display
> properties so that they don't show through overlays? I sort of doubt it at this point, but I might as well ask.

As I already mentioned, if your face for the display string doesn't
specify a background color, the hl-line background color will be used.

If that's not an option, perhaps you could do that dynamically, by
changing the face of the display string whenever the hl-line overlay
is on the line where you have your display string?

Or maybe someone else will have more clever ideas.

Message #29 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Travis Foster <tsfoster <at> mtu.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, 25348 <at> debbugs.gnu.org
Cc: Drew Adams <drew.adams <at> oracle.com>
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Thu, 5 Jan 2017 10:10:30 -0800

[Message part 1 (text/plain, inline)]

> As I already mentioned, if your face for the display string doesn't
> specify a background color, the hl-line background color will be used.

Yeah, I don't think that's an option. My use case is, I'm coloring tab
characters with different colors for multiple segments within the
character. So to do that, I'm setting the tab's display property to a
number of spaces equal to the width of the tab, and then I'm coloring the
spaces separately. But, the entire point is that I'm coloring whitespace,
so it has to set the background color.

> If that's not an option, perhaps you could do that dynamically, by
> changing the face of the display string whenever the hl-line overlay
> is on the line where you have your display string?

That's a possibility. I'll have to think about that.

Thanks for the help.

[Message part 2 (text/html, inline)]

Message #32 received at 25348 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Travis Foster <tsfoster <at> mtu.edu>, 25348 <at> debbugs.gnu.org,
 drew.adams <at> oracle.com
Subject: Re: bug#25348: `display` property faces are prioritized above overlays
Date: Sun, 29 Sep 2019 18:22:08 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> This is normal operation of the Emacs display engine, it has been like
> that since Emacs 21.  IOW, this is not a bug.

I think this was the conclusion here, so I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.