GNU bug report logs - #76658
Emacs 27.2; Prioritized invisible overlay is NOT taking precedence over non-prioritized display overlay

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76658 in the body.
You can then email your comments to 76658 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Emacs 27.2; Prioritized invisible overlay is NOT taking precedence
 over non-prioritized display overlay
Date: Fri, 28 Feb 2025 18:23:41 -0500

[Message part 1 (text/plain, inline)]

Hello,

This bug report is essentially taken from my Emacs Stack Exchange post:
https://emacs.stackexchange.com/questions/83169/why-doesnt-a-prioritized-invisible-overlay-take-precedence-over-non-prioritized

## Details

I have several overlays being applied to a buffer to replace the `display`
property of certain characters. There is also another overlay being applied
(conditionally) for making the nested items/text `invisible` in _addition_
to changing the `display` property. I set the overlay with the `invisible`
property to a higher priority (i.e. actually having a priority), but it is
not working as expected.

The character is _still_ being displayed with the alternative `display`
character while _also_ having the `invisible` property set to `t` from the
higher priority overlay.

The only thing that I could find that would _seem_ to explain this behavior
was an obscure Reddit thread:
https://www.reddit.com/r/emacs/comments/17nxc9s/problem_making_overlays_invisible/
whereby the `display` property apparently takes precedence over
`invisible`. This makes sense for non-prioritized overlays, however I
wouldn't imagine this would be correct when there is a `priority` property
applied.

## Reproducible Code/Data

Example code (using `ov` library) to demonstrate the problem:

```lisp
(ov 1 2 'display "•")
(ov 10 12 'display "★")
(ov 19 21 'display "★")
(ov 28 30 'display "★")
(ov 10 37 'invisible t 'priority 10)
```

Example data to show the issue (NOTE: - Org mode is irrelevant, it occurs
by executing the code against the same data in a scratch buffer):

```text
* Root A
** Sub A
** Sub B
** Sub C
* Root B
** Sub 1
** Sub 2
** Sub 3
** Sub 4
* Root C
** Sub I
** Sub II
* Root D
```

The even more confusing issue is that the effect _would_ seem to work as
expected when I change the `invisible` overlay start index by 1 less (e.g.
`(ov 9 37 'invisible t 'priority 10)`). This doesn't seem to logically be
consistent.

### Image of Example Code/Data Reproducing the Issue:
https://i.sstatic.net/8cAP29TK.png

*The character at index position 10 is visible despite having the
`invisible` property set to `t` and the `invisible` overlay having the
highest priority.*

## Investigation Notes

Even after reviewing the documentation for `overlays` and the `priority`
property (
https://www.gnu.org/software/emacs/manual/html_node/elisp/Overlay-Properties.html),
it _still_ does not make sense. The documentation says (emphasis mine):

> This property’s value determines the priority of the overlay. If you want
to specify a priority value, use either nil (or zero), or a positive
integer, or a cons of two values. Any other value triggers undefined
behavior.
>
> **The priority matters when two or more overlays cover the same character
and both specify the same property with different values; the one whose
priority value is higher overrides the other.** (For the face property, the
higher priority overlay’s value does not completely override the other
value; instead, its individual face attributes override the corresponding
face attributes of the face property whose priority is lower.) If two
overlays have the same priority value, and one is “nested” in the other
(i.e., covers fewer buffer or string positions), then the inner one will
prevail over the outer one. If neither is nested in the other then you
should not make assumptions about which overlay will prevail.
>
> **When a Lisp program puts overlays with defined priorities on text that
might have overlays without priorities, this could cause undesirable
results, because any overlay with a positive priority value will override
all the overlays without a priority.** Since most Emacs features that use
overlays don’t specify priorities for their overlays, integer priorities
should be used with care. Instead of using integer priorities and risk
overriding other overlays, you can use priority values of the form (primary
. secondary), where the primary value is used as described above, and
secondary is the fallback value used when primary and the nesting
considerations fail to resolve the precedence between overlays. In
particular, priority value (nil . n), with n a positive integer, enables
you to have the overlays ordered by priority when necessary without
completely overriding other overlays.
>
> Currently, all overlays take priority over text properties.

Even when reading the first sentence of the third paragraph in the
documentation, the issue _still_ occurs even if I defined a lower priority
for the `display` overlays.

NOTE: - The `buffer-invisibility-spec` is correctly set up to support
invisibility too, just in case anyone thought that might be the issue. It
is not. I am certain of it.

## Question

Can anyone explain why this issue only seems to occur when the overlay `(ov
10 37 'invisible t 'priority 10)` is applied but not when `(ov 9 37
'invisible t 'priority 10)` is applied (i.e. why does including the
preceding new line make the effect "work")? Is there a finer point of
nuance for overlays and the `priority`, `display`, and `invisible`
properties?

An ancillary note: I _can_ achieve the proper effect, with the proper
indexes, by explicitly setting the `display` property to an empty string on
the `invisible` overlay: `(ov 10 37 'invisible t 'priority 10 'display "")`.

However, despite this, I still want to properly understand things, why the
issue occurs/doesn't seem to make sense, etc. It's not an XY problem.

#### Standard environment disclosure:

- OS: MacOS
- Chip: Intel
- Emacs Version: 27.2.1
- Port: Mitsuharu Yamamoto / Railwaycat
- Other Notes: - No special distributions / etc.
- Relevant Common Packages Used:
    - ov
    - Standard internal Emacs packages

Thanks,
David

[Message part 2 (text/html, inline)]

Message #8 received at 76658 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
Cc: 76658 <at> debbugs.gnu.org
Subject: Re: bug#76658: Emacs 27.2;
 Prioritized invisible overlay is NOT taking precedence over
 non-prioritized display overlay
Date: Sat, 01 Mar 2025 12:35:49 +0200

tags 76658 notabug
thanks

> From: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
> Date: Fri, 28 Feb 2025 18:23:41 -0500
> 
> This bug report is essentially taken from my Emacs Stack Exchange post:
> https://emacs.stackexchange.com/questions/83169/why-doesnt-a-prioritized-invisible-overlay-take-precedence-over-non-prioritized

It is not easy to understand what exactly is the issue which you are
reporting and for which you are asking for explanations.  Both this
report and the linked postings have a lot of stuff, most of which is
AFAIU irrelevant.

My understanding is that the issue you are asking about is the effect
on display of having both 'display' and 'invisible' property start at
the same buffer position.  If this is not the issue you are asking
about, what's below may or may not make sense.

When both 'display' and 'invisible' properties start at the same
buffer position, and the 'display' property is a "replacing" property
(i.e. it instructs Emacs to show something else instead of buffer
text), then the 'display' property "wins", in the sense that the
invisible property is effectively ignored.  This happens due to how
the Emacs display engine processes properties:

  . it processes 'display' properties before 'invisible'
  . when either a replacing 'display' property or 'invisible' property
    is found, the display engine completely skips the text covered by
    the property, so any other properties in the same text are not
    processed

This should explain everything that you see.  In particular, overlay
priorities have nothing to do with this, since (as the ELisp manual
says) the priorities are only examined when two or more overlays have
the same property for the same buffer position, which is not the case
here.  Also, it explains why, if the 'invisible' property starts
before 'display', it makes all the text invisible, including the
overlay that specifies a 'display' property which shows an image.

If you sometimes need to have the image (defined via the 'display'
property) to vanish from display, simply remove the 'display'
property, or use the conditional 'display' property of the form
'(when CONDITION . DISPLAY-SPEC)' and change Lisp variables that
affect CONDITION.

Bottom line: this is not a bug.

Message #13 received at 76658 <at> debbugs.gnu.org (full text, mbox):

From: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76658 <at> debbugs.gnu.org
Subject: Re: bug#76658: Emacs 27.2; Prioritized invisible overlay is NOT
 taking precedence over non-prioritized display overlay
Date: Tue, 4 Mar 2025 23:28:55 -0500

[Message part 1 (text/plain, inline)]

Thanks for the response, Eli. Your understanding of the problem is correct.
Your response both makes sense as well and aligns with what I have
encountered in testing. Apologies for the extra information in my posts,
but I clearly was confused in my understanding of how the properties of
overlays interacted with each other and overlays between themselves.

After re-reading the documentation for an Overlay's `priority` property
several times before I asked my question, I had a suspicion that it was
only relevant on clashing properties that were declared across multiple
overlays (i.e. Overlay Priority is a tie-breaker at the individual property
level between the same properties, not at the overlay level between
overlays: first line of the second paragraph). I was unsure though and
thought the behavior I was experiencing was wrong/my understanding of the
documentation was incorrect. The clarification and confirmation is
appreciated.

The behavior around `display` taking precedence over `invisible` is
surprising ... somewhat. I am not sure if that is documented anywhere or
called out explicitly, I probably missed it if it was. However, stopping to
think about the context, it seems to make sense and is logically what would
be expected. There is still some doubt I have for certain scenarios and
what makes intuitive sense but that is more about me needing to more
clearly understand the display engine at a finer level than I currently do
(`xdisp.c` here I come).

The cases presented thus far make sense, the first overlay processed
essentially "wins" and if the starting positioning of an overlay is the
same as another with each specifying either `invisible` or `display`, the
`display` property value of whichever overlay "wins" (since the underlying
text is _essentially_ invisible for all intents and purposes based on
what/how `display` works, correct?). In other words, it could be thought of
as `display` is `invisible` with benefits: `hide but show something else
instead` OR the other way around, `invisible` is functionally equivalent to
just `display ""`?

To solve my issue, I did end up specifying a `display` property on the
`invisible` overlay with a condition similar to the condition applied for
`invisible`: `display (if XYZ "")`. It worked, however, at the time it just
felt "wrong" or "doing more than it should need to do for the effect
desired" (if that makes sense).

Thanks for the explanation, clarification, and confirmations Eli!

Much appreciated,
David

On Sat, Mar 1, 2025 at 5:35 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> tags 76658 notabug
> thanks
>
> > From: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
> > Date: Fri, 28 Feb 2025 18:23:41 -0500
> >
> > This bug report is essentially taken from my Emacs Stack Exchange post:
> >
> https://emacs.stackexchange.com/questions/83169/why-doesnt-a-prioritized-invisible-overlay-take-precedence-over-non-prioritized
>
> It is not easy to understand what exactly is the issue which you are
> reporting and for which you are asking for explanations.  Both this
> report and the linked postings have a lot of stuff, most of which is
> AFAIU irrelevant.
>
> My understanding is that the issue you are asking about is the effect
> on display of having both 'display' and 'invisible' property start at
> the same buffer position.  If this is not the issue you are asking
> about, what's below may or may not make sense.
>
> When both 'display' and 'invisible' properties start at the same
> buffer position, and the 'display' property is a "replacing" property
> (i.e. it instructs Emacs to show something else instead of buffer
> text), then the 'display' property "wins", in the sense that the
> invisible property is effectively ignored.  This happens due to how
> the Emacs display engine processes properties:
>
>   . it processes 'display' properties before 'invisible'
>   . when either a replacing 'display' property or 'invisible' property
>     is found, the display engine completely skips the text covered by
>     the property, so any other properties in the same text are not
>     processed
>
> This should explain everything that you see.  In particular, overlay
> priorities have nothing to do with this, since (as the ELisp manual
> says) the priorities are only examined when two or more overlays have
> the same property for the same buffer position, which is not the case
> here.  Also, it explains why, if the 'invisible' property starts
> before 'display', it makes all the text invisible, including the
> overlay that specifies a 'display' property which shows an image.
>
> If you sometimes need to have the image (defined via the 'display'
> property) to vanish from display, simply remove the 'display'
> property, or use the conditional 'display' property of the form
> '(when CONDITION . DISPLAY-SPEC)' and change Lisp variables that
> affect CONDITION.
>
> Bottom line: this is not a bug.
>

[Message part 2 (text/html, inline)]

Message #18 received at 76658-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
Cc: 76658-done <at> debbugs.gnu.org
Subject: Re: bug#76658: Emacs 27.2; Prioritized invisible overlay is NOT
 taking precedence over non-prioritized display overlay
Date: Wed, 05 Mar 2025 15:22:20 +0200

> From: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
> Date: Tue, 4 Mar 2025 23:28:55 -0500
> Cc: 76658 <at> debbugs.gnu.org
> 
> The behavior around `display` taking precedence over `invisible` is surprising ... somewhat. I am not sure if
> that is documented anywhere or called out explicitly, I probably missed it if it was.

It wasn't documented until now, but I added it notes to the ELisp
manual about it, since you are not the first one bumping into this
subtlety.

> The cases presented thus far make sense, the first overlay processed essentially "wins" and if the starting
> positioning of an overlay is the same as another with each specifying either `invisible` or `display`, the
> `display` property value of whichever overlay "wins" (since the underlying text is _essentially_ invisible for all
> intents and purposes based on what/how `display` works, correct?). In other words, it could be thought of as
> `display` is `invisible` with benefits: `hide but show something else instead` OR the other way around,
> `invisible` is functionally equivalent to just `display ""`?

Right.

> To solve my issue, I did end up specifying a `display` property on the `invisible` overlay with a condition
> similar to the condition applied for `invisible`: `display (if XYZ "")`. It worked, however, at the time it just felt
> "wrong" or "doing more than it should need to do for the effect desired" (if that makes sense).
> 
> Thanks for the explanation, clarification, and confirmations Eli!

Thanks, I'm therefore closing this bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.