GNU bug report logs - #48740
28.0.50; Composition text property is not always honoured

Previous Next

Package: emacs;

Reported by: Ihor Radchenko <yantar92 <at> gmail.com>

Date: Sun, 30 May 2021 07:32:01 UTC

Severity: normal

Tags: patch

Found in version 28.0.50

Fixed in version 28.1

Done: Stefan Kangas <stefan <at> marxist.se>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 48740 in the body.
You can then email your comments to 48740 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 30 May 2021 07:32:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ihor Radchenko <yantar92 <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Sun, 30 May 2021 07:32:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Composition text property is not always honoured
Date: Sun, 30 May 2021 15:35:51 +0800
[Message part 1 (text/plain, inline)]
Hello,

I encountered a very strange issue with text composition using
prettify-symbols-mode. It happens with text copied from an Org buffer to
Org agenda buffer when the composed text has >4 chars length.

Recipe:
1. git clone https://github.com/yantar92/org
2. cd /path/to/org
3. git checkout feature/org-agenda-fontify
4. make clean
5. make
6. emacs -Q -L ./lisp/ -l org -l /tmp/1.el ~/Org/inbox.org
5. M-x org-agenda < t
6. M-x org-todo on the first item selecting "ONGOING" state
7. Note that "ONGOING" keyword gets replaced with composed icon (as
   defined in 1.el). M-x describe-text-properties shows composition set
   to (0 7 [128119])
8. M-x org-agenda-redo-all
9. The "ONGOING" keyword is not replaced by composed icon anymore. Yet,
   M-x describe-text-properties still shows composition set to (0 7
   [128119])

The 1.el and inbox.org are attached.

Best,
Ihor

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, cairo version 1.16.0)
 of 2021-05-29 built on localhost
Repository revision: 2003a407c8bdf79dabe4f782039706d91c4fd9c5
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Gentoo/Linux

[inbox.org (application/vnd.lotus-organizer, inline)]
[1.el (application/emacs-lisp, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 30 May 2021 11:10:02 GMT) Full text and rfc822 format available.

Message #8 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50;
 Composition text property is not always honoured
Date: Sun, 30 May 2021 14:10:01 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> Date: Sun, 30 May 2021 15:35:51 +0800
> 
> I encountered a very strange issue with text composition using
> prettify-symbols-mode. It happens with text copied from an Org buffer to
> Org agenda buffer when the composed text has >4 chars length.

That probably means the composition property is somehow invalid or
something.

> Recipe:
> 1. git clone https://github.com/yantar92/org
> 2. cd /path/to/org
> 3. git checkout feature/org-agenda-fontify
> 4. make clean
> 5. make
> 6. emacs -Q -L ./lisp/ -l org -l /tmp/1.el ~/Org/inbox.org
> 5. M-x org-agenda < t
> 6. M-x org-todo on the first item selecting "ONGOING" state
> 7. Note that "ONGOING" keyword gets replaced with composed icon (as
>    defined in 1.el). M-x describe-text-properties shows composition set
>    to (0 7 [128119])
> 8. M-x org-agenda-redo-all
> 9. The "ONGOING" keyword is not replaced by composed icon anymore. Yet,
>    M-x describe-text-properties still shows composition set to (0 7
>    [128119])

Step 3 above fails here:

  $ git checkout feature/org-agenda-fontify
  error: pathspec 'feature/org-agenda-fontify' did not match any file(s) known to git.

Anyway, would it be possible to prepare a recipe that is simpler, and
doesn't involve checking out and building a large package?  That'd
make debugging this much less unpleasant.  TIA.

Alternatively, show the text in the problematic buffer with all the
text properties, overlays, and other stuff around the problematic
place, and maybe that alone will suffice (but in that case it would
probably also be easy to generate a simpler text case...)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 30 May 2021 13:05:01 GMT) Full text and rfc822 format available.

Message #11 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 30 May 2021 21:08:52 +0800
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:
> That probably means the composition property is somehow invalid or
> something.

Yeah. But I fail to understand what can be wrong there. The composition
property does not appear to change. Moreover, doing the same thing but
with older pretty-symbols-mode (not the built-in prettify-symbols-mode)
does not cause any issues despite the former is also using composition.

> Step 3 above fails here:
>
>   $ git checkout feature/org-agenda-fontify
>   error: pathspec 'feature/org-agenda-fontify' did not match any file(s) known to git.

Oops. Forgot to push the branch. Should work now.

> Anyway, would it be possible to prepare a recipe that is simpler, and
> doesn't involve checking out and building a large package?  That'd
> make debugging this much less unpleasant.  TIA.

Sorry, I had no luck making things simpler. In theory, what org-agenda
does (after my changes) is fontifying the org buffer and copying the
fontified text with all the text properties (including composition) into
agenda view, so that fontification is preserved. I tried to repeat what
agenda does, but failed to reproduce the problem. I tried to debug the
agenda code, but, as I mentioned, composition property is not removed or
changed. I simply don't understand what is going wrong.

> Alternatively, show the text in the problematic buffer with all the
> text properties, overlays, and other stuff around the problematic
> place, and maybe that alone will suffice (but in that case it would
> probably also be easy to generate a simpler text case...)

See the attached. I printed full string objects at steps 5, 6, and 8 for
the whole agenda buffer, the changed agenda line, and the output of
describe-text-properties for the first symbol in the changed
TODO-keyword.

Hope if helps. Though text properties in agenda buffers are messy.

Best,
Ihor

[agenda-step5.txt (text/plain, attachment)]
[agenda-step6.txt (text/plain, attachment)]
[agenda-step8.txt (text/plain, attachment)]
[agendaline-step5.txt (text/plain, attachment)]
[agendaline-step6.txt (text/plain, attachment)]
[agendaline-step8.txt (text/plain, attachment)]
[agendasymbol-step5.txt (text/plain, attachment)]
[agendasymbol-step6.txt (text/plain, attachment)]
[agendasymbol-step8.txt (text/plain, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 30 May 2021 13:53:02 GMT) Full text and rfc822 format available.

Message #14 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 30 May 2021 16:52:13 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Sun, 30 May 2021 21:08:52 +0800
> 
> > Step 3 above fails here:
> >
> >   $ git checkout feature/org-agenda-fontify
> >   error: pathspec 'feature/org-agenda-fontify' did not match any file(s) known to git.
> 
> Oops. Forgot to push the branch. Should work now.

It doesn't.  I'm now stuck at step 6, because there's no "item
selecting "ONGOING" state".  Perhaps because inbox.org says:

  #+SETUPFILE: /home/yantar92/Org/common.setup

?  (I also am not sure I understand what exactly item 6 wants me to
do...)

> > Anyway, would it be possible to prepare a recipe that is simpler, and
> > doesn't involve checking out and building a large package?  That'd
> > make debugging this much less unpleasant.  TIA.
> 
> Sorry, I had no luck making things simpler.

See above: I'm asking for a simpler test case because IME getting
stuff this complex to reproduce what you see on your machine is not
easy and full of frustration.  For example, you assume that the Emacs
I have installed, the one invoked by "make", is Emacs 28?  But that's
not what I have here.

> > Alternatively, show the text in the problematic buffer with all the
> > text properties, overlays, and other stuff around the problematic
> > place, and maybe that alone will suffice (but in that case it would
> > probably also be easy to generate a simpler text case...)
> 
> See the attached. I printed full string objects at steps 5, 6, and 8 for
> the whole agenda buffer, the changed agenda line, and the output of
> describe-text-properties for the first symbol in the changed
> TODO-keyword.
> 
> Hope if helps. Though text properties in agenda buffers are messy.

Ouch!  No, this doesn't help at all, sorry.  can you tell which of the
properties in the last step are in effect for the all buffer positions
of "ONGOING"?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 30 May 2021 15:09:02 GMT) Full text and rfc822 format available.

Message #17 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 30 May 2021 23:13:16 +0800
Eli Zaretskii <eliz <at> gnu.org> writes:

> It doesn't.  I'm now stuck at step 6, because there's no "item
> selecting "ONGOING" state".  Perhaps because inbox.org says:

Sorry. Probably that instruction is not very clearly written. What I
meant is moving cursor to the first TODO item in the agenda view. Then,
"t n" (switch todo state [t] from TODO to ONGOING [n]). The "t" is
default agenda shortcut to change todo state and "n" is defined in 1.el
as a shortcut to set "ONGOING" state.

>   #+SETUPFILE: /home/yantar92/Org/common.setup

It should not matter here. Also, the instruction have been reproduced by
at least one other user on different machine. See
https://orgmode.org/list/m2o8dbhut7.fsf <at> gmail.com/

> See above: I'm asking for a simpler test case because IME getting
> stuff this complex to reproduce what you see on your machine is not
> easy and full of frustration.  For example, you assume that the Emacs
> I have installed, the one invoked by "make", is Emacs 28?  But that's
> not what I have here.

I understand your concern and I unfortunately I cannot come up with
anything less complex. Also, I tried to test using Emacs master, Emacs
27, and Emacs 25. I was able to reproduce the issue with all the Emacs
versions.

> Ouch!  No, this doesn't help at all, sorry.  can you tell which of the
> properties in the last step are in effect for the all buffer positions
> of "ONGOING"?

The simplified text properties on the todo-keyword substring are the following:

At step 5 (first build of agenda):
For "TODO":
14 18 (org-heading t org-category \"inbox\" face org-todo fontified t tags (...) org-priority-highest 65 org-priority-lowest 67 time-of-day nil duration nil breadcrumbs nil txt #(...) level \" \" time \"\" extra \"\" format (...) dotime t done-face org-agenda-done org-not-done-regexp \"...\" org-todo-regexp \"...\" org-complex-heading-regexp \"...\" mouse-face highlight help-echo \"...\" org-marker #<marker (moves after insertion) at 292 in inbox.org> org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> priority 1001 ts-date nil type \"todo\" todo-state #(...) org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil)

At step 6 (after changing todo state from TODO to ONGOING):
For "ONG"
14 17 (todo-state #(...) type \"todo\" ts-date nil priority 1001 org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> org-marker #<marker (moves after insertion) at 292 in inbox.org> help-echo \"...\" mouse-face highlight org-complex-heading-regexp \"...\" org-todo-regexp \"...\" org-not-done-regexp \"...\" done-face org-agenda-done org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil dotime t format (...) extra \"\" time \"\" level \" \" txt #(...) breadcrumbs nil duration nil time-of-day nil org-priority-lowest 67 org-priority-highest 65 tags (...) org-category \"inbox\" face org-todo prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) org-todo-head #(...) fontified t org-heading t)
For "OING"
17 21 (todo-state #(...) type \"todo\" ts-date nil priority 1001 org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> org-marker #<marker (moves after insertion) at 292 in inbox.org> help-echo \"...\" mouse-face highlight org-complex-heading-regexp \"...\" org-todo-regexp \"...\" org-not-done-regexp \"...\" done-face org-agenda-done org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil dotime t format (...) extra \"\" time \"\" level \" \" txt #(...) breadcrumbs nil duration nil time-of-day nil org-priority-lowest 67 org-priority-highest 65 tags (...) org-category \"inbox\" face org-todo prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) org-todo-head #(\"TODO\" 0 4 (face org-todo)) fontified t org-heading t)

Note the composition part: prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119])

At step 8 (after org-agenda-redo-all):
For "ONG"
14 17 (org-heading t org-category \"inbox\" face org-todo prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) org-todo-head #(...) fontified t tags (...) org-priority-highest 65 org-priority-lowest 67 time-of-day nil duration nil breadcrumbs nil txt #(...) level \" \" time \"\" extra \"\" format (...) dotime t done-face org-agenda-done org-not-done-regexp \"...\" org-todo-regexp \"...\" org-complex-heading-regexp \"...\" mouse-face highlight help-echo \"...\" org-marker #<marker (moves after insertion) at 292 in inbox.org> org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> priority 1001 ts-date nil type \"todo\" todo-state #(...) org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil)
For "OING"
17 21 (org-heading t org-category \"inbox\" face org-todo prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) org-todo-head #(...) fontified t tags (...) org-priority-highest 65 org-priority-lowest 67 time-of-day nil duration nil breadcrumbs nil txt #(...) level \" \" time \"\" extra \"\" format (...) dotime t done-face org-agenda-done org-not-done-regexp \"...\" org-todo-regexp \"...\" org-complex-heading-regexp \"...\" mouse-face highlight help-echo \"...\" org-marker #<marker (moves after insertion) at 292 in inbox.org> org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> priority 1001 ts-date nil type \"todo\" todo-state #(...) org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil)

Note the composition part: prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119])
The composition part remain unchanged except position in the plist.

Best,
Ihor




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 30 May 2021 15:59:01 GMT) Full text and rfc822 format available.

Message #20 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 30 May 2021 18:58:20 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Sun, 30 May 2021 23:13:16 +0800
> 
> At step 8 (after org-agenda-redo-all):
> For "ONG"
> 14 17 (org-heading t org-category \"inbox\" face org-todo prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) org-todo-head #(...) fontified t tags (...) org-priority-highest 65 org-priority-lowest 67 time-of-day nil duration nil breadcrumbs nil txt #(...) level \" \" time \"\" extra \"\" format (...) dotime t done-face org-agenda-done org-not-done-regexp \"...\" org-todo-regexp \"...\" org-complex-heading-regexp \"...\" mouse-face highlight help-echo \"...\" org-marker #<marker (moves after insertion) at 292 in inbox.org> org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> priority 1001 ts-date nil type \"todo\" todo-state #(...) org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil)
> For "OING"
> 17 21 (org-heading t org-category \"inbox\" face org-todo prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) org-todo-head #(...) fontified t tags (...) org-priority-highest 65 org-priority-lowest 67 time-of-day nil duration nil breadcrumbs nil txt #(...) level \" \" time \"\" extra \"\" format (...) dotime t done-face org-agenda-done org-not-done-regexp \"...\" org-todo-regexp \"...\" org-complex-heading-regexp \"...\" mouse-face highlight help-echo \"...\" org-marker #<marker (moves after insertion) at 292 in inbox.org> org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> priority 1001 ts-date nil type \"todo\" todo-state #(...) org-agenda-type todo org-last-args nil org-redo-cmd (...) org-series-cmd nil)
> 
> Note the composition part: prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119])
> The composition part remain unchanged except position in the plist.

No, it is also subdivided into two separate regions, each one of 3
characters.  And that's why it doesn't work: the display engine
expects that the composed text should be covered by a single text
property which specifies the length (7 - 0 = 7) that is equal to the
length of the covered text (301 - 294 = 7).

So the problem is caused by whatever code in org-agenda-redo-all that
"breaks" the composition property into 2 separate adjacent properties.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Mon, 31 May 2021 12:56:02 GMT) Full text and rfc822 format available.

Message #23 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Mon, 31 May 2021 21:00:29 +0800
Eli Zaretskii <eliz <at> gnu.org> writes:

> No, it is also subdivided into two separate regions, each one of 3
> characters.  And that's why it doesn't work: the display engine
> expects that the composed text should be covered by a single text
> property which specifies the length (7 - 0 = 7) that is equal to the
> length of the covered text (301 - 294 = 7).
>
> So the problem is caused by whatever code in org-agenda-redo-all that
> "breaks" the composition property into 2 separate adjacent properties.

Yet, the same "broken" property does work in step 6... And in both cases
the composition properties of the two regions are same. The two
intervals are there because other properties in the list are not same,
as I understand.

The only possibility I can think about is that compositions in the two
intervals are 'eq at step 6, but only 'equal at step 8. (How on Earth
can one debug this???)

Best,
Ihor




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Mon, 31 May 2021 14:34:02 GMT) Full text and rfc822 format available.

Message #26 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Mon, 31 May 2021 17:32:50 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Mon, 31 May 2021 21:00:29 +0800
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > No, it is also subdivided into two separate regions, each one of 3
> > characters.  And that's why it doesn't work: the display engine
> > expects that the composed text should be covered by a single text
> > property which specifies the length (7 - 0 = 7) that is equal to the
> > length of the covered text (301 - 294 = 7).
> >
> > So the problem is caused by whatever code in org-agenda-redo-all that
> > "breaks" the composition property into 2 separate adjacent properties.
> 
> Yet, the same "broken" property does work in step 6...

No, AFAIU in step 6 a single property list covers the whole of the
word ONGOING.

> The two intervals are there because other properties in the list are
> not same, as I understand.

Exactly.  And that cannot work with the composition property.

> The only possibility I can think about is that compositions in the two
> intervals are 'eq at step 6, but only 'equal at step 8.

That cannot be the reason, because the display code doesn't test
'eq'uality.  I clearly see in the code that the problem is with
"partial" coverage of the text.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Tue, 01 Jun 2021 14:25:02 GMT) Full text and rfc822 format available.

Message #29 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Tue, 01 Jun 2021 22:28:50 +0800
Eli Zaretskii <eliz <at> gnu.org> writes:

>> Yet, the same "broken" property does work in step 6...
>
> No, AFAIU in step 6 a single property list covers the whole of the
> word ONGOING.

I feel that I am missing something then.

Stripping all unrelated properties from the plists, I get

At step 6 (after changing todo state from TODO to ONGOING):
For "ONG"
14 17 (... prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) ...)
For "OING"
17 21 (... prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) ...)

At step 8 (after org-agenda-redo-all):
For "ONG"
14 17 (... prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) ...)
For "OING"
17 21 (... prettify-symbols-end 301 prettify-symbols-start 294 composition (0 7 [128119]) ...)

step 6 and step 8 look identical for me, yet step 6 does have
composition being rendered, unlike step 8.

>> The two intervals are there because other properties in the list are
>> not same, as I understand.
>
> Exactly.  And that cannot work with the composition property.

Again, I feel that I miss something. Consider the following test case:

(switch-to-buffer "test")
(insert "asdasd")
(compose-region (point-min) (point-max) '(?a (bl . tl) ?| (bl . tl) ?b))
(put-text-property 1 3 'test 1)
(put-text-property 3 6 'test 2)
(buffer-string)

The output is

#("asdasd" 0 2 (composition (43 6 [97 72 124 72 98]) test 1) 2 5 (composition (43 6 [97 72 124 72 98]) test 2) 5 6 (composition (43 6 [97 72 124 72 98])))

Composition works fine here despite spanning over multiple intervals.
Again, I feel that I miss something in your explanation.

Best,
Ihor




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Tue, 01 Jun 2021 15:26:02 GMT) Full text and rfc822 format available.

Message #32 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Tue, 01 Jun 2021 18:25:17 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Tue, 01 Jun 2021 22:28:50 +0800
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> The two intervals are there because other properties in the list are
> >> not same, as I understand.
> >
> > Exactly.  And that cannot work with the composition property.
> 
> Again, I feel that I miss something. Consider the following test case:
> 
> (switch-to-buffer "test")
> (insert "asdasd")
> (compose-region (point-min) (point-max) '(?a (bl . tl) ?| (bl . tl) ?b))
> (put-text-property 1 3 'test 1)
> (put-text-property 3 6 'test 2)
> (buffer-string)
> 
> The output is
> 
> #("asdasd" 0 2 (composition (43 6 [97 72 124 72 98]) test 1) 2 5 (composition (43 6 [97 72 124 72 98]) test 2) 5 6 (composition (43 6 [97 72 124 72 98])))
> 
> Composition works fine here despite spanning over multiple intervals.
> Again, I feel that I miss something in your explanation.

We are mis-communicating.  You already stated correctly the reason for
the problem:

> The only possibility I can think about is that compositions in the two
> intervals are 'eq at step 6, but only 'equal at step 8. (How on Earth
> can one debug this???)

More accurately, it isn't the intervals that must be 'eq', it's the
values of the 'composition' property in the 2 adjacent intervals.  In
Step 6 (and in the example above) they are 'eq', in Step 8 they
aren't.  All the rest was my failed attempt to correct your inaccurate
description of what happens; forget about that if you want.

I don't know why the values of the 'composition' property become not
'eq'; perhaps some code does a copy-sequence or something?  Presumably
if you take a good look at what happens in org-agenda-redo-all and its
subroutines, you will find that code.  In general, when you are
dealing with static compositions, I'd suggest not to break the
composed region into several separate intervals with different
property lists: it's dangerous, as you see.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Tue, 01 Jun 2021 16:26:01 GMT) Full text and rfc822 format available.

Message #35 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: yantar92 <at> gmail.com
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50;
 Composition text property is not always honoured
Date: Tue, 01 Jun 2021 19:25:04 +0300
> Date: Tue, 01 Jun 2021 18:25:17 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 48740 <at> debbugs.gnu.org
> 
> I don't know why the values of the 'composition' property become not
> 'eq'; perhaps some code does a copy-sequence or something?

Btw, looking closer at the plist of the interval after
org-agenda-redo-all, I see that it is quite a mess.  For example, the
'composition' property appears there several times, not just once.

Here's the full value of the interval's plist that I see in the
debugger in Step 6:

  (org-heading t fontified t org-todo-head #("TODO" 0 4 (face org-todo)) composition (0 7 [128119]) prettify-symbols-start 294 prettify-symbols-end 301 face org-todo org-category "inbox" tags (#("SKIP" 0 4 (inherited t)) #("INBOX" 0 5 (inherited t)) #("BOOKMARK" 0 8 (org-category "inbox" keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) mouse-face highlight face (org-tag org-level-1) fontified t)) #("misc" 0 4 (org-category "inbox" keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) mouse-face highlight face (org-tag org-level-1) fontified t))) org-priority-highest 65 org-priority-lowest 67 time-of-day nil duration nil breadcrumbs nil txt #("TODO belluzj [Github] belluzj/fantasque-sans: A font family with a great monospaced variant for programmers. :SKIP:INBOX::BOOKMARK:misc:" 0 4 (fontified t face (org-todo org-level-1) org-category "inbox" org-heading t) 4 108 (fontified t face org-level-1 org-category "inbox" org-heading t) 108 110 (org-heading t) 110 114 (inherited t org-heading t) 114 115 (org-heading t) 115 120 (inherited t org-heading t) 120 122 (org-heading t) 122 130 (org-category "inbox" keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) mouse-face highlight face (org-tag org-level-1) fontified t org-heading t) 130 131 (org-heading t) 131 135 (fontified t face (org-tag org-level-1) mouse-face highlight keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) org-category "inbox" org-heading t) 135 136 (org-heading t)) level "" time "" extra "" format (((org-prefix-has-time nil) (org-prefix-has-tag nil) (org-prefix-category-length 12) (org-prefix-has-effort nil) (org-prefix-has-breadcrumbs nil)) (format " %s %s" (format "%s" (if (member category-icon '("" nil)) "" (concat category-icon "" (get-text-property 0 'extra-space category-icon)))) (format "%-12s" (if (member category '("" nil)) "" (concat category ":" (get-text-property 0 'extra-space category)))))) dotime t org-series-cmd nil org-redo-cmd (org-todo-list (or (and (numberp current-prefix-arg) current-prefix-arg) nil current-prefix-arg nil)) org-last-args nil org-agenda-type todo done-face org-agenda-done org-not-done-regexp "\\(DOING\\|HOLD\\|ONGOING\\|REVIEW\\|SOMEDAY\\|T\\(?:ICKLER\\|ODO\\)\\|WAITING\\)" org-todo-regexp "\\(CANCELLED\\|DO\\(?:ING\\|NE\\)\\|F\\(?:AILED\\|ROZEN\\)\\|HOLD\\|MERGED\\|ONGOING\\|REVIEW\\|SOMEDAY\\|T\\(?:ICKLER\\|ODO\\)\\|WAITING\\)" org-complex-heading-regexp "^\\(\\*+\\)\\(?: +\\(CANCELLED\\|DO\\(?:ING\\|NE\\)\\|F\\(?:AILED\\|ROZEN\\)\\|HOLD\\|MERGED\\|ONGOING\\|REVIEW\\|SOMEDAY\\|T\\(?:ICKLER\\|ODO\\)\\|WAITING\\)\\)?\\(?: +\\(\\[#.\\]\\)\\)?\\(?: +\\(.*?\\)\\)??\\(?:[ 	]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[
 	]*$" mouse-face highlight help-echo "mouse-2 or RET jump to org file ~/data/test-org/inbox.org" org-marker #<marker (moves after insertion) at 292 in inbox.org> org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> priority 1001 ts-date nil type "todo" todo-state #("TODO" 0 4 (fontified t face (org-todo org-level-1) org-category "inbox")))

And here's the value in Step 8:

  (org-series-cmd nil org-redo-cmd (org-todo-list (or (and (numberp current-prefix-arg) current-prefix-arg) nil current-prefix-arg nil)) org-last-args nil org-agenda-type todo todo-state #("ONGOING" 0 3 (fontified t org-todo-head #("TODO" 0 4 (face org-todo)) composition (0 7 [128119]) prettify-symbols-start 294 prettify-symbols-end 301 face (org-todo org-level-1) org-category "inbox") 3 7 (fontified t org-todo-head #("TODO" 0 4 (face org-todo)) composition (0 7 [128119]) prettify-symbols-start 294 prettify-symbols-end 301 face (org-todo org-level-1) org-category "inbox")) type "todo" ts-date nil priority 1001 org-hd-marker #<marker (moves after insertion) at 292 in inbox.org> org-marker #<marker (moves after insertion) at 292 in inbox.org> help-echo "mouse-2 or RET jump to org file ~/data/test-org/inbox.org" mouse-face highlight org-complex-heading-regexp "^\\(\\*+\\)\\(?: +\\(CANCELLED\\|DO\\(?:ING\\|NE\\)\\|F\\(?:AILED\\|ROZEN\\)\\|HOLD\\|MERGED\\|ONGOING\\|REVIEW\\|SOMEDAY\\|T\\(?:ICKLER\\|ODO\\)\\|WAITING\\)\\)?\\(?: +\\(\\[#.\\]\\)\\)?\\(?: +\\(.*?\\)\\)??\\(?:[ 	]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[
 	]*$" org-todo-regexp "\\(CANCELLED\\|DO\\(?:ING\\|NE\\)\\|F\\(?:AILED\\|ROZEN\\)\\|HOLD\\|MERGED\\|ONGOING\\|REVIEW\\|SOMEDAY\\|T\\(?:ICKLER\\|ODO\\)\\|WAITING\\)" org-not-done-regexp "\\(DOING\\|HOLD\\|ONGOING\\|REVIEW\\|SOMEDAY\\|T\\(?:ICKLER\\|ODO\\)\\|WAITING\\)" done-face org-agenda-done dotime t format (((org-prefix-has-time nil) (org-prefix-has-tag nil) (org-prefix-category-length 12) (org-prefix-has-effort nil) (org-prefix-has-breadcrumbs nil)) (format " %s %s" (format "%s" (if (member category-icon '("" nil)) "" (concat category-icon "" (get-text-property 0 'extra-space category-icon)))) (format "%-12s" (if (member category '("" nil)) "" (concat category ":" (get-text-property 0 'extra-space category)))))) extra "" time "" level " " txt #("ONGOING belluzj [Github] belluzj/fantasque-sans: A font family with a great monospaced variant for programmers. :SKIP:INBOX::BOOKMARK:misc:" 0 3 (fontified t org-todo-head #("TODO" 0 4 (face org-todo)) composition (0 7 [128119]) prettify-symbols-start 294 prettify-symbols-end 301 face (org-todo org-level-1) org-category "inbox" org-heading t) 3 7 (fontified t org-todo-head #("TODO" 0 4 (face org-todo)) composition (0 7 [128119]) prettify-symbols-start 294 prettify-symbols-end 301 face (org-todo org-level-1) org-category "inbox" org-heading t) 7 8 (fontified t org-todo-head #("TODO" 0 4 (face org-todo)) face org-level-1 org-category "inbox" org-heading t) 8 111 (fontified t org-category "inbox" org-todo-head #("TODO" 0 4 (face org-todo)) face org-level-1 org-heading t) 111 113 (org-heading t) 113 117 (inherited t org-heading t) 117 118 (org-heading t) 118 123 (inherited t org-heading t) 123 125 (org-heading t) 125 133 (keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) mouse-face highlight face (org-tag org-level-1) org-todo-head #("TODO" 0 4 (face org-todo)) org-category "inbox" fontified t org-heading t) 133 134 (org-heading t) 134 138 (fontified t org-category "inbox" org-todo-head #("TODO" 0 4 (face org-todo)) face (org-tag org-level-1) mouse-face highlight keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) org-heading t) 138 139 (org-heading t)) breadcrumbs nil duration nil time-of-day nil org-priority-lowest 67 org-priority-highest 65 tags (#("SKIP" 0 4 (inherited t)) #("INBOX" 0 5 (inherited t)) #("BOOKMARK" 0 8 (keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) mouse-face highlight face (org-tag org-level-1) org-todo-head #("TODO" 0 4 (face org-todo)) org-category "inbox" fontified t)) #("misc" 0 4 (keymap (keymap (follow-link . mouse-face) (mouse-3 . org-find-file-at-mouse) (mouse-2 . org-open-at-mouse)) mouse-face highlight face (org-tag org-level-1) org-todo-head #("TODO" 0 4 (face org-todo)) org-category "inbox" fontified t))) fontified t org-todo-head #("TODO" 0 4 (face org-todo)) composition (0 7 [128119]) prettify-symbols-start 294 prettify-symbols-end 301 face org-todo org-category "inbox" org-heading t)

As you see, the 'composition' property appears only once in Step 6,
but 5 times in Step 8.  This is probably part of the problem, or at
least its symptom.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 20 Jun 2021 11:26:01 GMT) Full text and rfc822 format available.

Message #38 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 20 Jun 2021 19:24:55 +0800
Eli Zaretskii <eliz <at> gnu.org> writes:

>> Composition works fine here despite spanning over multiple intervals.
>> Again, I feel that I miss something in your explanation.
>
> We are mis-communicating.  You already stated correctly the reason for
> the problem:

Thanks for the clarification.

> I don't know why the values of the 'composition' property become not
> 'eq'; perhaps some code does a copy-sequence or something?  Presumably
> if you take a good look at what happens in org-agenda-redo-all and its
> subroutines, you will find that code.  In general, when you are
> dealing with static compositions, I'd suggest not to break the
> composed region into several separate intervals with different
> property lists: it's dangerous, as you see.

I debugged the agenda code and found the place where the breakage
happened:

(concat
		   (substring x 0 (match-end 1))
                   (unless (string-empty-p org-agenda-todo-keyword-format)
		     (format org-agenda-todo-keyword-format
			     (match-string 2 x)))
                   ;; Remove `display' property as the icon could leak
		   ;; on the white space.
		   (org-add-props " " (org-plist-delete (text-properties-at 0 x)
			 				'display))
                   (substring x (match-end 3)))

Magically, replacing concat with format appears to fix the issue.

However, I still have no clue what is going on there. I tried to
investigate further and examined the text properties of the composed
ONGOING keyword in inbox.org buffer. It turned out that the composition
is broken in the inbox.org itself:

#("* ONGOING" 0 1 (...) 2 5 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10 face org-todo) 5 9 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10 face org-todo))

Yet, the ONGOING string is correctly replaced by 👷 symbol in inbox.org.
The replacement does not work only in agenda, *and only when concat is
used to construct the keyword*. I am completely confused about what is
going on.

Do you have any ideas about this?

Best,
Ihor






Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 20 Jun 2021 11:54:01 GMT) Full text and rfc822 format available.

Message #41 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 20 Jun 2021 14:53:57 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Sun, 20 Jun 2021 19:24:55 +0800
> 
> However, I still have no clue what is going on there. I tried to
> investigate further and examined the text properties of the composed
> ONGOING keyword in inbox.org buffer. It turned out that the composition
> is broken in the inbox.org itself:
> 
> #("* ONGOING" 0 1 (...) 2 5 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10 face org-todo) 5 9 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10 face org-todo))
> 
> Yet, the ONGOING string is correctly replaced by 👷 symbol in inbox.org.
> The replacement does not work only in agenda, *and only when concat is
> used to construct the keyword*. I am completely confused about what is
> going on.
> 
> Do you have any ideas about this?

Not sure I understand: ideas about what?  The basic problem is that
display of static compositions requires the values of the composition
property to be 'eq'.  Due to implementation details, when you break a
string with that property into 2 or more parts, the property value(s)
could get copied, in which case they will no longer be 'eq'.  What
else is needed to understand this problem?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 20 Jun 2021 13:17:01 GMT) Full text and rfc822 format available.

Message #44 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Sun, 20 Jun 2021 21:16:46 +0800
Eli Zaretskii <eliz <at> gnu.org> writes:

> Not sure I understand: ideas about what?  The basic problem is that
> display of static compositions requires the values of the composition
> property to be 'eq'.  Due to implementation details, when you break a
> string with that property into 2 or more parts, the property value(s)
> could get copied, in which case they will no longer be 'eq'.  What
> else is needed to understand this problem?

Sorry, I was not very clear describing what I observe.

I found two strange things:

1. After manually changing the todo state from TODO to ONGOING in
   inbox.org, the composition property appears to be broken. Yet, the
   ONGOING is replaced by 👷:

   #("* ONGOING" 0 1 (...) 2 5 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10
   face org-todo) 5 9 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10 face
   org-todo))

   I know no way to know if the property intervals are split because
   composition properties are not eq there is some other properties are
   not eq.

   Now, after writing this, I start to believe that composition is still
   eq in this kind of situation, because, as you have explained, the
   composition would not render otherwise.

2. The following code in org-agenda-highlight-todo unexpectedly breaks
   the composition into two intervals with composition values becoming
   not eq:

   (concat
		   (substring x 0 (match-end 1))
		   (unless (string-empty-p org-agenda-todo-keyword-format)
		     (format org-agenda-todo-keyword-format
			     (match-string 2 x)))
		   ;; Remove `display' property as the icon could leak
		   ;; on the white space.
		   (org-add-props " " (org-plist-delete (text-properties-at 0 x)
							'display))
		   (substring x (match-end 3)))

   During debugging, the composition property was always within a single
   interval except after running concat. Only the return value of concat
   had the composition split into to intervals.
   
   After replacing (concat ...) with (format "%s%s%s" ...), the
   composition was kept in a single interval and I found no issue with
   rendering.

   So, it appears to me that concat somehow messed up the composition
   proprety. May it be the case?

   I found a suspicious code in C concat function (fns.c:735):

   	  /* If successive arguments have properties, be sure that the
	     value of `composition' property be the copy.  */
	  if (last_to_end == textprops[argnum].to)
	    make_composition_value_copy (props);

    I can barely understand what is going on in the C code of concat,
    but if it copies the composition property in some cases, we might
    get the issue at hand.

Best,
Ihor




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Mon, 21 Jun 2021 12:19:01 GMT) Full text and rfc822 format available.

Message #47 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Mon, 21 Jun 2021 15:18:07 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Sun, 20 Jun 2021 21:16:46 +0800
> 
> 1. After manually changing the todo state from TODO to ONGOING in
>    inbox.org, the composition property appears to be broken. Yet, the
>    ONGOING is replaced by 👷:
> 
>    #("* ONGOING" 0 1 (...) 2 5 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10
>    face org-todo) 5 9 (... composition (0 7 [128119]) prettify-symbols-start 3 prettify-symbols-end 10 face
>    org-todo))
> 
>    I know no way to know if the property intervals are split because
>    composition properties are not eq there is some other properties are
>    not eq.
> 
>    Now, after writing this, I start to believe that composition is still
>    eq in this kind of situation, because, as you have explained, the
>    composition would not render otherwise.

Yes, it must be eq.

What I don't understand is why the property is broken into two
intervals.  You have only one word, ONGOING, so why is the property
divided into 2?

> 2. The following code in org-agenda-highlight-todo unexpectedly breaks
>    the composition into two intervals with composition values becoming
>    not eq:

Why is this code needed?  And why not put the property on the word
after concatenating, to avoid the issue?

>    So, it appears to me that concat somehow messed up the composition
>    proprety. May it be the case?
> 
>    I found a suspicious code in C concat function (fns.c:735):
> 
>    	  /* If successive arguments have properties, be sure that the
> 	     value of `composition' property be the copy.  */
> 	  if (last_to_end == textprops[argnum].to)
> 	    make_composition_value_copy (props);
> 
>     I can barely understand what is going on in the C code of concat,
>     but if it copies the composition property in some cases, we might
>     get the issue at hand.

It looks like some other use cases want to keep the compositions
separate when a string is generated by 'concat'.

I don't want to make low-level changes in how static compositions are
treated, so I'd prefer that this problem be fixed on the application
level.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Thu, 24 Jun 2021 14:36:01 GMT) Full text and rfc822 format available.

Message #50 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Thu, 24 Jun 2021 22:35:45 +0800
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:

> What I don't understand is why the property is broken into two
> intervals.  You have only one word, ONGOING, so why is the property
> divided into 2?

As I understand, the composition in the two intervals remains the same.
However, some other text properties differ, so the composition property
is "spread" across the two intervals.

>> 2. The following code in org-agenda-highlight-todo unexpectedly breaks
>>    the composition into two intervals with composition values becoming
>>    not eq:
>
> Why is this code needed?  And why not put the property on the word
> after concatenating, to avoid the issue?

The code formats todo keyword in agenda line according to
org-agenda-todo-keyword-format. The function does not know if the passed
string has composition property or not.

In any case, changing concat to format fixed the observed problem.

> It looks like some other use cases want to keep the compositions
> separate when a string is generated by 'concat'.
>
> I don't want to make low-level changes in how static compositions are
> treated, so I'd prefer that this problem be fixed on the application
> level.

I understand. However, the problem is quite unexpected. I do not see how
application can anticipate it. Probably, adding a note to docstring
would be useful? Something like in the attached patch.

Best,
Ihor

[0001-src-fns.c-Fconcat-Update-docstring.patch (text/x-diff, inline)]
From 4cf994cdb3cf9ca07ddfa53bc85d7daa07e7c9cc Mon Sep 17 00:00:00 2001
Message-Id: <4cf994cdb3cf9ca07ddfa53bc85d7daa07e7c9cc.1624545283.git.yantar92 <at> gmail.com>
From: Ihor Radchenko <yantar92 <at> gmail.com>
Date: Thu, 24 Jun 2021 22:33:08 +0800
Subject: [PATCH] src/fns.c (Fconcat): Update docstring

* src/fns.c (Fconcat): Note that composition values may not remain eq
in return value of concat.  See bug#48740.
---
 src/fns.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/src/fns.c b/src/fns.c
index a178216622..73669287c9 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -672,6 +672,9 @@ DEFUN ("concat", Fconcat, Sconcat, 0, MANY, 0,
        doc: /* Concatenate all the arguments and make the result a string.
 The result is a string whose elements are the elements of all the arguments.
 Each argument may be a string or a list or vector of characters (integers).
+
+Values of `composition' property of the result are not guaranteed to
+be `eq'.
 usage: (concat &rest SEQUENCES)  */)
   (ptrdiff_t nargs, Lisp_Object *args)
 {
-- 
2.31.1


Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Thu, 24 Jun 2021 16:07:02 GMT) Full text and rfc822 format available.

Message #53 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ihor Radchenko <yantar92 <at> gmail.com>
Cc: 48740 <at> debbugs.gnu.org
Subject: Re: bug#48740: 28.0.50; Composition text property is not always
 honoured
Date: Thu, 24 Jun 2021 19:06:41 +0300
> From: Ihor Radchenko <yantar92 <at> gmail.com>
> CC: 48740 <at> debbugs.gnu.org
> Date: Thu, 24 Jun 2021 22:35:45 +0800
> 
> 
> [1:text/plain Hide]
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > What I don't understand is why the property is broken into two
> > intervals.  You have only one word, ONGOING, so why is the property
> > divided into 2?
> 
> As I understand, the composition in the two intervals remains the same.
> However, some other text properties differ, so the composition property
> is "spread" across the two intervals.

OK, so we understand that.  But when Emacs splits the interval because
other properties differ, it takes care of copying the same property
value to both intervals, so the values are 'eq'.

> >> 2. The following code in org-agenda-highlight-todo unexpectedly breaks
> >>    the composition into two intervals with composition values becoming
> >>    not eq:
> >
> > Why is this code needed?  And why not put the property on the word
> > after concatenating, to avoid the issue?
> 
> The code formats todo keyword in agenda line according to
> org-agenda-todo-keyword-format. The function does not know if the passed
> string has composition property or not.

But the caller does.

> In any case, changing concat to format fixed the observed problem.

Yes, because 'format' attempts to preserve text properties.

> I understand. However, the problem is quite unexpected. I do not see how
> application can anticipate it. Probably, adding a note to docstring
> would be useful? Something like in the attached patch.

Fine with me, thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 24 Oct 2021 07:44:01 GMT) Full text and rfc822 format available.

Message #56 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org, Ihor Radchenko <yantar92 <at> gmail.com>
Subject: Re: bug#48740: 28.0.50;
 Composition text property is not always honoured
Date: Sun, 24 Oct 2021 00:42:56 -0700
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:

>> I understand. However, the problem is quite unexpected. I do not see how
>> application can anticipate it. Probably, adding a note to docstring
>> would be useful? Something like in the attached patch.
>
> Fine with me, thanks.

The patch here was never installed.  Should it be?

I've attached a version below with some minor touch-ups.
[0001-Doc-fix-for-concat.patch (text/x-diff, attachment)]

Added tag(s) patch. Request was from Stefan Kangas <stefan <at> marxist.se> to control <at> debbugs.gnu.org. (Sun, 24 Oct 2021 07:44:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 24 Oct 2021 12:05:02 GMT) Full text and rfc822 format available.

Message #61 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 48740 <at> debbugs.gnu.org, yantar92 <at> gmail.com
Subject: Re: bug#48740: 28.0.50;
 Composition text property is not always honoured
Date: Sun, 24 Oct 2021 15:04:23 +0300
> From: Stefan Kangas <stefan <at> marxist.se>
> Date: Sun, 24 Oct 2021 00:42:56 -0700
> Cc: Ihor Radchenko <yantar92 <at> gmail.com>, 48740 <at> debbugs.gnu.org
> 
> > Fine with me, thanks.
> 
> The patch here was never installed.  Should it be?
> 
> I've attached a version below with some minor touch-ups.

It's okay to install it, thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#48740; Package emacs. (Sun, 24 Oct 2021 12:28:02 GMT) Full text and rfc822 format available.

Message #64 received at 48740 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48740 <at> debbugs.gnu.org, yantar92 <at> gmail.com
Subject: Re: bug#48740: 28.0.50;
 Composition text property is not always honoured
Date: Sun, 24 Oct 2021 05:27:32 -0700
close 48740 28.1
thanks

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Stefan Kangas <stefan <at> marxist.se>
>> Date: Sun, 24 Oct 2021 00:42:56 -0700
>> Cc: Ihor Radchenko <yantar92 <at> gmail.com>, 48740 <at> debbugs.gnu.org
>>
>> > Fine with me, thanks.
>>
>> The patch here was never installed.  Should it be?
>>
>> I've attached a version below with some minor touch-ups.
>
> It's okay to install it, thanks.

Thanks, installed on emacs-28 (commit 817c929eda).

I'm consequently closing this bug report.




bug marked as fixed in version 28.1, send any further explanations to 48740 <at> debbugs.gnu.org and Ihor Radchenko <yantar92 <at> gmail.com> Request was from Stefan Kangas <stefan <at> marxist.se> to control <at> debbugs.gnu.org. (Sun, 24 Oct 2021 12:28:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 22 Nov 2021 12:24:07 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 155 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.