GNU bug report logs - #35536
27.0.50; Expose buffer's marker list to Elisp

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35536 in the body.
You can then email your comments to 35536 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.50; Expose buffer's marker list to Elisp
Date: Thu, 02 May 2019 16:44:52 +0100

[Message part 1 (text/plain, inline)]

Severity: wishlist
Tags: patch

[0001-Add-function-marker-list.patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

The question of listing a buffer's markers has been raised before:

https://debbugs.gnu.org/18#47
https://lists.gnu.org/archive/html/help-gnu-emacs/2016-06/msg00050.html
https://lists.gnu.org/archive/html/emacs-devel/2007-04/msg01391.html

I attach a patch implementing this based on BUF_MARKERS, as per Martin's
suggestion.  Any reasons not to expose such a function?

Thanks,

-- 
Basil

In GNU Emacs 27.0.50 (build 4, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-04-30 built on thunk
Repository revision: 910d170771ac74ab76d6dcb2dda3f3167e01b705
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux buster/sid

Message #8 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com, monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Thu, 02 May 2019 19:07:47 +0300

> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
> Date: Thu, 02 May 2019 16:44:52 +0100
> Cc: Mauro Aranda <maurooaranda <at> gmail.com>,
> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
> 
> I attach a patch implementing this based on BUF_MARKERS, as per Martin's
> suggestion.  Any reasons not to expose such a function?

I'm not yet convinced we need something like that, but in any case, is
the order important?  Because the code you propose produces a list in
reverse order.

More generally, I think we should discuss the need for this in more
detail.  Markers are used for several features, and there's internal
stuff like conversion from character to byte positions that depends on
them.  Changing markers could thus easily crash Emacs, especially if
it comes in some in-opportune moment.

It is possible that people actually need higher-level primitives that
manipulate markers internally.  We should first identify the use cases
where this could be needed, and then see how to help solving those use
cases by something like a new marker-related primitive.

Message #11 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com, monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Thu, 02 May 2019 17:51:12 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
>> Date: Thu, 02 May 2019 16:44:52 +0100
>> Cc: Mauro Aranda <maurooaranda <at> gmail.com>,
>> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
>> 
>> I attach a patch implementing this based on BUF_MARKERS, as per Martin's
>> suggestion.  Any reasons not to expose such a function?
>
> I'm not yet convinced we need something like that, but in any case, is
> the order important?  Because the code you propose produces a list in
> reverse order.

The order of the returned list is in increasing buffer position, thanks
to the call to Fsort.  Is that not a reasonable order?

> More generally, I think we should discuss the need for this in more
> detail.  Markers are used for several features, and there's internal
> stuff like conversion from character to byte positions that depends on
> them.  Changing markers could thus easily crash Emacs, especially if
> it comes in some in-opportune moment.

Are you saying that BUF_MARKERS could include markers created by
internal functions which could crash if these markers are changed across
calls to other Lisp functions?

If so, that sounds like a valid concern to a non-expert like me, but it
also sounds like a bug waiting to happen, given that other C code
also traverses and manipulates BUF_MARKERS.

If not, I don't see how manipulating markers returned by marker-list is
any worse than manipulating those created at the Lisp level, with the
usual and documented risks associated with manipulating markers not
owned by the caller.

> It is possible that people actually need higher-level primitives that
> manipulate markers internally.  We should first identify the use cases
> where this could be needed, and then see how to help solving those use
> cases by something like a new marker-related primitive.

I have yet to see a use-case for marker-list which can't be engineered
in a different way (other than as a replacement for the obsolete
buffer-has-markers-at, FWIW).  Perhaps some of the CCed parties might
have examples.

Thanks,

-- 
Basil

Message #14 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com, monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Thu, 02 May 2019 20:41:57 +0300

> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
> Cc: <35536 <at> debbugs.gnu.org>,  <maurooaranda <at> gmail.com>,  <monnier <at> iro.umontreal.ca>
> Date: Thu, 02 May 2019 17:51:12 +0100
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
> >> Date: Thu, 02 May 2019 16:44:52 +0100
> >> Cc: Mauro Aranda <maurooaranda <at> gmail.com>,
> >> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
> >> 
> >> I attach a patch implementing this based on BUF_MARKERS, as per Martin's
> >> suggestion.  Any reasons not to expose such a function?
> >
> > I'm not yet convinced we need something like that, but in any case, is
> > the order important?  Because the code you propose produces a list in
> > reverse order.
> 
> The order of the returned list is in increasing buffer position, thanks
> to the call to Fsort.  Is that not a reasonable order?

Sorry, missed the sort.  The question whether the order matters still
stands, though.

> > More generally, I think we should discuss the need for this in more
> > detail.  Markers are used for several features, and there's internal
> > stuff like conversion from character to byte positions that depends on
> > them.  Changing markers could thus easily crash Emacs, especially if
> > it comes in some in-opportune moment.
> 
> Are you saying that BUF_MARKERS could include markers created by
> internal functions which could crash if these markers are changed across
> calls to other Lisp functions?

Please don't forget that nowadays we call Lisp from many places in C,
like from redisplay.  We need to be very careful with this because I;m
quite sure the display code doesn't expect markers to change at least
in some of its paths.

> If so, that sounds like a valid concern to a non-expert like me, but it
> also sounds like a bug waiting to happen, given that other C code
> also traverses and manipulates BUF_MARKERS.

Emacs being designed using the MVC pattern, assumes that the buffers
(and thus markers) don't change while they are being displayed.  It
has some probes for when this might happen as result of calling some
hook, and when that is detected, we restart redisplay.  I'm saying
that enlarging the potential for such changes will need careful
auditing of code that didn't expect such changes until now.  It will
also necessarily slow down redisplay.  The question is: is that worth
the hassle?  If what is needed is some higher-level features, then
exposing markers to Lisp will unnecessarily force us to do all that
non-trivial auditing.  So I suggest that we discuss the needs before
coding, to see whether such low-level access to a central data
structure is really needed and justified.

> If not, I don't see how manipulating markers returned by marker-list is
> any worse than manipulating those created at the Lisp level, with the
> usual and documented risks associated with manipulating markers not
> owned by the caller.

Just reading the markers probably won't but do you really believe this
is the last word?  How many hours will it take for someone to ask for
a primitive to set the C-level markers as well, or request the ability
to map a function over all the markers?  If it's really needed, sure,
let's do it.  But is it?  Or are we doing that just because we can?

> I have yet to see a use-case for marker-list which can't be engineered
> in a different way (other than as a replacement for the obsolete
> buffer-has-markers-at, FWIW).

Well, the discussions you cited did express requirements whose
implementation with the existing facilities was either inconvenient or
restricted.  If these problems are still relevant, then why not try
providing some primitives to help them?

IOW, let me turn the table and ask: why would a Lisp program want to
get a list of all the markers in a buffer, especially those not
created from Lisp?

Message #17 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, Mauro Aranda <maurooaranda <at> gmail.com>,
 martin rudalics <rudalics <at> gmx.at>
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Thu, 02 May 2019 15:59:38 -0400

> I attach a patch implementing this based on BUF_MARKERS, as per Martin's
> suggestion.  Any reasons not to expose such a function?

AFAIK the main reason for such a function is so that you can implement
"replace" functions which preserves markers better than "insert+delete"
does, right?  Arguably `replace-buffer-contents` reduced this need, but
this is just one way to "guess" how to preserve the markers and for
specific replacements there are surely other approaches which would
work better, hence the desire to get access to the marker-list to write
ad-hoc solutions.

Random thoughts:
- I wouldn't expose a `(marker-list)` function but rather `(markers-in
  BEG END)` so you're not bothered by unrelated markers outside of
  the region of interest.
- The main problem I see is that some of the markers in BUF_MARKERS are
  "proper" markers, while others are just the markers that we happen to
  use in the current internal representation of overlays.
  If you can get your hands on those markers, you might end up breaking
  some invariants on which the C code relies (e.g. place the
  overlay-start after the overlay-end, or in a different buffer).
- I think the serious risks (e.g. crashes) are solvable.  E.g. there's
  room for an additional boolean field `lisp_marker` which could be used
  to distinguish those markers which can be safely returned (because
  they're normal Lisp-level markers already accessible from Lisp anyway)
  from the internal ones (such as those from overlays).
- Then we'd probably want to discussion whether markers used within
  `save-excursion` and friends should be marked as `lisp_marker` or not.

This said, as you say later:
> I have yet to see a use-case for marker-list which can't be engineered
> in a different way

So, whether it's worth the trouble: I don't know.


        Stefan

Message #20 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: contovob <at> tcd.ie, 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Thu, 02 May 2019 23:05:11 +0300

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Date: Thu, 02 May 2019 15:59:38 -0400
> Cc: 35536 <at> debbugs.gnu.org, Mauro Aranda <maurooaranda <at> gmail.com>
> 
> AFAIK the main reason for such a function is so that you can implement
> "replace" functions which preserves markers better than "insert+delete"
> does, right?

If this is the use case, perhaps we could provide a function to save
markers and then restore them, similar to current-window-configuration
and set-window-configuration, but with opaque handles returned to the
caller.

Message #23 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com, monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 03 May 2019 16:50:17 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
>> Cc: <35536 <at> debbugs.gnu.org>,  <maurooaranda <at> gmail.com>,  <monnier <at> iro.umontreal.ca>
>> Date: Thu, 02 May 2019 17:51:12 +0100
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> >> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
>> >> Date: Thu, 02 May 2019 16:44:52 +0100
>> >> Cc: Mauro Aranda <maurooaranda <at> gmail.com>,
>> >> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
>> >> 
>> >> I attach a patch implementing this based on BUF_MARKERS, as per Martin's
>> >> suggestion.  Any reasons not to expose such a function?
>> >
>> > I'm not yet convinced we need something like that, but in any case, is
>> > the order important?  Because the code you propose produces a list in
>> > reverse order.
>> 
>> The order of the returned list is in increasing buffer position, thanks
>> to the call to Fsort.  Is that not a reasonable order?
>
> Sorry, missed the sort.  The question whether the order matters still
> stands, though.

When asked for a list of markers between BEG and END, it makes sense to
me to return a list which ascends from BEG to END.  If it really
matters, we could either return the order of BUF_MARKERS unchanged, or
accept an additional argument which tells the function how to sort.

>> > More generally, I think we should discuss the need for this in more
>> > detail.  Markers are used for several features, and there's internal
>> > stuff like conversion from character to byte positions that depends on
>> > them.  Changing markers could thus easily crash Emacs, especially if
>> > it comes in some in-opportune moment.
>> 
>> Are you saying that BUF_MARKERS could include markers created by
>> internal functions which could crash if these markers are changed across
>> calls to other Lisp functions?
>
> Please don't forget that nowadays we call Lisp from many places in C,
> like from redisplay.  We need to be very careful with this because I;m
> quite sure the display code doesn't expect markers to change at least
> in some of its paths.

Noted.

>> If so, that sounds like a valid concern to a non-expert like me, but it
>> also sounds like a bug waiting to happen, given that other C code
>> also traverses and manipulates BUF_MARKERS.
>
> Emacs being designed using the MVC pattern, assumes that the buffers
> (and thus markers) don't change while they are being displayed.  It
> has some probes for when this might happen as result of calling some
> hook, and when that is detected, we restart redisplay.  I'm saying
> that enlarging the potential for such changes will need careful
> auditing of code that didn't expect such changes until now.  It will
> also necessarily slow down redisplay.  The question is: is that worth
> the hassle?  If what is needed is some higher-level features, then
> exposing markers to Lisp will unnecessarily force us to do all that
> non-trivial auditing.  So I suggest that we discuss the needs before
> coding, to see whether such low-level access to a central data
> structure is really needed and justified.

Thanks for explaining, sounds perfectly reasonable.  I'm not convinced
adding marker-list is worth the trouble.  (FWIW, I'm not eager to
blindly expose marker-list; I just opened a ticket to see what
reservations there are, and to discuss this question I've seen pop up
before.  A bit of code always gives discussions a bit more ground, and
it was an excuse for me to read a bit of the surrounding code.)

>> If not, I don't see how manipulating markers returned by marker-list is
>> any worse than manipulating those created at the Lisp level, with the
>> usual and documented risks associated with manipulating markers not
>> owned by the caller.
>
> Just reading the markers probably won't but do you really believe this
> is the last word?  How many hours will it take for someone to ask for
> a primitive to set the C-level markers as well, or request the ability
> to map a function over all the markers?  If it's really needed, sure,
> let's do it.  But is it?  Or are we doing that just because we can?

So far, the latter.

>> I have yet to see a use-case for marker-list which can't be engineered
>> in a different way (other than as a replacement for the obsolete
>> buffer-has-markers-at, FWIW).
>
> Well, the discussions you cited did express requirements whose
> implementation with the existing facilities was either inconvenient or
> restricted.  If these problems are still relevant, then why not try
> providing some primitives to help them?

A save+restore primitive like the one you suggested in your other
message sounds like it might do the trick without having to expose a
buffer's marker list to Lisp.

> IOW, let me turn the table and ask: why would a Lisp program want to
> get a list of all the markers in a buffer, especially those not
> created from Lisp?

As I say above, I don't have any use-cases which specifically need to
expose a buffer's marker list to Lisp, as opposed to using some other
approach.  The main call for marker-list in bug#18 could probably be
better solved with a different primitive.

Thanks,

-- 
Basil

Message #26 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 35536 <at> debbugs.gnu.org, Mauro Aranda <maurooaranda <at> gmail.com>,
 martin rudalics <rudalics <at> gmx.at>
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 03 May 2019 16:50:56 +0100

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> I attach a patch implementing this based on BUF_MARKERS, as per Martin's
>> suggestion.  Any reasons not to expose such a function?
>
> AFAIK the main reason for such a function is so that you can implement
> "replace" functions which preserves markers better than "insert+delete"
> does, right?

AIUI, yes.

> Random thoughts:
> - I wouldn't expose a `(marker-list)` function but rather `(markers-in
>   BEG END)` so you're not bothered by unrelated markers outside of
>   the region of interest.

The patch in the OP accepts optional BEG and END arguments for the
caller's typing convenience.

> - The main problem I see is that some of the markers in BUF_MARKERS are
>   "proper" markers, while others are just the markers that we happen to
>   use in the current internal representation of overlays.
>   If you can get your hands on those markers, you might end up breaking
>   some invariants on which the C code relies (e.g. place the
>   overlay-start after the overlay-end, or in a different buffer).
> - I think the serious risks (e.g. crashes) are solvable.  E.g. there's
>   room for an additional boolean field `lisp_marker` which could be used
>   to distinguish those markers which can be safely returned (because
>   they're normal Lisp-level markers already accessible from Lisp anyway)
>   from the internal ones (such as those from overlays).
> - Then we'd probably want to discussion whether markers used within
>   `save-excursion` and friends should be marked as `lisp_marker` or not.
>
> This said, as you say later:
>> I have yet to see a use-case for marker-list which can't be engineered
>> in a different way
>
> So, whether it's worth the trouble: I don't know.

Given a sufficiently sufficient save+restore primitive as per Eli's
suggestion, it's not looking worth the trouble.

Thanks,

-- 
Basil

Message #29 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com, monnier <at> iro.umontreal.ca
Subject: RE: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 3 May 2019 09:38:02 -0700 (PDT)

> When asked for a list of markers between BEG and END, it makes sense to
> me to return a list which ascends from BEG to END.
                            ^^^^^^^^^^^^^^^^^^^^^^^
IOW, in buffer-position order.

> If it really matters, we could either return the
> order of BUF_MARKERS unchanged,

Unchanged from what?

> or accept an additional argument which tells the
> function how to sort.

Have not really been following this thread, and
not weighing in on whether such a function is
needed or whether users need access to markers
created by C.

But as for the order of such a list: It's trivial
for users (any Lisp code) to sort by buffer position
or anything else, so why would the default order
be by buffer position?

What's _not_ available to users or Lisp code, I
think, is the order of marker creation or even the
order of last setting.  I'd think that
marker-creation order (either direction) would be
a better default sort order for this, no?

Message #32 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, Mauro Aranda <maurooaranda <at> gmail.com>,
 martin rudalics <rudalics <at> gmx.at>
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 03 May 2019 13:09:15 -0400

> Given a sufficiently sufficient save+restore primitive as per Eli's
> suggestion, it's not looking worth the trouble.

FWIW, I think exposing some new opaque "set of markers" plus primitives
to manipulate such objects is likely a lot more work, so I think from
a purely pragmatic point of view, `markers-in` might be a better option
*assuming* we can make it safe-enough (i.e. it can cause weird behaviors
if mis-used but no hard-crashes, and not if used "reasonably", e.g. if
it's not used to change the markers's buffers nor their relative ordering).

In Emacs, we're generally pretty happy to provide tools powerful enough
to shoot yourself in the foot, as long as such mishaps only happen when
you're clearly using the tool in a dangerous way.


        Stefan

Message #35 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 35536 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, maurooaranda <at> gmail.com,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 03 May 2019 18:22:39 +0100

Drew Adams <drew.adams <at> oracle.com> writes:

>> When asked for a list of markers between BEG and END, it makes sense to
>> me to return a list which ascends from BEG to END.
>                             ^^^^^^^^^^^^^^^^^^^^^^^
> IOW, in buffer-position order.

Yes.

>> If it really matters, we could either return the
>> order of BUF_MARKERS unchanged,
>
> Unchanged from what?

From the order returned by BUF_MARKERS, i.e. the internal chain of
markers pointing to the current buffer.  This order presumably reflects,
to an extent, the order in which markers were created/chained, but I'm
not sure about this.

>> or accept an additional argument which tells the
>> function how to sort.
>
> Have not really been following this thread, and
> not weighing in on whether such a function is
> needed or whether users need access to markers
> created by C.
>
> But as for the order of such a list: It's trivial
> for users (any Lisp code) to sort by buffer position
> or anything else, so why would the default order
> be by buffer position?

That is the order I would intuitively expect in any enumeration of a
partially ordered set of buffer artifacts in a given region, unless
otherwise stated.

What other order would make sense when talking about markers within a
given region?

> What's _not_ available to users or Lisp code, I
> think, is the order of marker creation or even the
> order of last setting.  I'd think that
> marker-creation order (either direction) would be
> a better default sort order for this, no?

Perhaps when enumerating markers pointing at a single position, yes.
But I think that ordering would make less sense when talking about
markers within a given region.  Assuming something like marker-list is
deemed a useful addition (which is not yet clear), perhaps there should
be two separate functions akin to overlays-in and overlays-at, with
different sorting options and/or default policies.

Thanks,

-- 
Basil

Message #38 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, maurooaranda <at> gmail.com,
 monnier <at> iro.umontreal.ca
Subject: RE: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 3 May 2019 10:31:17 -0700 (PDT)

I don't want to belabor this, but I'll reply once
about this.

> > But as for the order of such a list: It's trivial
> > for users (any Lisp code) to sort by buffer position
> > or anything else, so why would the default order
> > be by buffer position?
> 
> That is the order I would intuitively expect in any enumeration of a
> partially ordered set of buffer artifacts in a given region, unless
> otherwise stated.
> 
> What other order would make sense when talking about markers within a
> given region?

I think the order that makes sense as the (only?) way
to get the set of markers (including those from C) is
an order that we cannot get otherwise.  Creation order
is one such otherwise-unavailable order.

Anyone can get the buffer-position order at any time.
Or it can be provided as a function, if that's deemed
important.

> > What's _not_ available to users or Lisp code, I
> > think, is the order of marker creation or even the
> > order of last setting.  I'd think that
> > marker-creation order (either direction) would be
> > a better default sort order for this, no?
> 
> Perhaps when enumerating markers pointing at a single position, yes.
> But I think that ordering would make less sense when talking about
> markers within a given region.  Assuming something like marker-list is
> deemed a useful addition (which is not yet clear), perhaps there should
> be two separate functions akin to overlays-in and overlays-at, with
> different sorting options and/or default policies.

It's not about what's most immediately useful for
most imagined uses of the markers in a given region.

It's about _somehow_ getting a set of markers
(wherever, whether within some buffer-position range
or not) in their order of creation (or modification).

Such relative time information is otherwise lost
completely.  The markers themselves contain position
information.  They do not contain time information.

Again, though, I'm not saying anything about whether
we need such a function at all.  I'm just suggesting
that if we provide it it should probably provide info
that is not otherwise obtainable.  Creation time is
one such bit of info.

Message #41 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, maurooaranda <at> gmail.com,
 Drew Adams <drew.adams <at> oracle.com>
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 03 May 2019 13:39:15 -0400

>> What's _not_ available to users or Lisp code, I
>> think, is the order of marker creation or even the
>> order of last setting.

There's no reason for the C code to keep track of those things either.
So while the current implementation may happen to have the markers in
"creation order", it would be wrong to provide a primitive that promises
this ordering unless there's a *good* reason why we'd want to force
future Emacsen to also keep track of this info.  So far I don't see any
such reason.


        Stefan

Message #44 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>, "Basil L. Contovounesios"
 <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, maurooaranda <at> gmail.com
Subject: RE: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 3 May 2019 10:53:36 -0700 (PDT)

> >> What's _not_ available to users or Lisp code, I
> >> think, is the order of marker creation or even the
> >> order of last setting.
> 
> There's no reason for the C code to keep track of those things either.
> So while the current implementation may happen to have the markers in
> "creation order", it would be wrong to provide a primitive that promises
> this ordering unless there's a *good* reason why we'd want to force
> future Emacsen to also keep track of this info.  So far I don't see any
> such reason.

Fair enough.  I was thinking it might be a freebie.

E.g., I was thinking that the list to be returned
would just be created by, say, pushing new markers
onto it.  If that were the case then that order
could be useful, whether or not one could _depend_
on it.

Whether or not one should be able to depend on such
an order is a separate question.

If guaranteeing to provide such a default order is
too restrictive (and I agree that it probably would
be), but if the current implementation did mostly
return the set in such an order, then I'd vote for
not reordering it and just saying nothing about the
return order.

Consider `buffer-list', for example.  The order is
generally useful but we say nothing much about it,
directly.  (But we do show, in (elisp) `Buffer List',
how you can reorder the underlying buffer list.)

We also point out there of saying that `buffer-list'
"is not an internal Emacs data structure, and
modifying it has no effect on the order of buffers."

Message #47 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: "Basil L. Contovounesios" <contovob <at> tcd.ie>, 35536 <at> debbugs.gnu.org,
 Eli Zaretskii <eliz <at> gnu.org>, maurooaranda <at> gmail.com
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 03 May 2019 14:13:41 -0400

> Whether or not one should be able to depend on such
> an order is a separate question.

I'm fine with a `markers-in` which returns markers in the order found in
the C code.  But the docstring should state clearly that you can't rely
on this ordering.

I expect many uses won't care about the ordering anyway, so we don't
need to bother `sort`ing, although the performance cost of `sort`ing will
likely always be negligible.


        Stefan

Message #50 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: "Basil L. Contovounesios" <contovob <at> tcd.ie>, 35536 <at> debbugs.gnu.org,
 Eli Zaretskii <eliz <at> gnu.org>, maurooaranda <at> gmail.com
Subject: RE: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 3 May 2019 13:05:01 -0700 (PDT)

> > Whether or not one should be able to depend on such
> > an order is a separate question.
> 
> I'm fine with a `markers-in` which returns markers in the order found in
> the C code.  But the docstring should state clearly that you can't rely
> on this ordering.
> 
> I expect many uses won't care about the ordering anyway, so we don't
> need to bother `sort`ing, although the performance cost of `sort`ing will
> likely always be negligible.

Agreed on both counts.

Message #53 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Mauro Aranda <maurooaranda <at> gmail.com>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Fri, 3 May 2019 20:01:18 -0300

[Message part 1 (text/plain, inline)]

>>> I have yet to see a use-case for marker-list which can't be engineered
>>> in a different way (other than as a replacement for the obsolete
>>> buffer-has-markers-at, FWIW).
>>
>> Well, the discussions you cited did express requirements whose
>> implementation with the existing facilities was either inconvenient or
>> restricted.  If these problems are still relevant, then why not try
>> providing some primitives to help them?
>
> A save+restore primitive like the one you suggested in your other
> message sounds like it might do the trick without having to expose a
> buffer's marker list to Lisp.

Indeed.  I thought Martin was talking about something like this in his post
in bug#18.  Given a region where text is going to be replaced, save the
positions of markers that would be affected because of the delete+insert,
and then restore them.

>> IOW, let me turn the table and ask: why would a Lisp program want to
>> get a list of all the markers in a buffer, especially those not
>> created from Lisp?
>
> As I say above, I don't have any use-cases which specifically need to
> expose a buffer's marker list to Lisp, as opposed to using some other
> approach.  The main call for marker-list in bug#18 could probably be
> better solved with a different primitive.
>

When I said I didn't find anything at the Lisp level to get the markers,
that didn't fully express my thoughts.  I didn't mean it as a call for a
function to get that information (and certainly, I don't see a use for
getting information about markers created internally).  What I meant was
that I thought about that way of restoring markers, but had no way of
working on it (at least not with my current knowledge of C).

Best regards,
Mauro.

[Message part 2 (text/html, inline)]

Message #56 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: martin rudalics <rudalics <at> gmx.at>
To: Mauro Aranda <maurooaranda <at> gmail.com>,
 "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 35536 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Sat, 4 May 2019 19:34:08 +0200

>> A save+restore primitive like the one you suggested in your other
>> message sounds like it might do the trick without having to expose a
>> buffer's marker list to Lisp.
>
> Indeed.  I thought Martin was talking about something like this in his post
> in bug#18.  Given a region where text is going to be replaced, save the
> positions of markers that would be affected because of the delete+insert,
> and then restore them.

More precisely I would try to save the contexts of the old marker
positions similar to what we do with bookmarks and try to find these
contexts in the replaced region.  The important aspect is obviously
that this step can be skipped for regions left alone by compareseq (or
whatever was used) because in those (hopefully large) regions markers
are preserved automatically.  Some care would be needed for markers
that sit precisely at the bounds of these region and have the "wrong"
insertion type.  And maybe we should optionally give the cursor a
different color if it was not possible to restore it unequivocally.

The approach would be IMHO useful when reverting buffers after code
has been wrongly reindented or commented in or out.  I'm afraid that
it might not be overly useful for auto-reverted dired buffers and
buffers with many fine-grained modifications.

Which markers should be restored this way and whether the
corresponding code should be provided as a primitive or in Elisp is a
decision I leave to knowledgeable people.  I'd be already happy if
'point-marker' were handled that way.

martin

Message #59 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: contovob <at> tcd.ie, 35536 <at> debbugs.gnu.org, maurooaranda <at> gmail.com,
 drew.adams <at> oracle.com
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Sat, 04 May 2019 17:25:43 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I'm still opposed to making the internal list of markers accessible at all.
Markers can disappear from that list due to GC.

The only reason for having that list is so we can relocate the
markers.  It is not an interface that naturally should exist.

What exactly do people want to do with this list?
Let's look for another way.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Message #62 received at 35536 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Richard Stallman <rms <at> gnu.org>
Cc: contovob <at> tcd.ie, 35536 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, drew.adams <at> oracle.com,
 maurooaranda <at> gmail.com
Subject: Re: bug#35536: 27.0.50; Expose buffer's marker list to Elisp
Date: Mon, 16 Sep 2019 23:07:34 +0200

Richard Stallman <rms <at> gnu.org> writes:

> I'm still opposed to making the internal list of markers accessible at all.
> Markers can disappear from that list due to GC.
>
> The only reason for having that list is so we can relocate the
> markers.  It is not an interface that naturally should exist.
>
> What exactly do people want to do with this list?
> Let's look for another way.

I think the "rough consensus" here (i.e., Eli and Richard) is that we
don't really want to expose the marker list.  (This was also discussed
in a different context a couple of weeks ago, and the conclusion was the
same there.)

So I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.