GNU bug report logs - #76124
eglot buffer corruption

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76124 in the body.
You can then email your comments to 76124 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Johann Höchtl <johann.hoechtl <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: eglot buffer corruption
Date: Fri, 7 Feb 2025 21:14:54 +0100

[Message part 1 (text/plain, inline)]

When I edit a markdown file using markdown-mode AND enable eglot in that
buffer, at least one text operation corrupts the buffer.

When I am on an markdown list item:

- some text a
- some text b []  <-point

and press C-c <up> (which swaps the current list item with the preceding
one), the buffers content gets messed up: Parts of the preceding text are
not properly swapped but "bleeds into" the swapped line.

Possibly eglot and the interaction with "track-changes" is at fault: When I
eglot-shutdown, I never see this corruption.

This is not a visual problem: When I check the eglot logs changes
transfered to the language server or save and revert the buffer I see that
the buffer contains garbled text.

Using emacs as of commit 5e12843fa32150d2f18ce21fc6f3ae58732df6a7

[Message part 2 (text/html, inline)]

Message #8 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>,
 João Távora <joaotavora <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 08 Feb 2025 09:32:18 +0200

> From: Johann Höchtl <johann.hoechtl <at> gmail.com>
> Date: Fri, 7 Feb 2025 21:14:54 +0100
> 
> When I edit a markdown file using markdown-mode AND enable eglot in that buffer, at least one text
> operation corrupts the buffer.
> 
> When I am on an markdown list item:
> 
> - some text a
> - some text b []  <-point
> 
> and press C-c <up> (which swaps the current list item with the preceding one), the buffers content gets
> messed up: Parts of the preceding text are not properly swapped but "bleeds into" the swapped line.
> 
> Possibly eglot and the interaction with "track-changes" is at fault: When I eglot-shutdown, I never see this
> corruption.

I'm guess the information about the LSP server you are using might be
relevant?

> This is not a visual problem: When I check the eglot logs changes transfered to the language server or save
> and revert the buffer I see that the buffer contains garbled text.

Adding João and Stefan to the discussion.


> Using emacs as of commit 5e12843fa32150d2f18ce21fc6f3ae58732df6a7

Is that from the master branch of the Emacs Git repository?  (Your
report elided all version and configuration information, so it's hard
to tell.)

Thanks.

Message #11 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Johann Höchtl <johann.hoechtl <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: João Távora <joaotavora <at> gmail.com>,
 76124 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 8 Feb 2025 17:47:26 +0100

[Message part 1 (text/plain, inline)]

Am Sa., 8. Feb. 2025 um 08:32 Uhr schrieb Eli Zaretskii <eliz <at> gnu.org>:

> > From: Johann Höchtl <johann.hoechtl <at> gmail.com>
> > Date: Fri, 7 Feb 2025 21:14:54 +0100
> >
> > When I edit a markdown file using markdown-mode AND enable eglot in that
> buffer, at least one text
> > operation corrupts the buffer.
> >
> > When I am on an markdown list item:
> >
> > - some text a
> > - some text b []  <-point
> >
> > and press C-c <up> (which swaps the current list item with the preceding
> one), the buffers content gets
> > messed up: Parts of the preceding text are not properly swapped but
> "bleeds into" the swapped line.
> >
> > Possibly eglot and the interaction with "track-changes" is at fault:
> When I eglot-shutdown, I never see this
> > corruption.
>
> I'm guess the information about the LSP server you are using might be
> relevant?
>
>
I also do not know if this is relevant, but it is
https://github.com/mhersson/mpls


> > This is not a visual problem: When I check the eglot logs changes
> transfered to the language server or save
> > and revert the buffer I see that the buffer contains garbled text.
>
> Adding João and Stefan to the discussion.
>
>
> > Using emacs as of commit 5e12843fa32150d2f18ce21fc6f3ae58732df6a7
>
> Is that from the master branch of the Emacs Git repository?  (Your
> report elided all version and configuration information, so it's hard
> to tell.)
>
>

Does emacs-build-description provide enough details? The output is


In GNU Emacs 31.0.50 (build 1, x86_64-w64-mingw32) of 2025-02-02 built
 on fv-az521-368
Repository revision: 5e12843fa32150d2f18ce21fc6f3ae58732df6a7
Repository branch: HEAD
Windowing system distributor 'Microsoft Corp.', version 10.0.22631
System Description: Microsoft Windows 10 Enterprise (v10.0.2009.22631.4751)

Configured using:
 'configure
 --prefix=/d/a/emacs-build/emacs-build/pkg/5e12843-ucrt-x86_64
 'CFLAGS=-O2 -fno-semantic-interposition -floop-parallelize-all
 -ftree-parallelize-loops=4 -g3 ' --disable-build-details
 --without-dbus --enable-link-time-optimization --enable-build-details
 --with-compress-install --with-small-ja-dic --with-gif --with-gnutls
 --with-harfbuzz --with-jpeg --with-json --with-lcms2 --with-mps
 --with-native-compilation --with-png --with-rsvg --with-tree-sitter
 --with-xml2 --with-xpm --with-zlib --without-cairo --without-tiff'




> Thanks.
>

[Message part 2 (text/html, inline)]

Message #14 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 08 Feb 2025 19:04:24 +0200

> From: Johann Höchtl <johann.hoechtl <at> gmail.com>
> Date: Sat, 8 Feb 2025 17:47:26 +0100
> Cc: João Távora <joaotavora <at> gmail.com>, 
> 	Stefan Monnier <monnier <at> iro.umontreal.ca>, 76124 <at> debbugs.gnu.org
> 
>  I'm guess the information about the LSP server you are using might be
>  relevant?
> 
> I also do not know if this is relevant, but it is https://github.com/mhersson/mpls
>  
>  > This is not a visual problem: When I check the eglot logs changes transfered to the language server
>  or save
>  > and revert the buffer I see that the buffer contains garbled text.
> 
>  Adding João and Stefan to the discussion.
> 
>  > Using emacs as of commit 5e12843fa32150d2f18ce21fc6f3ae58732df6a7
> 
>  Is that from the master branch of the Emacs Git repository?  (Your
>  report elided all version and configuration information, so it's hard
>  to tell.)
> 
> Does emacs-build-description provide enough details?

It does, thanks.

Message #17 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 08 Feb 2025 19:55:03 +0000

Johann Höchtl <johann.hoechtl <at> gmail.com> writes:

On Sat, Feb 8, 2025 at 4:47 PM Johann Höchtl <johann.hoechtl <at> gmail.com> wrote:

>> > Possibly eglot and the interaction with "track-changes" is at
>> > fault: When I eglot-shutdown, I never see this corruption.
>> I'm guess the information about the LSP server you are using might be
>> relevant?
> I also do not know if this is relevant, but it is https://github.com/mhersson/mpls

It's normally extremely relevant, as the corruption related to
track-changes and before/after change functions (which you hint at) is
directly dependent on whether the server accepts partial updates or not.

>> > Using emacs as of commit 5e12843fa32150d2f18ce21fc6f3ae58732df6a7
>> Is that from the master branch of the Emacs Git repository?  (Your
>> report elided all version and configuration information, so it's hard
>> to tell.)

(Indeed, I don't see this commit anywhere.  What branch is this?)

Anyway, as to the problem itself, I've reproduced it.

TL/DR: Something in Eglot is making transpose-regions not work correctly
in file-less buffers.

Reproduction: 

It's kind of a fugitive bug, so here's the recipe:

- ensure you have a markdown LSP installed (marksman was easy to do)

- emacs -Q -f package-initialize /tmp/doesntexist.md -f eglot

  'package-initialize' is necessary to somehow bring in markdown.el,
  which isn't a part of Emacs.

- enter this text (disregarding this email's alignment)

  * one
    two
  * three
    four

Place text on first line and type C-c <down>, which invokes
'markdown-move-down'.  The window will scroll, when you scroll up,
you'll see the corruption.  The buffer becomes

  * three
  \@ four
  * one  four

Some notes:

- Doesn't seem to happen in file-supported buffers.

- I used the https://github.com/artempyanykh/marksman server

- track-changes.el is _not_ to blame since the most recent version of
  Eglot I'm using no longer uses that library.

- The 'marksman' server does _not_ use incremental updates, it always
  sends the full buffer text on every didChange update.

- Therefore, this is not the "normal" LSP update-related corruption
  issue.  (for context, in those issues what you see is that the server
  has a skewed image of what exactly is in the buffer, so it misses
  everything).

- I've not been able to reproduce it without Eglot.  So it seems to be
  somewhat related indeed.  It's as if Eglot and the
  'markdown-mode-down' function clashed for some reason.

- Very often, even in file-less buffers, the bug goes away and the C-c
  <up/down> switch works fine, even in Eglot.  I think actually saving
  the buffer into a file does it.

- Part of the garbled text is always invalid character NULL, which Emacs
  renders as ^@, I think.

- The problem happens simplify if you evaluate:

     (transpose-regions 1 12 13 27 nil)

  in that buffer text (the one, two... example.).

- You can also set marks on the regions and call transpose-regions
  interactively. So maybe that particular markdown.el function isn't to
  blame, but buffer settings done by 'markdown-mode' may still be.

- That's all I had time for.

Message #20 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: monnier <at> iro.umontreal.ca, João Távora
 <joaotavora <at> gmail.com>
Cc: johann.hoechtl <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 22 Feb 2025 11:25:00 +0200

Stefan, any comments?

> From: João Távora <joaotavora <at> gmail.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  Stefan Monnier
>  <monnier <at> iro.umontreal.ca>,  76124 <at> debbugs.gnu.org
> Date: Sat, 08 Feb 2025 19:55:03 +0000
> 
> Johann Höchtl <johann.hoechtl <at> gmail.com> writes:
> 
> On Sat, Feb 8, 2025 at 4:47 PM Johann Höchtl <johann.hoechtl <at> gmail.com> wrote:
> 
> >> > Possibly eglot and the interaction with "track-changes" is at
> >> > fault: When I eglot-shutdown, I never see this corruption.
> >> I'm guess the information about the LSP server you are using might be
> >> relevant?
> > I also do not know if this is relevant, but it is https://github.com/mhersson/mpls
> 
> It's normally extremely relevant, as the corruption related to
> track-changes and before/after change functions (which you hint at) is
> directly dependent on whether the server accepts partial updates or not.
> 
> >> > Using emacs as of commit 5e12843fa32150d2f18ce21fc6f3ae58732df6a7
> >> Is that from the master branch of the Emacs Git repository?  (Your
> >> report elided all version and configuration information, so it's hard
> >> to tell.)
> 
> (Indeed, I don't see this commit anywhere.  What branch is this?)
> 
> Anyway, as to the problem itself, I've reproduced it.
> 
> TL/DR: Something in Eglot is making transpose-regions not work correctly
> in file-less buffers.
> 
> Reproduction: 
> 
> It's kind of a fugitive bug, so here's the recipe:
> 
> - ensure you have a markdown LSP installed (marksman was easy to do)
> 
> - emacs -Q -f package-initialize /tmp/doesntexist.md -f eglot
> 
>   'package-initialize' is necessary to somehow bring in markdown.el,
>   which isn't a part of Emacs.
> 
> - enter this text (disregarding this email's alignment)
> 
>   * one
>     two
>   * three
>     four
> 
> Place text on first line and type C-c <down>, which invokes
> 'markdown-move-down'.  The window will scroll, when you scroll up,
> you'll see the corruption.  The buffer becomes
> 
>   * three
>   \@ four
>   * one  four
> 
> Some notes:
> 
> - Doesn't seem to happen in file-supported buffers.
> 
> - I used the https://github.com/artempyanykh/marksman server
> 
> - track-changes.el is _not_ to blame since the most recent version of
>   Eglot I'm using no longer uses that library.
> 
> - The 'marksman' server does _not_ use incremental updates, it always
>   sends the full buffer text on every didChange update.
> 
> - Therefore, this is not the "normal" LSP update-related corruption
>   issue.  (for context, in those issues what you see is that the server
>   has a skewed image of what exactly is in the buffer, so it misses
>   everything).
> 
> - I've not been able to reproduce it without Eglot.  So it seems to be
>   somewhat related indeed.  It's as if Eglot and the
>   'markdown-mode-down' function clashed for some reason.
> 
> - Very often, even in file-less buffers, the bug goes away and the C-c
>   <up/down> switch works fine, even in Eglot.  I think actually saving
>   the buffer into a file does it.
> 
> - Part of the garbled text is always invalid character NULL, which Emacs
>   renders as ^@, I think.
> 
> - The problem happens simplify if you evaluate:
> 
>      (transpose-regions 1 12 13 27 nil)
> 
>   in that buffer text (the one, two... example.).
> 
> - You can also set marks on the regions and call transpose-regions
>   interactively. So maybe that particular markdown.el function isn't to
>   blame, but buffer settings done by 'markdown-mode' may still be.
> 
> - That's all I had time for.
> 
> 

Message #23 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: johann.hoechtl <at> gmail.com,
 João Távora <joaotavora <at> gmail.com>,
 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sun, 23 Feb 2025 00:37:46 -0500

>> Place text on first line and type C-c <down>, which invokes
>> 'markdown-move-down'.  The window will scroll, when you scroll up,
>> you'll see the corruption.  The buffer becomes
>> 
>>   * three
>>   \@ four
>>   * one  four

This is very weird.  If at least the NUL byte was between the transposed
regions, but in the middle of one of the two?

>> - The problem happens simplify if you evaluate:
>>
>>      (transpose-regions 1 12 13 27 nil)
>>
>>   in that buffer text (the one, two... example.).

FWIW, I took a look at `transpose-regions` in case there was something
fishy in it.  I saw a problem with handling of text-properties in an
unrelated corner case (fixed on `master`), but nothing that would
explain the above.  🙁

>> - That's all I had time for.

Same here for now,


        Stefan

Message #26 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Johann Höchtl <johann.hoechtl <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 João Távora <joaotavora <at> gmail.com>, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Wed, 18 Jun 2025 10:55:55 +0200

[Message part 1 (text/plain, inline)]

Sorry for pinging, but buffer corruption is nothing to take easy and I was
bitten just now.

Am So., 23. Feb. 2025 um 06:38 Uhr schrieb Stefan Monnier <
monnier <at> iro.umontreal.ca>:

> >> Place text on first line and type C-c <down>, which invokes
> >> 'markdown-move-down'.  The window will scroll, when you scroll up,
> >> you'll see the corruption.  The buffer becomes
> >>
> >>   * three
> >>   \@ four
> >>   * one  four
>
> This is very weird.  If at least the NUL byte was between the transposed
> regions, but in the middle of one of the two?
>
> >> - The problem happens simplify if you evaluate:
> >>
> >>      (transpose-regions 1 12 13 27 nil)
> >>
> >>   in that buffer text (the one, two... example.).
>
> FWIW, I took a look at `transpose-regions` in case there was something
> fishy in it.  I saw a problem with handling of text-properties in an
> unrelated corner case (fixed on `master`), but nothing that would
> explain the above.  🙁
>
> >> - That's all I had time for.
>
> Same here for now,
>
>
>         Stefan
>
>

[Message part 2 (text/html, inline)]

Message #29 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 João Távora <joaotavora <at> gmail.com>,
 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Tue, 24 Jun 2025 16:48:00 -0400

> Sorry for pinging, but buffer corruption is nothing to take easy and I was
> bitten just now.

What LSP server were you using at the time?
[ Since João managed to reproduce it with `marksman` and you originally
  bumped into it with `mpls`, I presume the problem is actually not in
  the LSP server, but it's still useful info.  ]

Was it also in `markdown-mode`?

I haven't yet been able to reproduce it locally, because both `marksman`
and `mpls` both depend on build tools I don't have installed.

If you can reproduce it, I'd be curious to see the result of the
following:

- M-x load-library RET track-changes RET
- Set things up until just before reproducing the problem.
- M-: (track-changes-register #'ignore) RET
- M-: (setq track-changes-record-errors 'trace) RET
- Reproduce the problem (e.g. with `M-: (transpose-regions 1 12 13 27 nil)`).
- Show us the value of `track-changes--trace`.

Ideally do it in a small buffer and show us the content of the buffer
before and after the "reproduce" step (because the track will contain
info saying "inserted text between POS1 and POS2" so we need to look at
the buffer to know what that inserted text was).

Hopefully, this may give us a clue as to which piece of code inserts the
NUL chars (whether it's Eglot, Markdown mode, `transpose-regions`, or
yet something else).


        Stefan


> Am So., 23. Feb. 2025 um 06:38 Uhr schrieb Stefan Monnier <
> monnier <at> iro.umontreal.ca>:
>
>> >> Place text on first line and type C-c <down>, which invokes
>> >> 'markdown-move-down'.  The window will scroll, when you scroll up,
>> >> you'll see the corruption.  The buffer becomes
>> >>
>> >>   * three
>> >>   \@ four
>> >>   * one  four
>>
>> This is very weird.  If at least the NUL byte was between the transposed
>> regions, but in the middle of one of the two?
>>
>> >> - The problem happens simplify if you evaluate:
>> >>
>> >>      (transpose-regions 1 12 13 27 nil)
>> >>
>> >>   in that buffer text (the one, two... example.).
>>
>> FWIW, I took a look at `transpose-regions` in case there was something
>> fishy in it.  I saw a problem with handling of text-properties in an
>> unrelated corner case (fixed on `master`), but nothing that would
>> explain the above.  🙁
>>
>> >> - That's all I had time for.
>>
>> Same here for now,
>>
>>
>>         Stefan
>>
>>

Message #32 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: johann.hoechtl <at> gmail.com, Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 05 Jul 2025 10:56:49 +0300

Ping! Johann, could you please answer Stefan's questions?

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  João Távora
>  <joaotavora <at> gmail.com>,
>   76124 <at> debbugs.gnu.org
> Date: Tue, 24 Jun 2025 16:48:00 -0400
> 
> > Sorry for pinging, but buffer corruption is nothing to take easy and I was
> > bitten just now.
> 
> What LSP server were you using at the time?
> [ Since João managed to reproduce it with `marksman` and you originally
>   bumped into it with `mpls`, I presume the problem is actually not in
>   the LSP server, but it's still useful info.  ]
> 
> Was it also in `markdown-mode`?
> 
> I haven't yet been able to reproduce it locally, because both `marksman`
> and `mpls` both depend on build tools I don't have installed.
> 
> If you can reproduce it, I'd be curious to see the result of the
> following:
> 
> - M-x load-library RET track-changes RET
> - Set things up until just before reproducing the problem.
> - M-: (track-changes-register #'ignore) RET
> - M-: (setq track-changes-record-errors 'trace) RET
> - Reproduce the problem (e.g. with `M-: (transpose-regions 1 12 13 27 nil)`).
> - Show us the value of `track-changes--trace`.
> 
> Ideally do it in a small buffer and show us the content of the buffer
> before and after the "reproduce" step (because the track will contain
> info saying "inserted text between POS1 and POS2" so we need to look at
> the buffer to know what that inserted text was).
> 
> Hopefully, this may give us a clue as to which piece of code inserts the
> NUL chars (whether it's Eglot, Markdown mode, `transpose-regions`, or
> yet something else).
> 
> 
>         Stefan
> 
> 
> > Am So., 23. Feb. 2025 um 06:38 Uhr schrieb Stefan Monnier <
> > monnier <at> iro.umontreal.ca>:
> >
> >> >> Place text on first line and type C-c <down>, which invokes
> >> >> 'markdown-move-down'.  The window will scroll, when you scroll up,
> >> >> you'll see the corruption.  The buffer becomes
> >> >>
> >> >>   * three
> >> >>   \@ four
> >> >>   * one  four
> >>
> >> This is very weird.  If at least the NUL byte was between the transposed
> >> regions, but in the middle of one of the two?
> >>
> >> >> - The problem happens simplify if you evaluate:
> >> >>
> >> >>      (transpose-regions 1 12 13 27 nil)
> >> >>
> >> >>   in that buffer text (the one, two... example.).
> >>
> >> FWIW, I took a look at `transpose-regions` in case there was something
> >> fishy in it.  I saw a problem with handling of text-properties in an
> >> unrelated corner case (fixed on `master`), but nothing that would
> >> explain the above.  🙁
> >>
> >> >> - That's all I had time for.
> >>
> >> Same here for now,
> >>
> >>
> >>         Stefan
> >>
> >>
> 
> 

Message #35 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Johann Höchtl <johann.hoechtl <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 76124 <at> debbugs.gnu.org,
 joaotavora <at> gmail.com
Subject: Re: bug#76124: eglot buffer corruption
Date: Sun, 6 Jul 2025 20:46:04 +0200

[Message part 1 (text/plain, inline)]

Thank you very much @Stefan Monnier <monnier <at> iro.umontreal.ca>  for the
detailed information which helped me to produce a hopefully helpful trace
output.

The output of track-changes--trace is:

track-changes--trace is a variable defined in ‘track-changes.el’.

Its value is shown below.

Ring holding a trace of recent calls to the API.
Each call is recorded as a (BUFFER-NAME . BACKTRACE).

Value:
(8 10
   . [(#1="test.md" (t track-changes--trace nil nil)
 (t track-changes--before (24 24) nil)
 (t self-insert-command (1 111) nil)
 (t funcall-interactively (self-insert-command 1 111) nil)
 (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--after (24 25 0) nil)
  (t self-insert-command (1 111) nil)
  (t funcall-interactively (self-insert-command 1 111) nil)
  (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--before (25 25) nil)
  (t self-insert-command (1 117) nil)
  (t funcall-interactively (self-insert-command 1 117) nil)
  (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--after (25 26 0) nil)
  (t self-insert-command (1 117) nil)
  (t funcall-interactively (self-insert-command 1 117) nil)
  (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--before (26 26) nil)
  (t self-insert-command (1 114) nil)
  (t funcall-interactively (self-insert-command 1 114) nil)
  (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--after (26 27 0) nil)
  (t self-insert-command (1 114) nil)
  (t funcall-interactively (self-insert-command 1 114) nil)
  (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--before (1 27) nil)
  (t transpose-regions (1 12 13 27 nil) nil)
  (t markdown-move-list-item-down nil nil)
  (t funcall-interactively (markdown-move-list-item-down) nil)
  (t markdown-move-down nil nil)
  (t funcall-interactively (markdown-move-down) nil)
  (t command-execute (markdown-move-down) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--after (1 27 26) nil)
  (t transpose-regions (1 12 13 27 nil) nil)
  (t markdown-move-list-item-down nil nil)
  (t funcall-interactively (markdown-move-list-item-down) nil)
  (t markdown-move-down nil nil)
  (t funcall-interactively (markdown-move-down) nil)
  (t command-execute (markdown-move-down) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--before (23 23) nil)
  (t self-insert-command (1 102) nil)
  (t funcall-interactively (self-insert-command 1 102) nil)
  (t command-execute (self-insert-command) nil))
      (#1# (t track-changes--trace nil nil)
  (t track-changes--after (23 24 0) nil)
  (t self-insert-command (1 102) nil)
  (t funcall-interactively (self-insert-command 1 102) nil)
  (t command-execute (self-insert-command) nil))])


I followed the recipe of @João Távora <joaotavora <at> gmail.com>, thus buffer
before was:

- one
  two
- three
  four


After:  (as it seems to involve NUL values I converted to hexl-mode,
hopefully this doesn't garble the output further):

00000000: 2d20 7468 7265 650d 0a00 2066 6f75 720d  - three... four.
00000010: 0a2d 206f 6e65 2020 666f 7572            .- one  four


Best, Johann


Am Sa., 5. Juli 2025 um 09:56 Uhr schrieb Eli Zaretskii <eliz <at> gnu.org>:

> Ping! Johann, could you please answer Stefan's questions?
>
> > From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> > Cc: Eli Zaretskii <eliz <at> gnu.org>,  João Távora
> >  <joaotavora <at> gmail.com>,
> >   76124 <at> debbugs.gnu.org
> > Date: Tue, 24 Jun 2025 16:48:00 -0400
> >
> > > Sorry for pinging, but buffer corruption is nothing to take easy and I
> was
> > > bitten just now.
> >
> > What LSP server were you using at the time?
> > [ Since João managed to reproduce it with `marksman` and you originally
> >   bumped into it with `mpls`, I presume the problem is actually not in
> >   the LSP server, but it's still useful info.  ]
> >
> > Was it also in `markdown-mode`?
> >
> > I haven't yet been able to reproduce it locally, because both `marksman`
> > and `mpls` both depend on build tools I don't have installed.
> >
> > If you can reproduce it, I'd be curious to see the result of the
> > following:
> >
> > - M-x load-library RET track-changes RET
> > - Set things up until just before reproducing the problem.
> > - M-: (track-changes-register #'ignore) RET
> > - M-: (setq track-changes-record-errors 'trace) RET
> > - Reproduce the problem (e.g. with `M-: (transpose-regions 1 12 13 27
> nil)`).
> > - Show us the value of `track-changes--trace`.
> >
> > Ideally do it in a small buffer and show us the content of the buffer
> > before and after the "reproduce" step (because the track will contain
> > info saying "inserted text between POS1 and POS2" so we need to look at
> > the buffer to know what that inserted text was).
> >
> > Hopefully, this may give us a clue as to which piece of code inserts the
> > NUL chars (whether it's Eglot, Markdown mode, `transpose-regions`, or
> > yet something else).
> >
> >
> >         Stefan
> >
> >
> > > Am So., 23. Feb. 2025 um 06:38 Uhr schrieb Stefan Monnier <
> > > monnier <at> iro.umontreal.ca>:
> > >
> > >> >> Place text on first line and type C-c <down>, which invokes
> > >> >> 'markdown-move-down'.  The window will scroll, when you scroll up,
> > >> >> you'll see the corruption.  The buffer becomes
> > >> >>
> > >> >>   * three
> > >> >>   \@ four
> > >> >>   * one  four
> > >>
> > >> This is very weird.  If at least the NUL byte was between the
> transposed
> > >> regions, but in the middle of one of the two?
> > >>
> > >> >> - The problem happens simplify if you evaluate:
> > >> >>
> > >> >>      (transpose-regions 1 12 13 27 nil)
> > >> >>
> > >> >>   in that buffer text (the one, two... example.).
> > >>
> > >> FWIW, I took a look at `transpose-regions` in case there was something
> > >> fishy in it.  I saw a problem with handling of text-properties in an
> > >> unrelated corner case (fixed on `master`), but nothing that would
> > >> explain the above.  🙁
> > >>
> > >> >> - That's all I had time for.
> > >>
> > >> Same here for now,
> > >>
> > >>
> > >>         Stefan
> > >>
> > >>
> >
> >
>

[Message part 2 (text/html, inline)]

Message #38 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sun, 06 Jul 2025 17:55:04 -0400

[Message part 1 (text/plain, inline)]

> I followed the recipe of @João Távora <joaotavora <at> gmail.com>, thus buffer
> before was:
>
> - one
>   two
> - three
>   four

Hmm... I see nothing funny in the trace, and especially no involvement
of eglot, so it seems clear the problem comes from the C code of
`transpose-regions` (tho somehow triggered by something in Eglot's code).

Re-reading `transpose-regions` I see that there's a possibility for
trouble if the gap is moved by the `before-change-functions`, and that
could definitely explain the weird behavior.

I think the patch below is *right* but I'm not sure it fixes the problem
you're seeing.

Can you try it?


        Stefan

[transpose-regions.patch (text/x-diff, inline)]

diff --git a/src/editfns.c b/src/editfns.c
index 90bc7326c6f..be8aac673ce 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -4659,6 +4659,9 @@ DEFUN ("transpose-regions", Ftranspose_regions, Stranspose_regions, 4, 5,
 			     BUF_TS_LINECOL_POINT (current_buffer));
 #endif
 
+  /* Run the before-change-functions before we mess with the gap.  */
+  modify_text (start1, end2);
+
   /* Make sure the gap won't interfere, by moving it out of the text
      we will operate on.  */
   if (start1 < gap && gap < end2)
@@ -4703,7 +4706,6 @@ DEFUN ("transpose-regions", Ftranspose_regions, Stranspose_regions, 4, 5,
      enough to use as the temporary storage?  That would avoid an
      allocation... interesting.  Later, don't fool with it now.  */
 
-  modify_text (start1, end2);
   tmp_interval1 = copy_intervals (cur_intv, start1, len1);
   tmp_interval2 = copy_intervals (cur_intv, start2, len2);
   USE_SAFE_ALLOCA;

Message #41 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sun, 06 Jul 2025 19:19:50 -0400

> I think the patch below is *right* but I'm not sure it fixes the problem
> you're seeing.  Can you try it?

I'm now fairly convinced that this was indeed the culprit.
I pushed it to `master`, together with a test that produces a similar
misbehavior to the one you see (without using Eglot).
I'd still appreciate if you can confirm that you can't reproduce the
problem when this patch is installed.


        Stefan

Message #44 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Johann Höchtl <johann.hoechtl <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Mon, 7 Jul 2025 08:39:37 +0200

[Message part 1 (text/plain, inline)]

Unfortunately I do not have the means to test that out = build Emacs myself
including the patch. However I regularly update my Emacs to follow Head
from https://github.com/kiennq/emacs-build and as soon a new build arrives
I will re-test.

Am Mo., 7. Juli 2025 um 01:19 Uhr schrieb Stefan Monnier <
monnier <at> iro.umontreal.ca>:

> > I think the patch below is *right* but I'm not sure it fixes the problem
> > you're seeing.  Can you try it?
>
> I'm now fairly convinced that this was indeed the culprit.
> I pushed it to `master`, together with a test that produces a similar
> misbehavior to the one you see (without using Eglot).
> I'd still appreciate if you can confirm that you can't reproduce the
> problem when this patch is installed.
>
>
>         Stefan
>
>

[Message part 2 (text/html, inline)]

Message #47 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Mon, 07 Jul 2025 09:58:48 -0400

> Unfortunately I do not have the means to test that out = build Emacs myself
> including the patch. However I regularly update my Emacs to follow Head
> from https://github.com/kiennq/emacs-build and as soon a new build arrives
> I will re-test.

Perfect.  There's no hurry.


        Stefan

Message #50 received at 76124 <at> debbugs.gnu.org (full text, mbox):

From: Johann Höchtl <johann.hoechtl <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, joaotavora <at> gmail.com, 76124 <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 9 Aug 2025 19:59:46 +0200

[Message part 1 (text/plain, inline)]

Dear all,

I checked with a recent Emacs HEAD-build, which includes the fix of @Stefan
Monnier <monnier <at> iro.umontreal.ca>. Using the previous recipe to trigger
the error, I couldn't reproduce the corruption, the bug is fixed.

Best, Johann

Am Mo., 7. Juli 2025 um 15:58 Uhr schrieb Stefan Monnier <
monnier <at> iro.umontreal.ca>:

> > Unfortunately I do not have the means to test that out = build Emacs
> myself
> > including the patch. However I regularly update my Emacs to follow Head
> > from https://github.com/kiennq/emacs-build and as soon a new build
> arrives
> > I will re-test.
>
> Perfect.  There's no hurry.
>
>
>         Stefan
>
>

[Message part 2 (text/html, inline)]

Message #55 received at 76124-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Johann Höchtl <johann.hoechtl <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, joaotavora <at> gmail.com,
 76124-done <at> debbugs.gnu.org
Subject: Re: bug#76124: eglot buffer corruption
Date: Sat, 09 Aug 2025 17:06:41 -0400

> I checked with a recent Emacs HEAD-build, which includes the fix of @Stefan
> Monnier <monnier <at> iro.umontreal.ca>. Using the previous recipe to trigger
> the error, I couldn't reproduce the corruption, the bug is fixed.

Thanks for confirming, Johann,
Closing,


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.