GNU bug report logs - #41513
`compute-motion' can miscount buffer positions in the presence of 'before-string/'after-string overlays

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Stephen Bach <sjbach@HIDDEN>; dated Sun, 24 May 2020 18:45:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 41513 <at> debbugs.gnu.org:


Received: (at 41513) by debbugs.gnu.org; 25 May 2020 20:16:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon May 25 16:16:28 2020
Received: from localhost ([127.0.0.1]:42612 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jdJWO-0003Gb-5u
	for submit <at> debbugs.gnu.org; Mon, 25 May 2020 16:16:28 -0400
Received: from mail-lj1-f178.google.com ([209.85.208.178]:44529)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <sjbach@HIDDEN>) id 1jdJWL-00039C-Ui
 for 41513 <at> debbugs.gnu.org; Mon, 25 May 2020 16:16:27 -0400
Received: by mail-lj1-f178.google.com with SMTP id k5so21933321lji.11
 for <41513 <at> debbugs.gnu.org>; Mon, 25 May 2020 13:16:25 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sjbach.com; s=google;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=TzdnbNs1L66T3BaMkJOqrhQ8DgOf9MneMSskANbF/Fg=;
 b=H+isR/32nzoPAPzp5UoDc7MaSHQhJPQsnay9JptCxW6a+FGvQnFwLwonaTizYH1jht
 Va0QwUvD7Mcr5EXNHEqveM3H3t3mcb/sp2RZQY17K326Ai83vlO/zVX5d0rT3zRQKRPT
 7B+n5e4avP2kUK3YkB0gk7ZTFLi8qqHuraLnU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=TzdnbNs1L66T3BaMkJOqrhQ8DgOf9MneMSskANbF/Fg=;
 b=MEb4xxtEShEs3jynro3s7mCzXRYM0lsDfrEqb1/99cQlZd6bkGh7OIfjjmzrSohyOI
 7jYRwWDFMq4rVT05f+2Kd/TQ57vcOgDciR9SuaHXarxuPliAfMVYpGgDNEWXy41EG8Sa
 uQtPkPFIT3WVI9hNpgmb3lNe6vG5KLKCzv2WrCMP1za0T+IQFODdf8jwYkQErSnpyuKV
 t6j8jxrOX3GJvdlgenzFeCqufk2QVOm3BhfX4IJflADX93/h+Nxyr6XdEXlGGWHRzrCd
 zVGvkqDGpy9biMWNymMH+Bg8XHfsbgiTbJrjtJ86k0cA5YXTuoAr1Z8IWKfIk8O1Ecum
 AYNA==
X-Gm-Message-State: AOAM530HrevT22q7Yn64vsEJVv1HwULqrYtVm0G3a0gHaXO3ehDa2r9R
 eXci+1W6giPDuaOJVDuZGJboVrfdyEu0JyZGhKlPEC3GH94=
X-Google-Smtp-Source: ABdhPJy8Me9AWzAsPyUL98kVKWu7koJ68qlX32b7B8kjg09PbyQJNM2+Y1CiDahQgJXtMj3oSk72Q8wvfw79CrASRLo=
X-Received: by 2002:a2e:3818:: with SMTP id f24mr11472793lja.338.1590437779685; 
 Mon, 25 May 2020 13:16:19 -0700 (PDT)
MIME-Version: 1.0
References: <CAB5Eu-Pbd1xTYA3kynzjaryMBD_p3TbDF7VEJq1+fp24tMeX+A@HIDDEN>
 <83lflhw3og.fsf@HIDDEN>
 <CAB5Eu-N--usvQMe7kKQo8jcx7kAJbEqhSbpPNin0sdoFSbU9qQ@HIDDEN>
 <83y2pgujhk.fsf@HIDDEN>
In-Reply-To: <83y2pgujhk.fsf@HIDDEN>
From: Stephen Bach <sjbach@HIDDEN>
Date: Mon, 25 May 2020 16:16:08 -0400
Message-ID: <CAB5Eu-NJFop7fwMEwR=E5nkMJfDGKOoU_A5CxL3_XY79XwApjw@HIDDEN>
Subject: Re: bug#41513: `compute-motion' can miscount buffer positions in the
 presence of 'before-string/'after-string overlays
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41513
Cc: 41513 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Mon, May 25, 2020 at 11:22 AM Eli Zaretskii <eliz@HIDDEN> wrote:
>
> Please tell more.  What exactly would you like to measure?  Is it the
> width of each screen line? or just the maximum width of the longest
> line among those shown in the window? or something else?

The width of each screen line, kind-of. I'm writing a
terminal-compatible drawing library for displaying low-res
visualizations in the buffer/window:

   - https://sjbach.com/canvas-emacs-logo.mp4
   - https://sjbach.com/canvas-mario.mp4

To draw an image I apply temporary overlays to buffer ranges that
correspond to window coordinates which themselves correspond to the
"pixels" of the image.

A performance-sensitive part of this turns out to be when I walk between
`window-start' and `window-end' to figure out (1) the buffer positions
of the final characters of visible screen lines/visual rows, and (2) the
buffer positions underlying particular window coordinates that happen to
fall within the buffer text -- both of which I've been using
`compute-motion' to calculate.

Is `vertical-motion' a more reliable analogue to `compute-motion'? I see
it can take a (COLS . LINES) argument, so it appears similar.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#41513; Package emacs. Full text available.

Message received at 41513 <at> debbugs.gnu.org:


Received: (at 41513) by debbugs.gnu.org; 25 May 2020 15:23:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon May 25 11:23:11 2020
Received: from localhost ([127.0.0.1]:42171 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jdEwM-0007D5-M5
	for submit <at> debbugs.gnu.org; Mon, 25 May 2020 11:23:11 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48874)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1jdEwL-0007Ct-AZ
 for 41513 <at> debbugs.gnu.org; Mon, 25 May 2020 11:22:57 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:50966)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1jdEwF-00034m-V3; Mon, 25 May 2020 11:22:51 -0400
Received: from [176.228.60.248] (port=1039 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1jdEwE-00078c-QG; Mon, 25 May 2020 11:22:51 -0400
Date: Mon, 25 May 2020 18:23:03 +0300
Message-Id: <83y2pgujhk.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Stephen Bach <sjbach@HIDDEN>
In-Reply-To: <CAB5Eu-N--usvQMe7kKQo8jcx7kAJbEqhSbpPNin0sdoFSbU9qQ@HIDDEN>
 (message from Stephen Bach on Sun, 24 May 2020 15:55:09 -0400)
Subject: Re: bug#41513: `compute-motion' can miscount buffer positions in the
 presence of 'before-string/'after-string overlays
References: <CAB5Eu-Pbd1xTYA3kynzjaryMBD_p3TbDF7VEJq1+fp24tMeX+A@HIDDEN>
 <83lflhw3og.fsf@HIDDEN>
 <CAB5Eu-N--usvQMe7kKQo8jcx7kAJbEqhSbpPNin0sdoFSbU9qQ@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41513
Cc: 41513 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Stephen Bach <sjbach@HIDDEN>
> Date: Sun, 24 May 2020 15:55:09 -0400
> Cc: 41513 <at> debbugs.gnu.org
> 
> > There are other functions that can do the same job.  If you describe
> > what you need to do, I could try suggesting some alternatives that
> > will work.
> 
> Thanks Eli. Specifically, I'm measuring the jagged contour of a buffer's
> layout within a particular window, i.e. relating the final column of
> each visual row (as displayed within the window) to concrete buffer
> positions. This is notionally straightforward but complicated in
> practice, as you will know -- e.g. line continuation/truncation,
> `word-wrap', `visual-line-mode', `adaptive-wrap-prefix-mode',
> multi-columnar glyphs. Text properties or overlays using 'display, and
> so on.

Please tell more.  What exactly would you like to measure?  Is it the
width of each screen line? or just the maximum width of the longest
line among those shown in the window? or something else?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#41513; Package emacs. Full text available.

Message received at 41513 <at> debbugs.gnu.org:


Received: (at 41513) by debbugs.gnu.org; 24 May 2020 19:55:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun May 24 15:55:28 2020
Received: from localhost ([127.0.0.1]:38856 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jcwiW-00014F-E8
	for submit <at> debbugs.gnu.org; Sun, 24 May 2020 15:55:28 -0400
Received: from mail-lj1-f172.google.com ([209.85.208.172]:35166)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <sjbach@HIDDEN>) id 1jcwiU-000141-Tm
 for 41513 <at> debbugs.gnu.org; Sun, 24 May 2020 15:55:27 -0400
Received: by mail-lj1-f172.google.com with SMTP id c11so16433004ljn.2
 for <41513 <at> debbugs.gnu.org>; Sun, 24 May 2020 12:55:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sjbach.com; s=google;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=ycRywNeNJYNK6FS5LYNm6c4eeiK/EJ9tV4KmO6C1I6w=;
 b=T97S3y5pgwn4FIUOqNRz7hNqFGQqoMIUVi2FxjR+9OyazkptlYkZBrbj+GyMnDtG+h
 9Difybmo+yNhwAjKSdMABrt65Vi6+gDzEpxZfdnf0RooOnLz+8qkr53/4xIxQIywLu2z
 Xs9AeecaX34I/qhyImUuadm9kDNXu6sGeTDJ4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=ycRywNeNJYNK6FS5LYNm6c4eeiK/EJ9tV4KmO6C1I6w=;
 b=uAbF/cH23ol9QKo/rd8Rrke2ccMckIuFaZhBTcCUdcI19U/lUs0Djwu3YZEDktbw/p
 0EoAui4muTjV497Yt5do9ZS/McycsR9X9zuvj3baF+5G15PPKCpTVMOap6UW25O7W1kF
 VCuFtcKG1WvhIRrB7Ucm85K3PUwAZAYiVmfmDtiknRm3vO3aIYQk/Gvp9Nb1fJNXWxHh
 JurJwIY7Tgt1MFF8fajcXA7fcEfhMqYBZQ2fqtw6kZGxnoHoCwArlc46sd73WgKEukYG
 AV65nqaNgqQWfuoKMCek9P1FiAtgLn2Vc3elRQBExtvIkiY5CYWrzZPHEKbcasnF9S94
 katA==
X-Gm-Message-State: AOAM530DZ6ahAbyBL/xHkZMkt0Wdt8uVLM3Mg04FZlKizfnaLSn5ndpO
 6HtrjsqGK8bPkIFvc1VMD/+mybo9HtJLo3Kkq7Ki4w==
X-Google-Smtp-Source: ABdhPJyHTAaPwJBSfpWR/tsFPMRdiq8hQ49OVRpmTtf1VxHg979PVUKrZKRKEJsrzLTvIXtH2pN4fj7g4qxKIbirDAw=
X-Received: by 2002:a2e:3818:: with SMTP id f24mr9327207lja.338.1590350120502; 
 Sun, 24 May 2020 12:55:20 -0700 (PDT)
MIME-Version: 1.0
References: <CAB5Eu-Pbd1xTYA3kynzjaryMBD_p3TbDF7VEJq1+fp24tMeX+A@HIDDEN>
 <83lflhw3og.fsf@HIDDEN>
In-Reply-To: <83lflhw3og.fsf@HIDDEN>
From: Stephen Bach <sjbach@HIDDEN>
Date: Sun, 24 May 2020 15:55:09 -0400
Message-ID: <CAB5Eu-N--usvQMe7kKQo8jcx7kAJbEqhSbpPNin0sdoFSbU9qQ@HIDDEN>
Subject: Re: bug#41513: `compute-motion' can miscount buffer positions in the
 presence of 'before-string/'after-string overlays
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41513
Cc: 41513 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Sun, May 24, 2020 at 3:09 PM Eli Zaretskii <eliz@HIDDEN> wrote:
> Don't use compute-motion.  It is a remnant of the old pre-Emacs 21
> display engine, and almost certainly doesn't support tricky display
> features.
>
> There are other functions that can do the same job.  If you describe
> what you need to do, I could try suggesting some alternatives that
> will work.

Thanks Eli. Specifically, I'm measuring the jagged contour of a buffer's
layout within a particular window, i.e. relating the final column of
each visual row (as displayed within the window) to concrete buffer
positions. This is notionally straightforward but complicated in
practice, as you will know -- e.g. line continuation/truncation,
`word-wrap', `visual-line-mode', `adaptive-wrap-prefix-mode',
multi-columnar glyphs. Text properties or overlays using 'display, and
so on.

In my initial go at this I used `end-of-visual-line' calls within a
`save-excursion' but I found that to be a little slow given the
frequency I was making the calls and I was also uncomfortable about all
the cursor movement, perhaps irrationally. But maybe this is the correct
approach?

Re: `compute-motion', perhaps the manual or the docstring could include
a short note to dissuade developers from using it? Its complicated
interface may be sufficient to keep most people away, but on the other
hand a complicated interface can suggest efficiency and intricate
functionality.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#41513; Package emacs. Full text available.

Message received at 41513 <at> debbugs.gnu.org:


Received: (at 41513) by debbugs.gnu.org; 24 May 2020 19:09:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun May 24 15:09:18 2020
Received: from localhost ([127.0.0.1]:38743 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jcvzq-0008Ln-6t
	for submit <at> debbugs.gnu.org; Sun, 24 May 2020 15:09:18 -0400
Received: from eggs.gnu.org ([209.51.188.92]:56818)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1jcvzo-0008LZ-P5
 for 41513 <at> debbugs.gnu.org; Sun, 24 May 2020 15:09:17 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:36388)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1jcvzj-0004oI-9M; Sun, 24 May 2020 15:09:11 -0400
Received: from [176.228.60.248] (port=2077 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1jcvzi-0003Xk-NB; Sun, 24 May 2020 15:09:11 -0400
Date: Sun, 24 May 2020 22:09:19 +0300
Message-Id: <83lflhw3og.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Stephen Bach <sjbach@HIDDEN>
In-Reply-To: <CAB5Eu-Pbd1xTYA3kynzjaryMBD_p3TbDF7VEJq1+fp24tMeX+A@HIDDEN>
 (message from Stephen Bach on Sun, 24 May 2020 14:30:13 -0400)
Subject: Re: bug#41513: `compute-motion' can miscount buffer positions in the
 presence of 'before-string/'after-string overlays
References: <CAB5Eu-Pbd1xTYA3kynzjaryMBD_p3TbDF7VEJq1+fp24tMeX+A@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41513
Cc: 41513 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Stephen Bach <sjbach@HIDDEN>
> Date: Sun, 24 May 2020 14:30:13 -0400
> 
> The miscounting appears to exist at least as far back as Emacs 24. Might there be a mitigation/workaround?
> I like `compute-motion' despite its complicated signature and complicated return value (and despite its
> apparently rare use in the extended ecosystem) - it's fast and it involves no cursor movement to perform its
> measurements.

Don't use compute-motion.  It is a remnant of the old pre-Emacs 21
display engine, and almost certainly doesn't support tricky display
features.

There are other functions that can do the same job.  If you describe
what you need to do, I could try suggesting some alternatives that
will work.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#41513; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 24 May 2020 18:44:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun May 24 14:44:22 2020
Received: from localhost ([127.0.0.1]:38726 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jcvbh-0007eA-RM
	for submit <at> debbugs.gnu.org; Sun, 24 May 2020 14:44:22 -0400
Received: from lists.gnu.org ([209.51.188.17]:51450)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <sjbach@HIDDEN>) id 1jcvOJ-0007Hm-6L
 for submit <at> debbugs.gnu.org; Sun, 24 May 2020 14:30:31 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:60728)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <sjbach@HIDDEN>) id 1jcvOJ-0007zU-0H
 for bug-gnu-emacs@HIDDEN; Sun, 24 May 2020 14:30:31 -0400
Received: from mail-lj1-x22d.google.com ([2a00:1450:4864:20::22d]:34670)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <sjbach@HIDDEN>) id 1jcvOH-0006Oi-OW
 for bug-gnu-emacs@HIDDEN; Sun, 24 May 2020 14:30:30 -0400
Received: by mail-lj1-x22d.google.com with SMTP id b6so18491613ljj.1
 for <bug-gnu-emacs@HIDDEN>; Sun, 24 May 2020 11:30:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sjbach.com; s=google;
 h=mime-version:from:date:message-id:subject:to;
 bh=d1PMDCKh+cPkBSYBYZM2RwtI0tc8bLzrkxy7pbt6vtw=;
 b=fDrh6mUsiikbqtike4lxAZpjf+hXHpghqelCfbFYEctV6xNEGnjHuu6sPxoNBQKjYC
 49YibwXAYjUmF2EjRVRzz78WNajDx2Rxmo8IgOmFVFi8PIkuVHuNbNJUIEy66eIVIjkQ
 Y8ZTS8/KARoWW4qVUPP19ZOFIJEk7vsK10oeA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=d1PMDCKh+cPkBSYBYZM2RwtI0tc8bLzrkxy7pbt6vtw=;
 b=SLPOu1I4waW0sM885+jcYjhkDx6MOjUm6akrshQQYhj3972zfJlgtVHWnRbRiR81IE
 xuIJu75UQR8yYdjBm2zM1R/ONrWNX4YQvj613f8Ii5mzDORsoKeyzcZS/ou3K84Mrlxe
 aMujIqYBLDwY3CXpypzhjbwh5J6ZxxYbSa+ofHPQIZAcNKc+o0SJSIyKwj0e9nmFWVUX
 hfWwf4dG21pmZJRiTBVlNLshzx3ClJW3eIDhY7G7qDzJzSHk9oi5HBR/TR+esxbjRpnv
 mJM9S4xQAJCOLUdaPFUa43J1nmEGkcEIQ0134ydgkUDjDiDMitaqkwAc12n2C5rMJ63R
 yxug==
X-Gm-Message-State: AOAM5300IuK5q8Qlr4Lozn7q+tCcMdHmJp3p1Vi94tK1NiGe4CfRb8YD
 WM93q0JuTHikQrFqIV03oPJSptVLPf4rQakrzKioYIrN0AU=
X-Google-Smtp-Source: ABdhPJwQQ3xtYeDl/mVjLgQnOY2tH4XoyygkbINatOweczMf7rGwwieJ66tN7wfJZo7Sz9TOLoZver7p0d74hrrE44M=
X-Received: by 2002:a2e:3818:: with SMTP id f24mr9230424lja.338.1590345024648; 
 Sun, 24 May 2020 11:30:24 -0700 (PDT)
MIME-Version: 1.0
From: Stephen Bach <sjbach@HIDDEN>
Date: Sun, 24 May 2020 14:30:13 -0400
Message-ID: <CAB5Eu-Pbd1xTYA3kynzjaryMBD_p3TbDF7VEJq1+fp24tMeX+A@HIDDEN>
Subject: `compute-motion' can miscount buffer positions in the presence of
 'before-string/'after-string overlays
To: bug-gnu-emacs@HIDDEN
Content-Type: multipart/mixed; boundary="000000000000446d8405a6690ecb"
Received-SPF: pass client-ip=2a00:1450:4864:20::22d;
 envelope-from=sjbach@HIDDEN; helo=mail-lj1-x22d.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
 That's all we know.
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001,
 URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Sun, 24 May 2020 14:44:20 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

--000000000000446d8405a6690ecb
Content-Type: multipart/alternative; boundary="000000000000446d8205a6690ec9"

--000000000000446d8205a6690ec9
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

Please see the attached demonstration code, runnable with `emacs -Q' and
`eval-buffer'.

When `compute-motion' scans along a buffer range that covers an overlay
with a 'buffer-string or 'after-string property [*] having as its content a
sort-of tricky but not too tricky propertized string (described below),
`compute-motion' can/will return an incorrect POS value.

E.g. an overlay with this plist:

  '(before-string #("AAA" 0 3 (display ""))

This overlay is effectively a no-op for redisplay - the "AAA" string that
would have been printed at the start of the overlay interval is not printed
because its presentation is suppressed by the string's 'display property.
This is right and proper.

However, `compute-motion' appears to interpret this overlay as having an
effective visible length of -3 rather than 0. Ditto if the overlay had
instead had an 'after-string property with the same content. [*] The
effective length being negative suggests to me an overcorrection rather
than missing logic.

([*] Aside: I believe `compute-motion' sometimes also miscalculates
'display as an overlay property for cases where one might reasonably expect
it to work- but I limit this report to 'before-string and 'after-string.)

Popular packages such as flycheck and git-gutter use this pattern of
overlay to annotate the margin and the fringe. The pattern is increasingly
common which in turn increasingly limits the application of
`compute-motion' as a reliable exposed function.

The miscounting appears to exist at least as far back as Emacs 24. Might
there be a mitigation/workaround? I like `compute-motion' despite its
complicated signature and complicated return value (and despite its
apparently rare use in the extended ecosystem) - it's fast and it involves
no cursor movement to perform its measurements.

Thanks for reading.


Testcase output:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

-------------------------------------
Without overlay:
- =E2=80=98point=E2=80=99 at col 30: <31>
- buffer content on and after col 30: <30   35   40   45   50>
- posn coord at col 30: <(30 . 0)>
- buffer pos of col 30 according to =E2=80=98compute-motion=E2=80=99: <31> =
    [ok]
- coord of col 30 according to =E2=80=98compute-motion=E2=80=99: <(30 . 0)>
-------------------------------------
With =E2=80=99before-string overlay:
- =E2=80=98point=E2=80=99 at col 30: <31>
- buffer content on and after col 30: <30   35   40   45   50>
- posn coord at col 30: <(30 . 0)>
- buffer pos of col 30 according to =E2=80=98compute-motion=E2=80=99: <28> =
    <<<
INCONSISTENT
- coord of col 30 according to =E2=80=98compute-motion=E2=80=99: <(30 . 0)>
-------------------------------------
With =E2=80=99after-string overlay:
- =E2=80=98point=E2=80=99 at col 30: <31>
- buffer content on and after col 30: <30   35   40   45   50>
- posn coord at col 30: <(30 . 0)>
- buffer pos of col 30 according to =E2=80=98compute-motion=E2=80=99: <28> =
    <<<
INCONSISTENT
- coord of col 30 according to =E2=80=98compute-motion=E2=80=99: <(30 . 0)>

--000000000000446d8205a6690ec9
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<br><br>Please see the attached demonstration code, run=
nable with `emacs -Q&#39; and `eval-buffer&#39;.<br><br>When `compute-motio=
n&#39; scans along a buffer range that covers an overlay with a &#39;buffer=
-string or &#39;after-string property [*] having as its content a sort-of t=
ricky but not too tricky propertized string (described below), `compute-mot=
ion&#39; can/will return an incorrect POS value.<br><br>E.g. an overlay wit=
h this plist:<br><br>=C2=A0 &#39;(before-string #(&quot;AAA&quot; 0 3 (disp=
lay &quot;&quot;))<br><br>This overlay is effectively a no-op for redisplay=
 - the &quot;AAA&quot; string that would have been printed at the start of =
the overlay interval is not printed because its presentation is suppressed =
by the string&#39;s &#39;display property. This is right and proper.<br><br=
>However, `compute-motion&#39; appears to interpret this overlay as having =
an effective visible length of -3 rather than 0. Ditto if the overlay had i=
nstead had an &#39;after-string property with the same content. [*] The eff=
ective length being negative suggests to me an overcorrection rather than m=
issing logic.<br><br>([*] Aside: I believe `compute-motion&#39; sometimes a=
lso miscalculates &#39;display as an overlay property for cases where one m=
ight reasonably expect it to work- but I limit this report to &#39;before-s=
tring and &#39;after-string.)<br><br>Popular packages such as flycheck and =
git-gutter use this pattern of overlay to annotate the margin and the fring=
e. The pattern is increasingly common which in turn increasingly limits the=
 application of `compute-motion&#39; as a reliable exposed function.<br><br=
>The miscounting appears to exist at least as far back as Emacs 24. Might t=
here be a mitigation/workaround? I like `compute-motion&#39; despite its co=
mplicated signature and complicated return value (and despite its apparentl=
y rare use in the extended ecosystem) - it&#39;s fast and it involves no cu=
rsor movement to perform its measurements.<br><br>Thanks for reading.<div><=
br></div><div><br></div><div>Testcase output:</div><div>=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D<br><br>-------------------------------------<br>Wi=
thout overlay:<br>- =E2=80=98point=E2=80=99 at col 30: &lt;31&gt;<br>- buff=
er content on and after col 30: &lt;30 =C2=A0 35 =C2=A0 40 =C2=A0 45 =C2=A0=
 50&gt;<br>- posn coord at col 30: &lt;(30 . 0)&gt;<br>- buffer pos of col =
30 according to =E2=80=98compute-motion=E2=80=99: &lt;31&gt; =C2=A0 =C2=A0 =
[ok]<br>- coord of col 30 according to =E2=80=98compute-motion=E2=80=99: &l=
t;(30 . 0)&gt;<br>-------------------------------------<br>With =E2=80=99be=
fore-string overlay:<br>- =E2=80=98point=E2=80=99 at col 30: &lt;31&gt;<br>=
- buffer content on and after col 30: &lt;30 =C2=A0 35 =C2=A0 40 =C2=A0 45 =
=C2=A0 50&gt;<br>- posn coord at col 30: &lt;(30 . 0)&gt;<br>- buffer pos o=
f col 30 according to =E2=80=98compute-motion=E2=80=99: &lt;28&gt; =C2=A0 =
=C2=A0 &lt;&lt;&lt; INCONSISTENT<br>- coord of col 30 according to =E2=80=
=98compute-motion=E2=80=99: &lt;(30 . 0)&gt;<br>---------------------------=
----------<br>With =E2=80=99after-string overlay:<br>- =E2=80=98point=E2=80=
=99 at col 30: &lt;31&gt;<br>- buffer content on and after col 30: &lt;30 =
=C2=A0 35 =C2=A0 40 =C2=A0 45 =C2=A0 50&gt;<br>- posn coord at col 30: &lt;=
(30 . 0)&gt;<br>- buffer pos of col 30 according to =E2=80=98compute-motion=
=E2=80=99: &lt;28&gt; =C2=A0 =C2=A0 &lt;&lt;&lt; INCONSISTENT<br>- coord of=
 col 30 according to =E2=80=98compute-motion=E2=80=99: &lt;(30 . 0)&gt;<br>=
</div></div>

--000000000000446d8205a6690ec9--

--000000000000446d8405a6690ecb
Content-Type: application/octet-stream; name="compute-motion-testcase.el"
Content-Disposition: attachment; filename="compute-motion-testcase.el"
Content-Transfer-Encoding: base64
Content-ID: <f_kale2o7v0>
X-Attachment-Id: f_kale2o7v0

Ozs7IC0qLSBtb2RlOiBsaXNwLWludGVyYWN0aW9uIC0qLQoKOzsgU3VtbWFyeTogdXNlIGBjb21w
dXRlLW1vdGlvbicgdG8gbWVhc3VyZSB0aGUgYnVmZmVyIHBvc2l0aW9uIG9mIHRoZSB2aXN1YWwK
OzsgY29vcmRpbmF0ZSAnKDAgLiAzMCkgaW4gYSBuZXcgYnVmZmVyIHdpdGggc29tZSBjb250ZW50
LiBIZXJlLCBjb2x1bW4gMzAgaXMKOzsgYXQgKGFmdGVyKSBidWZmZXIgcG9zaXRpb24gMzEuCjs7
Cjs7IGBjb21wdXRlLW1vdGlvbicgY2FsY3VsYXRlcyB0aGlzIGNvcnJlY3RseSB1bmxlc3MgaXQg
c2NhbnMgb3ZlciBhCjs7ICdiZWZvcmUtc3RyaW5nIG9yICdhZnRlci1zdHJpbmcgb3ZlcmxheSBv
ZiB0aGUgc3R5bGUgc2hvd24gaW4KOzsgYHRlc3RjYXNlLWFwcGx5LW92ZXJsYXknIGluIHdoaWNo
IGNhc2UgaXQgcmV0dXJucyBidWZmZXIgcG9zaXRpb24gMjguCgooZGVmdW4gdGVzdGNhc2UtcmVj
cmVhdGUtYnVmZmVyICh0ZXN0Y2FzZS1idWZmZXIpCiAgKHdoZW4gKGdldC1idWZmZXIgdGVzdGNh
c2UtYnVmZmVyKQogICAgKGtpbGwtYnVmZmVyIHRlc3RjYXNlLWJ1ZmZlcikpCiAgKHdpdGgtY3Vy
cmVudC1idWZmZXIgKGdldC1idWZmZXItY3JlYXRlIHRlc3RjYXNlLWJ1ZmZlcikKICAgIChpbnNl
cnQgIiAgICAgNSAgICAxMCAgIDE1ICAgMjAgICAyNSAgIDMwICAgMzUgICA0MCAgIDQ1ICAgNTBc
biIpCiAgICAoZ290by1jaGFyIChwb2ludC1taW4pKSkpCgooZGVmdW4gdGVzdGNhc2UtbWVhc3Vy
ZS1hdC1jb2wtMzAgKHRlc3RjYXNlLWJ1ZmZlcikKICAobGV0ICgocHJldi1idWZmZXIgKGN1cnJl
bnQtYnVmZmVyKSkpCiAgICAoc3dpdGNoLXRvLWJ1ZmZlciB0ZXN0Y2FzZS1idWZmZXIpCiAgICAo
Z290by1jaGFyIChwb2ludC1taW4pKQogICAgKG1vdmUtdG8tY29sdW1uIDMwKQogICAgKHJlZGlz
cGxheSkgIDsgaGFjazogbWFrZSBwb3NuLWF0LXBvaW50IHJldHVybiBub24tbmlsCiAgICAobWVz
c2FnZSAiLSBgcG9pbnQnIGF0IGNvbCAzMDogPCVzPiIgKHBvaW50KSkKICAgIChtZXNzYWdlICIt
IGJ1ZmZlciBjb250ZW50IG9uIGFuZCBhZnRlciBjb2wgMzA6IDwlcz4iCiAgICAgICAgICAgICAo
YnVmZmVyLXN1YnN0cmluZyAocG9pbnQpIChsaW5lLWVuZC1wb3NpdGlvbikpKQogICAgKG1lc3Nh
Z2UgIi0gcG9zbiBjb29yZCBhdCBjb2wgMzA6IDwlcz4iIChwb3NuLWNvbC1yb3cgKHBvc24tYXQt
cG9pbnQpKSkKICAgIChwY2FzZS1sZXQgKChgKCxwb3MgLGNvbCAscm93IF8gXykKICAgICAgICAg
ICAgICAgICAoY29tcHV0ZS1tb3Rpb24gKHBvaW50LW1pbikKICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgJygwIC4gMCkKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKHBv
aW50LW1heCkKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKHBvc24tY29sLXJvdyAo
cG9zbi1hdC1wb2ludCkpCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5pbCBuaWwg
bmlsKSkpCiAgICAgIChtZXNzYWdlICItIGJ1ZmZlciBwb3Mgb2YgY29sIDMwIGFjY29yZGluZyB0
byBgY29tcHV0ZS1tb3Rpb24nOiA8JXM+ICVzIgogICAgICAgICAgICAgICBwb3MKICAgICAgICAg
ICAgICAgKGlmIChlcSBwb3MgKHBvaW50KSkKICAgICAgICAgICAgICAgICAgICIgICAgW29rXSIK
ICAgICAgICAgICAgICAgICAiICAgIDw8PCBJTkNPTlNJU1RFTlQiKSkKICAgICAgKG1lc3NhZ2Ug
Ii0gY29vcmQgb2YgY29sIDMwIGFjY29yZGluZyB0byBgY29tcHV0ZS1tb3Rpb24nOiA8JXM+Igog
ICAgICAgICAgICAgICAoY29ucyBjb2wgcm93KSkpCiAgICAoc3dpdGNoLXRvLWJ1ZmZlciBwcmV2
LWJ1ZmZlcikpKQoKKGRlZnVuIHRlc3RjYXNlLWFwcGx5LW92ZXJsYXkgKGJ1ZiBvdmVybGF5LXBv
cyBwcm9wLXN5bSkKICAod2l0aC1jdXJyZW50LWJ1ZmZlciBidWYKICAgIChsZXQqICgob2wgKG1h
a2Utb3ZlcmxheSBvdmVybGF5LXBvcyBvdmVybGF5LXBvcwogICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIChjdXJyZW50LWJ1ZmZlcikgbmlsIG5pbCkpKQogICAgICAob3ZlcmxheS1wdXQgb2wK
ICAgICAgICAgICAgICAgICAgIHByb3Atc3ltCiAgICAgICAgICAgICAgICAgICAocHJvcGVydGl6
ZSAiQUFBIiAnZGlzcGxheSAiIikpCiAgICAgIG9sKSkpCgoobGV0ICgodGVzdGNhc2UtYnVmZmVy
ICIqY29tcHV0ZS1tb3Rpb24gYnVnKiIpKQogIChtZXNzYWdlICJcbj09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PVxuIikKICAobGV0ICgocHJldi1idWZmZXIgKGN1cnJlbnQt
YnVmZmVyKSkpCiAgICAobWVzc2FnZSAiLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLSIpCiAgICAobWVzc2FnZSAiV2l0aG91dCBvdmVybGF5OiIpCiAgICAodGVzdGNhc2UtcmVj
cmVhdGUtYnVmZmVyIHRlc3RjYXNlLWJ1ZmZlcikKICAgICh0ZXN0Y2FzZS1tZWFzdXJlLWF0LWNv
bC0zMCB0ZXN0Y2FzZS1idWZmZXIpCgogICAgKG1lc3NhZ2UgIi0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0iKQogICAgKG1lc3NhZ2UgIldpdGggJ2JlZm9yZS1zdHJpbmcgb3Zl
cmxheToiKQogICAgKHRlc3RjYXNlLXJlY3JlYXRlLWJ1ZmZlciB0ZXN0Y2FzZS1idWZmZXIpCiAg
ICAodGVzdGNhc2UtYXBwbHktb3ZlcmxheSB0ZXN0Y2FzZS1idWZmZXIgMjAgJ2JlZm9yZS1zdHJp
bmcpCiAgICAodGVzdGNhc2UtbWVhc3VyZS1hdC1jb2wtMzAgdGVzdGNhc2UtYnVmZmVyKQoKICAg
IChtZXNzYWdlICItLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tIikKICAgICht
ZXNzYWdlICJXaXRoICdhZnRlci1zdHJpbmcgb3ZlcmxheToiKQogICAgKHRlc3RjYXNlLXJlY3Jl
YXRlLWJ1ZmZlciB0ZXN0Y2FzZS1idWZmZXIpCiAgICAodGVzdGNhc2UtYXBwbHktb3ZlcmxheSB0
ZXN0Y2FzZS1idWZmZXIgMjAgJ2FmdGVyLXN0cmluZykKICAgICh0ZXN0Y2FzZS1tZWFzdXJlLWF0
LWNvbC0zMCB0ZXN0Y2FzZS1idWZmZXIpCgogICAgKHN3aXRjaC10by1idWZmZXIgcHJldi1idWZm
ZXIpKQogIG5pbCkKCg==
--000000000000446d8405a6690ecb--




Acknowledgement sent to Stephen Bach <sjbach@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#41513; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 May 2020 20:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.