GNU bug report logs - #37467
26.3; whitespace-mode doesn't update BOB/EOB highlights

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Dale Sedivec <dale@HIDDEN>; merged with #42110; dated Fri, 20 Sep 2019 05:31:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.
Forcibly Merged 37467 42110. Request was from "Basil L. Contovounesios" <contovob@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
bug Marked as found in versions 25.3. Request was from "Basil L. Contovounesios" <contovob@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 37467 <at> debbugs.gnu.org:


Received: (at 37467) by debbugs.gnu.org; 9 Oct 2019 13:42:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 09 09:42:27 2019
Received: from localhost ([127.0.0.1]:52287 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iICEU-0005ZA-VL
	for submit <at> debbugs.gnu.org; Wed, 09 Oct 2019 09:42:27 -0400
Received: from mail-qt1-f178.google.com ([209.85.160.178]:41710)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <cseberino@HIDDEN>) id 1iICES-0005Yw-Pz
 for 37467 <at> debbugs.gnu.org; Wed, 09 Oct 2019 09:42:25 -0400
Received: by mail-qt1-f178.google.com with SMTP id v52so3433739qtb.8
 for <37467 <at> debbugs.gnu.org>; Wed, 09 Oct 2019 06:42:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=JUTf6BLgJexqUVX7S2BFRR1O2q6H3GsIxugxiS9HHpo=;
 b=NQZwVih09HGY9rmjk9d1Y89CaRidDkdBspUPCF78+XiY/UNeEsQHs4E9u3SmwLiyVy
 ssuM74YJC+fqugyMwIb2aauR9r9j6ZeEU/yaOE1pHADKa3yivFa29cGBOOACNLByugND
 JZdSFVYWuEtKuyL5GOuT7q9AOpKIRlkkqUjfdUU7DOqNoYP0gvupNFGb8htJt0GvOX+e
 FyCCRvkeXjAb8C5UMi7T5JRDsjMOarAFk9PKJFVmqiOpJlM7RAuHbJJIMW/4tQpe2SJ+
 y00CLNYzAJTmprI3zZTCx7Jm+oLkaFyyhbidmu77AX/LNPpZkRHYzZi00U52ABzYOKnl
 UWtA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=JUTf6BLgJexqUVX7S2BFRR1O2q6H3GsIxugxiS9HHpo=;
 b=p+RVNeLJM1//TT8/mjwtCOGLhjp2QdjkD13hIy8nKKR44EjA2Co3vls5dXBQoNbgsG
 mutGh8hm4NfpgUQFMX5piWFDqvmd+DWVsKVslrrYHLD40RsYDMUCLsoaBf+cLawZAG3n
 LVaeMconmM1FTGetdACIVjili6fPWEv8s33fCJqoyIO3atja8g2zbKvTPlgXPRSQbXMx
 ehYMWK2zJKmfM0bvbm67fZ8yI/g3VKNiY2vZWTRfbnYwJDPirkofmYcGpLW7OOhq8+ik
 xRkp+1hBuod0t/N7FH93Hp+93+ZgSaUP9of3bjCShBLxdUArG2u7WBkh/esSd+vGH1DK
 WTIA==
X-Gm-Message-State: APjAAAXv1G4xJGpjUHD0Y6e62O26VCzQHdbB6M0Ka3yO11Qoz0GEDEIp
 4Cpi7xD1nM+yhETrL55r1zujFwnPcubtRfkV2BvLNoq1QXs=
X-Google-Smtp-Source: APXvYqx1kYXD7B0l3UlJTi7x2Xzq6Xl10DEv6Ke0rOVybpeX+D4z9GcKSmA/jiULAOOVpQwMCwsIkc5fCge6bLysu7o=
X-Received: by 2002:a0c:e606:: with SMTP id z6mr3560403qvm.135.1570628538694; 
 Wed, 09 Oct 2019 06:42:18 -0700 (PDT)
MIME-Version: 1.0
From: Christian Seberino <cseberino@HIDDEN>
Date: Wed, 9 Oct 2019 08:42:07 -0500
Message-ID: <CAG5-5iJgf07BWx7sQR3x+be0Qf1B8tHgL758_c-sz18ChUTG2w@HIDDEN>
Subject: info from whitespace mode author to help fix bug
To: 37467 <at> debbugs.gnu.org
Content-Type: multipart/alternative; boundary="0000000000001f7e5805947a74c7"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 37467
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--0000000000001f7e5805947a74c7
Content-Type: text/plain; charset="UTF-8"

(Basically the EmacsWiki version does NOT show this bug but follow on
changes added this bug.)

From Vinicius Latorre.....

---------------------

I've just tested the steps in bug report #37467, my results:

1. Using whitespace-mode distributed with Emacs 27.0.50, the bug happens.

2. Using whitespace-mode in EmacsWiki, the bug does not happen.

I didn't see in deep to verify exactly what is going on.

Vinicius

PS1: The whitespace.el file distributed with Emacs has a lot of
modifications, probably it could start with whitespace.el in EmacsWiki and
adding one by one the modifications until reach the bug.

PS2: I saw that all save-match-data was removed from whitespace.el, this
could be a starting point, that is, add again save-match-data at original
point and check if the bug happens.

-----------------

--0000000000001f7e5805947a74c7
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=
=3D"gmail_signature"><div>(Basically the EmacsWiki version does NOT show th=
is bug but follow on changes added this bug.)</div><div><br></div><div>From=
 Vinicius Latorre.....</div><div><br></div><div>---------------------</div>=
<div dir=3D"ltr"><br></div><div dir=3D"ltr"><div>I&#39;ve just tested the s=
teps in bug report #37467, my results:<br><br></div><div>1. Using whitespac=
e-mode distributed with Emacs 27.0.50, the bug happens.</div><div><br></div=
><div>2. Using whitespace-mode in EmacsWiki, the bug does not happen.</div>=
<div><br></div><div>I didn&#39;t see in deep to verify exactly what is goin=
g on.</div><div><br></div><div>Vinicius</div><div><br></div><div>PS1: The w=
hitespace.el file distributed with Emacs has a lot of modifications, probab=
ly it could start with whitespace.el in EmacsWiki and adding one by one the=
 modifications until reach the bug.</div><div><br></div><div>PS2: I saw tha=
t all save-match-data was removed from whitespace.el, this could be a start=
ing point, that is, add again save-match-data at original point and check i=
f the bug happens.</div><div><br></div><div>-----------------</div></div></=
div></div>

--0000000000001f7e5805947a74c7--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#37467; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 20 Sep 2019 05:30:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 20 01:30:18 2019
Received: from localhost ([127.0.0.1]:56630 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iBBUn-0005Ea-TJ
	for submit <at> debbugs.gnu.org; Fri, 20 Sep 2019 01:30:18 -0400
Received: from lists.gnu.org ([209.51.188.17]:47133)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dale@HIDDEN>) id 1iBBUl-0005ES-NW
 for submit <at> debbugs.gnu.org; Fri, 20 Sep 2019 01:30:16 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:40373)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <dale@HIDDEN>) id 1iBBUj-0008MJ-Go
 for bug-gnu-emacs@HIDDEN; Fri, 20 Sep 2019 01:30:15 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED
 autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <dale@HIDDEN>) id 1iBBUh-00039p-GY
 for bug-gnu-emacs@HIDDEN; Fri, 20 Sep 2019 01:30:13 -0400
Received: from mail-io1-xd31.google.com ([2607:f8b0:4864:20::d31]:46827)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <dale@HIDDEN>) id 1iBBUh-00039M-2E
 for bug-gnu-emacs@HIDDEN; Fri, 20 Sep 2019 01:30:11 -0400
Received: by mail-io1-xd31.google.com with SMTP id c6so211705ioo.13
 for <bug-gnu-emacs@HIDDEN>; Thu, 19 Sep 2019 22:30:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=codefu-org.20150623.gappssmtp.com; s=20150623;
 h=from:content-transfer-encoding:mime-version:subject:message-id:date
 :to; bh=cZIAgbcJnO62DRZBkQ6Rh+GSLICKYngsuwVxqrA/5tI=;
 b=PB7iv8VSY5vuFKLyIciKUz/sXLdHRQgS9d5YyTQSFccPSgBPW0SagyElPP4toxpgdv
 XgWvzRqDGxxf4YohZjm/eMfEyKNFU1JmOJol/BAM8Y9lK3ARSXr/Yf12crycYiiiIEbu
 GnT0pMThD2IyJD2+EjTXfyV66a91ZsL+e4Dr2ftPaFbDBrT89gZUmobfTdtm2QeVY1Fs
 8QJBMLd8KJTtE2LTKophpMzf3SavyuursTRvDzQq2tKPL3v/mBEiOx5BLdcM9lGfcFAA
 0TQ717vOmsZbkDPEjgLT355QmZjHmB9i6X/dq1fNEbuWIX4SVFQ6ep7gVxxYtBfO+NFo
 pJag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:content-transfer-encoding:mime-version
 :subject:message-id:date:to;
 bh=cZIAgbcJnO62DRZBkQ6Rh+GSLICKYngsuwVxqrA/5tI=;
 b=cVgpG5qLgT4YrDYdA+cVXlfzFMmaeeZCbEl0JGAxHztZyzl/eDv+KXgzQMlGXyuOH/
 PdbFqXAZ4vFGclqlLMyMKdAfaqLpByj2U+mtoyGLoiyOfTaaMjwDj2ngZgjuCEA8c/jF
 mgC4kOuYoA20zssNyWUYaxPbGA943BTYbltC30uSAE0R06x0LgpvS1LV87L6s/iOfsGi
 Wk5OEy/OzQ4Vs0OFRHY6vZnp5uQ5ViB3b9UTHgxH79QKg+6pkZm+fdH6KmK9YwC0g1XG
 0nd/VP1RTGA6IQUAd5h0TqIWmBeZNUIBFA7hk9+VWvBirvO5TLszeVdO6K4NGoswzph1
 CHyw==
X-Gm-Message-State: APjAAAWbr/CJnp5yNqQAJU96gsAaElJ/66ZmKR0k9znJE9AcvAaMLdbu
 yp2cLqthVTeKblrgnxlbg0QkYfvkBOM0uQ==
X-Google-Smtp-Source: APXvYqylO7aotl1bVIz8eZnaS/waI0nHzQw6SkCXiV1phGXaht6nVDL05FRXVKR/3zoCV8PsjkcJAQ==
X-Received: by 2002:a6b:6110:: with SMTP id v16mr462206iob.199.1568957408930; 
 Thu, 19 Sep 2019 22:30:08 -0700 (PDT)
Received: from dale.caliginous.net (152.160.30.136.in-addr.arpa.
 [136.30.160.152])
 by smtp.gmail.com with ESMTPSA id l19sm725132iok.14.2019.09.19.22.30.08
 for <bug-gnu-emacs@HIDDEN>
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 19 Sep 2019 22:30:08 -0700 (PDT)
From: Dale Sedivec <dale@HIDDEN>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: 26.3; whitespace-mode doesn't update BOB/EOB highlights
Message-Id: <E96F6787-90A2-4841-8A5D-5D3215A84945@HIDDEN>
Date: Fri, 20 Sep 2019 00:30:07 -0500
To: bug-gnu-emacs@HIDDEN
X-Mailer: Apple Mail (2.3445.104.11)
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 2607:f8b0:4864:20::d31
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

whitespace-mode doesn't seem to un-highlight empty lines at BOB/EOB if =
you insert non-white-space characters in the middle of a run of blank =
lines.

Steps to reproduce:

1. emacs -Q

2. M-x whitespace-mode RET in the *scratch* buffer you start in.

3. Type C-2 RET C-2 C-p.  The bottom three lines of the buffer are now =
blank lines, point is on the penultimate line, and whitespace-mode has =
thus correctly highlighted the three blank lines at EOB with face =
whitespace-empty.

4. Type x.

To be clear, the buffer contents are now as follows, with point at "|":

~~~~~~
;; This buffer is for text that is not saved, and for Lisp evaluation.
;; To create a file, visit it with C-x C-f and enter text in its buffer.

x|

~~~~~~

Expected results: The bottom line is highlighted by whitespace-mode, no =
other lines are highlighted.

Observed results: The bottom line is correctly highlighted, and the =
highlighting is cleared from line 4 with the "x", but line 3 remains =
highlighted with face whitespace-empty.

There is a second bug too:

5. Type <backspace> to delete the x.

Expected results: The bottom lines 3=E2=80=935 are now highlighted with =
whitespace-empty.

Observed results: Only the bottom two blank lines 4=E2=80=935 are =
highlighted with whitespace-empty.  The adjacent blank line 3 is not =
highlighted.

Note that you can get line 3 correctly re-highlighted by moving to EOB =
then up one line (M-> C-p).

I believe you can reproduce this same behavior at the beginning of the =
buffer as well, using similar steps.

These bugs may seem obscure, but I hit them surprisingly often at EOB =
when programming: I enter a few blank lines at EOB as I prepare to =
append code to the buffer, but then stop and move upwards to make some =
changes, before returning to near-EOB and trying to enter some lines, at =
which point I am left with some incorrect whitespace-mode highlights.

My first guess at fixing this (only at EOB so far, not BOB) was to use =
font lock multiline:

* I set the font-lock-multiline property non-nil in =
whitespace-font-lock-keywords for whitespace-empty-at-eob-regexp.

* I make whitespace-mode install a new buffer-local function to =
font-lock-extend-region-functions to correctly identify the region for =
fontification at EOB.  (Note that (whitespace-looking-back =
whitespace-empty-at-eob-regexp nil) was handy here.)

However, this only solved the first problem, removing highlighting as =
desired at the end of step 4, above.  I have not yet been able to solve =
the problem of re-highlighting line 3 at the end of my step 5.  =
whitespace-post-command-hook is doing some work to determine what to =
refontify, and understanding of that function has thus far eluded me.  I =
suspect a solution for one or both of these bugs lies in there, however.

Thanks for your consideration!

Dale


In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.7.0, NS appkit-1671.60 =
Version 10.14.6 (Build 18G87))
of 2019-08-29 built on mojave.internal.macports.net
Windowing system distributor 'Apple', version 10.3.1671
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
'configure --prefix=3D/opt/local --without-dbus --without-gconf
--without-libotf --without-m17n-flt --without-gpm --with-gnutls
--with-xml2 --with-modules --infodir /opt/local/share/info/emacs
--with-ns --with-lcms2 'CFLAGS=3D-pipe -Os
=
-isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platf=
orm/Developer/SDKs/MacOSX10.14.sdk
-arch x86_64' 'CPPFLAGS=3D-I/opt/local/include
=
-isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platf=
orm/Developer/SDKs/MacOSX10.14.sdk'
'LDFLAGS=3D-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-no_pie
=
-Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOS=
X.platform/Developer/SDKs/MacOSX10.14.sdk
-arch x86_64''

Configured features:
JPEG NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES
THREADS LCMS2

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue cocoa ns
lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 203911 12801)
(symbols 48 20094 2)
(miscs 40 70 181)
(strings 32 28190 1477)
(string-bytes 1 764094)
(vectors 16 35016)
(vector-slots 8 723944 9708)
(floats 8 48 69)
(intervals 56 236 0)
(buffers 992 12))




Acknowledgement sent to Dale Sedivec <dale@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#37467; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 28 Jun 2020 16:30:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.