GNU bug report logs - #37467
26.3; whitespace-mode doesn't update BOB/EOB highlights

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37467 in the body.
You can then email your comments to 37467 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dale Sedivec <dale <at> codefu.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.3; whitespace-mode doesn't update BOB/EOB highlights
Date: Fri, 20 Sep 2019 00:30:07 -0500

whitespace-mode doesn't seem to un-highlight empty lines at BOB/EOB if you insert non-white-space characters in the middle of a run of blank lines.

Steps to reproduce:

1. emacs -Q

2. M-x whitespace-mode RET in the *scratch* buffer you start in.

3. Type C-2 RET C-2 C-p.  The bottom three lines of the buffer are now blank lines, point is on the penultimate line, and whitespace-mode has thus correctly highlighted the three blank lines at EOB with face whitespace-empty.

4. Type x.

To be clear, the buffer contents are now as follows, with point at "|":

~~~~~~
;; This buffer is for text that is not saved, and for Lisp evaluation.
;; To create a file, visit it with C-x C-f and enter text in its buffer.

x|

~~~~~~

Expected results: The bottom line is highlighted by whitespace-mode, no other lines are highlighted.

Observed results: The bottom line is correctly highlighted, and the highlighting is cleared from line 4 with the "x", but line 3 remains highlighted with face whitespace-empty.

There is a second bug too:

5. Type <backspace> to delete the x.

Expected results: The bottom lines 3–5 are now highlighted with whitespace-empty.

Observed results: Only the bottom two blank lines 4–5 are highlighted with whitespace-empty.  The adjacent blank line 3 is not highlighted.

Note that you can get line 3 correctly re-highlighted by moving to EOB then up one line (M-> C-p).

I believe you can reproduce this same behavior at the beginning of the buffer as well, using similar steps.

These bugs may seem obscure, but I hit them surprisingly often at EOB when programming: I enter a few blank lines at EOB as I prepare to append code to the buffer, but then stop and move upwards to make some changes, before returning to near-EOB and trying to enter some lines, at which point I am left with some incorrect whitespace-mode highlights.

My first guess at fixing this (only at EOB so far, not BOB) was to use font lock multiline:

* I set the font-lock-multiline property non-nil in whitespace-font-lock-keywords for whitespace-empty-at-eob-regexp.

* I make whitespace-mode install a new buffer-local function to font-lock-extend-region-functions to correctly identify the region for fontification at EOB.  (Note that (whitespace-looking-back whitespace-empty-at-eob-regexp nil) was handy here.)

However, this only solved the first problem, removing highlighting as desired at the end of step 4, above.  I have not yet been able to solve the problem of re-highlighting line 3 at the end of my step 5.  whitespace-post-command-hook is doing some work to determine what to refontify, and understanding of that function has thus far eluded me.  I suspect a solution for one or both of these bugs lies in there, however.

Thanks for your consideration!

Dale


In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.7.0, NS appkit-1671.60 Version 10.14.6 (Build 18G87))
of 2019-08-29 built on mojave.internal.macports.net
Windowing system distributor 'Apple', version 10.3.1671
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
'configure --prefix=/opt/local --without-dbus --without-gconf
--without-libotf --without-m17n-flt --without-gpm --with-gnutls
--with-xml2 --with-modules --infodir /opt/local/share/info/emacs
--with-ns --with-lcms2 'CFLAGS=-pipe -Os
-isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
-arch x86_64' 'CPPFLAGS=-I/opt/local/include
-isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk'
'LDFLAGS=-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-no_pie
-Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
-arch x86_64''

Configured features:
JPEG NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES
THREADS LCMS2

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue cocoa ns
lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 203911 12801)
(symbols 48 20094 2)
(miscs 40 70 181)
(strings 32 28190 1477)
(string-bytes 1 764094)
(vectors 16 35016)
(vector-slots 8 723944 9708)
(floats 8 48 69)
(intervals 56 236 0)
(buffers 992 12))

Message #8 received at 37467 <at> debbugs.gnu.org (full text, mbox):

From: Christian Seberino <cseberino <at> gmail.com>
To: 37467 <at> debbugs.gnu.org
Subject: info from whitespace mode author to help fix bug
Date: Wed, 9 Oct 2019 08:42:07 -0500

[Message part 1 (text/plain, inline)]

(Basically the EmacsWiki version does NOT show this bug but follow on
changes added this bug.)

From Vinicius Latorre.....

---------------------

I've just tested the steps in bug report #37467, my results:

1. Using whitespace-mode distributed with Emacs 27.0.50, the bug happens.

2. Using whitespace-mode in EmacsWiki, the bug does not happen.

I didn't see in deep to verify exactly what is going on.

Vinicius

PS1: The whitespace.el file distributed with Emacs has a lot of
modifications, probably it could start with whitespace.el in EmacsWiki and
adding one by one the modifications until reach the bug.

PS2: I saw that all save-match-data was removed from whitespace.el, this
could be a starting point, that is, add again save-match-data at original
point and check if the bug happens.

-----------------

[Message part 2 (text/html, inline)]

Message #15 received at 37467 <at> debbugs.gnu.org (full text, mbox):

From: Richard Hansen <rhansen <at> rhansen.org>
To: 37467 <at> debbugs.gnu.org
Subject: [PATCH] whitespace: Redo BoB/EoB empty line highlighting
Date: Wed, 29 Jun 2022 17:28:16 -0400

[Message part 1 (text/plain, inline)]

The attached patch series should fix this bug.

Patch 1:

    ert-x: New `ert-with-test-buffer-selected' convenience macro

    * lisp/emacs-lisp/ert-x.el (ert-with-test-buffer-selected): New
    convenience macro that extends `ert-with-test-buffer' by displaying
    the test buffer in a temporary selected window.  This makes it easier
    to simulate user input in the body via `execute-kbd-macro'.
    * test/lisp/emacs-lisp/ert-x-tests.el
    (ert-test-test-buffer-selected/*): Add tests.

Patch 2:

    ; whitespace: Delete unused `whitespace-font-lock-refontify' var

Patch 3:

    ; whitespace: Use `defvar-local' for buffer-local vars

Patch 4:

    ; whitespace: Fix indentation

Patch 5:

    whitespace: Reset `whitespace-buffer-changed' when refontifying

    * lisp/whitespace.el (whitespace-post-command-hook): Add missing reset
    of `whitespace-buffer-changed' back to nil between commands.

Patch 6:

    whitespace: Include empty final line in BoB empty match

    * lisp/whitespace.el (whitespace-empty-at-bob-regexp): Include any
    last line trailing whitespace in the BoB empty line match to ensure
    that those characters get highlighted.

Patch 7:

    whitespace: Redo BoB/EoB empty line highlighting

    * lisp/whitespace.el (whitespace--empty-at-bob-matcher,
    whitespace--empty-at-eob-matcher, whitespace--update-bob-eob,
    whitespace-color-off, whitespace-color-on,
    whitespace-empty-at-bob-regexp, whitespace-empty-at-eob-regexp,
    whitespace-looking-back, whitespace-post-command-hook): Redo the
    `empty' line highlighting logic to ensure that a buffer change causes
    all affected `empty' lines to become (un)highlighted (bug#37467).
    Also, for improved UX, don't highlight BoB empty lines at or below
    point (not just when point is at 1), or EoB empty lines at or above
    point (not just when point is `eobp').
    (whitespace-bob-marker, whitespace-eob-marker): Clarify documentation.
    * test/lisp/whitespace-tests.el (whitespace--with-test-buffer,
    whitespace--fu, whitespace-tests--empty-bob,
    whitespace-tests--empty-eob): Add tests.

[0001-ert-x-New-ert-with-test-buffer-selected-convenience-.patch (text/x-patch, attachment)]

[0002-whitespace-Delete-unused-whitespace-font-lock-refont.patch (text/x-patch, attachment)]

[0003-whitespace-Use-defvar-local-for-buffer-local-vars.patch (text/x-patch, attachment)]

[0004-whitespace-Fix-indentation.patch (text/x-patch, attachment)]

[0005-whitespace-Reset-whitespace-buffer-changed-when-refo.patch (text/x-patch, attachment)]

[0006-whitespace-Include-empty-final-line-in-BoB-empty-mat.patch (text/x-patch, attachment)]

[0007-whitespace-Redo-BoB-EoB-empty-line-highlighting.patch (text/x-patch, attachment)]

[OpenPGP_signature (application/pgp-signature, attachment)]

Message #18 received at 37467 <at> debbugs.gnu.org (full text, mbox):

From: Richard Hansen <rhansen <at> rhansen.org>
To: 37467 <at> debbugs.gnu.org
Cc: contovob <at> tcd.ie, cseberino <at> gmail.com, rrt <at> sc3d.org,
 robin_choudhury1 <at> protonmail.com, dale <at> codefu.org
Subject: Patches to fix Emacs whitespace-mode BoB/EoB empty line highlighting
Date: Sun, 17 Jul 2022 00:49:25 -0400

[Message part 1 (text/plain, inline)]

I recently posted some patches [1] that should fix this bug.  Would one of you kind individuals be willing to review and test them?

Thanks,
Richard

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37467#15

[OpenPGP_signature (application/pgp-signature, attachment)]

Message #23 received at 37467 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Richard Hansen <rhansen <at> rhansen.org>
Cc: 37467 <at> debbugs.gnu.org, 42110 <at> debbugs.gnu.org
Subject: Re: bug#42110: Whitespace Newline Face Doesn't Disappear
Date: Sun, 11 Sep 2022 13:46:14 +0200

Richard Hansen <rhansen <at> rhansen.org> writes:

> The attached patch series should fix this bug.

Thanks.  I've now tested the patch series, and it seems to fix the
reported problems, so I've now pushed it to Emacs 29.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.