GNU bug report logs - #49624
compilation message end-column function off-by-one bug

Previous Next

Package: emacs;

Reported by: Mattias Engdegård <mattiase <at> acm.org>

Date: Sun, 18 Jul 2021 19:01:02 UTC

Severity: normal

Done: Mattias Engdegård <mattiase <at> acm.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 49624 in the body.
You can then email your comments to 49624 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Sun, 18 Jul 2021 19:01:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mattias Engdegård <mattiase <at> acm.org>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Sun, 18 Jul 2021 19:01:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: bug-gnu-emacs <at> gnu.org
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
Subject: compilation message end-column function off-by-one bug
Date: Sun, 18 Jul 2021 21:00:41 +0200
[Message part 1 (text/plain, inline)]
Compilation message patterns (in compilation-regexp-alist and -alist) can indicate starting and ending line and column numbers, either by supplying regexp match numbers or functions that return the respective line/column numbers when called. In other words, the integer N can be understood as a shorthand for the function

(lambda () (and (match-beginning N) (string-to-number (match-string N))))

except that isn't true for the ending column where there is a difference of 1; an END-COL function returning 13 means that the error's last column is 12.

There does not seem to be a good justification for this nor is it documented so it's probably just a bug; proposed patch attached below. Does it warrant a mention in etc/NEWS?

[0001-Fix-off-by-one-error-in-compilation-rule-end-column-.patch (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Sun, 18 Jul 2021 19:11:01 GMT) Full text and rfc822 format available.

Message #8 received at 49624 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 49624 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Sun, 18 Jul 2021 22:10:05 +0300
> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sun, 18 Jul 2021 21:00:41 +0200
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, Juri Linkov <juri <at> linkov.net>
> 
> Compilation message patterns (in compilation-regexp-alist and -alist) can indicate starting and ending line and column numbers, either by supplying regexp match numbers or functions that return the respective line/column numbers when called. In other words, the integer N can be understood as a shorthand for the function
> 
> (lambda () (and (match-beginning N) (string-to-number (match-string N))))
> 
> except that isn't true for the ending column where there is a difference of 1; an END-COL function returning 13 means that the error's last column is 12.

Isn't that because END-COL is the first column _beyond_ the last
column?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Sun, 18 Jul 2021 19:14:02 GMT) Full text and rfc822 format available.

Message #11 received at 49624 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 49624 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Sun, 18 Jul 2021 21:13:36 +0200
18 juli 2021 kl. 21.10 skrev Eli Zaretskii <eliz <at> gnu.org>:

> Isn't that because END-COL is the first column _beyond_ the last
> column?

Yes, that is probably the origin of the bug. However as that is just the internal representation (column range as a half-open interval) it should not be visible to the user.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Sun, 18 Jul 2021 19:31:01 GMT) Full text and rfc822 format available.

Message #14 received at 49624 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 49624 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Sun, 18 Jul 2021 22:30:23 +0300
> Feedback-ID:mattiase <at> acm.or
> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sun, 18 Jul 2021 21:13:36 +0200
> Cc: 49624 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
> 
> 18 juli 2021 kl. 21.10 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> > Isn't that because END-COL is the first column _beyond_ the last
> > column?
> 
> Yes, that is probably the origin of the bug. However as that is just the internal representation (column range as a half-open interval) it should not be visible to the user.

??? Why not?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Mon, 19 Jul 2021 10:41:02 GMT) Full text and rfc822 format available.

Message #17 received at 49624 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 49624 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca, juri <at> linkov.net
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Mon, 19 Jul 2021 12:39:59 +0200
18 juli 2021 kl. 21.30 skrev Eli Zaretskii <eliz <at> gnu.org>:

> ??? Why not?

Not sure what you mean (and I don't really think it's something we disagree about) but in any case it's beside the point. Emacs should behave as expected by the user, as documented, and in a consistent way. Currently it does neither.

The more I think about it, the clearer it becomes that it's a plain bug which I'm making too much fuss about and should just go ahead and fix the stupid thing. It was mainly reported on the unlikely chance that Juri and/or Stefan would remember something interesting about the code, but that's secondary.





Reply sent to Mattias Engdegård <mattiase <at> acm.org>:
You have taken responsibility. (Fri, 23 Jul 2021 13:26:02 GMT) Full text and rfc822 format available.

Notification sent to Mattias Engdegård <mattiase <at> acm.org>:
bug acknowledged by developer. (Fri, 23 Jul 2021 13:26:02 GMT) Full text and rfc822 format available.

Message #22 received at 49624-done <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 49624-done <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Fri, 23 Jul 2021 15:24:55 +0200
19 juli 2021 kl. 23.43 skrev Juri Linkov <juri <at> linkov.net>:

>> Not even 9dc3a46a444a46e00ed3287a3174d73ed9511dac where the funcall for col and end-col originated?
>> In any case, no worry -- I'll deal with it.
> 
> My apologies.  This commit is so old that I even don't recognize it.
> I need to study this code again since I don't remember this part
> of compile.el.

Thank you, but it's not that important. Let's not overthink it -- I'm treating it as the bug it is and have pushed the obvious fix to master.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Mon, 26 Jul 2021 17:18:02 GMT) Full text and rfc822 format available.

Message #25 received at 49624 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: 49624 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Mon, 26 Jul 2021 20:06:20 +0300
>>> Not even 9dc3a46a444a46e00ed3287a3174d73ed9511dac
>>> where the funcall for col and end-col originated?
>>> In any case, no worry -- I'll deal with it.
>>
>> My apologies.  This commit is so old that I even don't recognize it.
>> I need to study this code again since I don't remember this part
>> of compile.el.
>
> Thank you, but it's not that important.  Let's not overthink it -- I'm
> treating it as the bug it is and have pushed the obvious fix to master.

I'm terribly sorry that it took so much time for me to realize
that my commit 9dc3a46a444a46e00ed3287a3174d73ed9511dac
was part of efforts to add column information to grep matches,
also I needed to inspect the history of column-related code in compile.el
to understand the reason of incrementing the value of end-col.
Here are my findings:

The commit c0090c20f88d1e8c99e9823db5b9cc25d98672bc with the log message

    (compilation-error-properties): Store one more than end-col, if present, so
    that transient-mark-mode will highlight last char too.

turned an exclusive upper bound (e.g. [4, 6) that highlighted 2 chars)
into an inclusive upper bound (e.g. [4, 6] that highlighted 3 chars)
on the assumption that most compilation tools report inclusive ranges.

Then without changing this logic in 9dc3a46a444a46e00ed3287a3174d73ed9511dac
I added a funcall without incrementing its result by 1
on the assumption that the function can return
an already inclusive result that doesn't need to offset by 1.

Now your commit aa5437493b1ca539409495ecdc54cf420ea110b9
broke the highlighting of columns in grep-regexp-alist,
so now visiting a grep match highlights an additional character
that is not part of the grep match.

Maybe there are more existing functions whose backward-compatibility
is broken now.  For example,

    (javac
     ,...
     1 2
     ,#'current-column
     (3))

uses ,#'current-column although not for end-col, so it's not affected.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#49624; Package emacs. (Mon, 26 Jul 2021 19:32:01 GMT) Full text and rfc822 format available.

Message #28 received at 49624 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 49624 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#49624: compilation message end-column function off-by-one bug
Date: Mon, 26 Jul 2021 21:30:53 +0200
26 juli 2021 kl. 19.06 skrev Juri Linkov <juri <at> linkov.net>:

>  my commit 9dc3a46a444a46e00ed3287a3174d73ed9511dac
> was part of efforts to add column information to grep matches,

> I added a funcall without incrementing its result by 1
> on the assumption that the function can return
> an already inclusive result that doesn't need to offset by 1.

Thank you for the thorough investigation!

> Now your commit aa5437493b1ca539409495ecdc54cf420ea110b9
> broke the highlighting of columns in grep-regexp-alist,
> so now visiting a grep match highlights an additional character
> that is not part of the grep match.

Good spotting! That one has now been fixed.

> Maybe there are more existing functions whose backward-compatibility
> is broken now.

In preparation of my change I went through lots and lots of external packages and found exactly none using a function for END-COL. This is unsurprising since the very ability to use functions for COL and END-COL wasn't documented until 2019.

Somehow I missed grep.el, but that too makes sense -- it's inside Emacs itself that the feature was most likely to be used since it was undocumented. I have now gone through the source tree once more, and there seem to be no more problems.





bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Tue, 24 Aug 2021 11:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 239 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.