GNU bug report logs - #60830
30.0.50; The *Compilation* buffer does not recognize Lua errors

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 60830 in the body.
You can then email your comments to 60830 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; The *Compilation* buffer does not recognize Lua errors
Date: Sun, 15 Jan 2023 12:33:47 +0100

REPRODUCTION STEPS:

1. Write the following in some buffer:

/usr/bin/lua: database.lua:31: assertion failed!
stack traceback:
	[C]: in function 'assert'
	database.lua:31: in field 'statement'
	database.lua:42: in field 'table'
	database.lua:55: in field 'row'
	database.lua:63: in field 'value'
	database.lua:68: in main chunk
	[C]: in ?

2. Enable the `compilation-mode'.

EXPECTED:

The Compilation mode recognizes the following as error loci:
  
- database.lua:31: (twice)
- database.lua:42:
- database.lua:55:
- database.lua:63:
- database.lua:68:

ACTUAL:

The Compilation mode finds no error loci.

NOTES:

The source files can any extension, such as `fnl' for Fennel.

Rudy


In GNU Emacs 30.0.50 (build 19, aarch64-apple-darwin22.1.0, NS appkit-2299.00
 Version 13.0.1 (Build 22A400)) of 2023-01-11 built on Rudolfs-MacBook-Air.local
Repository revision: c51bfef34a6bdefad048444c279ea5b63a6aada1
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2299
System Description:  macOS 13.0.1

Configured features:
ACL GLIB GNUTLS JSON LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS PDUMPER PNG RSVG
SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: notmuch-hello

Minor modes in effect:
  global-hi-lock-mode: t
  hi-lock-mode: t
  global-hl-todo-mode: t
  global-diff-hl-mode: t
  savehist-mode: t
  shell-dirtrack-mode: t
  sly-symbol-completion-mode: t
  pixel-scroll-precision-mode: t
  delete-selection-mode: t
  global-goto-address-mode: t
  goto-address-mode: t
  global-subword-mode: t
  subword-mode: t
  save-place-mode: t
  global-auto-revert-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/Users/salutis/.emacs.d/elpa/modus-themes-20230112.503/theme-loaddefs hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/theme-loaddefs
/Users/salutis/.emacs.d/elpa/transient-20230107.1528/transient hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/transient

Features:
(shadow sort tempo mail-extr emacsbug notmuch notmuch-tree notmuch-jump
notmuch-hello notmuch-show notmuch-print notmuch-crypto notmuch-mua
notmuch-message notmuch-draft notmuch-maildir-fcc notmuch-address
notmuch-company notmuch-parser notmuch-wash coolj icalendar diary-lib
diary-loaddefs notmuch-tag crm notmuch-lib notmuch-version notmuch-compat
message yank-media rfc822 mml mailabbrev gmm-utils mailheader mm-view mml-smime
mml-sec epa epg rfc6068 epg-config gnus-util smime gnutls puny dig mm-decode
mm-bodies mm-encode mail-parse rfc2231 em-unix em-term term disp-table ehelp
em-script em-prompt em-ls em-hist em-pred em-glob em-extpipe em-cmpl em-dirs
esh-var em-basic em-banner em-alias esh-mode eshell esh-cmd esh-ext esh-opt
esh-proc esh-io esh-arg esh-module esh-groups esh-util cl-print misearch
multi-isearch dired-aux hl-line dired-x ls-lisp dired dired-loaddefs make-mode
fennel-mode fennel-eldoc inf-lisp files-x vc-svn vc-git lua-mode advice
char-fold paredit edmacro kmacro hi-lock hl-todo compat diff-hl log-view
pcvs-util vc-dir ewoc cus-start vc vc-dispatcher diff-mode orderless pdf-loader
finder-inf savehist org-clock ob-makefile ob-latex ob-scheme geiser-impl
help-fns radix-tree geiser-custom geiser-base geiser ob-dot ob-R ob-C cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
ob-shell shell ob-java ob-sqlite ob-sql ob-org ob-lisp modus-operandi-theme
modus-themes sly easy-mmode sly-completion sly-buttons sly-messages sly-common
apropos etags fileloop generator xref arc-mode archive-mode hyperspec whitespace
display-fill-column-indicator display-line-numbers flyspell ispell fortune
flymake-proc flymake project compile text-property-search pixel-scroll cua-base
comp derived comp-cstr pcase warnings cl-extra help-mode delsel goto-addr
thingatpt cap-words superword subword saveplace cus-edit pp cus-load wid-edit
bug-reference autorevert filenotify org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-src ob-comint org-pcomplete pcomplete comint ansi-osc
ansi-color ring org-list org-footnote org-faces org-entities time-date noutline
outline icons ob-emacs-lisp ob-core ob-eval org-cycle org-table ol rx org-fold
org-fold-core org-keys oc org-loaddefs find-func cal-menu calendar cal-loaddefs
org-version org-compat org-macs format-spec sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils mmm-mode-autoloads org-drill-autoloads
bbdb-autoloads diff-hl-autoloads paredit-autoloads markdown-mode-autoloads
vterm-autoloads swift-mode-autoloads lua-mode-autoloads htmlize-autoloads
yaml-mode-autoloads auctex-autoloads tex-site kotlin-mode-autoloads
hl-todo-autoloads geiser-guile-autoloads modus-themes-autoloads
flymake-grammarly-autoloads grammarly-autoloads emms-autoloads
bnf-mode-autoloads fennel-mode-autoloads rainbow-mode-autoloads
request-autoloads magit-autoloads magit-section-autoloads git-commit-autoloads
transient-autoloads geiser-autoloads telega-autoloads
rainbow-identifiers-autoloads pdf-tools-autoloads sql-indent-autoloads
citar-autoloads citeproc-autoloads queue-autoloads tablist-autoloads
cape-autoloads hide-mode-line-autoloads orderless-autoloads
elfeed-tube-mpv-autoloads mpv-autoloads string-inflection-autoloads
visual-fill-column-autoloads parsebib-autoloads mentor-autoloads
url-scgi-autoloads async-autoloads xml-rpc-autoloads ebnf-mode-autoloads
graphviz-dot-mode-autoloads f-autoloads dash-autoloads sly-autoloads
persist-autoloads ess-autoloads websocket-autoloads sqlup-mode-autoloads
elfeed-tube-autoloads aio-autoloads elfeed-autoloads password-store-autoloads
with-editor-autoloads info compat-autoloads s-autoloads package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x map
byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic indonesian
philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet
lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop
case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded
button loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule custom widget
keymap hashtable-print-readable backquote threads kqueue cocoa ns lcms2
multi-tty make-network-process emacs)

Memory information:
((conses 16 705152 30738)
 (symbols 48 37553 37)
 (strings 32 201806 6173)
 (string-bytes 1 5391755)
 (vectors 16 61253)
 (vector-slots 8 827501 35559)
 (floats 8 220 464)
 (intervals 56 7615 148)
 (buffers 984 24))

-- 
"Be especially critical of any statement following the word
'obviously.'"
-- Anna Pell Wheeler, 1883-1966

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #10 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 60830 <at> debbugs.gnu.org
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Wed, 13 Sep 2023 07:55:30 -0700

Rudolf Adamkovič <salutis <at> me.com> writes:

> REPRODUCTION STEPS:
>
> 1. Write the following in some buffer:
>
> /usr/bin/lua: database.lua:31: assertion failed!
> stack traceback:
> 	[C]: in function 'assert'
> 	database.lua:31: in field 'statement'
> 	database.lua:42: in field 'table'
> 	database.lua:55: in field 'row'
> 	database.lua:63: in field 'value'
> 	database.lua:68: in main chunk
> 	[C]: in ?
>
> 2. Enable the `compilation-mode'.
>
> EXPECTED:
>
> The Compilation mode recognizes the following as error loci:
>
> - database.lua:31: (twice)
> - database.lua:42:
> - database.lua:55:
> - database.lua:63:
> - database.lua:68:
>
> ACTUAL:
>
> The Compilation mode finds no error loci.
>
> NOTES:
>
> The source files can any extension, such as `fnl' for Fennel.

This breaks because it doesn't follow the GNU standard's

      [PROGRAM:]FILE:LINE: MESSAGE

Any chance we could get the lua developers to follow that format,
instead?

See also:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58984#26

Message #13 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 60830 <at> debbugs.gnu.org
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Mon, 02 Oct 2023 14:04:01 +0200

Stefan Kangas <stefankangas <at> gmail.com> writes:

> This breaks because it doesn't follow the GNU standard's
>
>       [PROGRAM:]FILE:LINE: MESSAGE
>
> Any chance we could get the lua developers to follow that format,
> instead?

Thank you for the reply, Stefan.

So, we seem to have 2 problems here:

(1) the first line contains a space character after the 'PROGRAM:'
segment, which Lua developers might be willing to change perhaps,

and

(2) the stack trace lines are indented, which I doubt Lua developers
would want to change.

Would it make sense to ask Lua developers about fixing (1) only, or
would fixing (2) in Emacs also "magically" fix (1), if Emacs has to
handle Lua specifically anyway?

Rudy
-- 
"Mathematics takes us still further from what is human into the region
of absolute necessity, to which not only the actual world, but every
possible world, must conform."
-- Bertrand Russell, 1902

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #16 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 60830 <at> debbugs.gnu.org
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Mon, 2 Oct 2023 11:37:37 -0700

Rudolf Adamkovič <salutis <at> me.com> writes:

> Stefan Kangas <stefankangas <at> gmail.com> writes:
>
>> This breaks because it doesn't follow the GNU standard's
>>
>>       [PROGRAM:]FILE:LINE: MESSAGE
>>
>> Any chance we could get the lua developers to follow that format,
>> instead?
>
> Thank you for the reply, Stefan.
>
> So, we seem to have 2 problems here:
>
> (1) the first line contains a space character after the 'PROGRAM:'
> segment, which Lua developers might be willing to change perhaps,
>
> and
>
> (2) the stack trace lines are indented, which I doubt Lua developers
> would want to change.
>
> Would it make sense to ask Lua developers about fixing (1) only, or
> would fixing (2) in Emacs also "magically" fix (1), if Emacs has to
> handle Lua specifically anyway?

AFAICT, all that is needed is to fix the formatting you describe in (1).
That will give the desired behavior.

Regarding (2), I think compilation-mode already detects those as file
locations, with no changes.  At least it does on current master.

Message #19 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 60830 <at> debbugs.gnu.org
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Tue, 03 Oct 2023 10:12:05 +0200

[Message part 1 (text/plain, inline)]

Stefan Kangas <stefankangas <at> gmail.com> writes:

> [...], all that is needed is to fix the formatting you describe in
> (1). That will give the desired behavior. [...] Regarding (2), I think
> compilation-mode already detects those as file locations, with no
> changes.  At least it does on current master.

As for (2), I recompiled Emacs from 'master' (498d31e9f05) to verify
your claim, but it does seem to hold.  I can reproduce the bug under
'emacs -Q' by following the reproduction steps.

However!  I did some thinking and realized two important facts:

(A) Even if the GNU standard supported tab-indented errors, as per (2),
it would not help, as Lua stack frames are not errors.

(B) People use older Lua versions, which will work forever, as they are
written in ANSI C, and so even if (1) is fixed upstream, we must support
the current error format as well.

After I realized that (A) and (B) are true, I used my morning to write a
patch that solves the problem properly.  I am attaching the patch to
this message.  Do you think the patch could be merged upstream?

Thank you for your time!

Rudy

[0001-Make-the-Compilation-mode-recognize-Lua-errors.patch (text/x-patch, attachment)]

[Message part 3 (text/plain, inline)]

-- 
"Chop your own wood and it will warm you twice."
-- Henry Ford; Francis Kinloch, 1819; Henry David Thoreau, 1854

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #22 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 60830 <at> debbugs.gnu.org
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Tue, 3 Oct 2023 02:37:36 -0700

Rudolf Adamkovič <salutis <at> me.com> writes:

> As for (2), I recompiled Emacs from 'master' (498d31e9f05) to verify
> your claim, but it does seem to hold.  I can reproduce the bug under
> 'emacs -Q' by following the reproduction steps.
>
> However!  I did some thinking and realized two important facts:
>
> (A) Even if the GNU standard supported tab-indented errors, as per (2),
> it would not help, as Lua stack frames are not errors.

Yes, I noticed that too.  Is it a regression with regards to earlier
versions of Emacs?

> (B) People use older Lua versions, which will work forever, as they are
> written in ANSI C, and so even if (1) is fixed upstream, we must support
> the current error format as well.

I see the benefit of doing that, but I wouldn't say it so strongly that
we "must" do it.  We could also just say that people that want these
errors detected should upgrade lua.  Note that people would also need to
upgrade Emacs, to get this support.

> After I realized that (A) and (B) are true, I used my morning to write a
> patch that solves the problem properly.  I am attaching the patch to
> this message.  Do you think the patch could be merged upstream?

Mattias, what do you think of the below patch?

> Thank you for your time!

Thanks for working on this.

> From 3bbf4019df45548d1286ba9b0dffe8fadad2bdf1 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= <salutis <at> me.com>
> Date: Tue, 3 Oct 2023 09:07:40 +0200
> Subject: [PATCH] Make the Compilation mode recognize Lua errors
>
> Emacs comes with built-in support for the Lua programming language in
> the form of the Lua mode and now also the Lua Tree-sitter mode.  But,
> Lua support in Emacs is incomplete in that the Compilation mode does
> not recognize Lua errors, which is what this patch fixes.
>
> Reported as bug#60830.
>
> * lisp/progmodes/compile.el (compilation-error-regexp-alist-alist):
> Add regexps to aid Lua development, namely the 'lua' regexp that
> matches Lua errors and the 'lua-stack' regexp that matches Lua stack
> frames.
>
> * test/lisp/progmodes/compile-tests.el (compile-tests--test-regexps-data):
> (compile-test-error-regexps): Test the new 'lua' and 'lua-stack'
> regexps added to the 'compilation-error-regexp-alist-alist'.
> ---
>  lisp/progmodes/compile.el            |  5 +++++
>  test/lisp/progmodes/compile-tests.el | 13 +++++++++++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
> index f85cc0909dd..82f8280d76d 100644
> --- a/lisp/progmodes/compile.el
> +++ b/lisp/progmodes/compile.el
> @@ -362,6 +362,11 @@ compilation-error-regexp-alist-alist
>      (ruby-Test::Unit
>       "^    [[ ]?\\([^ (].*\\):\\([1-9][0-9]*\\)\\(\\]\\)?:in " 1 2)
>
> +    (lua
> +     "^[^: \n\t]+: \\([^: \n\t]+\\):\\([0-9]+\\): " 1 2 nil 2 1)
> +    (lua-stack
> +     "^\t\\([^: \n\t]+\\):\\([0-9]+\\): " 1 2 nil 0 1)
> +
>      (gmake
>       ;; Set GNU make error messages as INFO level.
>       ;; It starts with the name of the make program which is variable,
> diff --git a/test/lisp/progmodes/compile-tests.el b/test/lisp/progmodes/compile-tests.el
> index 078eef36774..889b3443993 100644
> --- a/test/lisp/progmodes/compile-tests.el
> +++ b/test/lisp/progmodes/compile-tests.el
> @@ -208,6 +208,15 @@ compile-tests--test-regexps-data
>       1 0 31 "/usr/include/c++/3.3/backward/iostream.h")
>      (gcc-include "                 from test_clt.cc:1:"
>       1 nil 1 "test_clt.cc")
> +    ;; Lua
> +    (lua "lua: database.lua:10: assertion failed!"
> +         6 nil 10 "database.lua")
> +    (lua "/usr/local/bin/lua: core/database.lua:20: assertion failed!"
> +         21 nil 20 "core/database.lua")
> +    (lua-stack "	database.lua:10: in field 'statement'"
> +                     2 nil 10 "database.lua" 0)
> +    (lua-stack "	core/database.lua:20: in field 'statement'"
> +                     2 nil 20 "core/database.lua" 0)
>      ;; gmake
>      (gmake "make: *** [Makefile:20: all] Error 2" 12 nil 20 "Makefile" 0)
>      (gmake "make[4]: *** [sub/make.mk:19: all] Error 127" 15 nil 19
> @@ -499,9 +508,9 @@ compile-test-error-regexps
>            (compilation-num-warnings-found 0)
>            (compilation-num-infos-found 0))
>        (mapc #'compile--test-error-line compile-tests--test-regexps-data)
> -      (should (eq compilation-num-errors-found 100))
> +      (should (eq compilation-num-errors-found 102))
>        (should (eq compilation-num-warnings-found 35))
> -      (should (eq compilation-num-infos-found 28)))))
> +      (should (eq compilation-num-infos-found 30)))))
>
>  (ert-deftest compile-test-grep-regexps ()
>    "Test the `grep-regexp-alist' regexps.
> --
> 2.37.1 (Apple Git-137.1)

Message #25 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 60830 <at> debbugs.gnu.org
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Tue, 03 Oct 2023 22:03:10 +0200

Stefan Kangas <stefankangas <at> gmail.com> writes:

>> (A) Even if the GNU standard supported tab-indented errors, as per (2),
>> it would not help, as Lua stack frames are not errors.
>
> Yes, I noticed that too.  Is it a regression with regards to earlier
> versions of Emacs?

I was unclear in what I wrote.  Let me clarify.

What I meant is that every Lua error is followed by a stack trace
indented with a tab character.  From the point of view of a Compilation
buffer, each such Lua error consists of exactly one *error* and sequence
of stack frames, each of which is an *info*.

Note that my patch handles all this correctly, and the tests exercise
both cases, errors and stack frames, and they exercise each case twice,
once with different data, to make sure all works now and in the future.

> I see the benefit of doing that, but I wouldn't say it so strongly that
> we "must" do it.  We could also just say that people that want these
> errors detected should upgrade lua.  Note that people would also need to
> upgrade Emacs, to get this support.

It was a "mathematical" must, as in "we must find X such that..." :)

As for upgrading Lua, please know that Lua is different from other
languages in that it introduces breaking changes even between minor
versions.  This does not break all Lua projects because the language is
nothing but a single ANSI C compilation unit [*] that will work
forever. In practice, if a Lua project has zero dependencies, or perhaps
only a few, the maintainers do not ever need to upgrade Lua.  This is a
key feature of Lua, not a bug.  Thus, if Emacs wants to serve Lua
developers well, it must recognize errors from older Lua versions as
well.

[*] When compiled via the 'one.c', otherwise it is multiple files.

Rudy
-- 
"Programming reliably -- must be an activity of an undeniably
mathematical nature […] You see, mathematics is about thinking, and
doing mathematics is always trying to think as well as possible."
-- Edsger W. Dijkstra, 1981

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #28 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Thu, 5 Oct 2023 13:27:23 +0200

3 okt. 2023 kl. 10.12 skrev Rudolf Adamkovič <salutis <at> me.com>:

> +    (lua
> +     "^[^: \n\t]+: \\([^: \n\t]+\\):\\([0-9]+\\): " 1 2 nil 2 1)

This pattern matches some lines normally matched by the `gnu` rule which comes later in the rule list. I don't know if it's a big problem -- it may cause misclassification of some messages which should be of 'warning' or 'info' type.

> +    (lua-stack
> +     "^\t\\([^: \n\t]+\\):\\([0-9]+\\): " 1 2 nil 0 1)

I'm actually slightly more bothered by this message because it would conflict with possible future (re-)relaxation of the `gnu` rule with respect to leading whitespace. The matter does come up from time to time, and it would be nice to at least have the option to do that.

In addition, there's this:

> (B) People use older Lua versions, which will work forever, as they are
> written in ANSI C, and so even if (1) is fixed upstream, we must support
> the current error format as well.

This also means that we may need to support an open set of Lua message types.

What about adding these rules but not enabling them by default? Is that too onerous?

The 'omake' rule has now been disabled by default for other reasons (it's something that has been discussed in the past), so there is at least precedence for doing this.

Message #31 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Thu, 05 Oct 2023 18:21:02 +0200

[Message part 1 (text/plain, inline)]

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> This pattern matches some lines normally matched by the `gnu` rule
> which comes later in the rule list. I don't know if it's a big problem
> -- it may cause misclassification of some messages which should be of
> 'warning' or 'info' type.

Good to know!  So, I studied 'luaL_traceback' in 'lauxlib.c' in Lua and
made the regular expression more precise.  No more mis-classification!

> I'm actually slightly more bothered by this message because it would
> conflict with possible future (re-)relaxation of the `gnu` rule with
> respect to leading whitespace. The matter does come up from time to
> time, and it would be nice to at least have the option to do that.

Ditto; I made other expression more precise as well!

> The 'omake' rule has now been disabled by default for other reasons
> (it's something that has been discussed in the past), so there is at
> least precedence for doing this.

/Note:/ I rebased on top of 'master' (and so the 'omake' patch).

See the new patch, with more precise regular expressions, attached
below.  While on it, I also added support for stack frames with no line
number, as per Lua source code.  Lastly, I added a NEWS entry, as well
as, improved the commit message a bit.

How about now? :)

Rudy

[0001-Make-the-Compilation-mode-recognize-Lua-errors.patch (text/x-patch, attachment)]

[Message part 3 (text/plain, inline)]

-- 
"If you're thinking without writing, you only think you're thinking."
--- Leslie Lamport

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #34 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Fri, 6 Oct 2023 13:38:37 +0200

5 okt. 2023 kl. 18.21 skrev Rudolf Adamkovič <salutis <at> me.com>:

> So, I studied 'luaL_traceback' in 'lauxlib.c' in Lua and
> made the regular expression more precise.  No more mis-classification!

That's the spirit!

> /usr/bin/lua: database.lua:31: assertion failed!
> stack traceback:
> 	[C]: in function 'assert'
> 	database.lua:31: in field 'statement'
> 	database.lua:42: in field 'table'
> 	database.lua:55: in field 'row'
> 	database.lua:63: in field 'value'
> 	database.lua:68: in main chunk
> 	[C]: in ?

Your rules incorrectly match "[C]" as a file name. This isn't just a matter of aesthetics; the user must be able to step through matches without tripping.

As to your implementation:

>     (lua
>      "^[^:\n\t]+: \\([^: \n\t]+\\):\\([0-9]+\\): .*?\nstack traceback:\n\t"
>      1 2 nil 2 1)
>     (lua-stack
>      "^\t\\([^:\n\t]+\\):\\(\\([0-9]+\\):\\)? in "
>      1 3 nil 0 1)

I see no reason for the non-greedy match in the first regexp, do you?

We try to make rules work with Windows file names (or names containing spaces, which are also somewhat common on Windows) where relevant and practical. Your patterns don't; can you argue that it's not useful for Lua or not possible to do so?

Don't keep capture groups that you don't use. (If you use rx then you don't have to think about it. If you don't, I'm probably going to translate your regexp to rx anyway.)

I'm still erring on having these rules disabled by default because they seem very close to unrelated compiler output. Would that be a major inconvenience?

> While on it, I also added support for stack frames with no line
> number, as per Lua source code.

That doesn't result in anything useful as far as I can tell.

>  Lastly, I added a NEWS entry, as well
> as, improved the commit message a bit.

Please include an addition to etc/compilation.txt as well.

Message #37 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Fri, 06 Oct 2023 09:21:04 -0400

> I'm still erring on having these rules disabled by default because they seem
> very close to unrelated compiler output. Would that be
> a major inconvenience?

Can we rely on the ordering in `compilation-error-regexp-alist` to give
precedence to the other rules?


        Stefan

Message #40 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Fri, 6 Oct 2023 16:00:13 +0200

6 okt. 2023 kl. 15.21 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:

> Can we rely on the ordering in `compilation-error-regexp-alist` to give
> precedence to the other rules?

Currently the ordering in `compilation-error-regexp-alist` (CERA for short) is the one used, but perhaps you mean the ordering in `compilation-error-regexp-alist-alist` (CERAA)?

I suppose we could compute a permutation of CERA that is consistent with CERAA and cache it to avoid doing it for every call to `compilation-parse-errors`, but what to do about non-symbol elements in CERA?

It's easiest to keep letting the user assume responsibility for ordering CERA correctly if customised. That said, the customisation mechanism doesn't work well for CERA. Perhaps it would be better to have a list of desired changes.

Message #43 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Fri, 06 Oct 2023 10:47:26 -0400

>> Can we rely on the ordering in `compilation-error-regexp-alist` to give
>> precedence to the other rules?
> Currently the ordering in `compilation-error-regexp-alist` (CERA for short)
> is the one used, but perhaps you mean the ordering in
> `compilation-error-regexp-alist-alist` (CERAA)?

No, I do mean the ordering in CERA: is there a reason not to use this
ordering to lower the precedence of those Lua rules instead of
disabling them?

Admittedly, there is a performance cost to having all those big regexps
active "just in case".  Short of convincing tool authors to stop the
madness and to settle on a standard format (seems the only accepted
such standard format, nowadays, is LSP 🙁), I wonder is we could come
with a way to figure out tell tale signs that particular tools might be
(or can't be) used.


        Stefan

Message #46 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Fri, 6 Oct 2023 18:20:45 +0200

6 okt. 2023 kl. 16.47 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:

> No, I do mean the ordering in CERA: is there a reason not to use this
> ordering to lower the precedence of those Lua rules instead of
> disabling them?

I'd say keeping rules active but placed where they never or only sometimes match can be seen as the worst of all choices: there is no indication that they are essentially inactive but things don't work right, and they still consume CPU.

What about powering ahead in the other direction and disable a whole lot of rarely-used rules? (No shortage of those.) Perhaps we could make a nicer interface for discovery and selection of available rules. The natural way is selecting a subset, not composing a list.

The rule names aren't ideal either and tell the user very little. We could augment each rule with a short description and an example of what they match, and perhaps information about rules that conflict.

> Admittedly, there is a performance cost to having all those big regexps
> active "just in case".  

Yes, and the number keeps growing so unless we start removing or disabling ones, it's only getting worse.

> I wonder is we could come
> with a way to figure out tell tale signs that particular tools might be
> (or can't be) used.

I can think of some dodgy heuristics that will make nobody happy but that's it.

Message #49 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Fri, 06 Oct 2023 12:49:39 -0400

>> I wonder is we could come with a way to figure out tell tale signs
>> that particular tools might be (or can't be) used.
> I can think of some dodgy heuristics that will make nobody happy but that's it.

How 'bout we only activate a handful of rules, and then use a "low
priority background" search for matches to the non-active rules and when
we find such a match we bring it to the users' attention that maybe they
should enable that extra rule?


        Stefan

Message #52 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sat, 7 Oct 2023 13:18:51 +0200

[Message part 1 (text/plain, inline)]

6 okt. 2023 kl. 18.49 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:

> How 'bout we only activate a handful of rules, and then use a "low
> priority background" search for matches to the non-active rules and when
> we find such a match we bring it to the users' attention that maybe they
> should enable that extra rule?

"It looks like you are compiling a Pascal program using a Sun compiler from 1989. Would you like me to parse the error messages for you?"

Even if we were to pull off the 'low priority background' part, it seems difficult to balance such a mechanism so that it's never annoying yet relevant when it needs to be.

In any case we would benefit from a better way to select rules. One problem is that packages (and users) go wild modifying both CERA and CERAA. Here is a patch that I'm not satisfied with but tries to make CERA easier to customise interactively.

Its main limitation is assuming that explicit rule specs precede symbols that refer to CERAA, which makes some kind of sense (new rules are added for a reason so they should take priority) but may not correspond to reality. Not sure what to do about that, but at least it's a start.

[cera-type.diff (application/octet-stream, attachment)]

Message #55 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sat, 07 Oct 2023 11:22:22 -0400

> Even if we were to pull off the 'low priority background' part,

I'm counting on you for this part.

> it seems difficult to balance such a mechanism so that it's never
> annoying yet relevant when it needs to be.

I suspect the "never annoying" is not too hard to get: those regexps are
currently matched so if they were annoying we'd know about it.

So just: if there's a match highlight it somehow and put a `help-echo`
(and maybe also a `mouse-2` binding) that tells how to activate it.

> In any case we would benefit from a better way to select rules.

Agreed.

> One problem is that packages (and users) go wild modifying both CERA
> and CERAA.  Here is a patch that I'm not satisfied with but tries to
> make CERA easier to customise interactively.

Looks OK to me, tho it will complain to the user if they have it set to
a shape that doesn't fit your mold :-(

> Its main limitation is assuming that explicit rule specs precede symbols
> that refer to CERAA, which makes some kind of sense (new rules are added for
> a reason so they should take priority) but may not correspond to
> reality.  Not sure what to do about that, but at least it's a start.

I wonder how common it is for people to set it "manually" in their
init file.  It seems more common to set it in ELisp packages.


        Stefan

Message #58 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sat, 07 Oct 2023 23:02:25 +0200

[Message part 1 (text/plain, inline)]

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> Your rules incorrectly match "[C]" as a file name. This isn't just a
> matter of aesthetics; the user must be able to step through matches
> without tripping.

Good catch!  I have two questions:

(1) How can I specify "this group cannot be '[C]'" in the regexp?
(2) How can I test this scenario in 'compile-tests.el'?

> I see no reason for the non-greedy match in the first regexp, do you?

I made the relevant groups greedy, but I wonder what difference it
makes.  Is there a way to test the change in 'compile-tests.el'?

> We try to make rules work with Windows file names (or names containing
> spaces, which are also somewhat common on Windows) where relevant and
> practical. Your patterns don't; can you argue that it's not useful for
> Lua or not possible to do so?

Good catch!  Fixed.

> Don't keep capture groups that you don't use.

Fixed and TIL about "shy groups".

> I'm still erring on having these rules disabled by default because
> they seem very close to unrelated compiler output. Would that be a
> major inconvenience?

I would understand if we did that for some long-dead compilers but not
for Lua.  I think we need a systematic approach to solve the performance
penalty and false positives.  Arbitrarily disabling languages that are
alive and well, and even supported by Emacs out of the box, such as Lua,
is not a good idea, IMHO.

>> While on it, I also added support for stack frames with no line
>> number, as per Lua source code.
>
> That doesn't result in anything useful as far as I can tell.

We still detect the file path/name, which is useful.  That said, Lua
stack frames typically include line numbers.

> Please include an addition to etc/compilation.txt as well.

Fixed.

[The current version of the patch is attached below.]

Rudy

[0001-Make-the-Compilation-mode-recognize-Lua-errors.patch (text/x-patch, attachment)]

[Message part 3 (text/plain, inline)]

-- 
"It is no paradox to say that in our most theoretical moods we may be
nearest to our most practical applications."
-- Alfred North Whitehead, 1861-1947

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #61 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sun, 8 Oct 2023 11:59:18 +0200

7 okt. 2023 kl. 23.02 skrev Rudolf Adamkovič <salutis <at> me.com>:

> (1) How can I specify "this group cannot be '[C]'" in the regexp?

Disallow files starting with a left square bracket?

> (2) How can I test this scenario in 'compile-tests.el'?

We can help out.

> I made the relevant groups greedy, but I wonder what difference it
> makes.

Only use non-greedy operators if they make the regexp significantly easier to express. Greedy should be the default.
Non-greedy operators are usually less efficient, and always raise eyebrows when read: why is this non-greedy?

Writing ".*?\n", as you did, makes no sense at all since "." will match anything except newline, and the result is identical to ".*\n" but less readable and maybe slower.

>> We try to make rules work with Windows file names (or names containing
>> spaces, which are also somewhat common on Windows) where relevant and
>> practical. Your patterns don't; can you argue that it's not useful for
>> Lua or not possible to do so?
> 
> Good catch!  Fixed.

Maybe you should test it just in case. (You don't need a Windows machine to do that but it adds authenticity).

> I would understand if we did that for some long-dead compilers but not
> for Lua.

Look, it's not a beauty contest. We'll do what we have to.

> We still detect the file path/name, which is useful.

The point is that it substantially increases the risks of false matches. Lua needs to be a good neighbour and can't just think of its own needs over anything else.

Message #64 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sun, 8 Oct 2023 12:45:36 +0200

7 okt. 2023 kl. 17.22 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:

> I suspect the "never annoying" is not too hard to get: those regexps are
> currently matched so if they were annoying we'd know about it.

Yes, but one of the points of this soft CERA reform is to allow rules in that may conflict with one another so this balance may not last.

> Looks OK to me, tho it will complain to the user if they have it set to
> a shape that doesn't fit your mold :-(

Indeed it does and I'm not sure what to do about it -- the widget system is a bit of a labyrinth to me.

Is there a way to somehow preprocess a value before the widget machinery tries to type-match it? If so, we could run CERA through a centrifuge that moves all symbols to the end of the list.

> I wonder how common it is for people to set it "manually" in their
> init file.  It seems more common to set it in ELisp packages.

Probably, but I have vivid memories of packages in the past that came with comments telling the user what to add to their ~/.emacs and a number of those may have survived.

Packages also vary in their practices: some add a rule to CERAA an its symbol to CERA, others just a pattern to CERA. Most push their additions to the front but some are shy and put theirs at the end.

Message #67 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sun, 8 Oct 2023 16:47:43 +0200

[Message part 1 (text/plain, inline)]

> Is there a way to somehow preprocess a value before the widget machinery tries to type-match it? If so, we could run CERA through a centrifuge that moves all symbols to the end of the list.

Of course there is, and here is the patch with the centrifuge installed.
(It may reorder the list if edited this way but again, this shouldn't be much of a problem in practice.)

[cera-type.diff (application/octet-stream, attachment)]

Message #70 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sun, 08 Oct 2023 11:49:03 -0400

> (1) How can I specify "this group cannot be '[C]'" in the regexp?

For true "regular expressions", you'd use conjunction and negation and
Bob's your uncle, but sadly, our regexps don't support conjunction nor
negation, so you're stuck doing it by hand, e.g.:

    "\\|\\[\\|[^[].*\\|\\[C\\|\\[[^C].*\\|\\[C[^]].*\\|\\[C].+"

Yes, it's hideous and it suffers from a really bad size-explosion
syndrome.  I don't even want to think about how to encode "this group
can match neither "joe" nor "henry".

>> I see no reason for the non-greedy match in the first regexp, do you?
> I made the relevant groups greedy, but I wonder what difference it
> makes.  Is there a way to test the change in 'compile-tests.el'?

    .*?\n

and

    .*\n

are equivalent because . can't match the same char as \n, so there's no
choice to be made about how many repetitions of . to use.

But the regexp compiler notices this fact (which it can use to avoid
using backtracking and thus work noticeably more efficiently) only if
you use ".*\n".


        Stefan

Message #73 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Thu, 12 Oct 2023 10:12:05 +0200

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> We can help out.

To quote you, "That's the spirit!" :)

Please see the to-do comment I left in the 'compile-tests.el'.  I think
we want 'compile--test-error-line' to support both positive and negative
test cases.

> Only use non-greedy operators [...]

I updated everything to my best knowledge.  If the regexps are still not
good enough, please quote the problematic fragments, so that I know
exactly where the problem is.

> Maybe you should test it just in case. (You don't need a Windows
> machine to do that but it adds authenticity).

I do not understand.  Are you saying that the tests I wrote are
insufficient?  If so, could you please elaborate?

> Look, it's not a beauty contest. We'll do what we have to.

How do false positives and performance tie to "beauty"?  I must have
miscommunicated badly.  What I wanted to say is that I imagine Lua
working out of the box, hence my hunch for the most precise regexps.

>> We still detect the file path/name, which is useful.
>
> The point is that it substantially increases the risks of false
> matches. Lua needs to be a good neighbour and can't just think of its
> own needs over anything else.

The sequence of characters is fairly unique, given each stack frame must
start with a tab and then contain ": in " after the file name.

Rudy
-- 
"Programming reliably -- must be an activity of an undeniably
mathematical nature […] You see, mathematics is about thinking, and
doing mathematics is always trying to think as well as possible."
-- Edsger W. Dijkstra, 1981

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #76 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Thu, 12 Oct 2023 10:14:46 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> For true "regular expressions", you'd use conjunction and negation and
> Bob's your uncle, but sadly, our regexps don't support conjunction nor
> negation, so you're stuck doing it by hand, e.g.:
>
>     "\\|\\[\\|[^[].*\\|\\[C\\|\\[[^C].*\\|\\[C[^]].*\\|\\[C].+"
>
> Yes, it's hideous and it suffers from a really bad size-explosion
> syndrome.  I don't even want to think about how to encode "this group
> can match neither "joe" nor "henry".

After a bit of thinking, I used a shy group

  \\(?:\\[C\\]\\|.......\\)
       ^^^^^^^   ^^^^^^^  
       NEGATIVE  POSITIVE

This either matches nothing or the groups in the POSITIVE alternative.

Rudy
-- 
"I love deadlines.  I love the whooshing noise they make as they go by."
-- Douglas Adams, The Salmon of Doubt, 2002

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #79 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Thu, 12 Oct 2023 10:17:10 +0200

[Message part 1 (text/plain, inline)]

Rudolf Adamkovič <salutis <at> me.com> writes:

> ...

And I forgot to attach the latest version of the patch. :)

Rudy

[0001-Make-the-Compilation-mode-recognize-Lua-errors.patch (text/x-patch, inline)]

From 715e9c353de17f9462842943e6000678ca8fab3a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= <salutis <at> me.com>
Date: Tue, 3 Oct 2023 09:07:40 +0200
Subject: [PATCH] Make the Compilation mode recognize Lua errors

Emacs comes with built-in support for the Lua programming language in
the form of the Lua mode and now also the Lua Tree-sitter mode.  This
patch further improves Lua support in Emacs by making the Compilation
mode recognize Lua errors and stack traces.

Reported as bug#60830.

* etc/compilation.txt (Lua): Add an example of a Lua error message
with a stack trace.

* lisp/progmodes/compile.el (compilation-error-regexp-alist-alist):
Add regexps to aid Lua development, namely the 'lua' regexp that
matches Lua errors and the 'lua-stack' regexp that matches Lua stack
frames.

* test/lisp/progmodes/compile-tests.el (compile-tests--test-regexps-data):
(compile-test-error-regexps): Test the new 'lua' and 'lua-stack'
regexps added to the 'compilation-error-regexp-alist-alist'.
---
 etc/NEWS                             |  5 +++++
 etc/compilation.txt                  | 13 +++++++++++++
 lisp/progmodes/compile.el            |  7 +++++++
 test/lisp/progmodes/compile-tests.el | 22 ++++++++++++++++++++--
 4 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 8b2bcaaf01d..d0df4df2b2e 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -319,6 +319,11 @@ This is because it partly acts by modifying other rules which may
 occasionally be surprising.  It can be re-enabled by adding 'omake' to
 'compilation-error-regexp-alist'.
 
+*** Lua errors and stack traces are now recognized.
+The compilation mode now recognizes Lua language errors and stack
+traces.  Every Lua error is recognized as a compilation error, and
+every Lua stack frame is recognized as a compilation info.
+
 ** VC
 
 ---
diff --git a/etc/compilation.txt b/etc/compilation.txt
index 5f6ecb09cc2..fa43c89879a 100644
--- a/etc/compilation.txt
+++ b/etc/compilation.txt
@@ -344,6 +344,19 @@ In /home/janneke/vc/guile/examples/gud-break.scm:
 1033: 0 [stderr "~a:hello world\n" (# # #)]
 
 
+* Lua
+
+/usr/bin/lua: database.lua:31: assertion failed!
+stack traceback:
+	[C]: in function 'assert'
+	database.lua:31: in field 'statement'
+	database.lua:42: in field 'table'
+	database.lua:55: in field 'row'
+	database.lua:63: in field 'value'
+	io.lua: in main chunk
+	[C]: in ?
+
+
 * Lucid Compiler, lcc 3.x
 
 symbol: lcc
diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index 9e441dbfcf7..9a2f633e3ec 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -362,6 +362,13 @@ compilation-error-regexp-alist-alist
     (ruby-Test::Unit
      "^    [[ ]?\\([^ (].*\\):\\([1-9][0-9]*\\)\\(\\]\\)?:in " 1 2)
 
+    (lua
+     "^[^:\n\t]+: \\([^:\n\t]+\\):\\([0-9]+\\): .+\nstack traceback:\n\t"
+     1 2 nil 2 1)
+    (lua-stack
+     "^\t\\(?:\\[C\\]:\\|\\([^:\n\t]+\\):\\(?:\\([0-9]+\\):\\)?\\) in "
+     1 2 nil 0 1)
+
     (gmake
      ;; Set GNU make error messages as INFO level.
      ;; It starts with the name of the make program which is variable,
diff --git a/test/lisp/progmodes/compile-tests.el b/test/lisp/progmodes/compile-tests.el
index d497644c389..2e917d3b614 100644
--- a/test/lisp/progmodes/compile-tests.el
+++ b/test/lisp/progmodes/compile-tests.el
@@ -206,6 +206,24 @@ compile-tests--test-regexps-data
      1 0 31 "/usr/include/c++/3.3/backward/iostream.h")
     (gcc-include "                 from test_clt.cc:1:"
      1 nil 1 "test_clt.cc")
+    ;; Lua
+    (lua "lua: database.lua:10: assertion failed!\nstack traceback:\n\t"
+         6 nil 10 "database.lua")
+    (lua "lua 5.4: database 2.lua:10: assertion failed!\nstack traceback:\n\t"
+         10 nil 10 "database 2.lua")
+    (lua "/usr/local/bin/lua: core/database.lua:20: assertion failed!\nstack traceback:\n\t"
+         21 nil 20 "core/database.lua")
+    (lua-stack "	database.lua: in field 'statement'"
+               2 nil nil "database.lua" 0)
+    (lua-stack "	database.lua:10: in field 'statement'"
+               2 nil 10 "database.lua" 0)
+    (lua-stack "	core/database.lua:20: in field 'statement'"
+               2 nil 20 "core/database.lua" 0)
+    (lua-stack "	database 2.lua: in field 'statement'"
+               2 nil nil "database 2.lua" 0)
+    ;; TODO Test the negative case:
+    ;; (lua-stack "	[C]: in field 'statement'"
+    ;;            nil nil nil nil nil)
     ;; gmake
     (gmake "make: *** [Makefile:20: all] Error 2" 12 nil 20 "Makefile" 0)
     (gmake "make[4]: *** [sub/make.mk:19: all] Error 127" 15 nil 19
@@ -507,9 +525,9 @@ compile-test-error-regexps
                    1 15 5 "alpha.c")))
         (compile--test-error-line test))
 
-      (should (eq compilation-num-errors-found 100))
+      (should (eq compilation-num-errors-found 103))
       (should (eq compilation-num-warnings-found 35))
-      (should (eq compilation-num-infos-found 28)))))
+      (should (eq compilation-num-infos-found 32)))))
 
 (ert-deftest compile-test-grep-regexps ()
   "Test the `grep-regexp-alist' regexps.
-- 
2.39.3 (Apple Git-145)

[Message part 3 (text/plain, inline)]

-- 
"Great minds discuss ideas; average minds discuss events; small minds discuss
people."
--- Anna Eleanor Roosevelt (1884-1962)

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #82 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Thu, 12 Oct 2023 16:32:51 +0200

12 okt. 2023 kl. 10.12 skrev Rudolf Adamkovič <salutis <at> me.com>:

>> Maybe you should test it just in case. (You don't need a Windows
>> machine to do that but it adds authenticity).
> 
> I do not understand.

Perhaps you could tell us exactly what you have tested that makes you confident in saying that Windows file names have been 'fixed'. Otherwise there is no way for us to know, because the tests in your patch give us no indication.

> How do false positives and performance tie to "beauty"?

It's a figure of speech.

By the way, precisely what versions of Lua use the message format you described? This information should go into the entry of compilation.txt (and the comments describing the new entries in compilation-error-regexp-alist-alist).
As you say, Lua versions tend to introduce incompatible changes because of how the language is used, and this could very well include the formatting of diagnostics.

Message #85 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Rudolf Adamkovič <salutis <at> me.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Sun, 10 Dec 2023 23:53:28 +0100

[Message part 1 (text/plain, inline)]

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> [...] Windows file names [...] tests [...] give us no indication.

Good catch!

I added more tests and updated the code to handle DOS/Windows paths.

>> How do false positives and performance tie to "beauty"?
>
> It's a figure of speech.

I see.

> [...] precisely what versions of Lua [...] information should go into
> the entry of compilation.txt (and the comments describing the new
> entries in compilation-error-regexp-alist-alist).

So that we can be 100% sure, today I compiled and tested the patch
against the last four versions of Lua, as well as, the famously speedy
fork LuaJIT.  It works correctly in all cases.  I added the version
information to the appropriate places.

I am attaching a new version of the patch.

Thank you!

Rudy

[0001-Make-the-Compilation-mode-recognize-Lua-errors.patch (text/x-patch, attachment)]

[Message part 3 (text/plain, inline)]

-- 
"Be especially critical of any statement following the word
'obviously.'"
-- Anna Pell Wheeler, 1883-1966

Rudolf Adamkovič <salutis <at> me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Message #90 received at 60830-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Rudolf Adamkovič <salutis <at> me.com>, 
 Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830-done <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Wed, 10 Jan 2024 05:48:51 -0800

Rudolf Adamkovič <salutis <at> me.com> writes:

> I am attaching a new version of the patch.

Thank you, installed on master (cd0eb055fd4) and closing the bug.

Message #93 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Wed, 10 Jan 2024 17:37:34 +0100

10 jan. 2024 kl. 14.48 skrev Stefan Kangas <stefankangas <at> gmail.com>:

> Thank you, installed on master (cd0eb055fd4) and closing the bug.

Sorry for falling behind on this bug, and thank you for helping me catch up.

However I wasn't actually finished here: the new regexps are still annoyingly close to other patterns that mismatches and/or bad performance cannot be ruled out, and it would be good to have a way to deal with it.

I'm tempted go about disabling large swaths of rules since for every useful contemporary rule like Gnu, Typescript and yes, Lua, there seem to be a half-dozen ones for niche and/or outright antiquated tools.

Anyway, I performed some minimum maintenance on the new Lua patterns but can't promise that they be enabled by default in the end -- it's not that they aren't useful but risk of interference.

(And it's probably high time I made a batch conversion of the remaining patterns to rx.)

Message #96 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Wed, 10 Jan 2024 09:09:11 -0800

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> 10 jan. 2024 kl. 14.48 skrev Stefan Kangas <stefankangas <at> gmail.com>:
>
>> Thank you, installed on master (cd0eb055fd4) and closing the bug.
>
> Sorry for falling behind on this bug, and thank you for helping me
> catch up.
>
> However I wasn't actually finished here: the new regexps are still
> annoyingly close to other patterns that mismatches and/or bad
> performance cannot be ruled out, and it would be good to have a way to
> deal with it.

Sorry, that was a misunderstanding on my part.  I thought we were done
discussing it.  Feel free to revert the change and reopen the bug, or
whatever else you think makes sense here.

> I'm tempted go about disabling large swaths of rules since for every
> useful contemporary rule like Gnu, Typescript and yes, Lua, there seem
> to be a half-dozen ones for niche and/or outright antiquated tools.
>
> Anyway, I performed some minimum maintenance on the new Lua patterns
> but can't promise that they be enabled by default in the end -- it's
> not that they aren't useful but risk of interference.

If we can't even enable new patterns due to risk of interference, then
it's clear that we've dug ourselves into a hole.

How about adding a new user option `compilation-enabled-errors' where
you could disable (or enable) the ones you are interested in, and then
generate `compilation-error-regexp-alist-alist' based on that?  That
would provide a convenient way for users to get back the support for
their trusty Jikes compiler or whatever.  We should probably also
support using it as a directory local variable somehow, so that projects
could set this only once.

> (And it's probably high time I made a batch conversion of the
> remaining patterns to rx.)

That would help maintenance, so I'm all for it.

Message #99 received at 60830 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 60830 <at> debbugs.gnu.org, Rudolf Adamkovič <salutis <at> me.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#60830: 30.0.50; The *Compilation* buffer does not recognize
 Lua errors
Date: Wed, 10 Jan 2024 18:45:24 +0100

10 jan. 2024 kl. 18.09 skrev Stefan Kangas <stefankangas <at> gmail.com>:

> Sorry, that was a misunderstanding on my part.  I thought we were done
> discussing it.  Feel free to revert the change and reopen the bug, or
> whatever else you think makes sense here.

No, I think we're done with the bug at hand (Lua messages) for now, even if it means that we may revisit the choices made later on.

> How about adding a new user option `compilation-enabled-errors' where
> you could disable (or enable) the ones you are interested in, and then
> generate `compilation-error-regexp-alist-alist' based on that?

See the (somewhat lengthy) history of this bug for some discussion about that. In short, we already have `compilation-error-regexp-alist` which basically serves that purpose, but is complicated by the fact that both users and packages edit it and `compilation-error-regexp-alist-alist`, often at the same time.

The question is how we can reinterpret CERA and CERAA in a compatible way that allows for easy user customisation but also for them to be extended in a modular way.

>> (And it's probably high time I made a batch conversion of the
>> remaining patterns to rx.)
> 
> That would help maintenance, so I'm all for it.

I'll do that a little later then.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.