GNU bug report logs - #49426
28.0.50; Byte-compilation can fail with large fill-column

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 49426 in the body.
You can then email your comments to 49426 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Byte-compilation can fail with large fill-column
Date: Tue, 06 Jul 2021 01:16:56 +0100

[Message part 1 (text/plain, inline)]

Tags: patch

As seen on the Dash issue tracker:

https://github.com/magnars/dash.el/issues/383

0. cd "$(mktemp -d)"
1. HOME="$PWD" XDG_CONFIG_HOME="$PWD/.config" emacs -Q
2. (setq-default
    package-archives '(("melpa" . "https://melpa.org/packages/"))
    fill-column #xffff)
3. C-x C-e
4. M-x package-install RET dash RET

Installation fails with the following in *Compile-Log*:

  Compiling file
  /tmp/tmp.IEFN9nE4Fs/.emacs.d/elpa/dash-20210704.1302/dash.el
  at Tue Jul 6 00:44:46 2021
  Entering directory
  ‘/tmp/tmp.IEFN9nE4Fs/.emacs.d/elpa/dash-20210704.1302/’
  dash.el:551:1: Error: Invalid regexp: "Invalid content of \\{\\}"

The same error occurs post-installation with 'M-x byte-compile-file' on
dash.el.  Either way, the error confusingly only shows up when Dash is
installed from MELPA:

  package-archives = '(("melpa" . "https://melpa.org/packages/"))

and not from GNU-devel ELPA:

  package-archives = '(("gnu" . "https://elpa.gnu.org/devel/"))

even though both contain identical versions of the file dash.el.  So I
guess it's some other part of the package .tar that gives rise to the
discrepancy.

Turning on debug-on-error doesn't give a backtrace, presumably because
displaying-byte-compile-warnings or similar demotes errors.

But a breakpoint in signal_or_quit reveals that
byte-compile--wide-docstring-p constructs an invalid regexp whose
repetition exceeds the new (since Emacs 27) RE_DUP_MAX limit of #xffff.

Maybe byte-compile--wide-docstring-p should defend itself against this
as follows?

[0001-Avoid-invalid-regexp-in-wide-docstring-check.patch (text/x-diff, inline)]

From 42de0117547a462ce1f161a3a9cae92fd322e9ec Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Date: Tue, 6 Jul 2021 01:05:41 +0100
Subject: [PATCH] Avoid invalid regexp in wide docstring check

* lisp/emacs-lisp/bytecomp.el (byte-compile--wide-docstring-p):
Avoid constructing an invalid regexp during byte-compilation by
limiting the number of columns to the current RE_DUP_MAX of 65535.
This protects against pathological values of fill-column, for
example.
---
 lisp/emacs-lisp/bytecomp.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 70999648d4..6970c8a505 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -1627,7 +1627,7 @@ byte-compile--wide-docstring-p
 `byte-compile--wide-docstring-substitution-len'.  Also ignore
 URLs."
   (string-match
-   (format "^.\\{%s,\\}$" (int-to-string (1+ col)))
+   (format "^.\\{%d,\\}$" (min (1+ col) #xffff)) ; Heed RE_DUP_MAX.
    (replace-regexp-in-string
     (rx (or
          ;; Ignore some URLs.
-- 
2.30.2

[Message part 3 (text/plain, inline)]

Thanks,

-- 
Basil

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2021-07-05 built on tia
Repository revision: 68276f6d30bbdc09cc26fb49d7f0c3aa4bce35f2
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-Og -ggdb' --config-cache
 --prefix=/home/blc/.local --enable-checking=structs
 --with-x-toolkit=lucid --with-file-notification=yes --with-x'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
X11 XAW3D XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail warnings compile comint
ansi-color ring info find-func easy-mmode autoload radix-tree lisp-mnt
tar-mode arc-mode archive-mode cus-edit pp cus-start cus-load wid-edit
mm-archive message dired dired-loaddefs rfc822 mml mml-sec epa derived
gnus-util rmail rmail-loaddefs text-property-search time-date mailabbrev
gmm-utils mailheader mm-decode mm-bodies mm-encode mail-utils gnutls
network-stream url-http mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr url-gw nsm rmc puny url-cache url-auth epg
epg-config finder-inf package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 243317 18548)
 (symbols 48 15849 1)
 (strings 32 83582 1296)
 (string-bytes 1 2065467)
 (vectors 16 24463)
 (vector-slots 8 318902 13721)
 (floats 8 39 454)
 (intervals 56 285 0)
 (buffers 992 15))

Message #8 received at 49426 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: 49426 <at> debbugs.gnu.org
Cc: Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#49426: 28.0.50; Byte-compilation can fail with large
 fill-column
Date: Tue, 06 Jul 2021 09:07:32 +0100

"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> The same error occurs post-installation with 'M-x byte-compile-file' on
> dash.el.  Either way, the error confusingly only shows up when Dash is
> installed from MELPA:
>
>   package-archives = '(("melpa" . "https://melpa.org/packages/"))
>
> and not from GNU-devel ELPA:
>
>   package-archives = '(("gnu" . "https://elpa.gnu.org/devel/"))
>
> even though both contain identical versions of the file dash.el.  So I
> guess it's some other part of the package .tar that gives rise to the
> discrepancy.

Duh, the GNU ELPA .tar includes a .dir-locals.el with a custom
fill-column, whereas the MELPA package installs only dash.el and its
Texinfo manual, so there is nothing to override the pathological default
value of fill-column.

In which case the recipe becomes much simpler:

0. emacs -Q
1. M-: (setq-default fill-column 9999999) RET
2. (defalias 'foo #'ignore "Foo.")
3. C-x C-s foo.el RET
4. M-x byte-compile-file RET foo.el RET

This gives:

Compiling file /home/blc/foo.el at Tue Jul  6 09:05:44 2021
foo.el:4:1: Error: Invalid regexp: "Invalid content of \\{\\}"

-- 
Basil

Message #11 received at 49426 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 49426 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#49426: 28.0.50; Byte-compilation can fail with large
 fill-column
Date: Tue, 06 Jul 2021 17:08:10 +0200

"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> Maybe byte-compile--wide-docstring-p should defend itself against this
> as follows?

Makes sense to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #16 received at 49426-done <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 49426-done <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#49426: 28.0.50; Byte-compilation can fail with large
 fill-column
Date: Tue, 06 Jul 2021 19:14:49 +0100

close 49426 28.1
quit

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> "Basil L. Contovounesios" <contovob <at> tcd.ie> writes:
>
>> Maybe byte-compile--wide-docstring-p should defend itself against this
>> as follows?
>
> Makes sense to me.

Thanks, pushed and closing.

Avoid invalid regexp in wide docstring check
044742bfe8 2021-07-06 18:56:15 +0100
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=044742bfe8c7c22e303242c40e16fbe9e564727a

-- 
Basil

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.