GNU bug report logs - #30635
No compiler warning if code forgets to require cl-lib

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30635 in the body.
You can then email your comments to 30635 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #3 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: submit <at> debbugs.gnu.org
Subject: No compiler warning if code forgets to require cl-lib
Date: Tue, 27 Feb 2018 13:45:57 -0500

Package: emacs
Version: 26.0.91

There's no compiler warning if a library uses cl-lib without requiring it.
I assume this is because bytecomp.el requires cl-lib.
Perhaps bytecomp can track cl-lib specially to work around this.

(I was wondering how the problem shown in 6288c3d went unspotted.)

Example:

foo.el:
(defun foo ()
  (cl-member-if 'cl-evenp '(1 2 3)))

emacs -Q -batch -f batch-byte-compile foo.el
  -> silently succeeds

emacs -Q -l foo.elc
M-: (foo)
 -> void-function cl-member-if

Message #6 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Mon, 19 Mar 2018 15:58:48 -0400

> There's no compiler warning if a library uses cl-lib without requiring it.
> I assume this is because bytecomp.el requires cl-lib.

Should we apply the patch below?


        Stefan


diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index b3ea9300b0..e75403d80d 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -124,17 +124,11 @@
 (require 'backquote)
 (require 'macroexp)
 (require 'cconv)
-(require 'cl-lib)
-
-;; During bootstrap, cl-loaddefs.el is not created yet, so loading cl-lib
-;; doesn't setup autoloads for things like cl-every, which is why we have to
-;; require cl-extra as well (bug#18804).
-(or (fboundp 'cl-every)
-    (require 'cl-extra))
-
-(or (fboundp 'defsubst)
-    ;; This really ought to be loaded already!
-    (load "byte-run"))
+(eval-when-compile
+  ;; We should refrain from loading cl-lib at run-time within the compiler
+  ;; code, otherwise we can't detect if a file forgets to (require 'cl-lib),
+  ;; as mentioned in bug#30635.
+  (require 'cl-lib))
 
 ;; The feature of compiling in a specific target Emacs version
 ;; has been turned off because compile time options are a bad idea.
@@ -3582,7 +3576,10 @@ byte-compile-and-folded
     (cond
      ((< l 3) (byte-compile-form `(progn ,(nth 1 form) t)))
      ((= l 3) (byte-compile-two-args form))
-     ((cl-every #'macroexp-copyable-p (nthcdr 2 form))
+     ;; This used to use `cl-every' but we need to avoid cl-lib at run-time as
+     ;; mentioned at the beginning of this file.
+     ((null (delq nil (mapcar (lambda (e) (not (macroexp-copyable-p e)))
+                              (nthcdr 2 form))))
       (byte-compile-form `(and (,(car form) ,(nth 1 form) ,(nth 2 form))
 			       (,(car form) ,@(nthcdr 2 form)))))
      (t (byte-compile-normal-call form)))))

Message #9 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Mon, 19 Mar 2018 16:12:28 -0400

Stefan Monnier wrote:

>> There's no compiler warning if a library uses cl-lib without requiring it.
>> I assume this is because bytecomp.el requires cl-lib.
>
> Should we apply the patch below?

Works for me, thanks.

It reveals a problem in isearch.el since 6bc78d5:

emacs -Q --eval "(setq search-exit-option 'append)"
C-s
 -> Error in pre-command-hook (isearch-pre-command-hook): (void-function cl-every)

Message #12 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Thu, 22 Mar 2018 17:53:01 -0400

>>> There's no compiler warning if a library uses cl-lib without requiring it.
>>> I assume this is because bytecomp.el requires cl-lib.
>> Should we apply the patch below?
> Works for me, thanks.

Doesn't work for me, OTOH:

    % src/emacs --batch --eval "(eval-after-load 'cl-lib '(debug t))"  -f batch-byte-compile lisp/emacs-lisp/bytecomp.el
    Debugger entered--beginning evaluation of function call form:
      (lambda nil (debug t))()
      eval-after-load-helper("/home/monnier/src/emacs/trunk/lisp/emacs-lisp/cl-lib.elc")
      run-hook-with-args(eval-after-load-helper "/home/monnier/src/emacs/trunk/lisp/emacs-lisp/cl-lib.elc")
      do-after-load-evaluation("/home/monnier/src/emacs/trunk/lisp/emacs-lisp/cl-lib.elc")
      require(cl-lib)
      require(seq)
      require(map)
      dir-locals-read-from-dir("/home/monnier/src/emacs/trunk/")
      hack-dir-local-variables()
      hack-local-variables(no-mode)
      normal-mode(t)
      byte-compile-file("lisp/emacs-lisp/bytecomp.el")
      batch-byte-compile-file("lisp/emacs-lisp/bytecomp.el")
      batch-byte-compile()
      command-line-1(("--eval" "(eval-after-load 'cl-lib '(debug t))" "-f" "batch-byte-compile" "lisp/emacs-lisp/bytecomp.el"))
      command-line()
      normal-top-level()

and if you look at `dir-locals-read-from-dir`, you'll see that it does
(require 'map) and then uses its `map-merge(-with)`.


        Stefan "the pressure to preload cl-lib is definitely mounting"

Message #15 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Thu, 22 Mar 2018 18:19:29 -0400

>>>> There's no compiler warning if a library uses cl-lib without requiring it.
>>>> I assume this is because bytecomp.el requires cl-lib.
>>> Should we apply the patch below?
>> Works for me, thanks.
> Doesn't work for me, OTOH:
[...]
> and if you look at `dir-locals-read-from-dir`, you'll see that it does
> (require 'map) and then uses its `map-merge(-with)`.

And if I tweak dir-locals-read-from-dir so it doesn't require `map`, and
then try to compile a file like gnus-cloud.el where I removed all the
`require`s, I still get:

    % src/emacs --batch --eval "(eval-after-load 'cl-lib '(debug t))"  -f batch-byte-compile lisp/gnus/gnus-cloud.el  
    Debugger entered--beginning evaluation of function call form:
      (lambda nil (debug t))()
      [...]
      require(cl-lib)
      byte-code("\300\301!\210\300\302!\210\300\303!\207" [require cl-lib macroexp gv] 2)
      cl--defsubst-expand((cl-x) (cl-block epg-context-armor (or (cl--struct-epg-context-p cl-x) (signal 'wrong-type-argument (list 'epg-context cl-x))) (aref cl-x 4)) nil nil nil context)
      epg-context-armor--cmacro((epg-context-armor context) context)
      apply(epg-context-armor--cmacro (epg-context-armor context) context)
      gv-get((epg-context-armor context) #f(compiled-function (getter setter) #<bytecode 0x57f35d>))
      [...]
      macroexpand-all((defalias 'gnus-cloud-encode-data ...) ...)
      [...]
      byte-compile-recurse-toplevel((defun gnus-cloud-encode-data ...) ...)

The end result is better tho; if I then add a call to `cl-every` into
that file, I do get a warning of the form:

    gnus/gnus-cloud.el:512:1:Warning: the function ‘cl-every’ might not be
        defined at runtime.

So I pushed the previous two patches to master, since I think they at
least partly fix this bug.


        Stefan

Message #18 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Thu, 22 Mar 2018 20:56:31 -0400

Stefan Monnier wrote:

> And if I tweak dir-locals-read-from-dir so it doesn't require `map`, and

Nice; this provides a measurable build speed-up by finally getting rid
of some of the slowdown introduced by support for multiple dir-locals.

(bug#22307)

Message #23 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: Glenn Morris <rgm <at> gnu.org>, 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Mon, 26 Mar 2018 01:51:22 +0100

Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:

> So I pushed the previous two patches to master, since I think they at
> least partly fix this bug.

I've bisected the following build error to your commit 97b7e58c4d "Try
and fix the more obvious sources of bug#30635" of 2018-03-22:

    [...]
    Dumping under the name emacs
    13102432 of 33554432 static heap bytes used
    99255 pure bytes used
    mv -f emacs bootstrap-emacs
    make -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
    make[2]: Entering directory '/home/blc/.local/src/emacs/lisp'
      ELC      emacs-lisp/bytecomp.elc
    Error reading dir-locals: (invalid-read-syntax "#")
    
    In toplevel form:
    emacs-lisp/bytecomp.el:124:1:Error: Symbol’s value as variable is void: =
    Makefile:301: recipe for target 'emacs-lisp/bytecomp.elc' failed
    make[2]: *** [emacs-lisp/bytecomp.elc] Error 1
    make[2]: Leaving directory '/home/blc/.local/src/emacs/lisp'
    Makefile:745: recipe for target 'bootstrap-emacs' failed
    make[1]: *** [bootstrap-emacs] Error 2
    make[1]: Leaving directory '/home/blc/.local/src/emacs/src'
    Makefile:418: recipe for target 'src' failed
    make: *** [src] Error 2

The immediately preceding incantation on my part is:

    make clean && ./configure [...] && make

where the configuration used follows my signature.

Subsequently invoking 'make boostrap' similarly barfs with:

    Dumping under the name emacs
    11966048 of 33554432 static heap bytes used
    2450001 pure bytes used
    Adding name emacs-27.0.50.1
    ln -f emacs bootstrap-emacs
    make[2]: Leaving directory '/home/blc/.local/src/emacs/src'
    make -C lisp all
    make[2]: Entering directory '/home/blc/.local/src/emacs/lisp'
    make -C ../leim all EMACS="../src/emacs"
    make[3]: Entering directory '/home/blc/.local/src/emacs/leim'
    make[3]: Nothing to be done for 'all'.
    make[3]: Leaving directory '/home/blc/.local/src/emacs/leim'
    make -C ../admin/grammars all EMACS="../../src/emacs"
    make[3]: Entering directory '/home/blc/.local/src/emacs/admin/grammars'
    make[3]: Nothing to be done for 'all'.
    make[3]: Leaving directory '/home/blc/.local/src/emacs/admin/grammars'
    make[3]: Entering directory '/home/blc/.local/src/emacs/lisp'
      ELC      emacs-lisp/eieio.elc
    
    In toplevel form:
    emacs-lisp/eieio.el:Error: Invalid read syntax: "#"
    Error reading dir-locals: (invalid-function "
    In toplevel form:
    emacs-lisp/eieio.el:Error: Invalid read syntax: \"#\"")
    emacs-lisp/eieio.el:52:4:Error: Symbol’s value as variable is void: =
    Makefile:301: recipe for target 'emacs-lisp/eieio.elc' failed
    make[3]: *** [emacs-lisp/eieio.elc] Error 1
    make[3]: Leaving directory '/home/blc/.local/src/emacs/lisp'
    Makefile:324: recipe for target 'compile-main' failed
    make[2]: *** [compile-main] Error 2
    make[2]: Leaving directory '/home/blc/.local/src/emacs/lisp'
    Makefile:405: recipe for target 'lisp' failed
    make[1]: *** [lisp] Error 2
    make[1]: Leaving directory '/home/blc/.local/src/emacs'
    Makefile:1099: recipe for target 'bootstrap' failed
    make: *** [bootstrap] Error 2

Any ideas on where I am or what is going wrong and how I can further
troubleshoot this?

TIA,

-- 
Basil

Configured using:
 'configure --prefix=/home/blc/.local --with-mailutils
 --with-x-toolkit=lucid --with-modules --with-file-notification=yes
 --with-x 'CFLAGS=-march=native -O2 -pipe'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 MODULES THREADS LIBSYSTEMD LCMS2

Message #26 received at 30635 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: Glenn Morris <rgm <at> gnu.org>, 30635 <at> debbugs.gnu.org
Subject: Re: bug#30635: No compiler warning if code forgets to require cl-lib
Date: Mon, 26 Mar 2018 02:30:29 +0100

"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> Any ideas on where I am or what is going wrong and how I can further
> troubleshoot this?

I ultimately nuked everything via 'git clean -fdx', as per INSTALL.REPO,
and the errors no longer appear.  Sorry about the noise.

-- 
Basil

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.