GNU bug report logs - #14120
invalid load-history in emacsen that CANNOT_DUMP

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 14120 in the body.
You can then email your comments to 14120 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: BT Templeton <bt <at> hcoop.net>
To: bug-gnu-emacs <at> gnu.org
Subject: invalid load-history in emacsen that CANNOT_DUMP
Date: Mon, 01 Apr 2013 18:47:09 -0400

"loadup.el" sets `current-load-list' to nil, apparently so that it will
have that value when restarting a dumped emacs. However, when Emacs is
built with CANNOT_DUMP defined, "loadup.el" evaluates `top-level'
instead of dumping. This can lead to invalid entries in `load-history'
if something is defined, required, etc. during evaluation of that form.

For example, load-history will contain `((require . tetris))' if
~/.emacs defines this hook:

(add-hook 'lisp-interaction-mode-hook
          (lambda () (require 'tetris)))

For `require', the problem can be fixed by also setting
`load-in-progress' to nil in "loadup.el", but this doesn't fix the bug
for other functions that change the load history.

-- 
Inteligenta persono lernas la lingvon Esperanton rapide kaj facile.
Esperanto estas moderna, kultura lingvo por la mondo. Simpla, fleksebla,
belsona, Esperanto estas la praktika solvo de la problemo de universala
interkompreno. Lernu la interlingvon Esperanton!

Message #8 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: BT Templeton <bt <at> hcoop.net>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Wed, 03 Apr 2013 14:29:40 -0400

> However, when Emacs is built with CANNOT_DUMP defined, "loadup.el"
> evaluates `top-level' instead of dumping.

FWIW, CANNOT_DUMP is a compilation option that is only meant to be used
as a temporary measure.  It suffers from various shortcomings (other
than the obvious startup cost).

So before we start trying to fix them, I'd like to know why you need to
use CANNOT_DUMP.


        Stefan

Message #17 received at 14120-done <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 14120-done <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Thu, 05 Sep 2019 20:09:52 -0400

Version: 27.1

The CANNOT_DUMP option no longer exists, and Emacs now uses a portable
dumper by default.

Message #24 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: 14120 <at> debbugs.gnu.org
Subject: [Robert Weiner] Emacs history selection non-string arguments
 causing failures for a long time
Date: Sun, 16 Feb 2020 15:55:12 -0500

[Message part 1 (message/rfc822, inline)]

From: Robert Weiner <rsw <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org
Subject: Emacs history selection non-string arguments causing failures for a
 long time
Date: Sun, 16 Feb 2020 01:14:48 -0500

[Message part 2 (text/plain, inline)]

Hi Stefan:

This bug #14120 in subr.el is still occurring in Emacs 27.0.50 because
leading entries like (require . info) rather than strings are being
injected into the history.  Since this has affected people for quite a
while, I hope you or someone else can add this one line change to the Emacs
27 branch and resolve it.

Bob

---------

*** subr-old.el 2020-02-16 01:05:56.000000000 -0500
--- subr.el 2020-02-16 01:06:28.000000000 -0500
***************
*** 4490,4496 ****
  (load-elt (and loads (car loads))))
      (save-match-data
        (while (and loads
!  (or (null (car load-elt))
       (not (string-match file-regexp (car load-elt)))))
  (setq loads (cdr loads)
       load-elt (and loads (car loads)))))
--- 4490,4499 ----
  (load-elt (and loads (car loads))))
      (save-match-data
        (while (and loads
!  ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=14120
!  ;; Avoid this bug still occuring in Emacs 27.0.50 by checking
!  ;; if load-elt is a string or not.
!  (or (not (stringp (car load-elt)))
       (not (string-match file-regexp (car load-elt)))))
  (setq loads (cdr loads)
       load-elt (and loads (car loads)))))

Message #27 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Robert Weiner <rsw <at> gnu.org>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: [Robert Weiner] Emacs history selection non-string
 arguments causing failures for a long time
Date: Sun, 16 Feb 2020 20:52:32 -0500

> This bug #14120 in subr.el is still occurring in Emacs 27.0.50 because
> leading entries like (require . info) rather than strings are being
> injected into the history.  Since this has affected people for quite a
> while, I hope you or someone else can add this one line change to the Emacs
> 27 branch and resolve it.

The original #14120 report was linked to the use of CANNOT_DUMP which is
not used any more, AFAICT, so we haven't really investigated this bug.

Concretely, I don't know how/where this shows up.  Do you have some
recipe (and/or backtrace)?

> !  ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=14120
> !  ;; Avoid this bug still occuring in Emacs 27.0.50 by checking
> !  ;; if load-elt is a string or not.
> !  (or (not (stringp (car load-elt)))
>        (not (string-match file-regexp (car load-elt)))))

The docstring of `load-history` says that (car load-elt) should never be
anything else than a string or nil.  So the right fix is likely
elsewhere.


        Stefan

Message #30 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Robert Weiner <rswgnu <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org, Robert Weiner <rsw <at> gnu.org>
Subject: Re: bug#14120: [Robert Weiner] Emacs history selection non-string
 arguments causing failures for a long time
Date: Sun, 16 Feb 2020 21:22:13 -0500

> On Feb 16, 2020, at 8:52 PM, Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:
> 
> The docstring of `load-history` says that (car load-elt) should never be
> anything else than a string or nil.  So the right fix is likely
> elsewhere.

Yes, I noticed that too, so maybe this is just a patch that allows history to work while filtering out these errant entries which may be as much as 20% of my total entries.  I hope someone can find the injection source.

Bob

Message #33 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org, Robert Weiner <rsw <at> gnu.org>
Subject: Re: bug#14120: [Robert Weiner] Emacs history selection non-string
 arguments causing failures for a long time
Date: Sun, 16 Feb 2020 22:18:40 -0500

Stefan Monnier wrote:

> The original #14120 report was linked to the use of CANNOT_DUMP which is
> not used any more, AFAICT, so we haven't really investigated this bug.
>
> Concretely, I don't know how/where this shows up.  Do you have some
> recipe (and/or backtrace)?

You could look at bugs 34094 and 34178, which, unlike 14120, are open
and not related to CANNOT_DUMP.

Message #36 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: rrandresf <at> gmail.com (Andrés Ramírez)
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: rrandresf <at> gmail.com, 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Mon, 17 Feb 2020 13:34:51 +0000

Hi Stefan.

> FWIW, CANNOT_DUMP is a compilation option that is only meant to be used
> as a temporary measure.  It suffers from various shortcomings (other
> than the obvious startup cost).

> So before we start trying to fix them, I'd like to know why you need to
> use CANNOT_DUMP.

I use CANNOT_DUMP on a daily basis on emacs 23.4 (the best emacs for
poor machines. it renders very quick. I would like to profile It against
latest emacs which I also use) It was compiled on on November 13 2019. I have suffered
the same issue as Templeton. So probably I have done the same workaround
as Templeton. See my function below:
--8<---------------cut here---------------start------------->8---
      (defun load-history-filename-element (file-regexp)
        "Get the first elt of `load-history' whose car matches FILE-REGEXP.
Return nil if there isn't one.
NOTE2ME: UPDATE stringp condition cos of error on tramp"
        (let* ((loads load-history)
               (load-elt (and loads (car loads))))
          (save-match-data
            (while (and loads
                        (or (null (and (car load-elt) (stringp (car load-elt))))
                            (not (string-match file-regexp (car load-elt)))))
              (setq loads (cdr loads)
                    load-elt (and loads (car loads)))))
          load-elt))
--8<---------------cut here---------------end--------------->8---

I have not reported the bug because. Emacs23 is very old emacs. And Also
most of Emacs developers believe It cann't compile with glibc 2.30. But
thanks to Gentoo packagers It does (Those guys have backported most of
the changes needed for Emacs23 to compile[1]. I have also upgraded tramp
built-in version there to 2.3.1. On x86 it compiles really fine. But on
armv7h It needs the CANNOT_DUMP compilation option. Because of an issue with
glibc on compilation it cores-dump).

I am answering the email probably just for the record. A couple of years
ago I was compiling very old code from nantuncked clipper (dbase like Prg
program). And I had run emacs20 from within DOS on dosbox (an
emulator) At that time. Anyone never knows ;).

Best Regards
[1] https://gitweb.gentoo.org/repo/gentoo.git/plain/app-editors/emacs/emacs-23.4-r14.ebuild

Message #39 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: rrandresf <at> gmail.com (Andrés Ramírez)
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Mon, 17 Feb 2020 10:09:59 -0500

>> FWIW, CANNOT_DUMP is a compilation option that is only meant to be used
>> as a temporary measure.  It suffers from various shortcomings (other
>> than the obvious startup cost).
>
>> So before we start trying to fix them, I'd like to know why you need to
>> use CANNOT_DUMP.
>
> I use CANNOT_DUMP on a daily basis on emacs 23.4 (the best emacs for
> poor machines. it renders very quick. I would like to profile It against
> latest emacs which I also use) It was compiled on on November 13 2019. I have suffered
> the same issue as Templeton.

Any chance you can try and investigate where/why/how that spurious value gets
pushed to `load-history`?

AFAICT the only place where `load-history` gets modified is in
lread.c:build_load_history.
Could you add some assertions there to try and catch the sucker?


        Stefan

Message #42 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: andrés ramírez <rrandresf <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Mon, 17 Feb 2020 17:14:54 +0000

Hi Stefan.

Stefan> Any chance you can try and investigate where/why/how that
Stefan> spurious value gets pushed to `load-history`?

Yes. I am going to try.

Stefan> AFAICT the only place where `load-history` gets modified is in
Stefan> lread.c:build_load_history.  Could you add some assertions
Stefan> there to try and catch the sucker?

I was aware of that. At that moment. I pointed to lread.c also. But this
seems too happen during startup (reading the dot emacs setup).

I am going to list what I would need to do for reaching the goal:
1. Recompile Emacs23 (for enabling debugging-symbols)
2.  Put a breakpoint on lread.c:build_load_history and watch how
many times it is called and with wich params. It seems a very long list
of params

3. After it. I suppose I should start emacs23 -Q  and try lo call one by
one lread.c:build_load_history with those params. And after every
function invocation I would need to check if that symbol(the spurious
one that corrupts the list) have got inside the
loads-history-list.

An idea: As I have a running emacs23 session right now.
--8<---------------cut here---------------start------------->8---
      (defun load-history-filename-element (file-regexp)
        "Get the first elt of `load-history' whose car matches FILE-REGEXP.
Return nil if there isn't one.
NOTE2ME: upd stringp condition cos of error on tramp"
        (let* ((loads load-history)
               (load-elt (and loads (car loads))))
          (save-match-data
            (while (and loads
                        (or (null (and (car load-elt) (stringp (car load-elt))))
                            (not (string-match file-regexp (car load-elt)))))
              (setq loads (cdr loads)
                    load-elt (and loads (car loads)))))
          load-elt))
 --8<---------------cut here---------------end--------------->8---

The function above could be modified for removing the stringp
validation. And every time load-history-filename-element is called when
an error happens that function show me the file on the history to which the spurious
symbol belongs to?

What Do You think?

AR

Message #45 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: andrés ramírez <rrandresf <at> gmail.com>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Mon, 17 Feb 2020 12:46:14 -0500

> 2.  Put a breakpoint on lread.c:build_load_history and watch how
> many times it is called and with wich params. It seems a very long list
> of params

It's called too often, that's not workable.
I was thinking of putting an assertion in there instead, maybe something
like the patch below.

> The function above could be modified for removing the stringp
> validation. And every time load-history-filename-element is called
> when an error happens that function show me the file on the history to
> which the spurious symbol belongs to?

The problem is that the file name is the string that's not there :-(

But maybe showing us the complete `load-history` when the error (in
load-history-filename-element) is caught might give us enough of a hint
(by looking at the entries nearby to try and infer the order of
operations at the time the broken entry was added)?


        Stefan


diff --git a/src/lread.c b/src/lread.c
index c124d5a1d8..7f5f1394c7 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -1876,8 +1876,14 @@ build_load_history (Lisp_Object filename, bool entire)
      front of load-history, the most-recently-loaded position.  Also
      do this if we didn't find an existing member for the file.  */
   if (entire || !foundit)
-    Vload_history = Fcons (Fnreverse (Vcurrent_load_list),
-			   Vload_history);
+    {
+      Lisp_Object tem = Fnreverse (Vcurrent_load_list);
+      eassert (EQ (filename, Fcar (tem)));
+      Vload_history = Fcons (tem, Vload_history);
+      /* FIXME: There should be an unbind_to right after calling us which
+         should re-establish the previous value of Vcurrent_load_list.  */
+      Vcurrent_load_list = Qt;
+    }
 }
 
 static void

Message #48 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: andrés ramírez <rrandresf <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Tue, 18 Feb 2020 01:01:35 +0000

[Message part 1 (text/plain, inline)]

Hi Stefan.

Stefan> It's called too often, that's not workable.  I was thinking of
Stefan> putting an assertion in there instead, maybe something like
Stefan> the patch below.

I have applied the patch. But no assert is trigered.
--8<---------------cut here---------------start------------->8---
GNU Emacs 23.4 (armv7l-unknown-linux-gnueabi)
 of 2020-02-18 on sacsa
--8<---------------cut here---------------end--------------->8---

Stefan> But maybe showing us the complete `load-history` when the
Stefan> error (in load-history-filename-element) is caught might give
Stefan> us enough of a hint (by looking at the entries nearby to try
Stefan> and infer the order of operations at the time the broken entry
Stefan> was added)?

I have compiled. And I am running the just compiled emacs23. The error
is this one:
--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (wrong-type-argument stringp (require . smartparens-config))
  string-match("\\(\\`\\|/\\)esh-util\\(\\.elc\\|\\.el\\)?\\(\\.gz\\)?\\'" (require . smartparens-config))
  (not (string-match file-regexp (car load-elt)))
  (or (null (and ...)) (not (string-match file-regexp ...)))
  (and loads (or (null ...) (not ...)))
  (while (and loads (or ... ...)) (setq loads (cdr loads) load-elt (and loads ...)))
  (progn (while (and loads ...) (setq loads ... load-elt ...)))
  (unwind-protect (progn (while ... ...)) (set-match-data save-match-data-internal (quote evaporate)))
  (let ((save-match-data-internal ...)) (unwind-protect (progn ...) (set-match-data save-match-data-internal ...)))
  (save-match-data (while (and loads ...) (setq loads ... load-elt ...)))
  (let* ((loads load-history) (load-elt ...)) (save-match-data (while ... ...)) load-elt)
  load-history-filename-element("\\(\\`\\|/\\)esh-util\\(\\.elc\\|\\.el\\)?\\(\\.gz\\)?\\'")
  eval-after-load("esh-util" (progn (add-hook (quote eshell-mode-hook) (quote tramp-eshell-directory-change)) (add-hook (quote eshell-directory-change-hook) (quote tramp-eshell-directory-change)) (add-hook (quote tramp-unload-hook) (lambda nil ... ...))))
  require(tramp)
  eval((require (quote tramp)))
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp nil nil)
--8<---------------cut here---------------end--------------->8---

and the load-history var is this one:

[emacs23-load-history-var.txt (text/plain, attachment)]

[Message part 3 (text/plain, inline)]

AR
ps: Let me know anything else I could do.

Message #51 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: andrés ramírez <rrandresf <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Tue, 18 Feb 2020 01:34:00 +0000

Hi Stefan. I have something to add.

AR> and the load-history var is this one: [2 file-var <text/plain
AR> (quoted-printable)>] load-history is a variable defined in `C
AR> source code'.  Its value is shown below.

Is the line below the one that is causing the issue?
--8<---------------cut here---------------start------------->8---
((require . smartparens-config))
--8<---------------cut here---------------end--------------->8---

BR
ps: If that is the case I need to share my emacs invocation and the
funct s-load-now/smartparens

Message #54 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: andrés ramírez <rrandresf <at> gmail.com>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Tue, 18 Feb 2020 08:40:36 -0500

> Hi Stefan. I have something to add.
>
> AR> and the load-history var is this one: [2 file-var <text/plain
> AR> (quoted-printable)>] load-history is a variable defined in `C
> AR> source code'.  Its value is shown below.
>
> Is the line below the one that is causing the issue?
> --8<---------------cut here---------------start------------->8---
> ((require . smartparens-config))
> --8<---------------cut here---------------end--------------->8---

Yes, this is the broken entry in `load-history`.
The question is how/why it got there.  The immediately
preceding/following entries don't give much hint, other than saying that
/home/olla/.emacs.d/my-noexternals.el might be happening nearby.

Can you grep for `smartparens-config` in all your Elisp files and show
us the surrounding Lisp code?


        Stefan

Message #57 received at 14120 <at> debbugs.gnu.org (full text, mbox):

From: andrés ramírez <rrandresf <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 14120 <at> debbugs.gnu.org
Subject: Re: bug#14120: invalid load-history in emacsen that CANNOT_DUMP
Date: Tue, 18 Feb 2020 13:50:13 +0000

Hi Stefan.

Stefan> Can you grep for `smartparens-config` in all your Elisp files
Stefan> and show us the surrounding Lisp code?

This is my emacs invoking line:
--8<---------------cut here---------------start------------->8---
gdb --args ./emacs -Q --eval "(progn (setq my-cli-not-init-pkgs-flag t)(load \"~/.emacs.d/init.el\") (s-load-now/god-mode)(s-load-now/ace-window-mode)(s-load-now/smartparens) (run-at-time \"1\" nil  #'find-file \"~/notes.md\"))"
--8<---------------cut here---------------end--------------->8---

This is s-load-now/smartparens
--8<---------------cut here---------------start------------->8---
(defun s-load-now/smartparens ()
  "load smartparent for emacs-23
  despues de instalarlo necesitaba un require para que funcione globalmente
just do smartparens-global-mode"
  (interactive)
  (when (file-exists-p "~/.emacs.d/elpa/dash/dash.el")
    (load-file "~/.emacs.d/elpa/dash/dash.el")
    (when (file-exists-p "~/.emacs.d/lisp/smartparens")
      (add-to-list 'load-path "~/.emacs.d/lisp/smartparens")
      (require (quote smartparens-config))
      (smartparens-global-mode)
      )))
 --8<---------------cut here---------------end--------------->8---

Best Regards

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.