GNU bug report logs - #46617
28.0.50; nativecomp: native compile cache is not invalidated when file is re-byte compiled and changes

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46617 in the body.
You can then email your comments to 46617 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Cc: Andrea Corallo <akrl <at> sdf.org>
Subject: 28.0.50; nativecomp: native compile cache is not invalidated when
 file is re-byte compiled and changes
Date: Thu, 18 Feb 2021 09:42:40 -0600

Create these two files in a directory, then run:

emacs -Q -l init.el

init.el:
(let ((repro
       (expand-file-name "repro.el" default-directory)))
  (defun some-fn ()
    1)
  (message "Byte and native compiling, some-fn = 1")
  (byte-compile-file repro)
  (native-compile repro)
  (load (concat repro "c") nil 'nomessage 'nosuffix)

  (message "Byte compiling, some-fn = 2")
  (defun some-fn ()
    2)
  (byte-compile-file repro)
  (load (concat repro "c") nil 'nomessage 'nosuffix)

  (message "Native compiling, some-fn = 2")
  (native-compile repro)
  (load (concat repro "c") nil 'nomessage 'nosuffix))
  
repro.el:
(eval
 `(if (equal
       (some-fn)
       ,(eval-when-compile (some-fn)))
      (message "EQUAL")
    (message "NOT EQUAL")))

You should see the output:

Byte and native compiling, some-fn = 1
Compiling /home/code/tmp/straight-repro/.emacs.d/repro.el...done
Wrote /home/code/tmp/straight-repro/.emacs.d/repro.elc
Compiling /home/code/tmp/straight-repro/.emacs.d/repro.el...done
EQUAL
Byte compiling, some-fn = 2
Compiling /home/code/tmp/straight-repro/.emacs.d/repro.el...done
Wrote /home/code/tmp/straight-repro/.emacs.d/repro.elc
NOT EQUAL
Native compiling, some-fn = 2
Compiling /home/code/tmp/straight-repro/.emacs.d/repro.el...done
EQUAL

I would expect that the second NOT EQUAL would be EQUAL because it would
load the byte-compiled file as it would deem the eln out-dated. Instead,
it loads the old eln despite the fact that the elc has changed.

In the wild, this causes issues downstream with the emacs-version change
detection mechanism in straight.el:

https://github.com/raxod502/straight.el/issues/697


In GNU Emacs 28.0.50 (build 4, x86_64-apple-darwin19.6.0, NS appkit-1894.60 Version 10.15.7 (Build 19H512))
 of 2021-02-17 built on aaron-sub.local
Repository revision: 24926a03cdaf36017df27e947abb02aa626b8a99
Repository branch: feature/native-comp
Windowing system distributor 'Apple', version 10.3.1894
System Description:  Mac OS X 10.15.7

Configured using:
 'configure 'LDFLAGS=-L/usr/local/lib/gcc/10
 -I/usr/local/opt/gcc/include -I/usr/local/opt/libgccjit/include
 -I/usr/local/opt/gmp/include -I/usr/local/opt/jpeg/include'
 'CFLAGS=-I/usr/local/opt/gcc/include -I/usr/local/opt/libgccjit/include
 -I/usr/local/opt/gmp/include -I/usr/local/opt/jpeg/include -g3 -Og'
 --disable-dependency-tracking --disable-silent-rules --with-xml2
 --with-gnutls --with-nativecomp --without-dbus --with-imagemagick
 --with-modules --with-rsvg --with-ns --disable-ns-self-contained'

Configured features:
ACL GLIB GMP GNUTLS IMAGEMAGICK JPEG JSON LCMS2 LIBXML2 MODULES
NATIVE_COMP NOTIFY KQUEUE NS PDUMPER PNG RSVG THREADS TIFF
TOOLKIT_SCROLL_BARS ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  selectrum-prescient-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  global-git-commit-mode: t
  transient-posframe-mode: t
  company-posframe-mode: t
  company-prescient-mode: t
  company-tng-mode: t
  company-mode: t
  show-paren-mode: t
  eval-sexp-fu-flash-mode: t
  eros-mode: t
  lispyville-mode: t
  lispy-mode: t
  elisp-def-mode: t
  sotlisp-mode: t
  speed-of-thought-mode: t
  dtrt-indent-mode: t
  form-feed-mode: t
  global-auto-revert-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  which-key-posframe-mode: t
  which-key-mode: t
  winner-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  smartparens-global-mode: t
  global-evil-collection-unimpaired-mode: t
  evil-collection-unimpaired-mode: t
  global-evil-mc-mode: t
  evil-mc-mode: t
  global-evil-matchit-mode: t
  evil-matchit-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  shell-dirtrack-mode: t
  evil-mode: t
  evil-local-mode: t
  envrc-global-mode: t
  envrc-mode: t
  recentf-mode: t
  projectile-mode: t
  persp-mode: t
  mini-frame-mode: t
  gcmh-mode: t
  selectrum-mode: t
  prescient-persist-mode: t
  +popup-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  save-place-mode: t
  ns-auto-titlebar-mode: t
  doom-modeline-mode: t
  leader-key-leader-override-mode: t
  global-leader-key-leader-override-mode: t
  delete-selection-mode: t
  savehist-mode: t
  xterm-mouse-mode: t
  override-global-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  auto-fill-function: yas--auto-fill
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/home/.emacs.d/straight/build/ivy/elpa hides /home/.emacs.d/straight/build/lispy/elpa
/home/.emacs.d/straight/build/org-plus-contrib/ob-stan hides /home/code/emacs/lisp/org/ob-stan
/home/.emacs.d/straight/build/org-plus-contrib/ob-comint hides /home/code/emacs/lisp/org/ob-comint
/home/.emacs.d/straight/build/org-plus-contrib/ob-exp hides /home/code/emacs/lisp/org/ob-exp
/home/.emacs.d/straight/build/org-plus-contrib/org-ctags hides /home/code/emacs/lisp/org/org-ctags
/home/.emacs.d/straight/build/org-plus-contrib/ob-emacs-lisp hides /home/code/emacs/lisp/org/ob-emacs-lisp
/home/.emacs.d/straight/build/org-plus-contrib/ox-texinfo hides /home/code/emacs/lisp/org/ox-texinfo
/home/.emacs.d/straight/build/org-plus-contrib/ol-irc hides /home/code/emacs/lisp/org/ol-irc
/home/.emacs.d/straight/build/org-plus-contrib/ob hides /home/code/emacs/lisp/org/ob
/home/.emacs.d/straight/build/org-plus-contrib/org-refile hides /home/code/emacs/lisp/org/org-refile
/home/.emacs.d/straight/build/org-plus-contrib/org-version hides /home/code/emacs/lisp/org/org-version
/home/.emacs.d/straight/build/org-plus-contrib/org-num hides /home/code/emacs/lisp/org/org-num
/home/.emacs.d/straight/build/org-plus-contrib/ol-mhe hides /home/code/emacs/lisp/org/ol-mhe
/home/.emacs.d/straight/build/org-plus-contrib/ob-shell hides /home/code/emacs/lisp/org/ob-shell
/home/.emacs.d/straight/build/org-plus-contrib/org-attach hides /home/code/emacs/lisp/org/org-attach
/home/.emacs.d/straight/build/org-plus-contrib/ob-C hides /home/code/emacs/lisp/org/ob-C
/home/.emacs.d/straight/build/org-plus-contrib/org-macs hides /home/code/emacs/lisp/org/org-macs
/home/.emacs.d/straight/build/org-plus-contrib/org-entities hides /home/code/emacs/lisp/org/org-entities
/home/.emacs.d/straight/build/org-plus-contrib/ob-dot hides /home/code/emacs/lisp/org/ob-dot
/home/.emacs.d/straight/build/org-plus-contrib/ob-sql hides /home/code/emacs/lisp/org/ob-sql
/home/.emacs.d/straight/build/org-plus-contrib/ol-eww hides /home/code/emacs/lisp/org/ol-eww
/home/.emacs.d/straight/build/org-plus-contrib/org-datetree hides /home/code/emacs/lisp/org/org-datetree
/home/.emacs.d/straight/build/org-plus-contrib/org-macro hides /home/code/emacs/lisp/org/org-macro
/home/.emacs.d/straight/build/org-plus-contrib/ob-eval hides /home/code/emacs/lisp/org/ob-eval
/home/.emacs.d/straight/build/org-plus-contrib/ob-haskell hides /home/code/emacs/lisp/org/ob-haskell
/home/.emacs.d/straight/build/org-plus-contrib/ob-asymptote hides /home/code/emacs/lisp/org/ob-asymptote
/home/.emacs.d/straight/build/org-plus-contrib/ob-abc hides /home/code/emacs/lisp/org/ob-abc
/home/.emacs.d/straight/build/org-plus-contrib/ox-org hides /home/code/emacs/lisp/org/ox-org
/home/.emacs.d/straight/build/org-plus-contrib/ol-rmail hides /home/code/emacs/lisp/org/ol-rmail
/home/.emacs.d/straight/build/org-plus-contrib/ob-awk hides /home/code/emacs/lisp/org/ob-awk
/home/.emacs.d/straight/build/org-plus-contrib/ob-groovy hides /home/code/emacs/lisp/org/ob-groovy
/home/.emacs.d/straight/build/org-plus-contrib/ob-shen hides /home/code/emacs/lisp/org/ob-shen
/home/.emacs.d/straight/build/org-plus-contrib/ox-icalendar hides /home/code/emacs/lisp/org/ox-icalendar
/home/.emacs.d/straight/build/org-plus-contrib/ob-octave hides /home/code/emacs/lisp/org/ob-octave
/home/.emacs.d/straight/build/org-plus-contrib/ob-scheme hides /home/code/emacs/lisp/org/ob-scheme
/home/.emacs.d/straight/build/org-plus-contrib/org-mobile hides /home/code/emacs/lisp/org/org-mobile
/home/.emacs.d/straight/build/org-plus-contrib/ob-processing hides /home/code/emacs/lisp/org/ob-processing
/home/.emacs.d/straight/build/org-plus-contrib/ob-ledger hides /home/code/emacs/lisp/org/ob-ledger
/home/.emacs.d/straight/build/org-plus-contrib/ob-picolisp hides /home/code/emacs/lisp/org/ob-picolisp
/home/.emacs.d/straight/build/org-plus-contrib/org-colview hides /home/code/emacs/lisp/org/org-colview
/home/.emacs.d/straight/build/org-plus-contrib/ob-R hides /home/code/emacs/lisp/org/ob-R
/home/.emacs.d/straight/build/org-plus-contrib/org-table hides /home/code/emacs/lisp/org/org-table
/home/.emacs.d/straight/build/org-plus-contrib/ox-html hides /home/code/emacs/lisp/org/ox-html
/home/.emacs.d/straight/build/org-plus-contrib/ob-fortran hides /home/code/emacs/lisp/org/ob-fortran
/home/.emacs.d/straight/build/org-plus-contrib/ol hides /home/code/emacs/lisp/org/ol
/home/.emacs.d/straight/build/org-plus-contrib/ob-plantuml hides /home/code/emacs/lisp/org/ob-plantuml
/home/.emacs.d/straight/build/org-plus-contrib/ol-docview hides /home/code/emacs/lisp/org/ol-docview
/home/.emacs.d/straight/build/org-plus-contrib/ob-perl hides /home/code/emacs/lisp/org/ob-perl
/home/.emacs.d/straight/build/org-plus-contrib/ob-sqlite hides /home/code/emacs/lisp/org/ob-sqlite
/home/.emacs.d/straight/build/org-plus-contrib/ob-sed hides /home/code/emacs/lisp/org/ob-sed
/home/.emacs.d/straight/build/org-plus-contrib/ob-ditaa hides /home/code/emacs/lisp/org/ob-ditaa
/home/.emacs.d/straight/build/org-plus-contrib/ob-ruby hides /home/code/emacs/lisp/org/ob-ruby
/home/.emacs.d/straight/build/org-plus-contrib/org-habit hides /home/code/emacs/lisp/org/org-habit
/home/.emacs.d/straight/build/org-plus-contrib/ol-gnus hides /home/code/emacs/lisp/org/ol-gnus
/home/.emacs.d/straight/build/org-plus-contrib/ob-screen hides /home/code/emacs/lisp/org/ob-screen
/home/.emacs.d/straight/build/org-plus-contrib/org-mouse hides /home/code/emacs/lisp/org/org-mouse
/home/.emacs.d/straight/build/org-plus-contrib/ob-css hides /home/code/emacs/lisp/org/ob-css
/home/.emacs.d/straight/build/org-plus-contrib/org-inlinetask hides /home/code/emacs/lisp/org/org-inlinetask
/home/.emacs.d/straight/build/org-plus-contrib/ob-lisp hides /home/code/emacs/lisp/org/ob-lisp
/home/.emacs.d/straight/build/org-plus-contrib/ol-eshell hides /home/code/emacs/lisp/org/ol-eshell
/home/.emacs.d/straight/build/org-plus-contrib/org-pcomplete hides /home/code/emacs/lisp/org/org-pcomplete
/home/.emacs.d/straight/build/org-plus-contrib/org-lint hides /home/code/emacs/lisp/org/org-lint
/home/.emacs.d/straight/build/org-plus-contrib/org-id hides /home/code/emacs/lisp/org/org-id
/home/.emacs.d/straight/build/org-plus-contrib/ob-vala hides /home/code/emacs/lisp/org/ob-vala
/home/.emacs.d/straight/build/org-plus-contrib/org-capture hides /home/code/emacs/lisp/org/org-capture
/home/.emacs.d/straight/build/org-plus-contrib/ob-sass hides /home/code/emacs/lisp/org/ob-sass
/home/.emacs.d/straight/build/org-plus-contrib/ob-tangle hides /home/code/emacs/lisp/org/ob-tangle
/home/.emacs.d/straight/build/org-plus-contrib/ob-calc hides /home/code/emacs/lisp/org/ob-calc
/home/.emacs.d/straight/build/org-plus-contrib/ob-java hides /home/code/emacs/lisp/org/ob-java
/home/.emacs.d/straight/build/org-plus-contrib/org-compat hides /home/code/emacs/lisp/org/org-compat
/home/.emacs.d/straight/build/org-plus-contrib/org-attach-git hides /home/code/emacs/lisp/org/org-attach-git
/home/.emacs.d/straight/build/org-plus-contrib/ox-beamer hides /home/code/emacs/lisp/org/ox-beamer
/home/.emacs.d/straight/build/org-plus-contrib/ob-coq hides /home/code/emacs/lisp/org/ob-coq
/home/.emacs.d/straight/build/org-plus-contrib/org-protocol hides /home/code/emacs/lisp/org/org-protocol
/home/.emacs.d/straight/build/org-plus-contrib/org-element hides /home/code/emacs/lisp/org/org-element
/home/.emacs.d/straight/build/org-plus-contrib/ob-lob hides /home/code/emacs/lisp/org/ob-lob
/home/.emacs.d/straight/build/org-plus-contrib/ob-mscgen hides /home/code/emacs/lisp/org/ob-mscgen
/home/.emacs.d/straight/build/org-plus-contrib/org-tempo hides /home/code/emacs/lisp/org/org-tempo
/home/.emacs.d/straight/build/org-plus-contrib/ob-python hides /home/code/emacs/lisp/org/ob-python
/home/.emacs.d/straight/build/org-plus-contrib/ob-latex hides /home/code/emacs/lisp/org/ob-latex
/home/.emacs.d/straight/build/org-plus-contrib/ol-w3m hides /home/code/emacs/lisp/org/ol-w3m
/home/.emacs.d/straight/build/org-plus-contrib/org-agenda hides /home/code/emacs/lisp/org/org-agenda
/home/.emacs.d/straight/build/org-plus-contrib/ob-ocaml hides /home/code/emacs/lisp/org/ob-ocaml
/home/.emacs.d/straight/build/org-plus-contrib/ob-ref hides /home/code/emacs/lisp/org/ob-ref
/home/.emacs.d/straight/build/org-plus-contrib/ob-lilypond hides /home/code/emacs/lisp/org/ob-lilypond
/home/.emacs.d/straight/build/org-plus-contrib/ob-table hides /home/code/emacs/lisp/org/ob-table
/home/.emacs.d/straight/build/org-plus-contrib/ob-clojure hides /home/code/emacs/lisp/org/ob-clojure
/home/.emacs.d/straight/build/org-plus-contrib/org-indent hides /home/code/emacs/lisp/org/org-indent
/home/.emacs.d/straight/build/org-plus-contrib/org-plot hides /home/code/emacs/lisp/org/org-plot
/home/.emacs.d/straight/build/org-plus-contrib/ox-latex hides /home/code/emacs/lisp/org/ox-latex
/home/.emacs.d/straight/build/org-plus-contrib/org-src hides /home/code/emacs/lisp/org/org-src
/home/.emacs.d/straight/build/org-plus-contrib/org-duration hides /home/code/emacs/lisp/org/org-duration
/home/.emacs.d/straight/build/org-plus-contrib/ob-makefile hides /home/code/emacs/lisp/org/ob-makefile
/home/.emacs.d/straight/build/org-plus-contrib/ol-info hides /home/code/emacs/lisp/org/ol-info
/home/.emacs.d/straight/build/org-plus-contrib/ob-ebnf hides /home/code/emacs/lisp/org/ob-ebnf
/home/.emacs.d/straight/build/org-plus-contrib/org-clock hides /home/code/emacs/lisp/org/org-clock
/home/.emacs.d/straight/build/org-plus-contrib/ob-forth hides /home/code/emacs/lisp/org/ob-forth
/home/.emacs.d/straight/build/org-plus-contrib/ox-odt hides /home/code/emacs/lisp/org/ox-odt
/home/.emacs.d/straight/build/org-plus-contrib/ox-publish hides /home/code/emacs/lisp/org/ox-publish
/home/.emacs.d/straight/build/org-plus-contrib/org-archive hides /home/code/emacs/lisp/org/org-archive
/home/.emacs.d/straight/build/org-plus-contrib/ob-org hides /home/code/emacs/lisp/org/ob-org
/home/.emacs.d/straight/build/org-plus-contrib/ob-lua hides /home/code/emacs/lisp/org/ob-lua
/home/.emacs.d/straight/build/org-plus-contrib/org-keys hides /home/code/emacs/lisp/org/org-keys
/home/.emacs.d/straight/build/org-plus-contrib/ob-eshell hides /home/code/emacs/lisp/org/ob-eshell
/home/.emacs.d/straight/build/org-plus-contrib/org-faces hides /home/code/emacs/lisp/org/org-faces
/home/.emacs.d/straight/build/org-plus-contrib/ox-man hides /home/code/emacs/lisp/org/ox-man
/home/.emacs.d/straight/build/org-plus-contrib/org-list hides /home/code/emacs/lisp/org/org-list
/home/.emacs.d/straight/build/org-plus-contrib/ox-md hides /home/code/emacs/lisp/org/ox-md
/home/.emacs.d/straight/build/org-plus-contrib/org-goto hides /home/code/emacs/lisp/org/org-goto
/home/.emacs.d/straight/build/org-plus-contrib/ol-bbdb hides /home/code/emacs/lisp/org/ol-bbdb
/home/.emacs.d/straight/build/org-plus-contrib/org-loaddefs hides /home/code/emacs/lisp/org/org-loaddefs
/home/.emacs.d/straight/build/org-plus-contrib/org hides /home/code/emacs/lisp/org/org
/home/.emacs.d/straight/build/org-plus-contrib/ol-bibtex hides /home/code/emacs/lisp/org/ol-bibtex
/home/.emacs.d/straight/build/org-plus-contrib/ob-J hides /home/code/emacs/lisp/org/ob-J
/home/.emacs.d/straight/build/org-plus-contrib/ox-ascii hides /home/code/emacs/lisp/org/ox-ascii
/home/.emacs.d/straight/build/org-plus-contrib/ob-matlab hides /home/code/emacs/lisp/org/ob-matlab
/home/.emacs.d/straight/build/org-plus-contrib/ox hides /home/code/emacs/lisp/org/ox
/home/.emacs.d/straight/build/org-plus-contrib/org-timer hides /home/code/emacs/lisp/org/org-timer
/home/.emacs.d/straight/build/org-plus-contrib/ob-core hides /home/code/emacs/lisp/org/ob-core
/home/.emacs.d/straight/build/org-plus-contrib/org-feed hides /home/code/emacs/lisp/org/org-feed
/home/.emacs.d/straight/build/org-plus-contrib/ob-io hides /home/code/emacs/lisp/org/ob-io
/home/.emacs.d/straight/build/org-plus-contrib/ob-gnuplot hides /home/code/emacs/lisp/org/ob-gnuplot
/home/.emacs.d/straight/build/org-plus-contrib/ob-js hides /home/code/emacs/lisp/org/ob-js
/home/.emacs.d/straight/build/org-plus-contrib/org-footnote hides /home/code/emacs/lisp/org/org-footnote
/home/.emacs.d/straight/build/org-plus-contrib/ob-maxima hides /home/code/emacs/lisp/org/ob-maxima
/home/.emacs.d/straight/build/org-plus-contrib/ob-hledger hides /home/code/emacs/lisp/org/ob-hledger
/home/.emacs.d/straight/build/org-plus-contrib/org-install hides /home/code/emacs/lisp/org/org-install
/home/.emacs.d/straight/build/org-plus-contrib/org-crypt hides /home/code/emacs/lisp/org/org-crypt
/home/.emacs.d/straight/build/let-alist/let-alist hides /home/code/emacs/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr emacsbug sendmail completing-read-xref
semantic/ia semantic/analyze/refs semantic/db-find semantic/db-ref
semantic/senator semantic/decorate pulse semantic/analyze semantic/sort
semantic/scope semantic/analyze/fcn semantic/ctxt semantic/format
semantic/tag-ls semantic/find cl-print shortdoc rspec-mode inf-ruby
smartparens-ruby ruby-mode sql view lsp-steep lsp-svelte lsp-sqls
lsp-yaml lsp-xml lsp-vimscript lsp-vhdl lsp-vetur lsp-html lsp-verilog
lsp-vala lsp-terraform lsp-tex lsp-sorbet lsp-solargraph lsp-rust lsp-rf
lsp-r lsp-purescript lsp-pyls lsp-pwsh lsp-php lsp-perl lsp-ocaml
lsp-nix lsp-nim lsp-lua lsp-kotlin lsp-json lsp-javascript lsp-haxe
lsp-groovy lsp-hack lsp-go lsp-completion lsp-gdscript lsp-fsharp
lsp-fortran lsp-eslint lsp-erlang lsp-elixir lsp-elm lsp-dockerfile
lsp-dhall lsp-css lsp-csharp gnutls lsp-crystal lsp-cmake lsp-clojure
lsp-clangd lsp-bash lsp-angular lsp-ada lsp-actionscript lsp-mode
lsp-protocol spinner network-stream smartparens-markdown markdown-mode
inline ht bindat smartparens-lua lua-mode checkdoc lisp-mnt rng-xsd
xsd-regexp rng-cmpct rng-nxml rng-valid nxml-mode nxml-outln nxml-rap
smartparens-html sgml-mode dired-aux org-journal org-crypt cal-move
consult-selectrum consult magit-bookmark bookmark tabify timezone
org-mac-link org-capture restart-emacs desktop frameset
evil-collection-helpful helpful trace org-goto sh-script smie json-mode
json-reformat json-snatcher smartparens-javascript js smartparens-c
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs cal-iso org-duration diary-lib diary-loaddefs org-agenda
char-fold selectrum-prescient selectrum minibuf-eldef executable
bug-reference face-remap evil-collection-magit magit-submodule
magit-obsolete magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
package url-handlers flycheck magit-repos magit-apply magit-wip
magit-log which-func imenu magit-diff smerge-mode git-commit log-edit
pcvs-util add-log magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor magit-mode transient-posframe
transient magit-git magit-section magit-utils crm info-look
dash-functional elisp-refs dired-subtree dired-hacks-utils
evil-collection-profiler profiler company-files company-capf
company-posframe company-prescient company-tng company paren
eval-sexp-fu eros lispyville lispy swiper ivy ivy-faces ivy-overlay
colir color lispy-inline avy etags fileloop evil-collection-edebug
edebug help-fns radix-tree lispy-tags semantic/db eieio-base
semantic/util-modes semantic/util semantic semantic/tag semantic/lex
semantic/fw mode-local cedet ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util iedit iedit-lib elisp-def
ert ewoc evil-collection-debug debug backtrace evil-collection-xref xref
sotlisp skeleton dtrt-indent vc-mtn vc-hg vc-git diff-mode vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher project org-appear
orgonomic form-feed ol-eww eww xdg url-queue mm-url ol-rmail ol-mhe
ol-irc ol-info ol-gnus nnselect gnus-search eieio-opt speedbar ezimage
dframe gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum shr
kinsoku svg dom browse-url gnus-group gnus-undo gnus-start gnus-dbus
dbus xml gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo parse-time
gnus-spec gnus-int gnus-range message rfc822 mml mml-sec epa epg
epg-config mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader
gnus-win gnus nnheader gnus-util rmail rmail-loaddefs mail-utils
ol-docview doc-view jka-compr image-mode exif evil-collection-dired
dired dired-loaddefs ol-bibtex bibtex iso8601 ol-bbdb ol-w3m
smartparens-org ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar ox-ascii ox-gfm ox-md ox-html table ox-publish ox
org-download org-attach org-id org-refile url-http async org-protocol
org-tempo tempo evil-org-agenda evil-org org-element avl-tree generator
ob-shell org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-footnote org-src ob-comint org-pcomplete org-list org-faces
org-entities time-date noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table ol org-keys org-compat org-macs org-loaddefs
format-spec find-func cal-menu calendar cal-loaddefs hide-mode-line
popup-mode-core autorevert filenotify yasnippet popup-mode-hacks
evil-collection-which-key which-key-posframe posframe which-key winner
url url-proxy url-privacy url-expand url-methods url-history mailcap
url-auth mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr url-cookie url-domsuf url-util url-gw url-parse auth-source
eieio eieio-core eieio-loaddefs password-cache url-vars nsm rmc puny
better-jumper smartparens-config smartparens-text smartparens evil-anzu
anzu evil-collection-unimpaired evil-collection-ibuffer
evil-collection-help evil-collection-custom evil-collection-process-menu
evil-collection annalist evil-mc evil-mc-command-execute
evil-mc-command-record evil-mc-cursor-make evil-mc-region
evil-mc-cursor-state evil-mc-undo evil-mc-vars evil-mc-known-commands
evil-mc-common evil-numbers evil-matchit evil-matchit-sdk evil-surround
cus-edit cus-start cus-load evil evil-integration evil-maps
evil-commands reveal flyspell ispell evil-jumps evil-command-window
evil-types evil-search evil-ex shell pcomplete evil-macros evil-repeat
evil-states evil-core evil-common windmove rect evil-digraphs evil-vars
envrc inheritenv json map recentf tree-widget wid-edit persp-projectile
projectile grep compile text-property-search comint ansi-color ibuf-ext
ibuffer ibuffer-loaddefs perspective advice thingatpt ido mini-frame
server gcmh modus-operandi-theme modus-themes gcmh-autoloads
explain-pause-mode-autoloads restclient-autoloads vterm-toggle-autoloads
vterm-autoloads dumb-jump-autoloads popup-autoloads
xterm-color-autoloads dockerfile-mode-autoloads yaml-mode-autoloads
json-mode-autoloads json-snatcher-autoloads json-reformat-autoloads
lua-mode-autoloads bundler-autoloads inf-ruby-autoloads
ruby-refactor-autoloads rspec-mode-autoloads
evil-ruby-text-objects-autoloads enh-ruby-mode-autoloads
sotlisp-autoloads elisp-def-autoloads lispyville-autoloads
lispy-autoloads zoutline-autoloads counsel-autoloads swiper-autoloads
ivy-autoloads iedit-autoloads eros-autoloads eval-sexp-fu-autoloads
eslintd-fix-autoloads web-mode-autoloads company-rg-autoloads
company-lsp-autoloads company-posframe-autoloads
company-prescient-autoloads company-autoloads git-link-autoloads
prettier-autoloads nvm-autoloads iter2-autoloads flycheck-autoloads
let-alist-autoloads lsp-ui-autoloads lsp-mode-autoloads
markdown-mode-autoloads spinner-autoloads ox-gfm-autoloads
org-pandoc-import-autoloads gnuplot-autoloads org-download-autoloads
async-autoloads org-journal-autoloads deft-autoloads org-roam-autoloads
emacsql-sqlite3-autoloads emacsql-autoloads orgonomic-autoloads
org-drill-autoloads persist-autoloads org-appear-autoloads
evil-org-autoloads transient-posframe-autoloads restart-emacs-autoloads
better-jumper-autoloads hydra ring lv buffer-move-autoloads
completing-read-xref-autoloads mini-frame-autoloads
embark-consult-autoloads embark-autoloads consult-autoloads
selectrum-prescient-autoloads selectrum-autoloads
persp-projectile-autoloads prescient prescient-autoloads
which-key-posframe-autoloads which-key-autoloads popup-mode
popup-mode-settings popup-mode-autoloads hide-mode-line-autoloads
wgrep-autoloads yasnippet-autoloads form-feed-autoloads
drag-stuff-autoloads smartparens-autoloads dtrt-indent-autoloads
ws-butler ws-butler-autoloads evil-anzu-autoloads anzu-autoloads
evil-collection-autoloads annalist-autoloads evil-mc-autoloads
evil-numbers-autoloads speeddating-autoloads evil-matchit-autoloads
evil-nerd-commenter-autoloads evil-visualstar-autoloads
evil-surround-autoloads pp undo-tree diff queue undo-tree-autoloads
queue-autoloads treemacs-perspective-autoloads perspective-autoloads
treemacs-magit-autoloads magit-autoloads git-commit-autoloads
with-editor-autoloads transient-autoloads
treemacs-all-the-icons-autoloads treemacs-projectile-autoloads
treemacs-evil-autoloads evil-autoloads goto-chg-autoloads
treemacs-autoloads cfrs-autoloads ht-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads projectile-autoloads
pkg-info-autoloads epl-autoloads saveplace rainbow-mode-autoloads
posframe-autoloads ns-auto-titlebar ns-auto-titlebar-autoloads
doom-modeline doom-modeline-segments let-alist doom-modeline-env
doom-modeline-core shrink-path f s all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons dash doom-modeline-autoloads
shrink-path-autoloads all-the-icons-autoloads modus-themes-autoloads
envrc-autoloads inheritenv-autoloads compdef derived compdef-autoloads
helpful-autoloads elisp-refs-autoloads f-autoloads s-autoloads
dash-functional-autoloads dired-subtree-autoloads
dired-hacks-utils-autoloads dash-autoloads use-package-bind-key
hydra-autoloads lv-autoloads leader-key bind-map leader-key-autoloads
bind-map-autoloads delsel savehist xt-mouse edmacro kmacro no-littering
no-littering-autoloads finder-inf bind-key easy-mmode use-package-core
use-package-autoloads bind-key-autoloads straight-autoloads info
straight display-line-numbers comp comp-cstr warnings subr-x rx cl-seq
cl-macs cl-extra help-mode easymenu seq byte-opt gv cl-loaddefs cl-lib
bytecomp byte-compile cconv iso-transl tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face pcase macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads kqueue
cocoa ns lcms2 multi-tty make-network-process nativecomp emacs)

Memory information:
((conses 16 1761473 271986)
 (symbols 48 80678 10)
 (strings 32 267395 57671)
 (string-bytes 1 11486389)
 (vectors 16 116952)
 (vector-slots 8 3083586 139837)
 (floats 8 2008 996)
 (intervals 56 60290 4855)
 (buffers 984 121))

Message #8 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: 46617 <at> debbugs.gnu.org
Cc: Andrea Corallo <akrl <at> sdf.org>
Subject: Re: 28.0.50; nativecomp: native compile cache is not invalidated when
 file is re-byte compiled and changes
Date: Wed, 24 Feb 2021 23:48:18 -0600

I just updated, recompiled and was able to launch Emacs without
incident using a previously compiled straight.el. Is it possible that
this was fixed on the native comp branch?

Aaron

Message #13 received at 46617-done <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 46617-done <at> debbugs.gnu.org
Subject: Re: 28.0.50; nativecomp: native compile cache is not invalidated
 when file is re-byte compiled and changes
Date: Thu, 25 Feb 2021 09:04:24 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> I just updated, recompiled and was able to launch Emacs without
> incident using a previously compiled straight.el. Is it possible that
> this was fixed on the native comp branch?
>
> Aaron

Hi Aaron,

Even tho I've no precise analysis on this most likely this is related to
the fact that I broke async compilation with 81b1013555 and fixed it
with 0ee1a16769.

Right I'm closing this (we can always reopen in case).

Thanks for reporting it!

  Andrea

Message #16 received at 46617-done <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 46617-done <at> debbugs.gnu.org
Subject: Re: 28.0.50; nativecomp: native compile cache is not invalidated when
 file is re-byte compiled and changes
Date: Thu, 25 Feb 2021 10:09:08 -0600

On Thu, Feb 25, 2021 at 3:04 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>
> Even tho I've no precise analysis on this most likely this is related to
> the fact that I broke async compilation with 81b1013555 and fixed it
> with 0ee1a16769.
>
> Right I'm closing this (we can always reopen in case).

Ah, I'm afraid we have to reopen. What you describe was the reason it
didn't repro for me this last time. It still does currently.

Message #21 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 46617 <at> debbugs.gnu.org
Subject: Re: 28.0.50; nativecomp: native compile cache is not invalidated
 when file is re-byte compiled and changes
Date: Thu, 25 Feb 2021 17:01:40 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Thu, Feb 25, 2021 at 3:04 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>>
>> Even tho I've no precise analysis on this most likely this is related to
>> the fact that I broke async compilation with 81b1013555 and fixed it
>> with 0ee1a16769.
>>
>> Right I'm closing this (we can always reopen in case).
>
> Ah, I'm afraid we have to reopen. What you describe was the reason it
> didn't repro for me this last time. It still does currently.

No problem, should be reopened by now :)

  Andrea

Message #24 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 46617 <at> debbugs.gnu.org, 46617-done <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Sun, 21 Mar 2021 17:27:30 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Thu, Feb 25, 2021 at 3:04 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>>
>> Even tho I've no precise analysis on this most likely this is related to
>> the fact that I broke async compilation with 81b1013555 and fixed it
>> with 0ee1a16769.
>>
>> Right I'm closing this (we can always reopen in case).
>
> Ah, I'm afraid we have to reopen. What you describe was the reason it
> didn't repro for me this last time. It still does currently.

Hi Aaron,

I had to slightly modify your init.el reproducer as we have changed the
semantic of `load' [1].

init.el:
=========
(let ((repro
       (expand-file-name "repro.el" default-directory)))
  (defun some-fn ()
    1)
  (message "Byte and native compiling, some-fn = 1")
  (byte-compile-file repro)
  (load (native-compile repro))

  (message "Byte compiling, some-fn = 2")
  (defun some-fn ()
    2)
  (byte-compile-file repro)
  (load (concat repro "c") nil 'nomessage 'nosuffix)

  (message "Native compiling, some-fn = 2")
  (load (native-compile repro)))
=========

This is still:

- defining `some-fn'
- native compiling and loading repro.el
- re-defining `some-fn'
- byte compiling and loading repro.el
- native compiling and loading repro.el

Running ./src/emacs -Q -l init.el in the *Messages* buffer I get:

=========
Byte and native compiling, some-fn = 1
Compiling /home/andcor03/emacs2/repro.el...done
Wrote /home/andcor03/emacs2/repro.elc
Compiling /home/andcor03/emacs2/repro.el...done
Loading /home/andcor03/.emacs.d/eln-cache/28.0.50-32137a9a/repro-fd364f11-b5afb0b4.eln (native compiled elisp)...
EQUAL
Loading /home/andcor03/.emacs.d/eln-cache/28.0.50-32137a9a/repro-fd364f11-b5afb0b4.eln (native compiled elisp)...done
Byte compiling, some-fn = 2
Compiling /home/andcor03/emacs2/repro.el...done
Wrote /home/andcor03/emacs2/repro.elc
EQUAL
Native compiling, some-fn = 2
Compiling /home/andcor03/emacs2/repro.el...done
Loading /home/andcor03/.emacs.d/eln-cache/28.0.50-32137a9a/repro-fd364f11-b5afb0b4.eln (native compiled elisp)...
EQUAL
Loading /home/andcor03/.emacs.d/eln-cache/28.0.50-32137a9a/repro-fd364f11-b5afb0b4.eln (native compiled elisp)...done
=========

I think what happened is that in the original reproducer the second load
of the .elc file still automatically loaded in place the corresponding
.eln as the source file didn't changed.  The issue here is that in the
compilation unit repro.elk is capturing some of the environment and this
is not accounted in the hash computation of the .eln file.

The new reproducer (well it does not reproduce much :) ) works because
now `load' called explicitly on a .elc file does not load automatically
the corresponding .eln.  Alternatively to overcome the issue still using
native compilation one has to force a new native compilation using
`native-compile.  WDYT?

Thanks

  Andrea

[1] <https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-03/msg01643.html>

Message #32 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 46617 <at> debbugs.gnu.org, 46617-done <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Sun, 21 Mar 2021 12:36:39 -0500

On Sun, Mar 21, 2021 at 12:27 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>
> Hi Aaron,
>
> I had to slightly modify your init.el reproducer as we have changed the
> semantic of `load' [1].

With this change, how do you specify that you want the eln to load if
it's there? If I understand correctly, this prevents loading the eln
with load unless it is specified explicitly? Or can you leave the
extension off when loading and that will prefer the eln, then elc,
then el?

> I think what happened is that in the original reproducer the second load
> of the .elc file still automatically loaded in place the corresponding
> .eln as the source file didn't changed.  The issue here is that in the
> compilation unit repro.elk is capturing some of the environment and this
> is not accounted in the hash computation of the .eln file.

The original repro simulated a restart without a restart. The real bug
(in straight.el) is that on start, it loads a byte compiled/native
compiled file that relies on outside environment. So, that may be
fixed now because I believe they load the elc explicitly, but that
means it'll never load the native compiled version. Ideally, they
would, but I don't know how to force loading byte compiled OR native
compiled in one command.


> The new reproducer (well it does not reproduce much :) ) works because
> now `load' called explicitly on a .elc file does not load automatically
> the corresponding .eln.  Alternatively to overcome the issue still using
> native compilation one has to force a new native compilation using
> `native-compile.  WDYT?

I think it sounds like straight.el will need to add detection for
native compilation, and any time they byte compile, also native
compile. I also think that this is extremely fragile and I wonder if
byte compiling should result in "invalidating" (deleting) the
previously corresponding eln. We essentially have a cache invalidation
that is relying purely on a cache key that is incomplete.

Thanks,

Aaron

Message #40 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Sun, 21 Mar 2021 17:59:23 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Sun, Mar 21, 2021 at 12:27 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>>
>> Hi Aaron,
>>
>> I had to slightly modify your init.el reproducer as we have changed the
>> semantic of `load' [1].
>
> With this change, how do you specify that you want the eln to load if
> it's there? If I understand correctly, this prevents loading the eln
> with load unless it is specified explicitly? Or can you leave the
> extension off when loading and that will prefer the eln, then elc,
> then el?

Exactly load by default tries to add the extentions to the FILE
argument.

>> I think what happened is that in the original reproducer the second load
>> of the .elc file still automatically loaded in place the corresponding
>> .eln as the source file didn't changed.  The issue here is that in the
>> compilation unit repro.elk is capturing some of the environment and this
>> is not accounted in the hash computation of the .eln file.
>
> The original repro simulated a restart without a restart. The real bug
> (in straight.el) is that on start, it loads a byte compiled/native
> compiled file that relies on outside environment. So, that may be
> fixed now because I believe they load the elc explicitly, but that
> means it'll never load the native compiled version. Ideally, they
> would, but I don't know how to force loading byte compiled OR native
> compiled in one command.

ATM to force the byte-code version to be used one can use `load' making
the .elc extension explicit (or without using the extension binding
`load-no-native' to non-nil).

To explicitly force load the native code version one has to do like:

(load (comp-el-to-eln-filename ".../foo.el"))

But this is limited as it works only if the .eln is in the first
directory of `com-eln-load-path', so probably we have to offer a more
complete solution.

>> The new reproducer (well it does not reproduce much :) ) works because
>> now `load' called explicitly on a .elc file does not load automatically
>> the corresponding .eln.  Alternatively to overcome the issue still using
>> native compilation one has to force a new native compilation using
>> `native-compile.  WDYT?
>
> I think it sounds like straight.el will need to add detection for
> native compilation, and any time they byte compile, also native
> compile. I also think that this is extremely fragile and I wonder if
> byte compiling should result in "invalidating" (deleting) the
> previously corresponding eln. We essentially have a cache invalidation
> that is relying purely on a cache key that is incomplete.

Mmhh maybe it's a good idea.  The other option would be to re-introduce
the constrain that, to be used in place, the .eln must be younger than
the .elc.

Thanks

  Andrea

Message #43 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Sun, 21 Mar 2021 20:54:04 +0000

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:

> Aaron Jensen <aaronjensen <at> gmail.com> writes:
>
>> On Sun, Mar 21, 2021 at 12:27 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>>>
>>> Hi Aaron,
>>>
>>> I had to slightly modify your init.el reproducer as we have changed the
>>> semantic of `load' [1].
>>
>> With this change, how do you specify that you want the eln to load if
>> it's there? If I understand correctly, this prevents loading the eln
>> with load unless it is specified explicitly? Or can you leave the
>> extension off when loading and that will prefer the eln, then elc,
>> then el?
>
> Exactly load by default tries to add the extentions to the FILE
> argument.
>
>>> I think what happened is that in the original reproducer the second load
>>> of the .elc file still automatically loaded in place the corresponding
>>> .eln as the source file didn't changed.  The issue here is that in the
>>> compilation unit repro.elk is capturing some of the environment and this
>>> is not accounted in the hash computation of the .eln file.
>>
>> The original repro simulated a restart without a restart. The real bug
>> (in straight.el) is that on start, it loads a byte compiled/native
>> compiled file that relies on outside environment. So, that may be
>> fixed now because I believe they load the elc explicitly, but that
>> means it'll never load the native compiled version. Ideally, they
>> would, but I don't know how to force loading byte compiled OR native
>> compiled in one command.
>
> ATM to force the byte-code version to be used one can use `load' making
> the .elc extension explicit (or without using the extension binding
> `load-no-native' to non-nil).
>
> To explicitly force load the native code version one has to do like:
>
> (load (comp-el-to-eln-filename ".../foo.el"))
>
> But this is limited as it works only if the .eln is in the first
> directory of `com-eln-load-path', so probably we have to offer a more
> complete solution.

Okay I've added `comp-lookup-eln' son now the user can search if a
certain .eln is in reach in `comp-eln-load-path' for a given .el.

So now one can use the following to force a native load.

(load (comp-lookup-eln ".../foo.el"))

  Andrea

Message #46 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Sun, 21 Mar 2021 19:44:45 -0500

On Sun, Mar 21, 2021 at 3:54 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>
> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>
>
> Okay I've added `comp-lookup-eln' son now the user can search if a
> certain .eln is in reach in `comp-eln-load-path' for a given .el.
>
> So now one can use the following to force a native load.
>
> (load (comp-lookup-eln ".../foo.el"))

Okay, and that would return nil if it's not native compiled, which
would be their cue to native compile it, yes? That's nice to be able
to force load it.

Without doing the modification date stuff you mentioned or purge
before byte compile, native compilation ends up having to be a concern
of anything that's doing byte compiling, which is unfortunate. Ideally
Straight.el could just change to doing a (load "foo") after the
explicit byte compile and get the automatic async native compilation.
They could also do a delete file if the result of comp-lookup-eln
exists, prior to or just after byte compiling, then at least they
won't have to native compile synchronously.

Thanks,

Aaron

Message #49 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Mon, 22 Mar 2021 07:57:40 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Sun, Mar 21, 2021 at 3:54 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>>
>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
>> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>>
>>
>> Okay I've added `comp-lookup-eln' son now the user can search if a
>> certain .eln is in reach in `comp-eln-load-path' for a given .el.
>>
>> So now one can use the following to force a native load.
>>
>> (load (comp-lookup-eln ".../foo.el"))
>
> Okay, and that would return nil if it's not native compiled, which
> would be their cue to native compile it, yes?

Yes `comp-lookup-eln' will return nil if the eln is not found.

> That's nice to be able
> to force load it.
>
> Without doing the modification date stuff you mentioned or purge
> before byte compile, native compilation ends up having to be a concern
> of anything that's doing byte compiling, which is unfortunate.

Say anything that's doing byte compiling deliberately capturing
environment values yes.

  Andrea

> Ideally
> Straight.el could just change to doing a (load "foo") after the
> explicit byte compile and get the automatic async native compilation.
> They could also do a delete file if the result of comp-lookup-eln
> exists, prior to or just after byte compiling, then at least they
> won't have to native compile synchronously.
>
> Thanks,
>
> Aaron

Message #52 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Fri, 26 Mar 2021 07:54:47 +0000

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:

> Aaron Jensen <aaronjensen <at> gmail.com> writes:
>
>> On Sun, Mar 21, 2021 at 3:54 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>>>
>>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
>>> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>>>
>>>
>>> Okay I've added `comp-lookup-eln' son now the user can search if a
>>> certain .eln is in reach in `comp-eln-load-path' for a given .el.
>>>
>>> So now one can use the following to force a native load.
>>>
>>> (load (comp-lookup-eln ".../foo.el"))
>>
>> Okay, and that would return nil if it's not native compiled, which
>> would be their cue to native compile it, yes?
>
> Yes `comp-lookup-eln' will return nil if the eln is not found.
>
>> That's nice to be able
>> to force load it.
>>
>> Without doing the modification date stuff you mentioned or purge
>> before byte compile, native compilation ends up having to be a concern
>> of anything that's doing byte compiling, which is unfortunate.
>
> Say anything that's doing byte compiling deliberately capturing
> environment values yes.
>
>   Andrea

I've reintroduced the file timestamp check on the eln files being loaded
with 79b8b6ca45.  This should protect us from loading stale eln that
captured some stale env during compilation.

I'm Ccing Andrew and this was previously discussed in bug#43532, (note
now .el and .eln timestamps can now be equal so it should work also for
the discussed case in bug#43532).

Please let us know if this change is problematic.

Thanks

  Andrea

Message #55 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Sun, 28 Mar 2021 13:24:49 -0500

On Fri, Mar 26, 2021 at 2:54 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>
> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>
> > Aaron Jensen <aaronjensen <at> gmail.com> writes:
> >
> >> On Sun, Mar 21, 2021 at 3:54 PM Andrea Corallo <akrl <at> sdf.org> wrote:
> >>>
> >>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> >>> text editors" <bug-gnu-emacs <at> gnu.org> writes:
> >>>
> >>>
> >>> Okay I've added `comp-lookup-eln' son now the user can search if a
> >>> certain .eln is in reach in `comp-eln-load-path' for a given .el.
> >>>
> >>> So now one can use the following to force a native load.
> >>>
> >>> (load (comp-lookup-eln ".../foo.el"))
> >>
> >> Okay, and that would return nil if it's not native compiled, which
> >> would be their cue to native compile it, yes?
> >
> > Yes `comp-lookup-eln' will return nil if the eln is not found.
> >
> >> That's nice to be able
> >> to force load it.
> >>
> >> Without doing the modification date stuff you mentioned or purge
> >> before byte compile, native compilation ends up having to be a concern
> >> of anything that's doing byte compiling, which is unfortunate.
> >
> > Say anything that's doing byte compiling deliberately capturing
> > environment values yes.
> >
> >   Andrea
>
> I've reintroduced the file timestamp check on the eln files being loaded
> with 79b8b6ca45.  This should protect us from loading stale eln that
> captured some stale env during compilation.

What does it compare the timestamp of the eln to? It doesn't appear
that it's the elc, which I believe is the only thing that would help
with the issue we're seeing in straight.el. Is it the el or something
else?

A related issue I've found is that now when you load an elc
explicitly, it triggers a native compile for that file regardless of
whether or not one has already been done.

One general question is, is the eln a compilation of the elc or the el?

Thanks,

Aaron

Message #58 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Mon, 29 Mar 2021 08:14:55 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Fri, Mar 26, 2021 at 2:54 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>>
>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
>> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>>
>> > Aaron Jensen <aaronjensen <at> gmail.com> writes:
>> >
>> >> On Sun, Mar 21, 2021 at 3:54 PM Andrea Corallo <akrl <at> sdf.org> wrote:
>> >>>
>> >>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
>> >>> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>> >>>
>> >>>
>> >>> Okay I've added `comp-lookup-eln' son now the user can search if a
>> >>> certain .eln is in reach in `comp-eln-load-path' for a given .el.
>> >>>
>> >>> So now one can use the following to force a native load.
>> >>>
>> >>> (load (comp-lookup-eln ".../foo.el"))
>> >>
>> >> Okay, and that would return nil if it's not native compiled, which
>> >> would be their cue to native compile it, yes?
>> >
>> > Yes `comp-lookup-eln' will return nil if the eln is not found.
>> >
>> >> That's nice to be able
>> >> to force load it.
>> >>
>> >> Without doing the modification date stuff you mentioned or purge
>> >> before byte compile, native compilation ends up having to be a concern
>> >> of anything that's doing byte compiling, which is unfortunate.
>> >
>> > Say anything that's doing byte compiling deliberately capturing
>> > environment values yes.
>> >
>> >   Andrea
>>
>> I've reintroduced the file timestamp check on the eln files being loaded
>> with 79b8b6ca45.  This should protect us from loading stale eln that
>> captured some stale env during compilation.
>
> What does it compare the timestamp of the eln to? It doesn't appear
> that it's the elc, which I believe is the only thing that would help
> with the issue we're seeing in straight.el. Is it the el or something
> else?

Compares against the .elc.

> A related issue I've found is that now when you load an elc
> explicitly, it triggers a native compile for that file regardless of
> whether or not one has already been done.

Do you have a reproducer I can look at?

> One general question is, is the eln a compilation of the elc or the
> el?

Native compilation uses as input always the source (.el) file.

Thanks

  Andrea

Message #61 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Mon, 29 Mar 2021 11:26:39 -0500

On Mon, Mar 29, 2021 at 3:15 AM Andrea Corallo <akrl <at> sdf.org> wrote:
> Compares against the .elc.

For some reason, I do not see this behavior. Touching the elc files
and restarting does not trigger a recompile of the elns.

> Do you have a reproducer I can look at?

Yes, you can follow straight.el's installation instructions. Create a
new .emacs.d/init.el with:

(defvar bootstrap-version)
(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el"
user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

Load emacs, let it install straight and do all native compilation.

Restart emacs. It will recompile straight.el every time you restart.

It will also recompile every time you do:

(load "~/.emacs.d/straight/repos/straight.el/straight.elc" nil
'nomessage 'nosuffix)

One thing of note here is that because of the way straight works there
ends up being two straight.elc files. One in straight/build/straight
and another in straight/repos/straight.el. The former has a
straight.el that symlinks to the latter, but the straight.elcs are
both concrete.

If you attempt to load the one in build, multiple "Compilation
finished." messages get written to the Async-native-compile-log, which
is odd as well:

(load "~/.emacs.d/straight/build/straight/straight.elc" nil 'nomessage
'nosuffix)

Thanks,

Aaron

Message #64 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Wed, 31 Mar 2021 10:22:29 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Mon, Mar 29, 2021 at 3:15 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>> Compares against the .elc.
>
> For some reason, I do not see this behavior. Touching the elc files
> and restarting does not trigger a recompile of the elns.

That's odd, I do.

>> Do you have a reproducer I can look at?
>
> Yes, you can follow straight.el's installation instructions. Create a
> new .emacs.d/init.el with:
>
> (defvar bootstrap-version)
> (let ((bootstrap-file
>        (expand-file-name "straight/repos/straight.el/bootstrap.el"
> user-emacs-directory))
>       (bootstrap-version 5))
>   (unless (file-exists-p bootstrap-file)
>     (with-current-buffer
>         (url-retrieve-synchronously
>          "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
>          'silent 'inhibit-cookies)
>       (goto-char (point-max))
>       (eval-print-last-sexp)))
>   (load bootstrap-file nil 'nomessage))
>
> Load emacs, let it install straight and do all native compilation.
>
> Restart emacs. It will recompile straight.el every time you restart.
>
> It will also recompile every time you do:
>
> (load "~/.emacs.d/straight/repos/straight.el/straight.elc" nil
> 'nomessage 'nosuffix)
>
> One thing of note here is that because of the way straight works there
> ends up being two straight.elc files. One in straight/build/straight
> and another in straight/repos/straight.el. The former has a
> straight.el that symlinks to the latter, but the straight.elcs are
> both concrete.

Okay I think I see what's the issue.

We have changed the `load' semantinc and now when the a .elc file is
explicitly requested for load we do not load the .eln even if present.
But at this point we don't have to trigger the native compilation for
that file.

I'll come-up with a fix.

Thanks

  Andrea

Message #67 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Wed, 31 Mar 2021 11:30:35 -0500

On Wed, Mar 31, 2021 at 5:22 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>
> Aaron Jensen <aaronjensen <at> gmail.com> writes:
>
> > On Mon, Mar 29, 2021 at 3:15 AM Andrea Corallo <akrl <at> sdf.org> wrote:
> >> Compares against the .elc.
> >
> > For some reason, I do not see this behavior. Touching the elc files
> > and restarting does not trigger a recompile of the elns.
>
> That's odd, I do.

Just tested again on master and it seems to work as it should. I must
be on an older build.

> Okay I think I see what's the issue.
>
> We have changed the `load' semantinc and now when the a .elc file is
> explicitly requested for load we do not load the .eln even if present.
> But at this point we don't have to trigger the native compilation for
> that file.
>
> I'll come-up with a fix.

Great, thank you.

Aaron

Message #70 received at 46617 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617 <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Wed, 31 Mar 2021 17:58:46 +0000

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> On Wed, Mar 31, 2021 at 5:22 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>>
>> Aaron Jensen <aaronjensen <at> gmail.com> writes:
>>
>> > On Mon, Mar 29, 2021 at 3:15 AM Andrea Corallo <akrl <at> sdf.org> wrote:
>> >> Compares against the .elc.
>> >
>> > For some reason, I do not see this behavior. Touching the elc files
>> > and restarting does not trigger a recompile of the elns.
>>
>> That's odd, I do.
>
> Just tested again on master and it seems to work as it should. I must
> be on an older build.
>
>> Okay I think I see what's the issue.
>>
>> We have changed the `load' semantinc and now when the a .elc file is
>> explicitly requested for load we do not load the .eln even if present.
>> But at this point we don't have to trigger the native compilation for
>> that file.
>>
>> I'll come-up with a fix.
>
> Great, thank you.

Right 613caa9527 should fix the described issue for me.  I've also tried
the straigh reproducer and I do not see recompilations during the second
startup.

Could you give it a try to verify it works for you too?

Thanks!

 Andrea

Message #75 received at 46617-done <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Andrew Whatson <whatson <at> gmail.com>, 46617-done <at> debbugs.gnu.org
Subject: Re: bug#46617: 28.0.50; nativecomp: native compile cache is not
 invalidated when file is re-byte compiled and changes
Date: Wed, 07 Apr 2021 18:48:44 +0000

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:

[...]

> Right 613caa9527 should fix the described issue for me.  I've also tried
> the straigh reproducer and I do not see recompilations during the second
> startup.
>
> Could you give it a try to verify it works for you too?
>
> Thanks!
>
>  Andrea

I'm closing this as seams to work for me.

Happy to re-open if necessary.

Thanks

  Andrea

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.