GNU bug report logs - #45433
bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish possibly miscompiled

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45433 in the body.
You can then email your comments to 45433 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; [feature/native-comp] batch-native-compiling ivy and
 ox-publish possibly miscompiles
Date: Fri, 25 Dec 2020 12:23:52 -0300

After compiling files individually with batch-native-compile, I get the
following errors when requiring the packages in my config file:

- ivy
Debugger entered--Lisp error: (void-variable fboundp)
  byte-code("\300\301\302\303!\203\f\0\303\202\n\0\304\305#\207" [defalias ivy--file-local-name fboundp file-local-name #f(compiled-function (file) #<bytecode 0x2005c2840d4428e>) "Compatibility shim for `file-local-name'.\nThe func..."] 4)
  require(ivy nil t)

- ox-publish (from org elpa)
Debugger entered--Lisp error: (invalid-function "A symbol, denoting if folders are sorted first in ...")
  "A symbol, denoting if folders are sorted first in site-maps.\n\nPossible values are `first', `last', `ignore' and nil.\nIf `first', folders will be sorted before files.\nIf `last', folders are sorted to the end after the files.\nIf `ignore', folders do not appear in the site-map.\nAny other value will mix files and folders.\n\nYou can overwrite this default per project in your\n`org-publish-project-alist', using `:sitemap-sort-folders'.\n\nThis variable is ignored when site-map style is `tree'."(:group org-export-publish :type (choice (const :tag "Folders before files" first) (const :tag "Folders after files" last) (const :tag "No folder in site-map" ignore) (const :tag "Mix folders and files" nil)) :version "26.1" :package-version (Org . "9.1") :safe symbolp)
  byte-code("\300\301\302\303\304\305\306\307&\7\210\310\311\312\313\314DD\315\306\301\316\317&\7\210\310\320\312\313\321DD\322\306\301\316\323&\7\210\310\324\312\313\325DD\326\306..." [custom-declare-group org-export-publish nil "Options for publishing a set of files." :tag "Org Publishing" :group org-export custom-declare-variable org-publish-project-alist funcall function #f(compiled-function () #<bytecode 0x1f4000153e91>) "Association list to control publishing behavior.\n\\..." :type alist org-publish-use-timestamps-flag #f(compiled-function () #<bytecode 0x1f4000153e5d>) "Non-nil means use timestamp checking to publish on..." boolean org-publish-timestamp-directory #f(compiled-function () #<bytecode -0x19384979f873f719>) "Name of directory in which to store publishing tim..." directory org-publish-list-skipped-files #f(compiled-function () #<bytecode 0x1f4000153e5d>) "Non-nil means show message about files *not* publi..." org-publish-sitemap-sort-files #f(compiled-function () #<bytecode 0x1f400c190add>) "Method to sort files in site-maps.\nPossible values..." symbol org-publish-sitemap-sort-folders #f(compiled-function () #<bytecode 0x1fb7ec096ac25>) "A symbol, denoting if folders are sorted first in ..." (choice (const :tag "Folders before files" first) (const :tag "Folders after files" last) (const :tag "No folder in site-map" ignore) (const :tag "Mix folders and files" nil)) :version "26.1" :package-version (Org . "9.1") :safe symbolp org-publish-sitemap-sort-ignore-case #f(compiled-function () #<bytecode 0x1f4000153e91>) "Non-nil when site-map sorting should ignore case.\n..."] 14)
  require(ox-publish)


Configured using:
 'configure
 --prefix=/nix/store/akx3lh6h8vyj63xgi396mylbhn3lmc4p-emacs-gcc-20201217.0
 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft
 --with-cairo --with-nativecomp'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY
INOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES NATIVE_COMP THREADS
LIBSYSTEMD JSON PDUMPER

Important settings:
  value of $EMACSLOADPATH: 
  value of $EMACSNATIVELOADPATH: /nix/store/mzk2nvl6yqgcfxi69mkyvkcx0bzy6awp-emacs-packages-deps/share/emacs/native-lisp::
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: IBuffer

Minor modes in effect:
  ace-window-display-mode: t
  org-roam-bibtex-mode: t
  org-roam-mode: t
  pdf-occur-ibuffer-minor-mode: t
  pdf-occur-global-minor-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  shell-dirtrack-mode: t
  global-fixmee-mode: t
  projectile-mode: t
  direnv-mode: t
  doom-modeline-mode: t
  display-time-mode: t
  display-battery-mode: t
  key-chord-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Memory information:
((conses 16 2437834 228693)
 (symbols 48 58024 0)
 (strings 32 323981 19871)
 (string-bytes 1 9492122)
 (vectors 16 403485)
 (vector-slots 8 5283857 166413)
 (floats 8 1042 857)
 (intervals 56 28601 6576)
 (buffers 984 36))

Message #8 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: 45433 <at> debbugs.gnu.org
Subject: bug#45433: 28.0.50; [feature/native-comp] batch-native-compiling
 ivy and ox-publish possibly miscompiles
Date: Fri, 25 Dec 2020 16:57:31 -0300

I tested with trunk. The above errors are still there, but I also got a
scarier one when requiring esxml-query, reproduced below. To clarify,
this is after running emacs --batch -f batch-native-compile individually
on every file in site-lisp, because that's what my package manager does.

Debugger entered--Lisp error: (error "binding stack not balanced (serious byte compiler ...")
  byte-code("\306\30\307\31\310\32\311\312\10\"\33\311\313\13\"\34\311\314\n\f#\35\311\315\n\f#\36 \316\36!\311\317\11\n\f$\36\"\311\320\11\n\f$\36#\311\321..." [h nl nonascii unicode escape nmstart "[0-9a-f]" "\n\\|\n\n\\|\n\\|\f" "[\200-�]" format "\\\\%s\\{1,6\\}[ \11\n\n\f]?" "\\(?:%s\\)\\|\\\\[ -~\200-�]" "[a-z_]\\|%s\\|\\(?:%s\\)" "[a-z0-9_-]\\|%s\\|\\(?:%s\\)" "[0-9]+\\|[0-9]*\\.[0-9]+" "\"\\(?:[\11 !#$%%&(-~]\\|\\\\\\(?:%s\\)\\|'\\|%s\\|\\(?:%s\\)\\)*..." "'\\(?:[\11 !#$%%&(-~]\\|\\\\\\(?:%s\\)\\|\"\\|%s\\|\\(?:%s\\)\\)*..." "[-]?\\(?:%s\\)\\(?:%s\\)*" "[-]?\\(?:%s\\)\\(?:%s\\)+" "\\(?:%s\\)+" (whitespace . "[ \11\n\n\f]+") string "\\(?:%s\\|%s\\)" ident hash "#%s" function "%s(" number dimension "\\(?:%s\\)%s" ((prefix-match . "\\^=") (suffix-match . "\\$=") (substring-match . "\\*=") (include-match . "~=") (dash-match . "|=") (comma . ",") (gt . ">") (plus . "\\+") (minus . "-") (tilde . "~") (asterisk . "\\*") (period . "\\.") (equals . "=") (colon . ":") (lbracket . "\\[") (rbracket . "\\]") (rparen . ")")) nmchar num string1 string2 unit name] 11)
  (defvar esxml--css-selector-token-matchers (byte-code "\306\30\307\31\310\32\311\312\10\"\33\311\313\13\"\34\311\314\n\f#\35\311\315\n\f#\36 \316\36!\311\317\11\n\f$\36\"\311\320\11\n\f$\36#\311\321..." [h nl nonascii unicode escape nmstart "[0-9a-f]" "\n\\|\n\n\\|\n\\|\f" "[\200-�]" format "\\\\%s\\{1,6\\}[ \11\n\n\f]?" "\\(?:%s\\)\\|\\\\[ -~\200-�]" "[a-z_]\\|%s\\|\\(?:%s\\)" "[a-z0-9_-]\\|%s\\|\\(?:%s\\)" "[0-9]+\\|[0-9]*\\.[0-9]+" "\"\\(?:[\11 !#$%%&(-~]\\|\\\\\\(?:%s\\)\\|'\\|%s\\|\\(?:%s\\)\\)*..." "'\\(?:[\11 !#$%%&(-~]\\|\\\\\\(?:%s\\)\\|\"\\|%s\\|\\(?:%s\\)\\)*..." "[-]?\\(?:%s\\)\\(?:%s\\)*" "[-]?\\(?:%s\\)\\(?:%s\\)+" "\\(?:%s\\)+" (whitespace . "[ \11\n\n\f]+") string "\\(?:%s\\|%s\\)" ident hash "#%s" function "%s(" number dimension "\\(?:%s\\)%s" ((prefix-match . "\\^=") (suffix-match . "\\$=") (substring-match . "\\*=") (include-match . "~=") (dash-match . "|=") (comma . ",") (gt . ">") (plus . "\\+") (minus . "-") (tilde . "~") (asterisk . "\\*") (period . "\\.") (equals . "=") (colon . ":") (lbracket . "\\[") (rbracket . "\\]") (rparen . ")")) nmchar num string1 string2 unit name] 11))
  require(esxml-query)

Message #11 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; [feature/native-comp]
 batch-native-compiling ivy and ox-publish possibly miscompiles
Date: Fri, 25 Dec 2020 20:52:15 +0000

Mauricio Collares <mauricio <at> collares.org> writes:

> I tested with trunk. The above errors are still there, but I also got a
> scarier one when requiring esxml-query, reproduced below. To clarify,
> this is after running emacs --batch -f batch-native-compile individually
> on every file in site-lisp, because that's what my package manager does.

Hi Mauricio,

you mean that on trunk you get the same error using `batch-byte-compile'
on both the mentioned files?

  Andrea

Message #14 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; [feature/native-comp]
 batch-native-compiling ivy and ox-publish possibly miscompiles
Date: Fri, 25 Dec 2020 18:41:13 -0300

Andrea Corallo <akrl <at> sdf.org> writes:
>
> Hi Mauricio,
>
> you mean that on trunk you get the same error using `batch-byte-compile'
> on both the mentioned files?
>
>   Andrea

Hi Andrea,

Sorry, I misspoke! All of the tests (including the one I will describe
below) were performed on the feature/native-comp branch, not trunk. What
I meant to say is that I retested using the latest commit, since my
original build was a week old. The error messages I get when requiring
ivy, ox-publish and esxml-query on the latest commit of the native-comp
branch match the three error messages I sent before.

I also tested if just using batch-byte-compile was enough to trigger the
error when requiring esxml-query, and it wasn't. It only happens when
using batch-native-compile. I didn't test deferred compilation, though.

Best,
Mauricio

Message #17 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Sat, 26 Dec 2020 20:54:59 +0000

Mauricio Collares <mauricio <at> collares.org> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>>
>> Hi Mauricio,
>>
>> you mean that on trunk you get the same error using `batch-byte-compile'
>> on both the mentioned files?
>>
>>   Andrea
>
> Hi Andrea,
>
> Sorry, I misspoke! All of the tests (including the one I will describe
> below) were performed on the feature/native-comp branch, not trunk. What
> I meant to say is that I retested using the latest commit, since my
> original build was a week old. The error messages I get when requiring
> ivy, ox-publish and esxml-query on the latest commit of the native-comp
> branch match the three error messages I sent before.
>
> I also tested if just using batch-byte-compile was enough to trigger the
> error when requiring esxml-query, and it wasn't. It only happens when
> using batch-native-compile. I didn't test deferred compilation, though.
>
> Best,
> Mauricio

Hi Mauricio,

I had a look, this issue is not native compiler related as I get the
same behavior byte compiling.

The trouble seems with `ivy-add-face-text-property' being defaliased,
ATM I'm not sure what is going on tho.

  Andrea

Message #22 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Sun, 27 Dec 2020 14:02:57 -0300

Andrea Corallo <akrl <at> sdf.org> writes:
>
> Hi Mauricio,
>
> I had a look, this issue is not native compiler related as I get the
> same behavior byte compiling.
>
> The trouble seems with `ivy-add-face-text-property' being defaliased,
> ATM I'm not sure what is going on tho.
>
>   Andrea

Hi Andrea,

I appreciate you taking a look at this and I apologise for being
insistent, but I am curious to isolate what's causing the difference of
behaviour here. Here's the exact set of steps I am following. In
~/test.el, I have these four lines:

(setq comp-deferred-compilation nil)
(add-to-list 'load-path "/home/collares")
(setq comp-eln-load-path '("/home/collares/"))
(require 'ivy)

I also have ivy's elisp files in my home directory (colir.el,
ivy-autoloads.el, ivy.el, ivy-faces.el, ivy-overlay.el,
ivy-pkg.el). With this, I get the following output:

$ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")" --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" -f batch-byte-compile ~/ivy.el                                                                              
$ emacs -Q --script ~/test.el

$ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")" --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" -f batch-native-compile ~/ivy.el                                                                            
$ emacs -Q --script ~/test.el
Debugger entered--Lisp error: (void-variable fboundp)
  byte-code("\300\301\302\303!\203\f\0\303\202\n\0\304\305#\207" [defalias ivy--file-local-name fboundp file-local-name #f(compiled-function (file) #<bytecode 0x163b76c1ebc309bd>) "Compatibility shim for `file-local-name'.\nThe func..."] 4)
  require(ivy)
  eval-buffer(#<buffer  *load*> nil "/home/collares/test.el" nil t)  ; Reading at buffer position 286
  load-with-code-conversion("/home/collares/test.el" "/home/collares/test.el" nil t)
  command-line-1(("-scriptload" "/home/collares/test.el"))
  command-line()
  normal-top-level()

Do you know what could be causing this difference in behaviour? I would
be happy to provide any debug information that's helpful.

Best,
Mauricio

Message #25 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 30 Dec 2020 10:54:30 -0300

Andrea Corallo <akrl <at> sdf.org> writes:

> Hi Mauricio,
>
> I had a look, this issue is not native compiler related as I get the
> same behavior byte compiling.
>
> The trouble seems with `ivy-add-face-text-property' being defaliased,
> ATM I'm not sure what is going on tho.
>
>   Andrea

Hi Andrea,

Here's an attempt at a minimized testcase:

* Contents of ~/ivy.el:

(defun ivy--call-marked (action)
  (let* ((prefix-len (length ivy-mark-prefix))
         (marked-candidates
          (mapcar
           (lambda (s)
             (let ((cand (substring s prefix-len)))
               (if ivy--directory
                   (expand-file-name cand ivy--directory)
                 cand)))
           ivy-marked-candidates))
         (multi-action (ivy--get-multi-action ivy-last)))))

(defalias 'ivy--file-local-name
  (if (fboundp 'file-local-name)
      #'file-local-name
    (lambda (file)
      (or (file-remote-p file 'localname) file))))

(provide 'ivy)

* Run the following three commands (note that skipping either command 1
  or command 2 makes the last command pass):

$ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
-f batch-byte-compile ~/ivy.el

$ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
--eval "(setq comp-eln-load-path '(\"/home/collares/\"))" -f batch-native-compile ~/ivy.el

$ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
--eval "(setq comp-eln-load-path '(\"/home/collares/\"))" --eval "(require 'ivy)"

With this, I still get the same error locally:

[... five undefined variable/function warnings, due to the code I
deleted to minimize the testcase ...] 
Debugger entered--Lisp error: (void-variable fboundp)
  byte-code("\300\301\302\303!\203\f\0\303\202\n\0\304\"\210\305\306!\207" [defalias ivy--file-local-name fboundp file-local-name #f(compiled-function (file) #<bytecode 0x5f7d1a5a33e66f3>) provide ivy] 4)
  require(ivy)
  command-line-1(("--eval" "(add-to-list 'load-path \"/home/collares\")" "--eval" "(setq comp-eln-load-path '(\"/home/collares/\"))" "--eval" "(require 'ivy)"))
  command-line()
  normal-top-level()

Best,
Mauricio

Message #28 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 30 Dec 2020 14:44:12 +0000

Mauricio Collares <mauricio <at> collares.org> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>
>> Hi Mauricio,
>>
>> I had a look, this issue is not native compiler related as I get the
>> same behavior byte compiling.
>>
>> The trouble seems with `ivy-add-face-text-property' being defaliased,
>> ATM I'm not sure what is going on tho.
>>
>>   Andrea
>
> Hi Andrea,
>
> Here's an attempt at a minimized testcase:
>
> * Contents of ~/ivy.el:
>
> (defun ivy--call-marked (action)
>   (let* ((prefix-len (length ivy-mark-prefix))
>          (marked-candidates
>           (mapcar
>            (lambda (s)
>              (let ((cand (substring s prefix-len)))
>                (if ivy--directory
>                    (expand-file-name cand ivy--directory)
>                  cand)))
>            ivy-marked-candidates))
>          (multi-action (ivy--get-multi-action ivy-last)))))
>
> (defalias 'ivy--file-local-name
>   (if (fboundp 'file-local-name)
>       #'file-local-name
>     (lambda (file)
>       (or (file-remote-p file 'localname) file))))
>
> (provide 'ivy)
>
> * Run the following three commands (note that skipping either command 1
>   or command 2 makes the last command pass):
>
> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
> -f batch-byte-compile ~/ivy.el
>
> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
> --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" -f batch-native-compile ~/ivy.el
>
> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
> --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" --eval "(require 'ivy)"
>
> With this, I still get the same error locally:
>
> [... five undefined variable/function warnings, due to the code I
> deleted to minimize the testcase ...] 
> Debugger entered--Lisp error: (void-variable fboundp)
>   byte-code("\300\301\302\303!\203\f\0\303\202\n\0\304\"\210\305\306!\207" [defalias ivy--file-local-name fboundp file-local-name #f(compiled-function (file) #<bytecode 0x5f7d1a5a33e66f3>) provide ivy] 4)
>   require(ivy)
>   command-line-1(("--eval" "(add-to-list 'load-path \"/home/collares\")" "--eval" "(setq comp-eln-load-path '(\"/home/collares/\"))" "--eval" "(require 'ivy)"))
>   command-line()
>   normal-top-level()
>
> Best,
> Mauricio

Hi Mauricio,

I had no time to progress with it but I want to look into it in the
coming days.

Thanks for reducing the test-case, this is greatly appreciated and will
certainly save me time.

  Andrea
  

Message #31 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Mon, 04 Jan 2021 22:12:20 +0000

Mauricio Collares <mauricio <at> collares.org> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>
>> Hi Mauricio,
>>
>> I had a look, this issue is not native compiler related as I get the
>> same behavior byte compiling.
>>
>> The trouble seems with `ivy-add-face-text-property' being defaliased,
>> ATM I'm not sure what is going on tho.
>>
>>   Andrea
>
> Hi Andrea,
>
> Here's an attempt at a minimized testcase:
>
> * Contents of ~/ivy.el:
>
> (defun ivy--call-marked (action)
>   (let* ((prefix-len (length ivy-mark-prefix))
>          (marked-candidates
>           (mapcar
>            (lambda (s)
>              (let ((cand (substring s prefix-len)))
>                (if ivy--directory
>                    (expand-file-name cand ivy--directory)
>                  cand)))
>            ivy-marked-candidates))
>          (multi-action (ivy--get-multi-action ivy-last)))))
>
> (defalias 'ivy--file-local-name
>   (if (fboundp 'file-local-name)
>       #'file-local-name
>     (lambda (file)
>       (or (file-remote-p file 'localname) file))))
>
> (provide 'ivy)
>
> * Run the following three commands (note that skipping either command 1
>   or command 2 makes the last command pass):
>
> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
> -f batch-byte-compile ~/ivy.el
>
> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
> --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" -f batch-native-compile ~/ivy.el
>
> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
> --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" --eval "(require 'ivy)"
>
> With this, I still get the same error locally:

Hi Mauricio,

thanks for reducing, I can reproduce this version too.

It looks more like a load issue then a miscompilation.  The interesting
part is that a priori I don't see how having the bytecode compiled
should influence loading the eln, very curious...

I'm calling the day but I'll prepare a debug build and look into it
further tomorrow.

  Andrea

Message #34 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Tue, 05 Jan 2021 09:24:28 -0300

Andrea Corallo <akrl <at> sdf.org> writes:
>
> Hi Mauricio,
>
> thanks for reducing, I can reproduce this version too.

Hi Andrea,

Very glad to hear this bug is reproducible, thanks a lot for taking a
look at it!

The esxml-query example (if it has the same root cause) is interesting
because of the byte-compiler assertion failure. To reproduce it, you can
follow the same steps as before with this in ~/ivy.el:

(defvar esxml--css-selector-token-matchers
  (let ((v1 "a")
        (v2 "a")
        (v3 "a")
        (v4 "a")
        (v5 "a")
        (v6 "a")
        (v7 "a")
        (v8 "a")
        (v9 "a")
        (v10 "a")
        (v11 "a")
        (v12 "a")
        (v13 "a"))
    `((function . ,(identity v13)))))

Running the same three commands as before then fails with:

Debugger entered--Lisp error: (error "binding stack not balanced (serious byte compiler ...")
  byte-code("\306\211\211\211\211\211\211\211\211\211\211\211\211\30\31\32\33\34\35\36\10\36\11\36\n\36\13\36\f\36\n\36\16\307\10.\nBC\207" [v13 v12 v11 v10 v9 v8 "a" function v7 v6 v5 v4 v3 v2 v1] 13)
  (defvar esxml--css-selector-token-matchers (byte-code "\306\211\211\211\211\211\211\211\211\211\211\211\211\30\31\32\33\34\35\36\10\36\11\36\n\36\13\36\f\36\n\36\16\307\10.\nBC\207" [v13 v12 v11 v10 v9 v8 "a" function v7 v6 v5 v4 v3 v2 v1] 13))

Best,
Mauricio

Message #37 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
 text editors" <bug-gnu-emacs <at> gnu.org>
Cc: Mauricio Collares <mauricio <at> collares.org>, 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 06 Jan 2021 00:13:36 +0000

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:

> Mauricio Collares <mauricio <at> collares.org> writes:
>
>> Andrea Corallo <akrl <at> sdf.org> writes:
>>
>>> Hi Mauricio,
>>>
>>> I had a look, this issue is not native compiler related as I get the
>>> same behavior byte compiling.
>>>
>>> The trouble seems with `ivy-add-face-text-property' being defaliased,
>>> ATM I'm not sure what is going on tho.
>>>
>>>   Andrea
>>
>> Hi Andrea,
>>
>> Here's an attempt at a minimized testcase:
>>
>> * Contents of ~/ivy.el:
>>
>> (defun ivy--call-marked (action)
>>   (let* ((prefix-len (length ivy-mark-prefix))
>>          (marked-candidates
>>           (mapcar
>>            (lambda (s)
>>              (let ((cand (substring s prefix-len)))
>>                (if ivy--directory
>>                    (expand-file-name cand ivy--directory)
>>                  cand)))
>>            ivy-marked-candidates))
>>          (multi-action (ivy--get-multi-action ivy-last)))))
>>
>> (defalias 'ivy--file-local-name
>>   (if (fboundp 'file-local-name)
>>       #'file-local-name
>>     (lambda (file)
>>       (or (file-remote-p file 'localname) file))))
>>
>> (provide 'ivy)
>>
>> * Run the following three commands (note that skipping either command 1
>>   or command 2 makes the last command pass):
>>
>> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
>> -f batch-byte-compile ~/ivy.el
>>
>> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
>> --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" -f batch-native-compile ~/ivy.el
>>
>> $ emacs -Q --batch --eval "(add-to-list 'load-path \"/home/collares\")"
>> --eval "(setq comp-eln-load-path '(\"/home/collares/\"))" --eval "(require 'ivy)"
>>
>> With this, I still get the same error locally:
>
> Hi Mauricio,
>
> thanks for reducing, I can reproduce this version too.
>
> It looks more like a load issue then a miscompilation.

All right I did some investigation.  This is how the top level form
looks in the .elc file:

(byte-code "\300\301\302\303!\203\f^@\303\202^M^@\304\"\210\305\306!\207" [defalias xxx--file-local-name fboundp file-local-name #[257 "\300^A\301\"\206^H^@\211\207" [file-
remote-p localname] 4 "\n\n(fn FILE)"] provide xxx] 4)

When we execute this from the .elc in exec_byte_code we have:

(gdb) p *(bytestr_data+10)
$43 = 13 '\r'

While loading from .eln we have:

(gdb) p *(bytestr_data+10)
$31 = 10 '\n'

ATM I've no idea why of this difference, I'm pretty sure in the eln we
have 13 so might be a reader miss-configuration?

I'll look into it more to see where the difference is coming from,
hopefully should be pretty narrowed at this stage.

  Andrea

Message #43 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: "Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of
 text editors" <bug-gnu-emacs <at> gnu.org>, 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 06 Jan 2021 09:03:35 -0300

Andrea Corallo <akrl <at> sdf.org> writes:
> All right I did some investigation.  This is how the top level form
> looks in the .elc file:
>
> (byte-code "\300\301\302\303!\203\f^@\303\202^M^@\304\"\210\305\306!\207" [defalias xxx--file-local-name fboundp file-local-name #[257 "\300^A\301\"\206^H^@\211\207" [file-
> remote-p localname] 4 "\n\n(fn FILE)"] provide xxx] 4)
>
> When we execute this from the .elc in exec_byte_code we have:
>
> (gdb) p *(bytestr_data+10)
> $43 = 13 '\r'
>
> While loading from .eln we have:
>
> (gdb) p *(bytestr_data+10)
> $31 = 10 '\n'
>
> ATM I've no idea why of this difference, I'm pretty sure in the eln we
> have 13 so might be a reader miss-configuration?
>
> I'll look into it more to see where the difference is coming from,
> hopefully should be pretty narrowed at this stage.

Hi Andrea,

You're definitely onto something here! The numbers 13 and 10 seem very
suspicious, and the esxml-query code also only fails with 13
variables. Maybe the .elc is being opened as a text file when
native-compiling and the newlines are being converted to Unix style when
reading?

Best,
Mauricio

Message #49 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: "Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of
 text editors" <bug-gnu-emacs <at> gnu.org>, 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 06 Jan 2021 13:06:25 +0000

Mauricio Collares <mauricio <at> collares.org> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>> All right I did some investigation.  This is how the top level form
>> looks in the .elc file:
>>
>> (byte-code
>> "\300\301\302\303!\203\f^@\303\202^M^@\304\"\210\305\306!\207"
>> [defalias xxx--file-local-name fboundp file-local-name #[257
>> "\300^A\301\"\206^H^@\211\207" [file-
>> remote-p localname] 4 "\n\n(fn FILE)"] provide xxx] 4)
>>
>> When we execute this from the .elc in exec_byte_code we have:
>>
>> (gdb) p *(bytestr_data+10)
>> $43 = 13 '\r'
>>
>> While loading from .eln we have:
>>
>> (gdb) p *(bytestr_data+10)
>> $31 = 10 '\n'
>>
>> ATM I've no idea why of this difference, I'm pretty sure in the eln we
>> have 13 so might be a reader miss-configuration?
>>
>> I'll look into it more to see where the difference is coming from,
>> hopefully should be pretty narrowed at this stage.
>
> Hi Andrea,
>
> You're definitely onto something here! The numbers 13 and 10 seem very
> suspicious, and the esxml-query code also only fails with 13
> variables. Maybe the .elc is being opened as a text file when
> native-compiling and the newlines are being converted to Unix style when
> reading?

Indeed this smells of EOL conversion as one is '/r' and the other '/n'.
Yesterday I tried quickly setting 'inhibit_eol_conversion' to true while
reading but had no success.  I might have been wrong as well and perhaps
the issue is while printing?

I'll look into it more this evening or tomorrow.

  Andrea

Message #55 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 06 Jan 2021 14:56:43 +0000

Hi Mauricio,

I've good news (got a small enlightenment), 33b8ce865f should fix :)
Would you like to give it a try?

Thanks!

  Andrea

Message #58 received at 45433 <at> debbugs.gnu.org (full text, mbox):

From: Mauricio Collares <mauricio <at> collares.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 45433 <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 06 Jan 2021 16:56:59 -0300

Andrea Corallo <akrl <at> sdf.org> writes:

> Hi Mauricio,
>
> I've good news (got a small enlightenment), 33b8ce865f should fix :)
> Would you like to give it a try?

Hi Andrea,

This fixed ivy, ox-publish and esxml-query for me, many thanks! Also,
thanks for the patience and fast replies.

By the way, congratulations on the excellent work you've been doing on
native compilation! It is extremely impressive and much
appreciated. Can't wait for it to get merged to trunk so most people can
benefit from it too :)

Best,
Mauricio

Message #63 received at 45433-done <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Mauricio Collares <mauricio <at> collares.org>
Cc: 45433-done <at> debbugs.gnu.org
Subject: Re: bug#45433: 28.0.50; byte/native-compiling ivy and ox-publish
 possibly miscompiled
Date: Wed, 06 Jan 2021 20:04:35 +0000

Mauricio Collares <mauricio <at> collares.org> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>
>> Hi Mauricio,
>>
>> I've good news (got a small enlightenment), 33b8ce865f should fix :)
>> Would you like to give it a try?
>
> Hi Andrea,
>
> This fixed ivy, ox-publish and esxml-query for me, many thanks! Also,
> thanks for the patience and fast replies.
>
> By the way, congratulations on the excellent work you've been doing on
> native compilation! It is extremely impressive and much
> appreciated. Can't wait for it to get merged to trunk so most people can
> benefit from it too :)

Thank you for reporting and helping to track down this bug!

Closing then

Thanks

  Andrea

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.